System

Abstract

To provide a system.SOLUTION: A system comprises: means for analyzing an input question; means for generating an answer only from specific information sources by using a generative AI model; means for calculating a probability of the validity of the answer by using the generative AI model; and means for presenting the generated answer, the calculated probability of the validity of the answer, and the information sources of the answer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional chatbots have the problem that they explicitly state that their answers are "incorrect" based on probabilistic theory, and that "humans cannot determine the correctness of their answers." In addition, it is difficult to clearly indicate to users the resources that led to the answer, and it is not possible to restrict answers to specific research paper databases, etc. [Means for solving the problem]

[0005] This invention provides a means to clearly indicate that a response is probabilistically "incorrect," a means to indicate that "a human cannot determine its validity," a means to clearly indicate the resources used to arrive at the response, or a means to restrict the response to only using a paper database, etc. This makes it easier for users to determine the validity of the chatbot's response and improves the reliability of the response. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15]FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] An embodiment of the present invention is realized as a chatbot system. This system includes an algorithm for generating answers to questions from users. This algorithm has the function of specifying what is "incorrect" probabilistically. Specifically, it generates possible answers to questions from users and probabilistically evaluates whether each answer is correct. This evaluation is performed by calculating the probability that each answer will be judged to be correct. "Embodiment Example 2"

[0029] Furthermore, this embodiment of the present invention has a function that indicates that "humans cannot determine the validity of the answers." Specifically, for each generated answer, the probability that the answer will be judged correct is presented to the user. This allows the user to judge the validity of the system's answers for themselves.

[0030] "Example 3"

[0031] Furthermore, this embodiment of the present invention has a function of clearly indicating the resources that led to the answer. Specifically, the system presents to the user the information sources (e.g., a specific paper database) that were referenced to generate the answer. This allows the user to know which information sources the system's answer is based on. The system can also impose restrictions such as limiting the answer to a specific paper database. This can improve the reliability of the system's answer.

[0032] The processing flow of each embodiment will be described below.

[0033] "Example 1"

[0034] Step 1: Receive a user question.

[0035] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[0036] Step 3: Probabilistically evaluate whether each generated answer is correct. This is done by calculating the probability that each answer is judged to be correct.

[0037] Step 4: Present the probabilistic evaluation results to the user, allowing the user to judge for themselves the validity of the system's answer.

[0038] "Example 2"

[0039] Step 1: Receive a user question.

[0040] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[0041] Step 3: For each generated answer, the user is presented with the probability that the answer is judged to be correct, allowing the user to judge for themselves the validity of the system's answers.

[0042] "Example 3"

[0043] Step 1: Receive a user question.

[0044] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[0045] Step 3: Present the user with the sources (e.g., a specific paper database) that were consulted to generate the answer, allowing the user to understand which sources the system based its answer on.

[0046] Step 4: Set constraints such as limiting answers to specific paper databases. This can improve the reliability of the system's answers.

[0047] Example 1

[0048] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] With conventional chatbot systems, it is difficult to guarantee the accuracy of answers to user questions, and there is a risk that incorrect information will be provided. Furthermore, since users lack a means to determine the accuracy of answers, there is a risk that they will believe incorrect information. Furthermore, since there is no way to clearly indicate the basis or reliability of the answer, it is difficult for users to evaluate the reliability of the answer. To solve these issues, a system is needed that can probabilistically evaluate the accuracy of answers and clearly state that answers with low accuracy are "likely incorrect."

[0050] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0051] In this invention, the server includes a means for receiving a question from a user, a means for generating multiple answers using a generative AI model, a means for probabilistically evaluating the accuracy of each answer, a means for indicating that answers with low accuracy are "likely incorrect" based on the evaluation results, and a means for displaying the organized answers to the user. This allows the user to receive information about the accuracy of the provided answers as probabilistically evaluated, reducing the risk of believing incorrect information. Furthermore, clearly indicating the reliability of the answers makes it easier for the user to evaluate the reliability of the answers.

[0052] The "means for receiving a question from a user" is a function that allows the system to receive a question entered by a user.

[0053] "Means for generating multiple answers using a generative AI model" is a function for automatically generating multiple answers to a user's question using a generative AI model.

[0054] The "means for probabilistically evaluating the accuracy of each answer" refers to an algorithm or method for probabilistically evaluating the accuracy of each generated answer.

[0055] "Means for clearly indicating that answers with low accuracy are 'likely incorrect' based on the evaluation results" is a function for displaying "likely incorrect" for answers that are judged to be low in accuracy based on the accuracy evaluation results.

[0056] The "means for displaying organized answers to the user" is a function for displaying answers organized based on the evaluation results to the user.

[0057] A "specific generative algorithm" is a specific algorithm used by a generative AI model to generate an answer to a user's question.

[0058] A "particular evaluation algorithm" is a particular algorithm used to probabilistically evaluate the accuracy of a generated answer.

[0059] This invention relates to a chatbot system that generates answers to user questions and probabilistically evaluates their accuracy. This system operates among three parties: a server, a terminal, and a user.

[0060] The server includes means for receiving questions from users, means for generating multiple answers using a generative AI model, means for probabilistically evaluating the accuracy of each answer, means for indicating that answers with low accuracy are "likely incorrect" based on the evaluation results, and means for displaying the organized answers to the user.

[0061] Specifically, when the server receives a question from a user, it generates multiple possible answers using a generative AI model (e.g., GPT-4 (registered trademark)). For each generated answer, the server probabilistically evaluates its accuracy. This evaluation is performed using a specific evaluation algorithm. Based on the evaluation results, answers with low accuracy are marked as "likely incorrect." Finally, the organized answers are displayed to the user.

[0062] For example, if a user asks "What is the diameter of the Earth?" the server might process the question as follows:

[0063] 1. A user types into the chatbot interface, "What is the diameter of the Earth?"

[0064] 2. The device sends this question to the server.

[0065] 3. The server receives the question and inputs it into the generative AI model (GPT-4).

[0066] 4. The generative AI model generates two answers: "The diameter of the Earth is approximately 12,742 kilometers" and "The diameter of the Earth is approximately 10,000 kilometers."

[0067] 5. The server rates the accuracy of each answer, rating answer 1 at 95% and answer 2 at 20%.

[0068] 6. Based on the evaluation results, the server marks answer 2 as "likely incorrect."

[0069] 7. The server sends the organized response to the device.

[0070] 8. The device displays to the user "The diameter of the Earth is approximately 12,742 kilometers" and "The diameter of the Earth is approximately 10,000 kilometers (likely incorrect)."

[0071] Example prompt sentence:

[0072] User: What is the diameter of the Earth?

[0073] Server: The diameter of the Earth is approximately 12,742 kilometers.

[0074] Server: The diameter of the Earth is approximately 10,000 kilometers. (likely incorrect)

[0075] This system allows users to receive information that probabilistically evaluates the accuracy of the answers provided, reducing the risk of believing incorrect information. In addition, by clearly indicating the reliability of the answers, users can more easily evaluate the reliability of the answers.

[0076] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0077] Step 1:

[0078] The user enters a question.

[0079] The user inputs a question into the chatbot's interface, for example, "What is the diameter of the Earth?" The input question is sent to the terminal.

[0080] Step 2:

[0081] The terminal sends a question to the server.

[0082] The terminal sends the question entered by the user to the server, where the question is converted into an appropriate format: the input is the user's question, and the output is the question sent to the server.

[0083] Step 3:

[0084] The server receives the query.

[0085] The server receives questions sent from the device. The received questions are prepared for the next processing step. The input is the question sent from the device, and the output is the question data to be input into the generative AI model.

[0086] Step 4:

[0087] The server inputs questions into the generative AI model.

[0088] The server inputs the received question into a generative AI model (e.g., GPT-4). At this time, the question is converted into a format that the generative AI model can understand. The input is the question data, and the output is the question input to the generative AI model.

[0089] Step 5:

[0090] A generative AI model generates multiple answers.

[0091] A generative AI model generates multiple answers to an input question. For example, it generates answers such as "The diameter of the Earth is approximately 12,742 kilometers" or "The diameter of the Earth is approximately 10,000 kilometers." The input is the question entered into the generative AI model, and the output is the multiple answers generated.

[0092] Step 6:

[0093] The server probabilistically rates the accuracy of each answer.

[0094] The server applies an algorithm to each generated answer to probabilistically evaluate its accuracy. For example, answer 1 may be evaluated as 95% accurate, answer 2 as 20% accurate, etc. The input is the generated answers, and the output is the accuracy evaluation result for each answer.

[0095] Step 7:

[0096] The server organizes the answers based on the evaluation results.

[0097] The server organizes each answer based on the evaluation results. Answers with low accuracy are marked as "likely incorrect." The input is the accuracy evaluation result of each answer, and the output is the organized answer.

[0098] Step 8:

[0099] The server sends the organized response to the terminal.

[0100] The server sends the organized answer to the terminal. The answer is formatted in a way that is easy for the user to understand. The input is the organized answer, and the output is the answer sent to the terminal.

[0101] Step 9:

[0102] The terminal displays the answer to the user.

[0103] The terminal displays the answer sent from the server to the user. For example, it might say, "The diameter of the Earth is approximately 12,742 kilometers." or "The diameter of the Earth is approximately 10,000 kilometers. (This is likely incorrect.)" The input is the answer sent from the server, and the output is the answer displayed to the user.

[0104] (Application example 1)

[0105] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] With conventional chatbot systems, it is difficult to guarantee the accuracy of answers to user questions, and there is a risk of providing incorrect information, especially for security-related questions. Furthermore, if an answer is likely to be inaccurate, the system lacks the functionality to notify the user, which can lead to the user believing the incorrect information. This not only undermines user trust, but also increases security risks.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0108] In this invention, the server includes a means for probabilistically specifying "incorrect," a means for generating multiple answers to a user's question, and a means for probabilistically evaluating the accuracy of each answer, thereby making it possible to provide the user with the most reliable answer and to notify the user when an answer may be inaccurate.

[0109] "Means to clearly indicate that an answer is probabilistically incorrect" is a function that clearly indicates, based on a probabilistic evaluation, when there is a possibility that the answer provided to the user may not be accurate.

[0110] The "means for indicating that the validity cannot be determined by a human" is a function that indicates that the validity of the answer provided cannot be easily confirmed by human judgment.

[0111] "Means for clearly indicating the resources that led to the answer" is a function that clearly indicates to the user which information sources or databases the generated answer is based on.

[0112] "Means for restricting answers to only paper databases, etc." is a function that limits the information sources used when generating answers to specific databases or papers.

[0113] The "means for generating multiple answers to a question from a user" is a function that generates multiple possible answers to a question entered by a user.

[0114] The "means for probabilistically evaluating the accuracy of each answer" is a function for evaluating the accuracy of each generated answer using a probabilistic method.

[0115] "Means for selecting the most reliable answer and providing it to the user" is a function for selecting the most reliable answer from the evaluated answers and providing it to the user.

[0116] "Means for notifying the user if the answer may not be accurate" is a function that notifies the user if the answer provided may not be accurate.

[0117] As an embodiment of the present invention, a system for installing a security assistant application on a smartphone will be described, which provides probabilistically evaluated answers to security questions posed by a user.

[0118] System configuration

[0119] This system uses the following hardware and software:

[0120] Hardware

[0121] Smartphone

[0122] software

[0123] Generative AI models (e.g., GPT-3 (registered trademark))

[0124] Python (registered trademark) program

[0125] Data processing and calculation

[0126] When the server receives a question from a user, it uses a generative AI model to generate multiple answers. The generated answers are then evaluated using a probabilistic method. Specifically, the accuracy of each answer is probabilistically evaluated, and the most reliable answer is selected based on that evaluation.

[0127] If the selected answer may not be accurate, we will notify the user so that they can determine the reliability of the information provided.

[0128] Specific examples

[0129] If a user asks, "How can I tell if my password has been compromised?" the system responds:

[0130] This answer may not be accurate: To check for password leaks, use our data leak checker.

[0131] Prompt Sentence Examples

[0132] An example of a prompt for the user to enter a question is:

[0133] User: How can I check if my password has been compromised?

[0134] Assistant: Use our data leak checker to check for password leaks.

[0135] In this way, users can get reliable answers to their security questions and be notified if the answers may not be accurate, allowing them to determine the reliability of the information provided.

[0136] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0137] Step 1:

[0138] The user enters their security questions using their smartphone.

[0139] Input: User question (e.g., "How can I check if my password has been leaked?")

[0140] Output: The user's question is sent to the server.

[0141] Step 2:

[0142] The server inputs the received user question into a generative AI model.

[0143] Input: User question

[0144] Output: Multiple answers generated by the generative AI model (e.g., 5 answers)

[0145] Step 3:

[0146] The server probabilistically evaluates the generated answers.

[0147] Input: Multiple answers generated by a generative AI model

[0148] Output: A probability value indicating the accuracy of each answer (e.g., a confidence score for each answer)

[0149] Step 4:

[0150] The server selects the most reliable answer from the evaluated answers.

[0151] Input: Probability values ​​indicating the accuracy of each answer

[0152] Output: The most reliable answer (e.g., the answer with the highest confidence score)

[0153] Step 5:

[0154] The server will notify the user if the selected answer may not be accurate.

[0155] Input: The most reliable answer and its reliability score

[0156] Output: The answer provided to the user and a notification that it may not be accurate (e.g., "This answer may not be accurate: Use a data leak checker to check if your password has been leaked.")

[0157] Step 6:

[0158] The user checks the answer provided by the server on their smartphone.

[0159] Input: The answer provided by the server and a notification that it may not be accurate

[0160] Output: Answer and notification for user to review

[0161] In this way, users can get reliable answers to their security questions and be notified if the answers may not be accurate, allowing them to determine the reliability of the information provided.

[0162] Example 2

[0163] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] Systems using conventional generative AI models have had the problem that it is difficult for users to judge the validity of the generated answers. Furthermore, they lack the functionality to display the probability of an answer to evaluate its validity, which makes it difficult for users to trust the system's answers. Furthermore, there are insufficient means to clearly indicate the resources that led to the answer, making it difficult for users to confirm the basis for the answer.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0166] In this invention, the server includes: a means for a user to input a prompt sentence; a means for a terminal to send the prompt sentence to the server; a means for the server to receive the prompt sentence and pass it to a generative AI model; a means for the generative AI model to generate an answer based on the prompt sentence; a means for the server to receive the generated answer and calculate the probability of evaluating its validity; a means for the server to present the answer and the probability of its validity to the user; a means to clearly indicate that the answer is probabilistically "incorrect"; a means to indicate that the answer is "unvalid for a human to determine its validity"; and a means to clearly indicate the resources that led to the answer or to restrict the answer to be based solely on a database. This allows the user to probabilistically evaluate the validity of the generated answer and confirm the basis for the answer.

[0167] A "user" is an entity that utilizes the system to enter prompts and receive generated answers.

[0168] A "terminal" is a device that a user uses to input a prompt and send it to a server. For example, this could be a computer or a smartphone.

[0169] A "server" is a computer system that receives prompt sentences, passes them to a generative AI model, and presents the generated answers to the user.

[0170] A "prompt sentence" is the text of a question or request that a user enters into the system.

[0171] A "generative AI model" is an artificial intelligence algorithm that generates answers based on prompts, such as a natural language processing model.

[0172] An "answer" is text that a generative AI model generates based on a prompt.

[0173] The "probability of evaluating correctness" is a numerical value indicating the probability that the generated answer will be judged to be correct.

[0174] A "means of probabilistically specifying 'incorrect'" is a method for probabilistically indicating the possibility that the generated answer is incorrect.

[0175] The "means of demonstrating that the validity cannot be determined by a human" is a method of providing information that allows a user to judge for themselves the validity of the generated answer.

[0176] "Resources" are the information sources or databases used to arrive at the generated answer.

[0177] The "means for restricting answers to be given only from a database" is a method for generating answers using only information obtained from a specific database.

[0178] This invention relates to a system that allows a user to input a prompt sentence, generates an answer using a generative AI model, and evaluates the validity of the answer. Specific embodiments of this system are described below.

[0179] First, the user inputs a prompt using a terminal. The terminal is a device such as a computer or smartphone, and is responsible for sending the prompt input by the user to the server. Examples of prompts include "What is the diameter of the Earth?" and "How many planets are there in the solar system?"

[0180] Next, the device sends the prompt sentence to the server. The server receives the prompt sentence and passes it to the generative AI model. For example, a natural language processing model is used as the generative AI model. Specifically, GPT-4 from OpenAI (registered trademark) is one such model.

[0181] The generative AI model generates an answer based on the received prompt, for example, "The diameter of the Earth is approximately 12,742 kilometers." This generated answer is returned to the server.

[0182] The server receives the generated answer and calculates the probability of evaluating its correctness. This probability is calculated based on the internal algorithm of the generative AI model. For example, if the probability that the generated answer will be judged correct is 85%, the server calculates this probability.

[0183] Finally, the server presents the answer and its probability of correctness to the user. For example, it displays the answer in the form of "The diameter of the Earth is approximately 12,742 kilometers. (Probability of correctness: 85%)." This allows the user to probabilistically evaluate the correctness of the generated answer and check the basis of the answer.

[0184] This system includes a means to clearly indicate that a result is probabilistically "incorrect," a means to show that the "correctness cannot be determined by humans," a means to clearly indicate the resources that led to the answer, or a means to restrict the answer to only the database. This allows users to trust the system's answers and use them appropriately.

[0185] As a concrete example, if a user inputs the prompt "What is the diameter of the Earth?", the server will use the generative AI model to generate the answer "The diameter of the Earth is approximately 12,742 kilometers," evaluate the probability of its correctness as 85%, and present it to the user. In this way, the user can use the system while checking the correctness of the answer.

[0186] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0187] Step 1:

[0188] The user enters a prompt statement.

[0189] The user types a question or request into an input field on the terminal, for example, "What is the diameter of the Earth?" The entered prompt is temporarily stored in the terminal's memory.

[0190] Step 2:

[0191] The terminal sends the prompt to the server.

[0192] The device sends the input prompt text to the server as an HTTP POST request. At this time, the device sends data using an Internet connection. The input data is the prompt text, and the output is a notification to the server that transmission has been completed.

[0193] Step 3:

[0194] The server receives the prompt sentence and passes it to the generative AI model.

[0195] The server receives the prompt sentence sent from the terminal. After receiving it, the server passes this prompt sentence to the generative AI model in an appropriate format (for example, JSON format). The input data is the prompt sentence, and the output is the input data to the generative AI model.

[0196] Step 4:

[0197] A generative AI model generates an answer based on the prompt.

[0198] A generative AI model generates an answer based on the prompt it receives, for example, "The diameter of the Earth is approximately 12,742 kilometers." The input data is the prompt, and the output is the generated answer.

[0199] Step 5:

[0200] The server receives the generated answer and calculates the probability of its correctness.

[0201] The server receives the answer generated by the generative AI model. Then, the server calculates the probability to evaluate the correctness of this answer. This probability is calculated based on the internal algorithm of the generative AI model. The input data is the generated answer, and the output is the probability of correctness.

[0202] Step 6:

[0203] The server presents the answer and its probability of correctness to the user.

[0204] The server presents the generated answer and its probability of correctness to the user. For example, it displays "The diameter of the Earth is approximately 12,742 kilometers. (Probability of correctness: 85%)" on a web page. The input data is the generated answer and its probability of correctness, and the output is the display to the user.

[0205] (Application example 2)

[0206] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0207] Conventional generative AI models do not provide users with information to determine the legitimacy of the generated answers, making it difficult for users to evaluate the reliability of the answers. Furthermore, while they are required to provide appropriate answers to security-related questions, the problem is that users cannot determine the legitimacy of the answers. This creates a risk that users will act based on incorrect information.

[0208] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0209] In this invention, the server includes a means for clearly indicating that an answer is probabilistically "incorrect," a means for indicating that "the validity of the answer cannot be determined by humans," a means for specifying the resources used to arrive at the answer or a means for restricting the answer to only a paper database, etc., a means for presenting to the user the probability that each generated answer is judged to be correct, and a means for the generative AI model to generate an answer to a question entered by the user. This allows the user to confirm the validity of the generated answer as a probability, and to evaluate the reliability of the answer.

[0210] "Means for specifying probabilistically incorrect" is a means for probabilistically indicating the possibility that the generated answer is incorrect.

[0211] The "means for demonstrating that the validity cannot be determined by a human" is a means for demonstrating that the validity of the generated answer cannot be easily determined by a human.

[0212] "Means for clearly indicating the resources that led to the answer" refers to means for clearly indicating what information sources or databases the generated answer is based on.

[0213] The "means for restricting answers to be given only in the paper database, etc." is a means for restricting the generated answers to be given only in a specific database, for example, a paper database.

[0214] "Means for presenting to the user, for each generated answer, the probability that the answer will be judged to be correct" refers to means for calculating the probability that the generated answer will be judged to be correct and presenting that probability to the user.

[0215] "Means by which a generative AI model generates an answer to a question entered by a user" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[0216] In the system for implementing this invention, a generative AI model generates an answer to a question entered by a user and presents the validity of the answer as a probability. A specific embodiment of this system is described below.

[0217] System Configuration

[0218] The system is configured using the following hardware and software.

[0219] Hardware: Smartphone

[0220] Software: Python, OpenAI API

[0221] Data processing and calculation

[0222] 1. User enters question:

[0223] Users enter security questions through a smartphone application, such as "Is this password secure?"

[0224] 2. Generate prompt sentence:

[0225] The smartphone application converts the user's question into a prompt, which looks like this:

[0226] User Question: Is this password secure?

[0227] answer:

[0228] 3. Invoke the generative AI model:

[0229] The smartphone application sends the generated prompt sentences to OpenAI's generative AI model (e.g., GPT-3), which generates an appropriate answer.

[0230] 4. Probability calculation of the correctness of the answer:

[0231] The correctness of the generated answer is calculated as a probability. Using a hypothetical logic, it is determined to be correct with, for example, an 85% probability.

[0232] 5. User Presentation:

[0233] The smartphone application presents the generated answer and its probability of correctness to the user, for example, "Answer: Yes, it's safe. Probability of correctness: 85%."

[0234] Specific examples

[0235] If a user types the question "Is this password secure?", the system will:

[0236] 1. Convert the user's question into a prompt.

[0237] User Question: Is this password secure?

[0238] answer:

[0239] 2. Invoke a generative AI model to generate an answer.

[0240] 3. Calculate the correctness of your answer as a probability.

[0241] 4. Present the user with "Answer: Yes, it's safe. Probability of validity: 85%."

[0242] In this way, the user can check the validity of the generated answer as a probability, and can evaluate the reliability of the answer.

[0243] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0244] Step 1:

[0245] The user launches the application on their smartphone and enters security questions.

[0246] Input: User question (e.g. "Is this password secure?")

[0247] Output: The text of the user's question

[0248] Specific behavior: The user enters a question into the application's input field and presses the submit button.

[0249] Step 2:

[0250] The terminal receives the user's question and generates a prompt sentence.

[0251] Input: User question text

[0252] Output: Prompt text (e.g. "User Question: Is this password secure?\nAnswer:")

[0253] What it does: The application formats the user's question and creates a prompt suitable for the generative AI model.

[0254] Step 3:

[0255] The device sends the generated prompt to OpenAI's generative AI model to generate an answer.

[0256] Input: prompt statement

[0257] Output: The answer from the generative AI model (e.g., "Yes, it's safe")

[0258] What it does: The application calls the OpenAI API to send prompts and get answers.

[0259] Step 4:

[0260] The terminal calculates the correctness of the generated answer as a probability.

[0261] Input: Answer from a generative AI model

[0262] Output: Probability of the answer being correct (e.g. 85%)

[0263] What it does: The application uses hypothetical logic to calculate the probability of the answer being correct.

[0264] Step 5:

[0265] The terminal presents the generated answer and its probability of correctness to the user.

[0266] Input: Answer from generative AI model, probability of answer correctness

[0267] Output: Information presented to the user (e.g., "Answer: Yes, it's safe. Probability of validity: 85%")

[0268] Specific behavior: The application displays the generated answer and its probability of correctness on the screen and presents it to the user.

[0269] Example 3

[0270] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0271] In answer generation systems using conventional generative AI models, it is difficult for users to judge the legitimacy and reliability of the generated answers. In addition, because the information sources that led to the answer are not made clear, users cannot confirm the basis for the answer. Furthermore, because it is not possible to restrict the use of only specific information sources, the reliability of the answer may be reduced. There is a need to solve these problems.

[0272] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0273] In this invention, the server includes means for probabilistically indicating that the answer is "incorrect," means for indicating that the "legitimateness cannot be determined by humans," means for indicating the information sources that led to the answer, means for restricting the answer to only specific information sources, means for receiving and analyzing the user's question, means for generating an answer using a generative AI model, and means for presenting the generated answer and its information source to the user. This allows the user to confirm the legitimacy and reliability of the generated answer and clarify the basis for the answer. Furthermore, restricting the use of only specific information sources can improve the reliability of the answer.

[0274] The "means for clearly indicating that the answer is probabilistically incorrect" is a means for notifying the user that the generated answer may be probabilistically incorrect.

[0275] The "means for indicating that the validity cannot be determined by a human" is a means for indicating to the user that the validity of the generated answer cannot be easily determined by a human.

[0276] The "means for clearly indicating the information source that led to the answer" is a means for clearly indicating to the user which information source the generated answer is based on.

[0277] The "means for restricting the system to generate an answer using only a specific information source" is a means for restricting the system to generate an answer using only a specific information source.

[0278] The "means for receiving and analyzing a user's question" is a means for receiving a question entered by a user and analyzing that question.

[0279] "Means for generating an answer using a generative AI model" refers to means for generating an answer to a user's question using a generative AI model.

[0280] The "means for presenting the generated answer and its information source to the user" is a means for presenting the generated answer and the information source that led to the answer to the user.

[0281] This invention is a system that uses a generative AI model to generate answers to user questions and clearly indicates the information sources that led to the answers. Specific embodiments of this system are described below.

[0282] Hardware and software used

[0283] 1. Hardware

[0284] Server: Use a server with a powerful processor and sufficient memory. For example, use a server with a server-grade processor.

[0285] Terminal: A device that allows users to input questions. For example, a device that can connect to the Internet, such as a PC or smartphone.

[0286] 2. Software

[0287] Natural language processing libraries: used to parse the question. For example, use libraries such as SpaCy or NLTK.

[0288] Generative AI models: used to generate answers, for example, OpenAI's GPT-3.

[0289] Database API: Used to reference information sources, for example, using the API of a scientific literature database or a medical literature database.

[0290] System Operation

[0291] 1. User inputs a question

[0292] The user uses the device to input a question. For example, they open a browser and type "Please tell me about papers on the latest AI technology" into the system interface.

[0293] 2. Receiving and analyzing questions

[0294] The server receives the user's question and analyzes it using natural language processing (NLP) technology. For example, the server receives the question and analyzes the intent of the question using the SpaCy library. Specifically, it extracts the keywords "latest AI technology" and "paper."

[0295] 3. References to sources

[0296] The server refers to a specified information source (e.g., a specific paper database). For example, the server calls the API of a scientific and technical literature database to search for papers on "the latest AI technology."

[0297] 4. Answer generation using generative AI models

[0298] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate an answer based on the referenced information source. For example, the server inputs a prompt to GPT-3, such as "Please tell me papers about the latest AI technology," and obtains the generated answer. Specifically, it generates the answer, "Paper A was published in 2023."

[0299] 5. Source citation

[0300] The server presents the information source that led to the answer to the user. For example, the server may add information such as "This answer is based on a scientific and technical literature database."

[0301] 6. Presenting the Answer

[0302] The server sends the generated answer and its source to the user's device and displays it to the user. For example, the user's device might display a message such as, "Paper A, published in 2023, is a paper on the latest AI technology. This answer is based on a scientific and technical literature database."

[0303] Specific examples

[0304] User question input

[0305] Prompt: "Please tell me some papers about the latest AI technology."

[0306] Answer generation using generative AI models

[0307] Answer: "Paper A, published in 2023, is a paper on the latest AI technology."

[0308] Clarification of sources

[0309] Source: "This answer is based on scientific and technical literature databases."

[0310] In this way, the user can know which information source the system's answer is based on and can confirm the reliability of the answer. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0311] Step 1:

[0312] The user uses the terminal to enter a question.

[0313] Input: The user types a question into the terminal interface.

[0314] Specific operation: The user opens a browser and types "Please tell me papers about the latest AI technology" into the system interface.

[0315] Output: The user's question is sent from the terminal to the server.

[0316] Step 2:

[0317] The server receives the user's question and analyzes it using natural language processing (NLP) techniques.

[0318] Input: User question (e.g., "Please tell me papers about the latest AI technology.")

[0319] Specific operation: The server receives the question and analyzes the intent of the question using the SpaCy library. Specifically, it extracts the keywords "latest AI technology" and "paper."

[0320] Output: Extracted keywords and analysis results are obtained.

[0321] Step 3:

[0322] The server consults the specified information source (e.g., a particular research database).

[0323] Input: Extracted keywords and analysis results (e.g., "latest AI technology," "paper")

[0324] Specific operation: The server calls the API of a scientific and technical literature database and searches for papers on "latest AI technology."

[0325] Output: The search results will give you a list of relevant papers.

[0326] Step 4:

[0327] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate an answer based on the referenced sources.

[0328] Input: A list of papers obtained as search results and a prompt (e.g., "Please tell me papers about the latest AI technologies")

[0329] Specific operation: The server inputs the prompt and search results to GPT-3 and obtains the generated answer. Specifically, it generates the answer "Paper A was published in 2023."

[0330] Output: You get the generated answer.

[0331] Step 5:

[0332] The server presents the user with the information sources that led to the answer.

[0333] Input: The generated answer and its source (e.g., "Paper A was published in 2023. This answer is based on a scientific and technical literature database.")

[0334] Specific behavior: The server appends the source of the generated answer.

[0335] Output: Answers with source citations.

[0336] Step 6:

[0337] The server generates an answer and sends the answer and its source to the user's terminal and displays it to the user.

[0338] Input: Answer with source information (e.g., "Paper A was published in 2023. This answer is based on a scientific and technical literature database.")

[0339] Specific operation: The server sends a response to the user's device, specifying the source of the information.

[0340] Output: The answer is displayed on the user's terminal.

[0341] In this way, the user can see which sources the system's answers are based on and can verify the reliability of the answers.

[0342] (Application example 3)

[0343] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] In conventional information provision systems, it was difficult for users to judge the reliability of the information provided, and doubts often arose about the accuracy and reliability of the information, especially when the source of the information, such as a news article, was not clearly stated. In addition, there was a lack of means to probabilistically determine the legitimacy of the information, which created the risk that users would make decisions based on incorrect information. This could undermine user trust.

[0345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0346] In this invention, the server includes a means for probabilistically indicating that a result is "incorrect," a means for indicating that the "validity cannot be determined by humans," a means for indicating the resources used to arrive at the answer or a means for restricting the answer to only using a paper database, etc., and a means for indicating the source of each news article to increase its reliability. This allows users to easily determine the reliability of the information provided and avoid making decisions based on incorrect information. Furthermore, indicating the source of the information increases user trust.

[0347] "Means of clearly stating that information is probabilistically incorrect" refers to a means of using a specific algorithm to determine the accuracy of information probabilistically, and as a result, clearly stating that there is a possibility that the information is incorrect.

[0348] "Means for demonstrating that the legitimacy cannot be determined by humans" refers to means for demonstrating that the legitimacy of the information provided cannot be easily determined by human judgment.

[0349] "Means for indicating the resources that led to the answer" means a means for showing the user which information sources the system-generated answer is based on.

[0350] "Means for imposing a constraint that answers be given only from a research paper database, etc." means a means for imposing a constraint that the system generate answers by referring only to a specific database (for example, a research paper database).

[0351] "Means of clearly indicating which sources each article is based on in order to increase the reliability of news articles" refers to means of clearly indicating to users which sources each article is based on in order to increase the reliability of news articles.

[0352] As an embodiment of the present invention, a news distribution application will be described as an example. The server includes means for probabilistically indicating that a result is "incorrect," means for indicating that "the validity of the result cannot be determined by humans," means for indicating the resources that led to the answer or means for restricting the answer to only a database of papers, etc., and means for indicating which information source each news article is based on in order to increase the reliability of the news articles.

[0353] Explanation of program processing

[0354] The server uses Python requests and the BeautifulSoup library to retrieve news articles from the website. Specifically, the NewsApp class retrieves news articles, and the NewsArticle class holds information about each article. The fetch_news method retrieves news articles from the specified URL, and the display_articles method displays the retrieved articles.

[0355] Hardware and software used

[0356] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[0357] Software: Python, requests library, BeautifulSoup library

[0358] Data processing and calculation

[0359] 1. Data retrieval: The server retrieves the HTML data of the news article from the specified URL.

[0360] 2. Data Analysis: Parse the HTML data using the BeautifulSoup library to extract article titles, content, and sources.

[0361] 3. Save data: Save the extracted data as an instance of the NewsArticle class.

[0362] 4. Data display: Use the display_articles method to display the article title, content, and source on the user's device.

[0363] Specific examples

[0364] For example, if a news distribution app retrieves a news article from the URL "https: / / example.com / news1", it inputs the following prompt sentence into the generative AI model.

[0365] Prompt Sentence Examples

[0366] A news distribution app should retrieve news articles from the URL "https: / / example.com / news1" and display the article title, content, and source.

[0367] Using this prompt, the generative AI model can retrieve news articles from the specified URL and provide reliable information to the user.

[0368] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0369] Step 1:

[0370] The server receives the URL of the news article specified by the user.

[0371] Input: User-specified news article URL

[0372] Output: The URL of the news article to be retrieved.

[0373] What it does: When a user enters the URL of a news article into an application input field and presses the submit button, the server receives the URL.

[0374] Step 2:

[0375] The server retrieves HTML data from the specified URL using the requests library.

[0376] Input: News article URL

[0377] Output: Retrieved HTML data

[0378] Specific operation: The server executes requests.get(URL) to retrieve HTML data from the specified URL.

[0379] Step 3:

[0380] The server uses the BeautifulSoup library to parse the retrieved HTML data and extract the title, content, and source of the news article.

[0381] Input: Retrieved HTML data

[0382] Output: Title, content, and source of extracted news articles

[0383] Specific operation: The server parses the HTML data using BeautifulSoup, extracts the title with soup.find('h1').text, the content with soup.find('div', class_='article-content').text, and the URL itself as the source of information.

[0384] Step 4:

[0385] The server stores the extracted data as an instance of the NewsArticle class.

[0386] Input: Title, content, and source of extracted news articles

[0387] Output: An instance of the NewsArticle class

[0388] Specific operation: The server calls the constructor of the NewsArticle class and saves the extracted data in the instance.

[0389] Step 5:

[0390] The server transmits the stored instance of the news article to the user's terminal for display.

[0391] Input: An instance of the NewsArticle class

[0392] Output: The title, content, and source of the news article displayed on the user's device

[0393] Specific operation: The server calls the display_articles method to send the title, content, and source of the news article to the user terminal and display it.

[0394] Step 6:

[0395] The user checks the title, content, and source of the displayed news article.

[0396] Input: Title, content, and source of news article displayed on user device

[0397] Output: User confirmation result

[0398] What it does: The user checks the title, content, and source of a news article displayed on the device to determine the reliability of the information.

[0399] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0400] "Example 1"

[0401] The present invention is a system that includes an emotion engine that recognizes a user's emotions. Specifically, it receives a question from a user and generates possible answers to that question. In the answer generation process, a specific algorithm is used to generate possible answers to the question. The correctness of each generated answer is probabilistically evaluated, and the evaluation results are presented to the user. Furthermore, the information source (e.g., a specific paper database) referenced to generate the answer is presented to the user. This allows the user to know which information source the system's answer is based on. It is also possible to set restrictions, such as limiting answers to specific paper databases.

[0402] "Example 2"

[0403] The emotion engine recognizes the user's emotional state and adjusts responses based on that emotional state. For example, if the user feels angry, the system will provide them with more polite responses. If the user feels happy, the system will provide them with more cheerful responses. In this way, the emotion engine provides a user experience that takes the user's emotional state into account.

[0404] "Example 3"

[0405] As a concrete example, suppose a user asks, "What's the weather like today?" The system generates potential answers to this question and probabilistically evaluates the accuracy of each answer. It then presents the results of this evaluation to the user along with the information source (e.g., a specific weather database) that was used to generate the answer. Additionally, the emotion engine recognizes the user's emotional state and adjusts the answer based on that emotional state. For example, if the user feels happy, the system might provide a more cheerful answer such as, "It's sunny today. It's going to be a great day!"

[0406] The processing flow of each embodiment will be described below.

[0407] "Example 1"

[0408] Step 1: The system receives a question from the user.

[0409] Step 2: The system uses a specific algorithm to generate possible answers to the question.

[0410] Step 3: Probabilistically evaluate whether each generated answer is correct.

[0411] Step 4: Present the user with the results of the assessment and the sources consulted to generate the answer (e.g., a specific research database).

[0412] Step 5: It is also possible to set restrictions such as limiting responses to specific paper databases. "Example 2"

[0413] Step 1: The emotion engine recognizes the user's emotional state.

[0414] Step 2: Adjust the response based on the user's emotional state. For example, if the user feels angry, the system will provide a more polite response to the user.

[0415] Step 3: If the system senses that the user is happy, it will provide a more enjoyable response to that user.

[0416] "Example 3"

[0417] Step 1: Suppose a user asks, "What's the weather like today?"

[0418] Step 2: The system generates possible answers to this question and probabilistically evaluates the correctness of each answer.

[0419] Step 3: Present the user with the evaluation results and the information sources (e.g., a specific weather database) that were consulted to generate the answer.

[0420] Step 4: The emotion engine recognizes the user's emotional state and adjusts the answer based on that emotional state. For example, if the user feels happy, the system will provide a more cheerful answer such as "It's sunny today. It looks like it's going to be a great day!"

[0421] Example 1

[0422] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0423] Conventional chatbot systems lacked a means to evaluate the accuracy and reliability of answers to user questions, making it difficult for users to determine the reliability of the information provided. Furthermore, they lacked a function to clearly indicate the source of the answer, meaning users could not confirm the basis for the answer. Furthermore, they lacked a function to probabilistically evaluate the accuracy of the answer, making it difficult for users to understand the reliability of the answer.

[0424] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0425] In this invention, the server includes means for receiving a question from a user, means for inputting the received question as a prompt to a generative AI model, means for generating multiple answers from the generative AI model, means for probabilistically evaluating the accuracy of each generated answer, means for presenting an answer including the evaluation result and a reference information source to the user, means for probabilistically indicating that the answer is "incorrect," means for indicating that the "validity cannot be determined by a human," and means for indicating the resources that led to the answer or for restricting the answer to only using a database, etc. This allows the user to evaluate the reliability of the information provided and confirm the basis for the answer.

[0426] A "user" is an entity that inputs questions to the system and receives answers.

[0427] A "server" is a device or system that receives questions from users, generates answers using a generative AI model, and presents them to the user along with evaluation results.

[0428] A "generative AI model" is an artificial intelligence model used to generate multiple answers to a user question.

[0429] A "prompt" is a question or instruction entered into a generative AI model; it is the input data that the model uses to generate an answer.

[0430] "Probabilistic evaluation" means evaluating the accuracy of each generated answer based on probability theory.

[0431] The "evaluation result" is the result of a probabilistic evaluation of the accuracy of the generated answer.

[0432] A "reference source" is a source such as a database or literature that provides information on which the generated answer is based.

[0433] "Specifying that the answer is probabilistically incorrect" means specifying, based on probability theory, that the generated answer may be incorrect.

[0434] "Demonstrating that the validity cannot be determined by a human" means demonstrating that the validity of the generated answer cannot be easily determined by a human.

[0435] "Resources" are the information sources or databases consulted to generate the answer.

[0436] A "database" is a collection of information that is systematically organized and stored so that it can be searched and used as needed.

[0437] This invention relates to a chatbot system that generates answers to user questions. This system receives questions from users, generates answers using a generative AI model, probabilistically evaluates the accuracy of the answers, and presents them to the user.

[0438] The server provides an interface for receiving questions from users. When a user inputs a question, the device sends the question to the server. The server then inputs the received question as a prompt into a generative AI model (e.g., GPT-4). The generative AI model generates multiple answers to the question.

[0439] The server probabilistically evaluates the accuracy of each generated answer. This evaluation is performed using a specific algorithm. For example, the answer "The latest treatment is monoclonal antibody therapy" is evaluated as being correct 80% of the time. Along with this evaluation result, the server presents the user with a reference source (e.g., a specific paper database). This allows the user to know which source the system's answer is based on.

[0440] As a concrete example, consider the case where a user asks, "What is the latest treatment for COVID-19?" The device sends this question to the server. The server uses a generative AI model (GPT-4) to generate possible answers. For example, answers such as "The latest treatment is monoclonal antibody therapy" or "The latest treatment is mRNA vaccine" are generated. The server probabilistically evaluates the accuracy of each answer and sends the answer, including the evaluation result and reference source, to the device. The device displays the message to the user, "The latest treatment is monoclonal antibody therapy. This information is based on a specific paper database. There is an 80% chance that this answer is correct."

[0441] This system allows users to evaluate the reliability of the information provided and check the basis of the answer. Also, by probabilistically evaluating the accuracy of the answer, users can easily understand the reliability of the answer. Furthermore, by clearly indicating the reference source, users can check the basis of the answer.

[0442] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0443] Step 1:

[0444] The user enters a question.

[0445] The user enters a question through the device's interface, for example, "What is the latest treatment for COVID-19?" The entered question is displayed in an input field on the device.

[0446] Step 2:

[0447] The terminal sends a question to the server.

[0448] The terminal sends the question entered by the user to the server using a protocol such as an HTTP request. The input data is the user's question, and the output data is the question sent to the server.

[0449] Step 3:

[0450] The server receives the question and inputs it as a prompt into the generative AI model.

[0451] The server inputs the question received from the device as a prompt to the generative AI model (e.g., GPT-4). This prompt is converted into a format that the model can easily understand. The input data is the user's question, and the output data is the prompt that is input to the generative AI model.

[0452] Step 4:

[0453] The server generates multiple answers from the generative AI model.

[0454] The server generates multiple answers from the generative AI model, such as "The latest treatment is monoclonal antibody therapy" or "The latest treatment is mRNA vaccine." The input data is the prompt, and the output data is the generated multiple answers.

[0455] Step 5:

[0456] The server probabilistically rates the accuracy of each answer.

[0457] The server probabilistically evaluates the accuracy of each generated answer. This evaluation is performed using a specific algorithm. For example, the answer "monoclonal antibody therapy" is evaluated as being correct 80% of the time. The input data are the generated answers, and the output data are the evaluation results of the accuracy of each answer.

[0458] Step 6:

[0459] The server sends a response including the evaluation result and the reference information source to the terminal.

[0460] The server sends a response including the evaluation result and a reference source (e.g., a specific paper database) to the terminal. This transmission is also performed using a protocol such as HTTP response. The input data is the response including the evaluation result and the reference source, and the output data is the response sent to the terminal.

[0461] Step 7:

[0462] The terminal displays the answer to the user.

[0463] The terminal displays the answer received from the server to the user. For example, it may say, "The latest treatment is monoclonal antibody therapy. This information is based on a specific research paper database. There is an 80% chance that this answer is correct." The input data is the answer received from the server, and the output data is the answer displayed to the user.

[0464] (Application example 1)

[0465] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0466] Conventional chatbot systems lacked the ability to probabilistically evaluate the accuracy of answers to user questions, making it difficult for users to determine the reliability of the information provided. They also lacked the ability to recognize users' emotions and provide appropriate feedback based on them, resulting in a poor user experience. Furthermore, they lacked the ability to clearly indicate the resources that led to the answer, making it difficult for users to verify the source of the information. These issues need to be resolved.

[0467] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0468] In this invention, the server includes a means for probabilistically indicating that the answer is "incorrect," a means for indicating that the "validity cannot be determined by humans," a means for indicating the resources that led to the answer or a means for restricting the answer to only a paper database, a means for recognizing the user's emotions, and a means for providing appropriate feedback based on the emotion recognition results. This allows the user to probabilistically evaluate the reliability of the provided information and receive appropriate feedback based on their emotions. Furthermore, by indicating the resources that led to the answer, the user can confirm the source of the information.

[0469] "Means to clearly state that something is probabilistically incorrect" is a function that probabilistically evaluates the accuracy of answers to user questions and presents the evaluation results to the user.

[0470] "Means to show that a human cannot determine the validity" is a function that clearly shows that a human cannot easily determine whether the generated answer is correct.

[0471] "Means for clearly indicating the resources that led to the answer" is a function that shows the user which information sources the generated answer is based on.

[0472] "Means for restricting answers to be generated only from a research paper database, etc." is a function for restricting answers to be generated only from information obtained from a specific database.

[0473] "Means for recognizing user emotions" is a function that analyzes emotions from the user's questions and input and obtains the results.

[0474] "Means for providing appropriate feedback based on emotion recognition results" is a function that provides appropriate answers or actions based on the results of user emotion analysis.

[0475] A system for implementing this invention includes a server, a user terminal, and a network. The server executes a program for generating answers to user questions and probabilistically evaluating their accuracy. Specifically, the process is carried out in the following manner.

[0476] First, the user device receives a question from the user and sends it to the server. The server analyzes the received question and generates possible answers using a generative AI model. In doing so, it uses a specific algorithm to probabilistically evaluate the accuracy of each answer. The evaluation results are presented to the user as a means of clearly indicating which answers are probabilistically "incorrect."

[0477] Next, the server uses an emotion engine to recognize emotions from the user's question. This emotion engine is implemented, for example, using the pipeline in the transformers library. Based on the emotion recognition results, the server generates appropriate feedback and provides it to the user.

[0478] Furthermore, the server clearly indicates which sources the generated answers are based on, allowing users to verify the reliability of the answers. Sources include specific databases and research papers.

[0479] As a concrete example, if a user asks "How can I protect myself from being hacked?" the server will generate the following result:

[0480] User: How can I protect myself from being hacked?

[0481] System: Install antivirus software (78% confidence).

[0482] Reference sources: Security expert blogs, latest security papers

[0483] Sentiment Analysis: Positive (85%)

[0484] In this way, users can probabilistically evaluate the reliability of the provided information and receive appropriate emotional feedback. Furthermore, by clearly citing the resources that led to the answer, users can verify the source of the information.

[0485] The hardware used includes user devices (smartphones, tablets, PCs, etc.) and servers, and the software used includes generative AI models, emotion engines (such as pipelines in the transformers library), and database access libraries.

[0486] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0487] Step 1:

[0488] The user terminal receives a question from the user.

[0489] Input: User question (e.g., "How can I protect myself from being hacked?")

[0490] Output: Question data

[0491] Specific behavior: The user enters a question into the input field on the device and presses the send button.

[0492] Step 2:

[0493] The user terminal transmits the received question data to the server.

[0494] Input: Question data

[0495] Output: Request to server

[0496] Specific operation: The user terminal sends the question data to the server as an HTTP request.

[0497] Step 3:

[0498] The server analyzes the received question data and generates possible answers using a generative AI model.

[0499] Input: Question data

[0500] Output: A list of generated answers

[0501] Specific operation: The server inputs question data into a natural language processing model and generates multiple answer candidates.

[0502] Step 4:

[0503] The server uses a specific algorithm to probabilistically assess the accuracy of each answer.

[0504] Input: Generated answer list

[0505] Output: List of answers with probability ratings

[0506] Specific operation: The server applies a probabilistic evaluation algorithm to each answer and calculates the confidence of each answer.

[0507] Step 5:

[0508] The server uses an emotion engine to recognize emotions from the content of the user's question.

[0509] Input: Question data

[0510] Output: Emotion recognition result

[0511] Specific operation: The server inputs the question data into an emotion engine (e.g., the pipeline of the transformers library) and performs emotion analysis.

[0512] Step 6:

[0513] The server generates appropriate feedback based on the emotion recognition results.

[0514] Input: Emotion recognition results, answer list with probability evaluation

[0515] Output: Answer with feedback

[0516] Specific operation: Based on the emotion recognition results, the server selects an answer that includes appropriate feedback for the user.

[0517] Step 7:

[0518] The server specifies the source of the generated answer.

[0519] Input: List of answers with probability ratings

[0520] Output: List of answers with sources

[0521] Specific operation: The server adds a reference source to each answer and generates data to present to the user.

[0522] Step 8:

[0523] The server transmits the final response data to the user terminal.

[0524] Input: Sourced Answer List

[0525] Output: Response to user terminal

[0526] Specific operation: The server sends the final answer data to the user terminal as an HTTP response.

[0527] Step 9:

[0528] The user terminal displays the received response data to the user.

[0529] Input: Response from the server

[0530] Output: What the user sees

[0531] Specific operation: The user terminal displays the received response data on the screen and provides it to the user.

[0532] Example 2

[0533] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0534] Conventional systems have problems in that it is difficult for users to judge the validity of the generated answers and they are unable to respond appropriately depending on the user's emotional state, which can lead to a poor user experience and a loss of system reliability.

[0535] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0536] In this invention, the server includes means for probabilistically indicating that the answer is "incorrect," means for indicating that the "validity cannot be determined by a human," means for indicating the resources that led to the answer, means for recognizing the user's emotional state, means for adjusting the answer based on the recognized emotional state, and means for presenting the user with the probability of the validity of the generated answer. This allows the user to determine for themselves the validity of the generated answer, and further enables appropriate responses according to the user's emotional state.

[0537] "Means for specifying probabilistically incorrect" is a means for probabilistically indicating the possibility that the generated answer is incorrect.

[0538] "Means for demonstrating that the validity cannot be determined by a human" refers to means for demonstrating that the validity of the generated answer cannot be easily determined by a user.

[0539] "Means for indicating the resources that led to the answer" means a means for showing the user which information sources or databases the generated answer is based on.

[0540] The "means for imposing a constraint that answers are to be obtained only from a database, etc." is a means for imposing a constraint that answers are to be generated by obtaining information only from a specific database.

[0541] "Means for recognizing a user's emotional state" refers to means for analyzing and recognizing a user's emotional state from their input and actions.

[0542] The "means for adjusting a response based on a recognized emotional state" refers to a means for appropriately adjusting a generated response in accordance with the recognized emotional state of the user.

[0543] The "means for presenting to the user the probability of the correctness of the generated answer" is a means for showing to the user the probability that the generated answer will be judged to be correct.

[0544] This invention is a system equipped with an emotion engine that recognizes the user's emotional state and adjusts answers based on that emotion. It also has a function that displays to the user the probability that each generated answer is judged to be correct, allowing the user to judge the validity of the system's answers for themselves.

[0545] Hardware and software used

[0546] Hardware: Servers, user devices (PCs, smartphones, etc.)

[0547] Software: Generative AI models (e.g., GPT-4), emotion recognition engines

[0548] Explanation of program processing

[0549] 1. Accepting user input

[0550] Users enter questions and comments through the device.

[0551] Example: User types "What's the weather like today?"

[0552] 2. Recognizing emotional states

[0553] The server uses an emotion recognition engine to parse the emotional state from the user's input.

[0554] Example: If the user's input indicates anger, the emotion recognition engine will determine "Anger".

[0555] 3. Generate answers

[0556] The server uses a generative AI model to generate answers to user input.

[0557] Example: Generate "The weather is sunny today."

[0558] 4. Adjust your responses based on emotions

[0559] The server uses an emotion engine to tailor the response based on the user's emotional state.

[0560] Example: If the user is angry, adjust your response to a more polite one: "The weather is sunny today. Let me know if I need any help."

[0561] 5. Probability of the correctness of the answer

[0562] The server calculates the probability that the generated answer is correct and presents it to the user.

[0563] Example: Display "Today's weather is sunny (accuracy: 95%)."

[0564] 6. Providing answers to users

[0565] The server sends the adjusted answer and its probability of correctness to the user's terminal and displays it to the user.

[0566] Example: The user sees the message "Today's weather is sunny (95% accuracy). Let us know if we need any help."

[0567] Examples of concrete examples and prompts

[0568] Examples:

[0569] The user types, "I'm having trouble at work."

[0570] The server uses an emotion recognition engine to detect "sadness."

[0571] The server uses a generative AI model to generate a response: "That's tough. Is there anything I can help you with?"

[0572] The server uses an emotion engine to adjust based on "sadness" to say, "That's terrible. Is there anything I can help you with? Let me hear your story."

[0573] The server calculates the probability as "Accuracy: 90%" and displays it to the user.

[0574] The server sends the adjusted answer and its probability of correctness to the user's terminal, where it is displayed on the user's screen.

[0575] Example prompt sentence:

[0576] "If a user types 'I'm having trouble at work,' explain how the emotion recognition engine detects 'sadness,' the generative AI model generates an appropriate response, and the emotion engine adjusts that response."

[0577] In this way, the system takes into account the user's emotional state and provides justification for the answer, providing a more personalized experience for the user.

[0578] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0579] Step 1:

[0580] Accepting user input

[0581] Users enter questions and comments through the device.

[0582] Input: User types "What's the weather like today?"

[0583] Specific behavior: A user types "What's the weather like today?" into a chat app on their smartphone and presses the send button.

[0584] Output: User input data is sent to the server.

[0585] Step 2:

[0586] Recognition of emotional states

[0587] The server uses an emotion recognition engine to parse the emotional state from the user's input.

[0588] Input: User input data "What's the weather like today?"

[0589] Specific operation: The server calls the emotion recognition engine and analyzes the user's text data.

[0590] Data processing: Text data is input into an emotion recognition engine to determine the emotional state.

[0591] Output: Emotional state is determined to be "neutral".

[0592] Step 3:

[0593] Generate answers

[0594] The server uses the generative AI model to generate answers to the user's input.

[0595] Input: User input data "What's the weather like today?" and emotional state "Neutral"

[0596] What happens: The server invokes the generative AI model and uses the user's input data as prompts.

[0597] Data computation: Generative AI models generate answers based on prompts.

[0598] Output: The answer "The weather is sunny today" is produced.

[0599] Step 4:

[0600] Adjusting responses based on emotion

[0601] The server uses an emotion engine to tailor the response based on the user's emotional state.

[0602] Input: Generated answer "The weather is sunny today" and emotional state "Neutral"

[0603] What it does: The server invokes the emotion engine and adjusts the generated answers based on the emotional state.

[0604] Data processing: Use the answers as they are depending on the emotional state.

[0605] Output: You get the adjusted answer "The weather is sunny today."

[0606] Step 5:

[0607] Probability of the answer being correct

[0608] The server calculates the probability that the generated answer is correct and presents it to the user.

[0609] Input: Adjusted answer "The weather is sunny today"

[0610] What happens: The server invokes an algorithm that calculates the probability of correctness and evaluates the accuracy of the answer.

[0611] Data calculation: Calculate the accuracy of the answer as a probability.

[0612] Output: A 95% probability of accuracy is obtained.

[0613] Step 6:

[0614] Providing answers to users

[0615] The server sends the adjusted answer and its probability of correctness to the user's terminal and displays it to the user.

[0616] Input: Calibrated answer "The weather is sunny today" and probability of accuracy "95%"

[0617] Specific operation: The server sends the response data to the user device.

[0618] Output: The user will see "Today's weather is sunny (accuracy: 95%)".

[0619] (Application example 2)

[0620] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0621] Conventional systems provide uniform answers without considering the user's emotional state, which can lead to an inconsistent user experience. Furthermore, users lack the means to determine the legitimacy of the generated answers, which creates the risk of believing incorrect information. Furthermore, there is little transparency regarding the resources that led to the answer, making it difficult for users to verify the reliability of the information.

[0622] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0623] In this invention, the server includes means for probabilistically indicating that an answer is "incorrect," means for indicating that the "validity cannot be determined by a human," means for indicating the resources that led to the answer, means for recognizing the emotional state of the user and adjusting the answer based on that emotional state, and means for presenting to the user the probability that each generated answer will be determined to be correct. This makes it possible to provide an appropriate answer according to the user's emotional state and enable the user to determine the validity of the answer.

[0624] "Means for clearly stating that a given answer is probabilistically incorrect" refers to a means for probabilistically evaluating whether the generated answer is correct and clearly indicating the result to the user.

[0625] "Means for showing that 'humans cannot determine the validity'" refers to means for showing the user that the validity of the generated answer is difficult for humans to determine.

[0626] "Means for indicating the resources that led to the answer" means a means for indicating to the user which information sources or databases the generated answer is based on.

[0627] "Means for recognizing a user's emotional state and adjusting responses based on that emotional state" refers to means for recognizing a user's emotions in real time and adjusting the content and tone of responses according to those emotions.

[0628] "Means for presenting to the user the probability that each generated answer will be judged to be correct" refers to means for calculating the validity of the generated answer as a probability and presenting that probability to the user.

[0629] The system for implementing this invention has an emotion engine that recognizes the user's emotional state and adjusts answers based on that emotional state. It also has the function of presenting the user with the probability that each generated answer is judged to be correct. A specific embodiment of this system is described below.

[0630] Hardware and software used

[0631] Hardware: Smartphone

[0632] software:

[0633] EmotionRecognizer: A library for recognizing a user's emotional state

[0634] ContentRecommender: A library for recommending content based on emotional states

[0635] Data processing and calculation

[0636] emotion recognition

[0637] The server passes the user's input text to the EmotionRecognizer library to recognize their emotional state. For example, if a user types, "I'm feeling very happy today," the system will parse the text and recognize the emotion "happy."

[0638] Content Recommendation

[0639] The server passes the recognized emotional state to the ContentRecommender library, which recommends content appropriate to that emotion. For example, comedy movies are recommended for the emotion "happy." The server also calculates the validity of the recommended content as a probability and presents it to the user. For example, a comedy movie recommendation shows an 85% probability.

[0640] display

[0641] The device displays the recommended content and its probability of validity to the user, allowing the user to judge the validity of the recommended content for themselves.

[0642] Specific examples

[0643] If a user types "I'm feeling very happy today," the system will recognize it as "happy" and recommend a comedy movie with an 85% accuracy rate.

[0644] Prompt Sentence Examples

[0645] "Please describe how you feel: I'm feeling very happy today!"

[0646] "Please describe how you feel: I'm a bit sad and lonely."

[0647] In this way, emotion-responsive content delivery apps provide optimal content based on the user's emotional state and display its validity as a probability, allowing users to make their own judgments.

[0648] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0649] Step 1:

[0650] The user inputs their emotional state into the terminal, for example, "I feel very happy today." This input text becomes the input data for the next processing step.

[0651] Step 2:

[0652] The device sends the input text to the server. The server passes the received text to the EmotionRecognizer library to recognize the emotional state. For example, the emotion "happy" is recognized from the text "I'm feeling very happy today." This emotional state becomes the input data for the next processing step.

[0653] Step 3:

[0654] The server passes the recognized emotional state to the ContentRecommender library, which recommends content appropriate to that emotion. For example, comedy movies are recommended for the emotion "happy." This recommended content becomes input data for the next processing step.

[0655] Step 4:

[0656] The server calculates the validity of the recommended content as a probability. For example, it calculates a probability of 85% for a comedy movie recommendation. This probability becomes the input data for the next processing step.

[0657] Step 5:

[0658] The server sends the recommended content and its probability of validity to the device. The device displays the received content and probability to the user. For example, it displays "comedy movie" with a probability of "85%." This allows the user to judge the validity of the recommended content for themselves.

[0659] In this way, by clarifying the specific operations performed at each processing step and their inputs and outputs, it becomes easier to understand the processing flow of the entire system.

[0660] Example 3

[0661] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0662] In conventional systems, it is difficult for users to judge the accuracy and reliability of the generated answers, and the sources of information on which the answers are based are not clearly stated, which often leaves users feeling uneasy. Furthermore, the system does not provide personalized answers that reflect the user's emotional state, which limits the user experience.

[0663] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0664] In this invention, the server includes a means for probabilistically indicating that the answer is "incorrect," a means for indicating that the "correctness cannot be determined by a human," a means for indicating the resources that led to the answer, a means for restricting the answer to a specific database, a means for recognizing the user's emotional state, and a means for adjusting the answer based on the user's emotional state. This allows the user to confirm the accuracy and reliability of the generated answer and to know the source of the answer's basis. Furthermore, personalized answers based on the user's emotional state are provided, enabling a more satisfying user experience.

[0665] The "means for clearly indicating that the answer is probabilistically incorrect" is a means for probabilistically evaluating the possibility that the generated answer is incorrect and presenting the evaluation results to the user.

[0666] "Means for demonstrating that the validity cannot be determined by humans" refers to means for clearly indicating that the validity of the generated answer cannot be easily confirmed by human judgment.

[0667] The "means for clearly indicating the resources that led to the answer" is a means for presenting to the user the information sources that are the basis for the generated answer.

[0668] The "means for imposing a constraint that answers are only made using a specific database" is a means for imposing a constraint that only a specific database is referenced when generating an answer.

[0669] The "means for recognizing the emotional state of the user" is a means for analyzing the user's input and dialogue history to recognize the user's emotional state.

[0670] The "means for adjusting a response based on the emotional state of the user" refers to a means for adjusting a generated response in accordance with the recognized emotional state of the user.

[0671] This invention is a system in which a user inputs a question using a terminal, and a server generates an answer to that question using a generative AI model. The system ensures the accuracy and reliability of the answer to the user's question, and also provides a personalized answer according to the user's emotional state.

[0672] Hardware and software used

[0673] Hardware: Servers, terminals

[0674] Software: Generative AI model, emotion engine, database access module

[0675] Processing flow

[0676] 1. Accepting user questions

[0677] A user uses a terminal to input a question into the system.

[0678] Specific behavior: The user enters "What's the weather like today?" into the device's input field and presses the send button.

[0679] 2. Question Analysis

[0680] The server receives the user's question and generates a prompt to input into the generative AI model.

[0681] Specific operation: The server analyzes the question "What's the weather like today?" and converts it into a format suitable for the generative AI model. For example, it generates a prompt sentence such as "What's the weather like today?"

[0682] 3. Generate answers

[0683] The server uses a generative AI model to generate possible answers to the question.

[0684] Specific operation: The server inputs a prompt sentence into the generative AI model, and the model generates the answer "It's sunny today."

[0685] 4. References to sources

[0686] Identify the information source (e.g., a particular weather database) that the server consulted to generate the answer.

[0687] Specific behavior: Record that the server referred to the weather database when generating the answer "It's sunny today."

[0688] 5. Evaluating your answers

[0689] The server probabilistically evaluates the accuracy of the generated answer.

[0690] Specific operation: The server performs a probability calculation based on data from the weather database to evaluate the accuracy of the answer "It's sunny today" as 90%.

[0691] 6. Citation of sources

[0692] The server presents the user with the generated answer and the information sources that were consulted to generate the answer.

[0693] Specific behavior: The server displays to the user, "It's sunny today. Source: Weather database."

[0694] 7. Recognizing emotional states

[0695] The server uses an emotion engine to recognize the user's emotional state.

[0696] Specific operation: The server analyzes the user's input and past dialogue history, and the emotion engine recognizes that the user is happy.

[0697] 8. Adjusting your answers

[0698] The server adjusts the generated answers based on the user's emotional state.

[0699] What happens: The server adjusts the answer "It's sunny today" to "It's sunny today. It's going to be a great day!"

[0700] 9. Presentation of the final answer

[0701] The server presents the adjusted final answer to the user.

[0702] Specific behavior: The server displays on the user's device, "It's sunny today. It looks like it's going to be a great day! Source: Weather database."

[0703] Specific examples

[0704] User Questions

[0705] User: "What's the weather like today?"

[0706] Prompt Sentence Examples

[0707] Prompt: "What's the weather like today?"

[0708] In this way, the system provides reliable answers to the user's questions and clearly indicates the source of the answer, giving the user a sense of security. Furthermore, the system adjusts the answer depending on the user's emotional state, providing a more personalized experience. The flow of the identification process in Example 3 will be described with reference to FIG. 21.

[0709] Step 1:

[0710] Accepting user questions

[0711] Input: The user enters a question into the device's input field and presses the submit button.

[0712] Specific behavior: The user types "What's the weather like today?" and presses the send button.

[0713] Output: The user's question is sent to the server.

[0714] Step 2:

[0715] Question Analysis

[0716] Input: The server receives the user's question.

[0717] What it does: The server parses the question "What's the weather like today?" and converts it into a format suitable for the generative AI model.

[0718] Data processing: Analyze the question and generate a prompt to input into the generative AI model.

[0719] Output: The prompt sentence "What's the weather like today?" is generated.

[0720] Step 3:

[0721] Generate answers

[0722] Input: The prompt sentence generated by the server is input to the generative AI model.

[0723] Specific operation: The server inputs the prompt sentence "What is the weather like today?" to the generated AI model.

[0724] Data computation: A generative AI model generates an answer based on the prompt.

[0725] Output: Produces the answer "It's sunny today."

[0726] Step 4:

[0727] References to sources

[0728] Input: The server receives the generated answer.

[0729] Specific operation: When the server generates the answer "It's sunny today," it refers to the weather database.

[0730] Data processing: Check the sources that support the answers generated.

[0731] Output: Records that the weather database was consulted.

[0732] Step 5:

[0733] Evaluating answers

[0734] Input: The server receives the generated answer and its source.

[0735] Specific operation: The server probabilistically evaluates the accuracy of the answer "It's sunny today."

[0736] Data calculation: Probability calculations are performed based on data from the weather database.

[0737] Output: "It's sunny today." is rated as 90% accurate.

[0738] Step 6:

[0739] Source of information

[0740] Input: The server receives the evaluated answer.

[0741] Specific behavior: The server displays to the user, "It's sunny today. Source: Weather database."

[0742] Data processing: Combining the answers and their sources and converting them into a format that can be presented to the user.

[0743] Output: Produces the message "It's sunny today. Source: Weather database".

[0744] Step 7:

[0745] Recognition of emotional states

[0746] Input: The server receives user input and past interaction history.

[0747] Specific operation: The server uses the emotion engine to recognize the user's emotional state.

[0748] Data computation: Analyzes user input and interaction history to identify emotional state.

[0749] Output: The user is perceived as happy.

[0750] Step 8:

[0751] Adjusting your answers

[0752] Input: The server receives the recognized emotional state of the user and the generated answer.

[0753] What happens: The server adjusts the answer "It's sunny today" to "It's sunny today. It's going to be a great day!"

[0754] Data manipulation: tailoring answers based on the user's emotional state.

[0755] Output: The adjusted answer is generated.

[0756] Step 9:

[0757] Providing the final answer

[0758] Input: The server receives the adjusted answer.

[0759] Specific behavior: The server displays on the user's device, "It's sunny today. It looks like it's going to be a great day! Source: Weather database."

[0760] Data processing: Combining tailored answers and their sources into a format that can be presented to the user.

[0761] Output: The message "It's sunny today. It's going to be a great day! Source: Weather database" is displayed on the user's terminal.

[0762] (Application example 3)

[0763] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0764] Conventional information provision systems have difficulty providing reliable answers to user questions, and lack the functionality to clearly indicate the source of the answer, making it difficult for users to judge the reliability of the answer.Furthermore, answers are not adjusted taking into account the user's emotional state, which has led to issues with the user experience not being improved.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0766] In this invention, the server includes a means for probabilistically indicating that a response is "incorrect," a means for indicating that the "validity cannot be determined by a human," a means for indicating the resources used to arrive at the answer or a means for restricting the answer to sources only, and a means for recognizing the user's emotional state and adjusting the answer based on that emotional state. This allows the server to provide a highly reliable answer to a user's question and clearly indicate the sources on which the answer is based, enabling the user to determine the reliability of the answer. Furthermore, the adjustment of the answer taking the user's emotional state into account improves the user experience.

[0767] "Means for clearly indicating that an answer is probabilistically incorrect" is a function that uses a specific algorithm to determine the probability that an answer is incorrect and clearly indicates the result.

[0768] The "means for showing that the validity cannot be determined by a human" is a function for showing that the validity of an answer cannot be easily determined by a human.

[0769] "Means for clearly indicating the resources that led to the answer" is a function that shows the user the information sources that were consulted to generate the answer.

[0770] The "means for restricting answers to be generated only from information sources, etc." is a function for restricting answers to be generated only from specific information sources.

[0771] "Means for recognizing the user's emotional state and adjusting the response based on that emotional state" refers to a function that recognizes the user's emotional state using an emotion engine or the like and adjusts the content and expression of the response according to that emotional state.

[0772] As an embodiment of the present invention, we will explain a smart content assistant system. This system provides reliable answers to user questions, clearly citing the source of the answer. It also recognizes the user's emotional state and adjusts the answer based on that emotional state.

[0773] Hardware and software used

[0774] Hardware: Smartphones, smart glasses, head-mounted displays

[0775] Software: Python, sentiment analysis library (e.g., sentiment_analysis), information source acquisition library (e.g., information_sources)

[0776] System configuration

[0777] 1. User Interface: Provides an interface for users to input questions. This can run on a smartphone, smart glasses, or a head-mounted display.

[0778] 2. Source Acquisition Module: Generates answers from reliable sources to the user's questions. This module retrieves information from specific databases and generates answers.

[0779] 3. Sentiment Analysis Module: Recognizes the emotional state of the user and tailors the answer based on that emotional state. This module uses a sentiment analysis library to analyze the user's emotions.

[0780] 4. Answer generation module: Integrates data from the source acquisition module and sentiment analysis module to generate the final answer.

[0781] Data processing and calculation

[0782] 1. Question input: The user inputs a question, for example, "What's the weather like today?"

[0783] 2. Answer generation from information sources: The information source acquisition module retrieves the answer to the question from a specific database, for example, the answer "It's sunny today" from a weather database.

[0784] 3. Sentiment Analysis: The sentiment analysis module analyzes the user's emotional state. For example, if the user types "happy," the emotional state will be recognized as "positive."

[0785] 4. Answer Adjustment: Based on the results of the sentiment analysis, the answer generation module adjusts the answer, for example, "It's sunny today. It looks like it's going to be a great day!"

[0786] 5. Display results: Show the user the tailored answer and the referenced sources.

[0787] Specific examples

[0788] Question: "What's the weather like today?"

[0789] Emotional state: "happy"

[0790] Prompt Sentence Examples

[0791] Enter your question: What's the weather like today?

[0792] Please enter your current emotional state (e.g. happy, sad): happy

[0793] Example output

[0794] Answer: It's sunny today. It's going to be a great day!

[0795] Referenced sources: [Weather Database 1, Weather Database 2]

[0796] In this way, the smart content assistant system can provide reliable answers to user questions, clearly cite the sources behind the answers, and improve the user experience by adjusting the answers based on the user's emotional state.

[0797] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0798] Step 1:

[0799] The user enters a question.

[0800] The user uses a device (smartphone, smart glasses, head-mounted display) to input a question, for example, "What's the weather like today?" The input question is sent to the system.

[0801] Input: User question (e.g. "What's the weather like today?")

[0802] Output: Question data sent to the system

[0803] Step 2:

[0804] The server generates the answer from the information source.

[0805] The server uses a source acquisition module to retrieve the answer to the question from a particular database, for example, retrieving the answer "It's sunny today" from a weather database.

[0806] Input: User question data

[0807] Output: Answer data obtained from the source (e.g., "It's sunny today")

[0808] Step 3:

[0809] The server recognizes the user's emotional state.

[0810] The user uses the terminal to input their current emotional state, for example, "happy." The server uses the emotion analysis module to analyze the user's emotional state.

[0811] Input: User's emotional state (e.g. "happy")

[0812] Output: Sentiment analysis result (e.g. "positive")

[0813] Step 4:

[0814] The server adjusts the answer.

[0815] Based on the results of the sentiment analysis, the server uses an answer generation module to tailor the answer, for example, "It's sunny today. It's going to be a great day!"

[0816] Input: Response data obtained from sources, sentiment analysis results

[0817] Output: Adjusted response data (e.g., "It's sunny today. It's going to be a great day!")

[0818] Step 5:

[0819] Specifies the resource from which the server arrived at the answer.

[0820] The server will then present the information sources it referenced to generate the answer to the user, for example, "Weather Database 1, Weather Database 2."

[0821] Input: Response data obtained from sources

[0822] Output: Resource information leading to the answer (e.g., "Weather Database 1, Weather Database 2")

[0823] Step 6:

[0824] The server displays the results to the user.

[0825] The server displays the adjusted answer and the referenced sources to the user, who can then check the final answer and its rationale through their device.

[0826] Input: Adjusted response data, resource information leading to the response

[0827] Output: The final answer and source shown to the user (e.g., "It's sunny today. It's going to be a great day! Sources consulted: Weather Database 1, Weather Database 2")

[0828] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0829] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0830] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0831] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0832] [Second embodiment]

[0833] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0834] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0835] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0836] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0837] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0838] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0839] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0840] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0841] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0842] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0843] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0844] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0845] "Example 1"

[0846] An embodiment of the present invention is realized as a chatbot system. This system includes an algorithm for generating answers to questions from users. This algorithm has the function of specifying what is "incorrect" probabilistically. Specifically, it generates possible answers to questions from users and probabilistically evaluates whether each answer is correct. This evaluation is performed by calculating the probability that each answer will be judged to be correct. "Embodiment Example 2"

[0847] Furthermore, this embodiment of the present invention has a function that indicates that "humans cannot determine the validity of the answers." Specifically, for each generated answer, the probability that the answer will be judged correct is presented to the user. This allows the user to judge the validity of the system's answers for themselves.

[0848] "Example 3"

[0849] Furthermore, this embodiment of the present invention has a function of clearly indicating the resources that led to the answer. Specifically, the system presents to the user the information sources (e.g., a specific paper database) that were referenced to generate the answer. This allows the user to know which information sources the system's answer is based on. The system can also impose restrictions such as limiting the answer to a specific paper database. This can improve the reliability of the system's answer.

[0850] The processing flow of each embodiment will be described below.

[0851] "Example 1"

[0852] Step 1: Receive a user question.

[0853] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[0854] Step 3: Probabilistically evaluate whether each generated answer is correct. This is done by calculating the probability that each answer is judged to be correct.

[0855] Step 4: Present the probabilistic evaluation results to the user, allowing the user to judge for themselves the validity of the system's answer.

[0856] "Example 2"

[0857] Step 1: Receive a user question.

[0858] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[0859] Step 3: For each generated answer, the user is presented with the probability that the answer is judged to be correct, allowing the user to judge for themselves the validity of the system's answers.

[0860] "Example 3"

[0861] Step 1: Receive a user question.

[0862] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[0863] Step 3: Present the user with the sources (e.g., a specific paper database) that were consulted to generate the answer, allowing the user to understand which sources the system based its answer on.

[0864] Step 4: Set constraints such as limiting answers to specific paper databases. This can improve the reliability of the system's answers.

[0865] Example 1

[0866] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0867] With conventional chatbot systems, it is difficult to guarantee the accuracy of answers to user questions, and there is a risk that incorrect information will be provided. Furthermore, since users lack a means to determine the accuracy of answers, there is a risk that they will believe incorrect information. Furthermore, since there is no way to clearly indicate the basis or reliability of the answer, it is difficult for users to evaluate the reliability of the answer. To solve these issues, a system is needed that can probabilistically evaluate the accuracy of answers and clearly state that answers with low accuracy are "likely incorrect."

[0868] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0869] In this invention, the server includes a means for receiving a question from a user, a means for generating multiple answers using a generative AI model, a means for probabilistically evaluating the accuracy of each answer, a means for indicating that answers with low accuracy are "likely incorrect" based on the evaluation results, and a means for displaying the organized answers to the user. This allows the user to receive information about the accuracy of the provided answers as probabilistically evaluated, reducing the risk of believing incorrect information. Furthermore, clearly indicating the reliability of the answers makes it easier for the user to evaluate the reliability of the answers.

[0870] The "means for receiving a question from a user" is a function that allows the system to receive a question entered by a user.

[0871] "Means for generating multiple answers using a generative AI model" is a function for automatically generating multiple answers to a user's question using a generative AI model.

[0872] The "means for probabilistically evaluating the accuracy of each answer" refers to an algorithm or method for probabilistically evaluating the accuracy of each generated answer.

[0873] "Means for clearly indicating that answers with low accuracy are 'likely incorrect' based on the evaluation results" is a function for displaying "likely incorrect" for answers that are judged to be low in accuracy based on the accuracy evaluation results.

[0874] The "means for displaying organized answers to the user" is a function for displaying answers organized based on the evaluation results to the user.

[0875] A "specific generative algorithm" is a specific algorithm used by a generative AI model to generate an answer to a user's question.

[0876] A "particular evaluation algorithm" is a particular algorithm used to probabilistically evaluate the accuracy of a generated answer.

[0877] This invention relates to a chatbot system that generates answers to user questions and probabilistically evaluates their accuracy. This system operates among three parties: a server, a terminal, and a user.

[0878] The server includes means for receiving questions from users, means for generating multiple answers using a generative AI model, means for probabilistically evaluating the accuracy of each answer, means for indicating that answers with low accuracy are "likely incorrect" based on the evaluation results, and means for displaying the organized answers to the user.

[0879] Specifically, when the server receives a question from a user, it uses a generative AI model (e.g., GPT-4) to generate multiple possible answers. For each generated answer, the server probabilistically evaluates its accuracy. This evaluation is performed using a specific evaluation algorithm. Based on the evaluation results, answers with low accuracy are marked as "likely incorrect." Finally, the organized answers are displayed to the user.

[0880] For example, if a user asks "What is the diameter of the Earth?" the server might process the question as follows:

[0881] 1. A user types into the chatbot interface, "What is the diameter of the Earth?"

[0882] 2. The device sends this question to the server.

[0883] 3. The server receives the question and inputs it into the generative AI model (GPT-4).

[0884] 4. The generative AI model generates two answers: "The diameter of the Earth is approximately 12,742 kilometers" and "The diameter of the Earth is approximately 10,000 kilometers."

[0885] 5. The server rates the accuracy of each answer, rating answer 1 at 95% and answer 2 at 20%.

[0886] 6. Based on the evaluation results, the server marks answer 2 as "likely incorrect."

[0887] 7. The server sends the organized response to the device.

[0888] 8. The device displays to the user "The diameter of the Earth is approximately 12,742 kilometers" and "The diameter of the Earth is approximately 10,000 kilometers (likely incorrect)."

[0889] Example prompt sentence:

[0890] User: What is the diameter of the Earth?

[0891] Server: The diameter of the Earth is approximately 12,742 kilometers.

[0892] Server: The diameter of the Earth is approximately 10,000 kilometers. (likely incorrect)

[0893] This system allows users to receive information that probabilistically evaluates the accuracy of the answers provided, reducing the risk of believing incorrect information. In addition, by clearly indicating the reliability of the answers, users can more easily evaluate the reliability of the answers.

[0894] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0895] Step 1:

[0896] The user enters a question.

[0897] The user inputs a question into the chatbot's interface, for example, "What is the diameter of the Earth?" The input question is sent to the terminal.

[0898] Step 2:

[0899] The terminal sends a question to the server.

[0900] The terminal sends the question entered by the user to the server, where the question is converted into an appropriate format: the input is the user's question, and the output is the question sent to the server.

[0901] Step 3:

[0902] The server receives the query.

[0903] The server receives questions sent from the device. The received questions are prepared for the next processing step. The input is the question sent from the device, and the output is the question data to be input into the generative AI model.

[0904] Step 4:

[0905] The server inputs questions into the generative AI model.

[0906] The server inputs the received question into a generative AI model (e.g., GPT-4). At this time, the question is converted into a format that the generative AI model can understand. The input is the question data, and the output is the question input to the generative AI model.

[0907] Step 5:

[0908] A generative AI model generates multiple answers.

[0909] A generative AI model generates multiple answers to an input question. For example, it generates answers such as "The diameter of the Earth is approximately 12,742 kilometers" or "The diameter of the Earth is approximately 10,000 kilometers." The input is the question entered into the generative AI model, and the output is the multiple answers generated.

[0910] Step 6:

[0911] The server probabilistically rates the accuracy of each answer.

[0912] The server applies an algorithm to each generated answer to probabilistically evaluate its accuracy. For example, answer 1 may be evaluated as 95% accurate, answer 2 as 20% accurate, etc. The input is the generated answers, and the output is the accuracy evaluation result for each answer.

[0913] Step 7:

[0914] The server organizes the answers based on the evaluation results.

[0915] The server organizes each answer based on the evaluation results. Answers with low accuracy are marked as "likely incorrect." The input is the accuracy evaluation result of each answer, and the output is the organized answer.

[0916] Step 8:

[0917] The server sends the organized response to the terminal.

[0918] The server sends the organized answer to the terminal. The answer is formatted in a way that is easy for the user to understand. The input is the organized answer, and the output is the answer sent to the terminal.

[0919] Step 9:

[0920] The terminal displays the answer to the user.

[0921] The terminal displays the answer sent from the server to the user. For example, it might say, "The diameter of the Earth is approximately 12,742 kilometers." or "The diameter of the Earth is approximately 10,000 kilometers. (This is likely incorrect.)" The input is the answer sent from the server, and the output is the answer displayed to the user.

[0922] (Application example 1)

[0923] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0924] With conventional chatbot systems, it is difficult to guarantee the accuracy of answers to user questions, and there is a risk of providing incorrect information, especially for security-related questions. Furthermore, if an answer is likely to be inaccurate, the system lacks the functionality to notify the user, which can lead to the user believing the incorrect information. This not only undermines user trust, but also increases security risks.

[0925] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0926] In this invention, the server includes a means for probabilistically specifying "incorrect," a means for generating multiple answers to a user's question, and a means for probabilistically evaluating the accuracy of each answer, thereby making it possible to provide the user with the most reliable answer and to notify the user when an answer may be inaccurate.

[0927] "Means to clearly indicate that an answer is probabilistically incorrect" is a function that clearly indicates, based on a probabilistic evaluation, when there is a possibility that the answer provided to the user may not be accurate.

[0928] The "means for indicating that the validity cannot be determined by a human" is a function that indicates that the validity of the answer provided cannot be easily confirmed by human judgment.

[0929] "Means for clearly indicating the resources that led to the answer" is a function that clearly indicates to the user which information sources or databases the generated answer is based on.

[0930] "Means for restricting answers to only paper databases, etc." is a function that limits the information sources used when generating answers to specific databases or papers.

[0931] The "means for generating multiple answers to a question from a user" is a function that generates multiple possible answers to a question entered by a user.

[0932] The "means for probabilistically evaluating the accuracy of each answer" is a function for evaluating the accuracy of each generated answer using a probabilistic method.

[0933] "Means for selecting the most reliable answer and providing it to the user" is a function for selecting the most reliable answer from the evaluated answers and providing it to the user.

[0934] "Means for notifying the user if the answer may not be accurate" is a function that notifies the user if the answer provided may not be accurate.

[0935] As an embodiment of the present invention, a system for installing a security assistant application on a smartphone will be described, which provides probabilistically evaluated answers to security questions posed by a user.

[0936] System configuration

[0937] This system uses the following hardware and software:

[0938] Hardware

[0939] Smartphone

[0940] software

[0941] Generative AI models (e.g., GPT-3)

[0942] Python Program

[0943] Data processing and calculation

[0944] When the server receives a question from a user, it uses a generative AI model to generate multiple answers. The generated answers are then evaluated using a probabilistic method. Specifically, the accuracy of each answer is probabilistically evaluated, and the most reliable answer is selected based on that evaluation.

[0945] If the selected answer may not be accurate, we will notify the user so that they can determine the reliability of the information provided.

[0946] Specific examples

[0947] If a user asks, "How can I tell if my password has been compromised?" the system responds:

[0948] This answer may not be accurate: To check for password leaks, use our data leak checker.

[0949] Prompt Sentence Examples

[0950] An example of a prompt for the user to enter a question is:

[0951] User: How can I check if my password has been compromised?

[0952] Assistant: Use our data leak checker to check for password leaks.

[0953] In this way, users can get reliable answers to their security questions and be notified if the answers may not be accurate, allowing them to determine the reliability of the information provided.

[0954] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0955] Step 1:

[0956] The user enters their security questions using their smartphone.

[0957] Input: User question (e.g., "How can I check if my password has been leaked?")

[0958] Output: The user's question is sent to the server.

[0959] Step 2:

[0960] The server inputs the received user question into a generative AI model.

[0961] Input: User question

[0962] Output: Multiple answers generated by the generative AI model (e.g., 5 answers)

[0963] Step 3:

[0964] The server probabilistically evaluates the generated answers.

[0965] Input: Multiple answers generated by a generative AI model

[0966] Output: A probability value indicating the accuracy of each answer (e.g., a confidence score for each answer)

[0967] Step 4:

[0968] The server selects the most reliable answer from the evaluated answers.

[0969] Input: Probability values ​​indicating the accuracy of each answer

[0970] Output: The most reliable answer (e.g., the answer with the highest confidence score)

[0971] Step 5:

[0972] The server will notify the user if the selected answer may not be accurate.

[0973] Input: The most reliable answer and its reliability score

[0974] Output: The answer provided to the user and a notification that it may not be accurate (e.g., "This answer may not be accurate: Use a data leak checker to check if your password has been leaked.")

[0975] Step 6:

[0976] The user checks the answer provided by the server on their smartphone.

[0977] Input: The answer provided by the server and a notification that it may not be accurate

[0978] Output: Answer and notification for user to review

[0979] In this way, users can get reliable answers to their security questions and be notified if the answers may not be accurate, allowing them to determine the reliability of the information provided.

[0980] Example 2

[0981] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0982] Systems using conventional generative AI models have had the problem that it is difficult for users to judge the validity of the generated answers. Furthermore, they lack the functionality to display the probability of an answer to evaluate its validity, which makes it difficult for users to trust the system's answers. Furthermore, there are insufficient means to clearly indicate the resources that led to the answer, making it difficult for users to confirm the basis for the answer.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0984] In this invention, the server includes: a means for a user to input a prompt sentence; a means for a terminal to send the prompt sentence to the server; a means for the server to receive the prompt sentence and pass it to a generative AI model; a means for the generative AI model to generate an answer based on the prompt sentence; a means for the server to receive the generated answer and calculate the probability of evaluating its validity; a means for the server to present the answer and the probability of its validity to the user; a means to clearly indicate that the answer is probabilistically "incorrect"; a means to indicate that the answer is "unvalid for a human to determine its validity"; and a means to clearly indicate the resources that led to the answer or to restrict the answer to be based solely on a database. This allows the user to probabilistically evaluate the validity of the generated answer and confirm the basis for the answer.

[0985] A "user" is an entity that utilizes the system to enter prompts and receive generated answers.

[0986] A "terminal" is a device that a user uses to input a prompt and send it to a server. For example, this could be a computer or a smartphone.

[0987] A "server" is a computer system that receives prompt sentences, passes them to a generative AI model, and presents the generated answers to the user.

[0988] A "prompt sentence" is the text of a question or request that a user enters into the system.

[0989] A "generative AI model" is an artificial intelligence algorithm that generates answers based on prompts, such as a natural language processing model.

[0990] An "answer" is text that a generative AI model generates based on a prompt.

[0991] The "probability of evaluating correctness" is a numerical value indicating the probability that the generated answer will be judged to be correct.

[0992] A "means of probabilistically specifying 'incorrect'" is a method for probabilistically indicating the possibility that the generated answer is incorrect.

[0993] The "means of demonstrating that the validity cannot be determined by a human" is a method of providing information that allows a user to judge for themselves the validity of the generated answer.

[0994] "Resources" are the information sources or databases used to arrive at the generated answer.

[0995] The "means for restricting answers to be given only from a database" is a method for generating answers using only information obtained from a specific database.

[0996] This invention relates to a system that allows a user to input a prompt sentence, generates an answer using a generative AI model, and evaluates the validity of the answer. Specific embodiments of this system are described below.

[0997] First, the user inputs a prompt using a terminal. The terminal is a device such as a computer or smartphone, and is responsible for sending the prompt input by the user to the server. Examples of prompts include "What is the diameter of the Earth?" and "How many planets are there in the solar system?"

[0998] Next, the device sends the prompt sentence to the server. The server receives the prompt sentence and passes it to a generative AI model. A natural language processing model, such as OpenAI's GPT-4, is used as the generative AI model.

[0999] The generative AI model generates an answer based on the received prompt, for example, "The diameter of the Earth is approximately 12,742 kilometers." This generated answer is returned to the server.

[1000] The server receives the generated answer and calculates the probability of evaluating its correctness. This probability is calculated based on the internal algorithm of the generative AI model. For example, if the probability that the generated answer will be judged correct is 85%, the server calculates this probability.

[1001] Finally, the server presents the answer and its probability of correctness to the user. For example, it displays the answer in the form of "The diameter of the Earth is approximately 12,742 kilometers. (Probability of correctness: 85%)." This allows the user to probabilistically evaluate the correctness of the generated answer and check the basis of the answer.

[1002] This system includes a means to clearly indicate that a result is probabilistically "incorrect," a means to show that the "correctness cannot be determined by humans," a means to clearly indicate the resources that led to the answer, or a means to restrict the answer to only the database. This allows users to trust the system's answers and use them appropriately.

[1003] As a concrete example, if a user inputs the prompt "What is the diameter of the Earth?", the server will use the generative AI model to generate the answer "The diameter of the Earth is approximately 12,742 kilometers," evaluate the probability of its correctness as 85%, and present it to the user. In this way, the user can use the system while checking the correctness of the answer.

[1004] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1005] Step 1:

[1006] The user enters a prompt statement.

[1007] The user types a question or request into an input field on the terminal, for example, "What is the diameter of the Earth?" The entered prompt is temporarily stored in the terminal's memory.

[1008] Step 2:

[1009] The terminal sends the prompt to the server.

[1010] The device sends the input prompt text to the server as an HTTP POST request. At this time, the device sends data using an Internet connection. The input data is the prompt text, and the output is a notification to the server that transmission has been completed.

[1011] Step 3:

[1012] The server receives the prompt sentence and passes it to the generative AI model.

[1013] The server receives the prompt sentence sent from the terminal. After receiving it, the server passes this prompt sentence to the generative AI model in an appropriate format (for example, JSON format). The input data is the prompt sentence, and the output is the input data to the generative AI model.

[1014] Step 4:

[1015] A generative AI model generates an answer based on the prompt.

[1016] A generative AI model generates an answer based on the prompt it receives, for example, "The diameter of the Earth is approximately 12,742 kilometers." The input data is the prompt, and the output is the generated answer.

[1017] Step 5:

[1018] The server receives the generated answer and calculates the probability of its correctness.

[1019] The server receives the answer generated by the generative AI model. Then, the server calculates the probability to evaluate the correctness of this answer. This probability is calculated based on the internal algorithm of the generative AI model. The input data is the generated answer, and the output is the probability of correctness.

[1020] Step 6:

[1021] The server presents the answer and its probability of correctness to the user.

[1022] The server presents the generated answer and its probability of correctness to the user. For example, it displays "The diameter of the Earth is approximately 12,742 kilometers. (Probability of correctness: 85%)" on a web page. The input data is the generated answer and its probability of correctness, and the output is the display to the user.

[1023] (Application example 2)

[1024] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1025] Conventional generative AI models do not provide users with information to determine the legitimacy of the generated answers, making it difficult for users to evaluate the reliability of the answers. Furthermore, while they are required to provide appropriate answers to security-related questions, the problem is that users cannot determine the legitimacy of the answers. This creates a risk that users will act based on incorrect information.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1027] In this invention, the server includes a means for clearly indicating that an answer is probabilistically "incorrect," a means for indicating that "the validity of the answer cannot be determined by humans," a means for specifying the resources used to arrive at the answer or a means for restricting the answer to only a paper database, etc., a means for presenting to the user the probability that each generated answer is judged to be correct, and a means for the generative AI model to generate an answer to a question entered by the user. This allows the user to confirm the validity of the generated answer as a probability, and to evaluate the reliability of the answer.

[1028] "Means for specifying probabilistically incorrect" is a means for probabilistically indicating the possibility that the generated answer is incorrect.

[1029] The "means for demonstrating that the validity cannot be determined by a human" is a means for demonstrating that the validity of the generated answer cannot be easily determined by a human.

[1030] "Means for clearly indicating the resources that led to the answer" refers to means for clearly indicating what information sources or databases the generated answer is based on.

[1031] The "means for restricting answers to be given only in the paper database, etc." is a means for restricting the generated answers to be given only in a specific database, for example, a paper database.

[1032] "Means for presenting to the user, for each generated answer, the probability that the answer will be judged to be correct" refers to means for calculating the probability that the generated answer will be judged to be correct and presenting that probability to the user.

[1033] "Means by which a generative AI model generates an answer to a question entered by a user" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[1034] In the system for implementing this invention, a generative AI model generates an answer to a question entered by a user and presents the validity of the answer as a probability. A specific embodiment of this system is described below.

[1035] System Configuration

[1036] The system is configured using the following hardware and software.

[1037] Hardware: Smartphone

[1038] Software: Python, OpenAI API

[1039] Data processing and calculation

[1040] 1. User enters question:

[1041] Users enter security questions through a smartphone application, such as "Is this password secure?"

[1042] 2. Generate prompt sentence:

[1043] The smartphone application converts the user's question into a prompt, which looks like this:

[1044] User Question: Is this password secure?

[1045] answer:

[1046] 3. Invoke the generative AI model:

[1047] The smartphone application sends the generated prompt sentences to OpenAI's generative AI model (e.g., GPT-3), which generates an appropriate answer.

[1048] 4. Probability calculation of the correctness of the answer:

[1049] The correctness of the generated answer is calculated as a probability. Using a hypothetical logic, it is determined to be correct with, for example, an 85% probability.

[1050] 5. User Presentation:

[1051] The smartphone application presents the generated answer and its probability of correctness to the user, for example, "Answer: Yes, it's safe. Probability of correctness: 85%."

[1052] Specific examples

[1053] If a user types the question "Is this password secure?", the system will:

[1054] 1. Convert the user's question into a prompt.

[1055] User Question: Is this password secure?

[1056] answer:

[1057] 2. Invoke a generative AI model to generate an answer.

[1058] 3. Calculate the correctness of your answer as a probability.

[1059] 4. Present the user with "Answer: Yes, it's safe. Probability of validity: 85%."

[1060] In this way, the user can check the validity of the generated answer as a probability, and can evaluate the reliability of the answer.

[1061] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1062] Step 1:

[1063] The user launches the application on their smartphone and enters security questions.

[1064] Input: User question (e.g. "Is this password secure?")

[1065] Output: The text of the user's question

[1066] Specific behavior: The user enters a question into the application's input field and presses the submit button.

[1067] Step 2:

[1068] The terminal receives the user's question and generates a prompt sentence.

[1069] Input: User question text

[1070] Output: Prompt text (e.g. "User Question: Is this password secure?\nAnswer:")

[1071] What it does: The application formats the user's question and creates a prompt suitable for the generative AI model.

[1072] Step 3:

[1073] The device sends the generated prompt to OpenAI's generative AI model to generate an answer.

[1074] Input: prompt statement

[1075] Output: The answer from the generative AI model (e.g., "Yes, it's safe")

[1076] What it does: The application calls the OpenAI API to send prompts and get answers.

[1077] Step 4:

[1078] The terminal calculates the correctness of the generated answer as a probability.

[1079] Input: Answer from a generative AI model

[1080] Output: Probability of the answer being correct (e.g. 85%)

[1081] What it does: The application uses hypothetical logic to calculate the probability of the answer being correct.

[1082] Step 5:

[1083] The terminal presents the generated answer and its probability of correctness to the user.

[1084] Input: Answer from generative AI model, probability of answer correctness

[1085] Output: Information presented to the user (e.g., "Answer: Yes, it's safe. Probability of validity: 85%")

[1086] Specific behavior: The application displays the generated answer and its probability of correctness on the screen and presents it to the user.

[1087] Example 3

[1088] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1089] In answer generation systems using conventional generative AI models, it is difficult for users to judge the legitimacy and reliability of the generated answers. In addition, because the information sources that led to the answer are not made clear, users cannot confirm the basis for the answer. Furthermore, because it is not possible to restrict the use of only specific information sources, the reliability of the answer may be reduced. There is a need to solve these problems.

[1090] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1091] In this invention, the server includes means for probabilistically indicating that the answer is "incorrect," means for indicating that the "legitimateness cannot be determined by humans," means for indicating the information sources that led to the answer, means for restricting the answer to only specific information sources, means for receiving and analyzing the user's question, means for generating an answer using a generative AI model, and means for presenting the generated answer and its information source to the user. This allows the user to confirm the legitimacy and reliability of the generated answer and clarify the basis for the answer. Furthermore, restricting the use of only specific information sources can improve the reliability of the answer.

[1092] The "means for clearly indicating that the answer is probabilistically incorrect" is a means for notifying the user that the generated answer may be probabilistically incorrect.

[1093] The "means for indicating that the validity cannot be determined by a human" is a means for indicating to the user that the validity of the generated answer cannot be easily determined by a human.

[1094] The "means for clearly indicating the information source that led to the answer" is a means for clearly indicating to the user which information source the generated answer is based on.

[1095] The "means for restricting the system to generate an answer using only a specific information source" is a means for restricting the system to generate an answer using only a specific information source.

[1096] The "means for receiving and analyzing a user's question" is a means for receiving a question entered by a user and analyzing that question.

[1097] "Means for generating an answer using a generative AI model" refers to means for generating an answer to a user's question using a generative AI model.

[1098] The "means for presenting the generated answer and its information source to the user" is a means for presenting the generated answer and the information source that led to the answer to the user.

[1099] This invention is a system that uses a generative AI model to generate answers to user questions and clearly indicates the information sources that led to the answers. Specific embodiments of this system are described below.

[1100] Hardware and software used

[1101] 1. Hardware

[1102] Server: Use a server with a powerful processor and sufficient memory. For example, use a server with a server-grade processor.

[1103] Terminal: A device that allows users to input questions. For example, a device that can connect to the Internet, such as a PC or smartphone.

[1104] 2. Software

[1105] Natural language processing libraries: used to parse the question. For example, use libraries such as SpaCy or NLTK.

[1106] Generative AI models: used to generate answers, for example, OpenAI's GPT-3.

[1107] Database API: Used to reference information sources, for example, using the API of a scientific literature database or a medical literature database.

[1108] System Operation

[1109] 1. User inputs a question

[1110] The user uses the device to input a question. For example, they open a browser and type "Please tell me about papers on the latest AI technology" into the system interface.

[1111] 2. Receiving and analyzing questions

[1112] The server receives the user's question and analyzes it using natural language processing (NLP) technology. For example, the server receives the question and analyzes the intent of the question using the SpaCy library. Specifically, it extracts the keywords "latest AI technology" and "paper."

[1113] 3. References to sources

[1114] The server refers to a specified information source (e.g., a specific paper database). For example, the server calls the API of a scientific and technical literature database to search for papers on "the latest AI technology."

[1115] 4. Answer generation using generative AI models

[1116] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate an answer based on the referenced information source. For example, the server inputs a prompt to GPT-3, such as "Please tell me papers about the latest AI technology," and obtains the generated answer. Specifically, it generates the answer, "Paper A was published in 2023."

[1117] 5. Source citation

[1118] The server presents the information source that led to the answer to the user. For example, the server may add information such as "This answer is based on a scientific and technical literature database."

[1119] 6. Presenting the Answer

[1120] The server sends the generated answer and its source to the user's device and displays it to the user. For example, the user's device might display a message such as, "Paper A, published in 2023, is a paper on the latest AI technology. This answer is based on a scientific and technical literature database."

[1121] Specific examples

[1122] User question input

[1123] Prompt: "Please tell me some papers about the latest AI technology."

[1124] Answer generation using generative AI models

[1125] Answer: "Paper A, published in 2023, is a paper on the latest AI technology."

[1126] Clarification of sources

[1127] Source: "This answer is based on scientific and technical literature databases."

[1128] In this way, the user can know which information source the system's answer is based on and can confirm the reliability of the answer. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1129] Step 1:

[1130] The user uses the terminal to enter a question.

[1131] Input: The user types a question into the terminal interface.

[1132] Specific operation: The user opens a browser and types "Please tell me papers about the latest AI technology" into the system interface.

[1133] Output: The user's question is sent from the terminal to the server.

[1134] Step 2:

[1135] The server receives the user's question and analyzes it using natural language processing (NLP) techniques.

[1136] Input: User question (e.g., "Please tell me papers about the latest AI technology.")

[1137] Specific operation: The server receives the question and analyzes the intent of the question using the SpaCy library. Specifically, it extracts the keywords "latest AI technology" and "paper."

[1138] Output: Extracted keywords and analysis results are obtained.

[1139] Step 3:

[1140] The server consults the specified information source (e.g., a particular research database).

[1141] Input: Extracted keywords and analysis results (e.g., "latest AI technology," "paper")

[1142] Specific operation: The server calls the API of a scientific and technical literature database and searches for papers on "latest AI technology."

[1143] Output: The search results will give you a list of relevant papers.

[1144] Step 4:

[1145] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate an answer based on the referenced sources.

[1146] Input: A list of papers obtained as search results and a prompt (e.g., "Please tell me papers about the latest AI technologies")

[1147] Specific operation: The server inputs the prompt and search results to GPT-3 and obtains the generated answer. Specifically, it generates the answer "Paper A was published in 2023."

[1148] Output: You get the generated answer.

[1149] Step 5:

[1150] The server presents the user with the information sources that led to the answer.

[1151] Input: The generated answer and its source (e.g., "Paper A was published in 2023. This answer is based on a scientific and technical literature database.")

[1152] Specific behavior: The server appends the source of the generated answer.

[1153] Output: Answers with source citations.

[1154] Step 6:

[1155] The server generates an answer and sends the answer and its source to the user's terminal and displays it to the user.

[1156] Input: Answer with source information (e.g., "Paper A was published in 2023. This answer is based on a scientific and technical literature database.")

[1157] Specific operation: The server sends a response to the user's device, specifying the source of the information.

[1158] Output: The answer is displayed on the user's terminal.

[1159] In this way, the user can see which sources the system's answers are based on and can verify the reliability of the answers.

[1160] (Application example 3)

[1161] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1162] In conventional information provision systems, it was difficult for users to judge the reliability of the information provided, and doubts often arose about the accuracy and reliability of the information, especially when the source of the information, such as a news article, was not clearly stated. In addition, there was a lack of means to probabilistically determine the legitimacy of the information, which created the risk that users would make decisions based on incorrect information. This could undermine user trust.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1164] In this invention, the server includes a means for probabilistically indicating that a result is "incorrect," a means for indicating that the "validity cannot be determined by humans," a means for indicating the resources used to arrive at the answer or a means for restricting the answer to only using a paper database, etc., and a means for indicating the source of each news article to increase its reliability. This allows users to easily determine the reliability of the information provided and avoid making decisions based on incorrect information. Furthermore, indicating the source of the information increases user trust.

[1165] "Means of clearly stating that information is probabilistically incorrect" refers to a means of using a specific algorithm to determine the accuracy of information probabilistically, and as a result, clearly stating that there is a possibility that the information is incorrect.

[1166] "Means for demonstrating that the legitimacy cannot be determined by humans" refers to means for demonstrating that the legitimacy of the information provided cannot be easily determined by human judgment.

[1167] "Means for indicating the resources that led to the answer" means a means for showing the user which information sources the system-generated answer is based on.

[1168] "Means for imposing a constraint that answers be given only from a research paper database, etc." means a means for imposing a constraint that the system generate answers by referring only to a specific database (for example, a research paper database).

[1169] "Means of clearly indicating which sources each article is based on in order to increase the reliability of news articles" refers to means of clearly indicating to users which sources each article is based on in order to increase the reliability of news articles.

[1170] As an embodiment of the present invention, a news distribution application will be described as an example. The server includes means for probabilistically indicating that a result is "incorrect," means for indicating that "the validity of the result cannot be determined by humans," means for indicating the resources that led to the answer or means for restricting the answer to only a database of papers, etc., and means for indicating which information source each news article is based on in order to increase the reliability of the news articles.

[1171] Explanation of program processing

[1172] The server uses Python requests and the BeautifulSoup library to retrieve news articles from the website. Specifically, the NewsApp class retrieves news articles, and the NewsArticle class holds information about each article. The fetch_news method retrieves news articles from the specified URL, and the display_articles method displays the retrieved articles.

[1173] Hardware and software used

[1174] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1175] Software: Python, requests library, BeautifulSoup library

[1176] Data processing and calculation

[1177] 1. Data retrieval: The server retrieves the HTML data of the news article from the specified URL.

[1178] 2. Data Analysis: Parse the HTML data using the BeautifulSoup library to extract article titles, content, and sources.

[1179] 3. Save data: Save the extracted data as an instance of the NewsArticle class.

[1180] 4. Data display: Use the display_articles method to display the article title, content, and source on the user's device.

[1181] Specific examples

[1182] For example, if a news distribution app retrieves a news article from the URL "https: / / example.com / news1", it inputs the following prompt sentence into the generative AI model.

[1183] Prompt Sentence Examples

[1184] A news distribution app should retrieve news articles from the URL "https: / / example.com / news1" and display the article title, content, and source.

[1185] Using this prompt, the generative AI model can retrieve news articles from the specified URL and provide reliable information to the user.

[1186] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1187] Step 1:

[1188] The server receives the URL of the news article specified by the user.

[1189] Input: User-specified news article URL

[1190] Output: The URL of the news article to be retrieved.

[1191] What it does: When a user enters the URL of a news article into an application input field and presses the submit button, the server receives the URL.

[1192] Step 2:

[1193] The server retrieves HTML data from the specified URL using the requests library.

[1194] Input: News article URL

[1195] Output: Retrieved HTML data

[1196] Specific operation: The server executes requests.get(URL) to retrieve HTML data from the specified URL.

[1197] Step 3:

[1198] The server uses the BeautifulSoup library to parse the retrieved HTML data and extract the title, content, and source of the news article.

[1199] Input: Retrieved HTML data

[1200] Output: Title, content, and source of extracted news articles

[1201] Specific operation: The server parses the HTML data using BeautifulSoup, extracts the title with soup.find('h1').text, the content with soup.find('div', class_='article-content').text, and the URL itself as the source of information.

[1202] Step 4:

[1203] The server stores the extracted data as an instance of the NewsArticle class.

[1204] Input: Title, content, and source of extracted news articles

[1205] Output: An instance of the NewsArticle class

[1206] Specific operation: The server calls the constructor of the NewsArticle class and saves the extracted data in the instance.

[1207] Step 5:

[1208] The server transmits the stored instance of the news article to the user's terminal for display.

[1209] Input: An instance of the NewsArticle class

[1210] Output: The title, content, and source of the news article displayed on the user's device

[1211] Specific operation: The server calls the display_articles method to send the title, content, and source of the news article to the user terminal and display it.

[1212] Step 6:

[1213] The user checks the title, content, and source of the displayed news article.

[1214] Input: Title, content, and source of news article displayed on user device

[1215] Output: User confirmation result

[1216] What it does: The user checks the title, content, and source of a news article displayed on the device to determine the reliability of the information.

[1217] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1218] "Example 1"

[1219] The present invention is a system that includes an emotion engine that recognizes a user's emotions. Specifically, it receives a question from a user and generates possible answers to that question. In the answer generation process, a specific algorithm is used to generate possible answers to the question. The correctness of each generated answer is probabilistically evaluated, and the evaluation results are presented to the user. Furthermore, the information source (e.g., a specific paper database) referenced to generate the answer is presented to the user. This allows the user to know which information source the system's answer is based on. It is also possible to set restrictions, such as limiting answers to specific paper databases.

[1220] "Example 2"

[1221] The emotion engine recognizes the user's emotional state and adjusts responses based on that emotional state. For example, if the user feels angry, the system will provide them with more polite responses. If the user feels happy, the system will provide them with more cheerful responses. In this way, the emotion engine provides a user experience that takes the user's emotional state into account.

[1222] "Example 3"

[1223] As a concrete example, suppose a user asks, "What's the weather like today?" The system generates potential answers to this question and probabilistically evaluates the accuracy of each answer. It then presents the results of this evaluation to the user along with the information source (e.g., a specific weather database) that was used to generate the answer. Additionally, the emotion engine recognizes the user's emotional state and adjusts the answer based on that emotional state. For example, if the user feels happy, the system might provide a more cheerful answer such as, "It's sunny today. It's going to be a great day!"

[1224] The processing flow of each embodiment will be described below.

[1225] "Example 1"

[1226] Step 1: The system receives a question from the user.

[1227] Step 2: The system uses a specific algorithm to generate possible answers to the question.

[1228] Step 3: Probabilistically evaluate whether each generated answer is correct.

[1229] Step 4: Present the user with the results of the assessment and the sources consulted to generate the answer (e.g., a specific research database).

[1230] Step 5: It is also possible to set restrictions such as limiting responses to specific paper databases. "Example 2"

[1231] Step 1: The emotion engine recognizes the user's emotional state.

[1232] Step 2: Adjust the response based on the user's emotional state. For example, if the user feels angry, the system will provide a more polite response to the user.

[1233] Step 3: If the system senses that the user is happy, it will provide a more enjoyable response to that user.

[1234] "Example 3"

[1235] Step 1: Suppose a user asks, "What's the weather like today?"

[1236] Step 2: The system generates possible answers to this question and probabilistically evaluates the correctness of each answer.

[1237] Step 3: Present the user with the evaluation results and the information sources (e.g., a specific weather database) that were consulted to generate the answer.

[1238] Step 4: The emotion engine recognizes the user's emotional state and adjusts the answer based on that emotional state. For example, if the user feels happy, the system will provide a more cheerful answer such as "It's sunny today. It looks like it's going to be a great day!"

[1239] Example 1

[1240] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1241] Conventional chatbot systems lacked a means to evaluate the accuracy and reliability of answers to user questions, making it difficult for users to determine the reliability of the information provided. Furthermore, they lacked a function to clearly indicate the source of the answer, meaning users could not confirm the basis for the answer. Furthermore, they lacked a function to probabilistically evaluate the accuracy of the answer, making it difficult for users to understand the reliability of the answer.

[1242] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1243] In this invention, the server includes means for receiving a question from a user, means for inputting the received question as a prompt to a generative AI model, means for generating multiple answers from the generative AI model, means for probabilistically evaluating the accuracy of each generated answer, means for presenting an answer including the evaluation result and a reference information source to the user, means for probabilistically indicating that the answer is "incorrect," means for indicating that the "validity cannot be determined by a human," and means for indicating the resources that led to the answer or for restricting the answer to only using a database, etc. This allows the user to evaluate the reliability of the information provided and confirm the basis for the answer.

[1244] A "user" is an entity that inputs questions to the system and receives answers.

[1245] A "server" is a device or system that receives questions from users, generates answers using a generative AI model, and presents them to the user along with evaluation results.

[1246] A "generative AI model" is an artificial intelligence model used to generate multiple answers to a user question.

[1247] A "prompt" is a question or instruction entered into a generative AI model; it is the input data that the model uses to generate an answer.

[1248] "Probabilistic evaluation" means evaluating the accuracy of each generated answer based on probability theory.

[1249] The "evaluation result" is the result of a probabilistic evaluation of the accuracy of the generated answer.

[1250] A "reference source" is a source such as a database or literature that provides information on which the generated answer is based.

[1251] "Specifying that the answer is probabilistically incorrect" means specifying, based on probability theory, that the generated answer may be incorrect.

[1252] "Demonstrating that the validity cannot be determined by a human" means demonstrating that the validity of the generated answer cannot be easily determined by a human.

[1253] "Resources" are the information sources or databases consulted to generate the answer.

[1254] A "database" is a collection of information that is systematically organized and stored so that it can be searched and used as needed.

[1255] This invention relates to a chatbot system that generates answers to user questions. This system receives questions from users, generates answers using a generative AI model, probabilistically evaluates the accuracy of the answers, and presents them to the user.

[1256] The server provides an interface for receiving questions from users. When a user inputs a question, the device sends the question to the server. The server then inputs the received question as a prompt into a generative AI model (e.g., GPT-4). The generative AI model generates multiple answers to the question.

[1257] The server probabilistically evaluates the accuracy of each generated answer. This evaluation is performed using a specific algorithm. For example, the answer "The latest treatment is monoclonal antibody therapy" is evaluated as being correct 80% of the time. Along with this evaluation result, the server presents the user with a reference source (e.g., a specific paper database). This allows the user to know which source the system's answer is based on.

[1258] As a concrete example, consider the case where a user asks, "What is the latest treatment for COVID-19?" The device sends this question to the server. The server uses a generative AI model (GPT-4) to generate possible answers. For example, answers such as "The latest treatment is monoclonal antibody therapy" or "The latest treatment is mRNA vaccine" are generated. The server probabilistically evaluates the accuracy of each answer and sends the answer, including the evaluation result and reference source, to the device. The device displays the message to the user, "The latest treatment is monoclonal antibody therapy. This information is based on a specific paper database. There is an 80% chance that this answer is correct."

[1259] This system allows users to evaluate the reliability of the information provided and check the basis of the answer. Also, by probabilistically evaluating the accuracy of the answer, users can easily understand the reliability of the answer. Furthermore, by clearly indicating the reference source, users can check the basis of the answer.

[1260] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1261] Step 1:

[1262] The user enters a question.

[1263] The user enters a question through the device's interface, for example, "What is the latest treatment for COVID-19?" The entered question is displayed in an input field on the device.

[1264] Step 2:

[1265] The terminal sends a question to the server.

[1266] The terminal sends the question entered by the user to the server using a protocol such as an HTTP request. The input data is the user's question, and the output data is the question sent to the server.

[1267] Step 3:

[1268] The server receives the question and inputs it as a prompt into the generative AI model.

[1269] The server inputs the question received from the device as a prompt to the generative AI model (e.g., GPT-4). This prompt is converted into a format that the model can easily understand. The input data is the user's question, and the output data is the prompt that is input to the generative AI model.

[1270] Step 4:

[1271] The server generates multiple answers from the generative AI model.

[1272] The server generates multiple answers from the generative AI model, such as "The latest treatment is monoclonal antibody therapy" or "The latest treatment is mRNA vaccine." The input data is the prompt, and the output data is the generated multiple answers.

[1273] Step 5:

[1274] The server probabilistically rates the accuracy of each answer.

[1275] The server probabilistically evaluates the accuracy of each generated answer. This evaluation is performed using a specific algorithm. For example, the answer "monoclonal antibody therapy" is evaluated as being correct 80% of the time. The input data are the generated answers, and the output data are the evaluation results of the accuracy of each answer.

[1276] Step 6:

[1277] The server sends a response including the evaluation result and the reference information source to the terminal.

[1278] The server sends a response including the evaluation result and a reference source (e.g., a specific paper database) to the terminal. This transmission is also performed using a protocol such as HTTP response. The input data is the response including the evaluation result and the reference source, and the output data is the response sent to the terminal.

[1279] Step 7:

[1280] The terminal displays the answer to the user.

[1281] The terminal displays the answer received from the server to the user. For example, it may say, "The latest treatment is monoclonal antibody therapy. This information is based on a specific research paper database. There is an 80% chance that this answer is correct." The input data is the answer received from the server, and the output data is the answer displayed to the user.

[1282] (Application example 1)

[1283] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1284] Conventional chatbot systems lacked the ability to probabilistically evaluate the accuracy of answers to user questions, making it difficult for users to determine the reliability of the information provided. They also lacked the ability to recognize users' emotions and provide appropriate feedback based on them, resulting in a poor user experience. Furthermore, they lacked the ability to clearly indicate the resources that led to the answer, making it difficult for users to verify the source of the information. These issues need to be resolved.

[1285] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1286] In this invention, the server includes a means for probabilistically indicating that the answer is "incorrect," a means for indicating that the "validity cannot be determined by humans," a means for indicating the resources that led to the answer or a means for restricting the answer to only a paper database, a means for recognizing the user's emotions, and a means for providing appropriate feedback based on the emotion recognition results. This allows the user to probabilistically evaluate the reliability of the provided information and receive appropriate feedback based on their emotions. Furthermore, by indicating the resources that led to the answer, the user can confirm the source of the information.

[1287] "Means to clearly state that something is probabilistically incorrect" is a function that probabilistically evaluates the accuracy of answers to user questions and presents the evaluation results to the user.

[1288] "Means to show that a human cannot determine the validity" is a function that clearly shows that a human cannot easily determine whether the generated answer is correct.

[1289] "Means for clearly indicating the resources that led to the answer" is a function that shows the user which information sources the generated answer is based on.

[1290] "Means for restricting answers to be generated only from a research paper database, etc." is a function for restricting answers to be generated only from information obtained from a specific database.

[1291] "Means for recognizing user emotions" is a function that analyzes emotions from the user's questions and input and obtains the results.

[1292] "Means for providing appropriate feedback based on emotion recognition results" is a function that provides appropriate answers or actions based on the results of user emotion analysis.

[1293] A system for implementing this invention includes a server, a user terminal, and a network. The server executes a program for generating answers to user questions and probabilistically evaluating their accuracy. Specifically, the process is carried out in the following manner.

[1294] First, the user device receives a question from the user and sends it to the server. The server analyzes the received question and generates possible answers using a generative AI model. In doing so, it uses a specific algorithm to probabilistically evaluate the accuracy of each answer. The evaluation results are presented to the user as a means of clearly indicating which answers are probabilistically "incorrect."

[1295] Next, the server uses an emotion engine to recognize emotions from the user's question. This emotion engine is implemented, for example, using the pipeline in the transformers library. Based on the emotion recognition results, the server generates appropriate feedback and provides it to the user.

[1296] Furthermore, the server clearly indicates which sources the generated answers are based on, allowing users to verify the reliability of the answers. Sources include specific databases and research papers.

[1297] As a concrete example, if a user asks "How can I protect myself from being hacked?" the server will generate the following result:

[1298] User: How can I protect myself from being hacked?

[1299] System: Install antivirus software (78% confidence).

[1300] Reference sources: Security expert blogs, latest security papers

[1301] Sentiment Analysis: Positive (85%)

[1302] In this way, users can probabilistically evaluate the reliability of the provided information and receive appropriate emotional feedback. Furthermore, by clearly citing the resources that led to the answer, users can verify the source of the information.

[1303] The hardware used includes user devices (smartphones, tablets, PCs, etc.) and servers, and the software used includes generative AI models, emotion engines (such as pipelines in the transformers library), and database access libraries.

[1304] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1305] Step 1:

[1306] The user terminal receives a question from the user.

[1307] Input: User question (e.g., "How can I protect myself from being hacked?")

[1308] Output: Question data

[1309] Specific behavior: The user enters a question into the input field on the device and presses the send button.

[1310] Step 2:

[1311] The user terminal transmits the received question data to the server.

[1312] Input: Question data

[1313] Output: Request to server

[1314] Specific operation: The user terminal sends the question data to the server as an HTTP request.

[1315] Step 3:

[1316] The server analyzes the received question data and generates possible answers using a generative AI model.

[1317] Input: Question data

[1318] Output: A list of generated answers

[1319] Specific operation: The server inputs question data into a natural language processing model and generates multiple answer candidates.

[1320] Step 4:

[1321] The server uses a specific algorithm to probabilistically assess the accuracy of each answer.

[1322] Input: Generated answer list

[1323] Output: List of answers with probability ratings

[1324] Specific operation: The server applies a probabilistic evaluation algorithm to each answer and calculates the confidence of each answer.

[1325] Step 5:

[1326] The server uses an emotion engine to recognize emotions from the content of the user's question.

[1327] Input: Question data

[1328] Output: Emotion recognition result

[1329] Specific operation: The server inputs the question data into an emotion engine (e.g., the pipeline of the transformers library) and performs emotion analysis.

[1330] Step 6:

[1331] The server generates appropriate feedback based on the emotion recognition results.

[1332] Input: Emotion recognition results, answer list with probability evaluation

[1333] Output: Answer with feedback

[1334] Specific operation: Based on the emotion recognition results, the server selects an answer that includes appropriate feedback for the user.

[1335] Step 7:

[1336] The server specifies the source of the generated answer.

[1337] Input: List of answers with probability ratings

[1338] Output: List of answers with sources

[1339] Specific operation: The server adds a reference source to each answer and generates data to present to the user.

[1340] Step 8:

[1341] The server transmits the final response data to the user terminal.

[1342] Input: Sourced Answer List

[1343] Output: Response to user terminal

[1344] Specific operation: The server sends the final answer data to the user terminal as an HTTP response.

[1345] Step 9:

[1346] The user terminal displays the received response data to the user.

[1347] Input: Response from the server

[1348] Output: What the user sees

[1349] Specific operation: The user terminal displays the received response data on the screen and provides it to the user.

[1350] Example 2

[1351] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1352] Conventional systems have problems in that it is difficult for users to judge the validity of the generated answers and they are unable to respond appropriately depending on the user's emotional state, which can lead to a poor user experience and a loss of system reliability.

[1353] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1354] In this invention, the server includes means for probabilistically indicating that the answer is "incorrect," means for indicating that the "validity cannot be determined by a human," means for indicating the resources that led to the answer, means for recognizing the user's emotional state, means for adjusting the answer based on the recognized emotional state, and means for presenting the user with the probability of the validity of the generated answer. This allows the user to determine for themselves the validity of the generated answer, and further enables appropriate responses according to the user's emotional state.

[1355] "Means for specifying probabilistically incorrect" is a means for probabilistically indicating the possibility that the generated answer is incorrect.

[1356] "Means for demonstrating that the validity cannot be determined by a human" refers to means for demonstrating that the validity of the generated answer cannot be easily determined by a user.

[1357] "Means for indicating the resources that led to the answer" means a means for showing the user which information sources or databases the generated answer is based on.

[1358] The "means for imposing a constraint that answers are to be obtained only from a database, etc." is a means for imposing a constraint that answers are to be generated by obtaining information only from a specific database.

[1359] "Means for recognizing a user's emotional state" refers to means for analyzing and recognizing a user's emotional state from their input and actions.

[1360] The "means for adjusting a response based on a recognized emotional state" refers to a means for appropriately adjusting a generated response in accordance with the recognized emotional state of the user.

[1361] The "means for presenting to the user the probability of the correctness of the generated answer" is a means for showing to the user the probability that the generated answer will be judged to be correct.

[1362] This invention is a system equipped with an emotion engine that recognizes the user's emotional state and adjusts answers based on that emotion. It also has a function that displays to the user the probability that each generated answer is judged to be correct, allowing the user to judge the validity of the system's answers for themselves.

[1363] Hardware and software used

[1364] Hardware: Servers, user devices (PCs, smartphones, etc.)

[1365] Software: Generative AI models (e.g., GPT-4), emotion recognition engines

[1366] Explanation of program processing

[1367] 1. Accepting user input

[1368] Users enter questions and comments through the device.

[1369] Example: User types "What's the weather like today?"

[1370] 2. Recognizing emotional states

[1371] The server uses an emotion recognition engine to parse the emotional state from the user's input.

[1372] Example: If the user's input indicates anger, the emotion recognition engine will determine "Anger".

[1373] 3. Generate answers

[1374] The server uses a generative AI model to generate answers to user input.

[1375] Example: Generate "The weather is sunny today."

[1376] 4. Adjust your responses based on emotions

[1377] The server uses an emotion engine to tailor the response based on the user's emotional state.

[1378] Example: If the user is angry, adjust your response to a more polite one: "The weather is sunny today. Let me know if I need any help."

[1379] 5. Probability of the correctness of the answer

[1380] The server calculates the probability that the generated answer is correct and presents it to the user.

[1381] Example: Display "Today's weather is sunny (accuracy: 95%)."

[1382] 6. Providing answers to users

[1383] The server sends the adjusted answer and its probability of correctness to the user's terminal and displays it to the user.

[1384] Example: The user sees the message "Today's weather is sunny (95% accuracy). Let us know if we need any help."

[1385] Examples of concrete examples and prompts

[1386] Examples:

[1387] The user types, "I'm having trouble at work."

[1388] The server uses an emotion recognition engine to detect "sadness."

[1389] The server uses a generative AI model to generate a response: "That's tough. Is there anything I can help you with?"

[1390] The server uses an emotion engine to adjust based on "sadness" to say, "That's terrible. Is there anything I can help you with? Let me hear your story."

[1391] The server calculates the probability as "Accuracy: 90%" and displays it to the user.

[1392] The server sends the adjusted answer and its probability of correctness to the user's terminal, where it is displayed on the user's screen.

[1393] Example prompt sentence:

[1394] "If a user types 'I'm having trouble at work,' explain how the emotion recognition engine detects 'sadness,' the generative AI model generates an appropriate response, and the emotion engine adjusts that response."

[1395] In this way, the system takes into account the user's emotional state and provides justification for the answer, providing a more personalized experience for the user.

[1396] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1397] Step 1:

[1398] Accepting user input

[1399] Users enter questions and comments through the device.

[1400] Input: User types "What's the weather like today?"

[1401] Specific behavior: A user types "What's the weather like today?" into a chat app on their smartphone and presses the send button.

[1402] Output: User input data is sent to the server.

[1403] Step 2:

[1404] Recognition of emotional states

[1405] The server uses an emotion recognition engine to parse the emotional state from the user's input.

[1406] Input: User input data "What's the weather like today?"

[1407] Specific operation: The server calls the emotion recognition engine and analyzes the user's text data.

[1408] Data processing: Text data is input into an emotion recognition engine to determine the emotional state.

[1409] Output: Emotional state is determined to be "neutral".

[1410] Step 3:

[1411] Generate answers

[1412] The server uses the generative AI model to generate answers to the user's input.

[1413] Input: User input data "What's the weather like today?" and emotional state "Neutral"

[1414] What happens: The server invokes the generative AI model and uses the user's input data as prompts.

[1415] Data computation: Generative AI models generate answers based on prompts.

[1416] Output: The answer "The weather is sunny today" is produced.

[1417] Step 4:

[1418] Adjusting responses based on emotion

[1419] The server uses an emotion engine to tailor the response based on the user's emotional state.

[1420] Input: Generated answer "The weather is sunny today" and emotional state "Neutral"

[1421] What it does: The server invokes the emotion engine and adjusts the generated answers based on the emotional state.

[1422] Data processing: Use the answers as they are depending on the emotional state.

[1423] Output: You get the adjusted answer "The weather is sunny today."

[1424] Step 5:

[1425] Probability of the answer being correct

[1426] The server calculates the probability that the generated answer is correct and presents it to the user.

[1427] Input: Adjusted answer "The weather is sunny today"

[1428] What happens: The server invokes an algorithm that calculates the probability of correctness and evaluates the accuracy of the answer.

[1429] Data calculation: Calculate the accuracy of the answer as a probability.

[1430] Output: A 95% probability of accuracy is obtained.

[1431] Step 6:

[1432] Providing answers to users

[1433] The server sends the adjusted answer and its probability of correctness to the user's terminal and displays it to the user.

[1434] Input: Calibrated answer "The weather is sunny today" and probability of accuracy "95%"

[1435] Specific operation: The server sends the response data to the user device.

[1436] Output: The user will see "Today's weather is sunny (accuracy: 95%)".

[1437] (Application example 2)

[1438] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1439] Conventional systems provide uniform answers without considering the user's emotional state, which can lead to an inconsistent user experience. Furthermore, users lack the means to determine the legitimacy of the generated answers, which creates the risk of believing incorrect information. Furthermore, there is little transparency regarding the resources that led to the answer, making it difficult for users to verify the reliability of the information.

[1440] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1441] In this invention, the server includes means for probabilistically indicating that an answer is "incorrect," means for indicating that the "validity cannot be determined by a human," means for indicating the resources that led to the answer, means for recognizing the emotional state of the user and adjusting the answer based on that emotional state, and means for presenting to the user the probability that each generated answer will be determined to be correct. This makes it possible to provide an appropriate answer according to the user's emotional state and enable the user to determine the validity of the answer.

[1442] "Means for clearly stating that a given answer is probabilistically incorrect" refers to a means for probabilistically evaluating whether the generated answer is correct and clearly indicating the result to the user.

[1443] "Means for showing that 'humans cannot determine the validity'" refers to means for showing the user that the validity of the generated answer is difficult for humans to determine.

[1444] "Means for indicating the resources that led to the answer" means a means for indicating to the user which information sources or databases the generated answer is based on.

[1445] "Means for recognizing a user's emotional state and adjusting responses based on that emotional state" refers to means for recognizing a user's emotions in real time and adjusting the content and tone of responses according to those emotions.

[1446] "Means for presenting to the user the probability that each generated answer will be judged to be correct" refers to means for calculating the validity of the generated answer as a probability and presenting that probability to the user.

[1447] The system for implementing this invention has an emotion engine that recognizes the user's emotional state and adjusts answers based on that emotional state. It also has the function of presenting the user with the probability that each generated answer is judged to be correct. A specific embodiment of this system is described below.

[1448] Hardware and software used

[1449] Hardware: Smartphone

[1450] software:

[1451] EmotionRecognizer: A library for recognizing a user's emotional state

[1452] ContentRecommender: A library for recommending content based on emotional states

[1453] Data processing and calculation

[1454] emotion recognition

[1455] The server passes the user's input text to the EmotionRecognizer library to recognize their emotional state. For example, if a user types, "I'm feeling very happy today," the system will parse the text and recognize the emotion "happy."

[1456] Content Recommendation

[1457] The server passes the recognized emotional state to the ContentRecommender library, which recommends content appropriate to that emotion. For example, comedy movies are recommended for the emotion "happy." The server also calculates the validity of the recommended content as a probability and presents it to the user. For example, a comedy movie recommendation shows an 85% probability.

[1458] display

[1459] The device displays the recommended content and its probability of validity to the user, allowing the user to judge the validity of the recommended content for themselves.

[1460] Specific examples

[1461] If a user types "I'm feeling very happy today," the system will recognize it as "happy" and recommend a comedy movie with an 85% accuracy rate.

[1462] Prompt Sentence Examples

[1463] "Please describe how you feel: I'm feeling very happy today!"

[1464] "Please describe how you feel: I'm a bit sad and lonely."

[1465] In this way, emotion-responsive content delivery apps provide optimal content based on the user's emotional state and display its validity as a probability, allowing users to make their own judgments.

[1466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1467] Step 1:

[1468] The user inputs their emotional state into the terminal, for example, "I feel very happy today." This input text becomes the input data for the next processing step.

[1469] Step 2:

[1470] The device sends the input text to the server. The server passes the received text to the EmotionRecognizer library to recognize the emotional state. For example, the emotion "happy" is recognized from the text "I'm feeling very happy today." This emotional state becomes the input data for the next processing step.

[1471] Step 3:

[1472] The server passes the recognized emotional state to the ContentRecommender library, which recommends content appropriate to that emotion. For example, comedy movies are recommended for the emotion "happy." This recommended content becomes the input data for the next processing step.

[1473] Step 4:

[1474] The server calculates the validity of the recommended content as a probability. For example, it calculates a probability of 85% for a comedy movie recommendation. This probability becomes the input data for the next processing step.

[1475] Step 5:

[1476] The server sends the recommended content and its probability of validity to the device. The device displays the received content and probability to the user. For example, it displays "comedy movie" with a probability of "85%." This allows the user to judge the validity of the recommended content for themselves.

[1477] In this way, by clarifying the specific operations performed at each processing step and their inputs and outputs, it becomes easier to understand the processing flow of the entire system.

[1478] Example 3

[1479] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1480] In conventional systems, it is difficult for users to judge the accuracy and reliability of the generated answers, and the sources of information on which the answers are based are not clearly stated, which often leaves users feeling uneasy. Furthermore, the system does not provide personalized answers that reflect the user's emotional state, which limits the user experience.

[1481] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1482] In this invention, the server includes a means for probabilistically indicating that the answer is "incorrect," a means for indicating that the "correctness cannot be determined by a human," a means for indicating the resources that led to the answer, a means for restricting the answer to a specific database, a means for recognizing the user's emotional state, and a means for adjusting the answer based on the user's emotional state. This allows the user to confirm the accuracy and reliability of the generated answer and to know the source of the answer's basis. Furthermore, personalized answers based on the user's emotional state are provided, enabling a more satisfying user experience.

[1483] The "means for clearly indicating that the answer is probabilistically incorrect" is a means for probabilistically evaluating the possibility that the generated answer is incorrect and presenting the evaluation results to the user.

[1484] "Means for demonstrating that the validity cannot be determined by humans" refers to means for clearly indicating that the validity of the generated answer cannot be easily confirmed by human judgment.

[1485] The "means for clearly indicating the resources that led to the answer" is a means for presenting to the user the information sources that are the basis for the generated answer.

[1486] The "means for imposing a constraint that answers are only made using a specific database" is a means for imposing a constraint that only a specific database is referenced when generating an answer.

[1487] The "means for recognizing the emotional state of the user" is a means for analyzing the user's input and dialogue history to recognize the user's emotional state.

[1488] The "means for adjusting a response based on the emotional state of the user" refers to a means for adjusting a generated response in accordance with the recognized emotional state of the user.

[1489] This invention is a system in which a user inputs a question using a terminal, and a server generates an answer to that question using a generative AI model. The system ensures the accuracy and reliability of the answer to the user's question, and also provides a personalized answer according to the user's emotional state.

[1490] Hardware and software used

[1491] Hardware: Servers, terminals

[1492] Software: Generative AI model, emotion engine, database access module

[1493] Processing flow

[1494] 1. Accepting user questions

[1495] A user uses a terminal to input a question into the system.

[1496] Specific behavior: The user enters "What's the weather like today?" into the device's input field and presses the send button.

[1497] 2. Question Analysis

[1498] The server receives the user's question and generates a prompt to input into the generative AI model.

[1499] Specific operation: The server analyzes the question "What's the weather like today?" and converts it into a format suitable for the generative AI model. For example, it generates a prompt sentence such as "What's the weather like today?"

[1500] 3. Answer Generation

[1501] The server uses a generative AI model to generate possible answers to the question.

[1502] Specific operation: The server inputs a prompt sentence into the generative AI model, and the model generates the answer "It's sunny today."

[1503] 4. References to sources

[1504] Identify the information source (e.g., a particular weather database) that the server consulted to generate the answer.

[1505] Specific behavior: Record that the server referred to the weather database when generating the answer "It's sunny today."

[1506] 5. Evaluating your answers

[1507] The server probabilistically evaluates the accuracy of the generated answer.

[1508] Specific operation: The server performs a probability calculation based on data from the weather database to evaluate the accuracy of the answer "It's sunny today" as 90%.

[1509] 6. Citation of sources

[1510] The server presents the user with the generated answer and the information sources that were consulted to generate the answer.

[1511] Specific behavior: The server displays to the user, "It's sunny today. Source: Weather database."

[1512] 7. Recognizing emotional states

[1513] The server uses an emotion engine to recognize the user's emotional state.

[1514] Specific operation: The server analyzes the user's input and past dialogue history, and the emotion engine recognizes that the user is happy.

[1515] 8. Adjusting your answers

[1516] The server adjusts the generated answers based on the user's emotional state.

[1517] What happens: The server adjusts the answer "It's sunny today" to "It's sunny today. It's going to be a great day!"

[1518] 9. Presentation of the final answer

[1519] The server presents the adjusted final answer to the user.

[1520] Specific behavior: The server displays on the user's device, "It's sunny today. It looks like it's going to be a great day! Source: Weather database."

[1521] Specific examples

[1522] User Questions

[1523] User: "What's the weather like today?"

[1524] Prompt Sentence Examples

[1525] Prompt: "What's the weather like today?"

[1526] In this way, the system provides reliable answers to the user's questions and clearly indicates the source of the answer, giving the user a sense of security. Furthermore, the system adjusts the answer depending on the user's emotional state, providing a more personalized experience. The flow of the identification process in Example 3 will be described with reference to FIG. 21.

[1527] Step 1:

[1528] Accepting user questions

[1529] Input: The user enters a question into the device's input field and presses the submit button.

[1530] Specific behavior: The user types "What's the weather like today?" and presses the send button.

[1531] Output: The user's question is sent to the server.

[1532] Step 2:

[1533] Question Analysis

[1534] Input: The server receives the user's question.

[1535] What it does: The server parses the question "What's the weather like today?" and converts it into a format suitable for the generative AI model.

[1536] Data processing: Analyze the question and generate a prompt to input into the generative AI model.

[1537] Output: The prompt sentence "What's the weather like today?" is generated.

[1538] Step 3:

[1539] Generate answers

[1540] Input: The prompt sentence generated by the server is input to the generative AI model.

[1541] Specific operation: The server inputs the prompt sentence "What is the weather like today?" to the generated AI model.

[1542] Data computation: A generative AI model generates an answer based on the prompt.

[1543] Output: Produces the answer "It's sunny today."

[1544] Step 4:

[1545] References to sources

[1546] Input: The server receives the generated answer.

[1547] Specific operation: When the server generates the answer "It's sunny today," it refers to the weather database.

[1548] Data processing: Check the sources that support the answers generated.

[1549] Output: Records that the weather database was consulted.

[1550] Step 5:

[1551] Evaluating answers

[1552] Input: The server receives the generated answer and its source.

[1553] Specific operation: The server probabilistically evaluates the accuracy of the answer "It's sunny today."

[1554] Data calculation: Probability calculations are performed based on data from the weather database.

[1555] Output: "It's sunny today." is rated as 90% accurate.

[1556] Step 6:

[1557] Source of information

[1558] Input: The server receives the evaluated answer.

[1559] Specific behavior: The server displays to the user, "It's sunny today. Source: Weather database."

[1560] Data processing: Combining the answers and their sources and converting them into a format that can be presented to the user.

[1561] Output: Produces the message "It's sunny today. Source: Weather database".

[1562] Step 7:

[1563] Recognition of emotional states

[1564] Input: The server receives user input and past interaction history.

[1565] Specific operation: The server uses the emotion engine to recognize the user's emotional state.

[1566] Data computation: Analyzes user input and interaction history to identify emotional state.

[1567] Output: The user is perceived as happy.

[1568] Step 8:

[1569] Adjusting your answers

[1570] Input: The server receives the recognized emotional state of the user and the generated answer.

[1571] What happens: The server adjusts the answer "It's sunny today" to "It's sunny today. It's going to be a great day!"

[1572] Data manipulation: tailoring answers based on the user's emotional state.

[1573] Output: The adjusted answer is generated.

[1574] Step 9:

[1575] Providing the final answer

[1576] Input: The server receives the adjusted answer.

[1577] Specific behavior: The server displays on the user's device, "It's sunny today. It looks like it's going to be a great day! Source: Weather database."

[1578] Data processing: Combining tailored answers and their sources into a format that can be presented to the user.

[1579] Output: The message "It's sunny today. It's going to be a great day! Source: Weather database" is displayed on the user's terminal.

[1580] (Application example 3)

[1581] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1582] Conventional information provision systems have difficulty providing reliable answers to user questions, and lack the functionality to clearly indicate the source of the answer, making it difficult for users to judge the reliability of the answer.Furthermore, answers are not adjusted taking into account the user's emotional state, which has led to issues with the user experience not being improved.

[1583] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1584] In this invention, the server includes a means for probabilistically indicating that a response is "incorrect," a means for indicating that the "validity cannot be determined by a human," a means for indicating the resources used to arrive at the answer or a means for restricting the answer to sources only, and a means for recognizing the user's emotional state and adjusting the answer based on that emotional state. This allows the server to provide a highly reliable answer to a user's question and clearly indicate the sources on which the answer is based, enabling the user to determine the reliability of the answer. Furthermore, the adjustment of the answer taking the user's emotional state into account improves the user experience.

[1585] "Means for clearly indicating that an answer is probabilistically incorrect" is a function that uses a specific algorithm to determine the probability that an answer is incorrect and clearly indicates the result.

[1586] The "means for showing that the validity cannot be determined by a human" is a function for showing that the validity of an answer cannot be easily determined by a human.

[1587] "Means for clearly indicating the resources that led to the answer" is a function that shows the user the information sources that were consulted to generate the answer.

[1588] The "means for restricting answers to be generated only from information sources" is a function for restricting answers to be generated only from specific information sources.

[1589] "Means for recognizing the user's emotional state and adjusting the response based on that emotional state" refers to a function that recognizes the user's emotional state using an emotion engine or the like and adjusts the content and expression of the response according to that emotional state.

[1590] As an embodiment of the present invention, we will explain a smart content assistant system. This system provides reliable answers to user questions, clearly citing the source of the answer. It also recognizes the user's emotional state and adjusts the answer based on that emotional state.

[1591] Hardware and software used

[1592] Hardware: Smartphones, smart glasses, head-mounted displays

[1593] Software: Python, sentiment analysis library (e.g., sentiment_analysis), information source acquisition library (e.g., information_sources)

[1594] System configuration

[1595] 1. User Interface: Provides an interface for users to input questions. This can run on a smartphone, smart glasses, or a head-mounted display.

[1596] 2. Source Acquisition Module: Generates answers from reliable sources to users' questions. This module retrieves information from specific databases and generates answers.

[1597] 3. Sentiment Analysis Module: Recognizes the emotional state of the user and tailors the answer based on that emotional state. This module uses a sentiment analysis library to analyze the user's emotions.

[1598] 4. Answer generation module: Integrates data from the source acquisition module and sentiment analysis module to generate the final answer.

[1599] Data processing and calculation

[1600] 1. Question input: The user inputs a question, for example, "What's the weather like today?"

[1601] 2. Answer generation from information sources: The information source acquisition module retrieves the answer to the question from a specific database, for example, the answer "It's sunny today" from a weather database.

[1602] 3. Sentiment Analysis: The sentiment analysis module analyzes the user's emotional state. For example, if the user types "happy," the emotional state will be recognized as "positive."

[1603] 4. Answer Adjustment: Based on the results of the sentiment analysis, the answer generation module adjusts the answer, for example, "It's sunny today. It looks like it's going to be a great day!"

[1604] 5. Display results: Show the user the tailored answer and the referenced sources.

[1605] Specific examples

[1606] Question: "What's the weather like today?"

[1607] Emotional state: "happy"

[1608] Prompt Sentence Examples

[1609] Enter your question: What's the weather like today?

[1610] Please enter your current emotional state (e.g. happy, sad): happy

[1611] Example output

[1612] Answer: It's sunny today. It's going to be a great day!

[1613] Referenced sources: [Weather Database 1, Weather Database 2]

[1614] In this way, the smart content assistant system can provide reliable answers to user questions, clearly cite the sources behind the answers, and improve the user experience by adjusting the answers based on the user's emotional state.

[1615] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1616] Step 1:

[1617] The user enters a question.

[1618] The user uses a device (smartphone, smart glasses, head-mounted display) to input a question, for example, "What's the weather like today?" The input question is sent to the system.

[1619] Input: User question (e.g. "What's the weather like today?")

[1620] Output: Question data sent to the system

[1621] Step 2:

[1622] The server generates the answer from the information source.

[1623] The server uses a source acquisition module to retrieve the answer to the question from a particular database, for example, retrieving the answer "It's sunny today" from a weather database.

[1624] Input: User question data

[1625] Output: Answer data obtained from the source (e.g., "It's sunny today")

[1626] Step 3:

[1627] The server recognizes the user's emotional state.

[1628] The user uses the terminal to input their current emotional state, for example, "happy." The server uses the emotion analysis module to analyze the user's emotional state.

[1629] Input: User's emotional state (e.g. "happy")

[1630] Output: Sentiment analysis result (e.g. "positive")

[1631] Step 4:

[1632] The server adjusts the answer.

[1633] Based on the results of the sentiment analysis, the server uses an answer generation module to tailor the answer, for example, "It's sunny today. It's going to be a great day!"

[1634] Input: Response data obtained from sources, sentiment analysis results

[1635] Output: Adjusted response data (e.g., "It's sunny today. It's going to be a great day!")

[1636] Step 5:

[1637] Specifies the resource from which the server arrived at the answer.

[1638] The server will then present the information sources it referenced to generate the answer to the user, for example, "Weather Database 1, Weather Database 2."

[1639] Input: Response data obtained from sources

[1640] Output: Resource information leading to the answer (e.g., "Weather Database 1, Weather Database 2")

[1641] Step 6:

[1642] The server displays the results to the user.

[1643] The server displays the adjusted answer and the referenced sources to the user, who can then check the final answer and its rationale through their device.

[1644] Input: Adjusted response data, resource information leading to the response

[1645] Output: The final answer and source shown to the user (e.g., "It's sunny today. It's going to be a great day! Sources consulted: Weather Database 1, Weather Database 2")

[1646] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1647] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1648] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1649] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1650] [Third embodiment]

[1651] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1652] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1653] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1654] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1655] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1656] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1657] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1658] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1659] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1660] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1661] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1662] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1663] "Example 1"

[1664] An embodiment of the present invention is realized as a chatbot system. This system includes an algorithm for generating answers to questions from users. This algorithm has the function of specifying what is "incorrect" probabilistically. Specifically, it generates possible answers to questions from users and probabilistically evaluates whether each answer is correct. This evaluation is performed by calculating the probability that each answer will be judged to be correct. "Embodiment Example 2"

[1665] Furthermore, this embodiment of the present invention has a function that indicates that "humans cannot determine the validity of the answers." Specifically, for each generated answer, the probability that the answer will be judged correct is presented to the user. This allows the user to judge the validity of the system's answers for themselves.

[1666] "Example 3"

[1667] Furthermore, this embodiment of the present invention has a function of clearly indicating the resources that led to the answer. Specifically, the system presents to the user the information sources (e.g., a specific paper database) that were referenced to generate the answer. This allows the user to know which information sources the system's answer is based on. The system can also impose restrictions such as limiting the answer to a specific paper database. This can improve the reliability of the system's answer.

[1668] The processing flow of each embodiment will be described below.

[1669] "Example 1"

[1670] Step 1: Receive a user question.

[1671] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[1672] Step 3: Probabilistically evaluate whether each generated answer is correct. This is done by calculating the probability that each answer is judged to be correct.

[1673] Step 4: Present the probabilistic evaluation results to the user, allowing the user to judge for themselves the validity of the system's answer.

[1674] "Example 2"

[1675] Step 1: Receive a user question.

[1676] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[1677] Step 3: For each generated answer, the user is presented with the probability that the answer is judged to be correct, allowing the user to judge for themselves the validity of the system's answers.

[1678] "Example 3"

[1679] Step 1: Receive a user question.

[1680] Step 2: Generate possible answers to the question. In this step, a specific algorithm is used to generate possible answers to the question.

[1681] Step 3: Present the user with the sources (e.g., a specific paper database) that were consulted to generate the answer, allowing the user to understand which sources the system based its answer on.

[1682] Step 4: Set constraints such as limiting answers to specific paper databases. This can improve the reliability of the system's answers.

[1683] Example 1

[1684] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1685] With conventional chatbot systems, it is difficult to guarantee the accuracy of answers to user questions, and there is a risk that incorrect information will be provided. Furthermore, since users lack a means to determine the accuracy of answers, there is a risk that they will believe incorrect information. Furthermore, since there is no way to clearly indicate the basis or reliability of the answer, it is difficult for users to evaluate the reliability of the answer. To solve these issues, a system is needed that can probabilistically evaluate the accuracy of answers and clearly state that answers with low accuracy are "likely incorrect."

[1686] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1687] In this invention, the server includes a means for receiving a question from a user, a means for generating multiple answers using a generative AI model, a means for probabilistically evaluating the accuracy of each answer, a means for indicating that answers with low accuracy are "likely incorrect" based on the evaluation results, and a means for displaying the organized answers to the user. This allows the user to receive information about the accuracy of the provided answers as probabilistically evaluated, reducing the risk of believing incorrect information. Furthermore, clearly indicating the reliability of the answers makes it easier for the user to evaluate the reliability of the answers.

[1688] The "means for receiving a question from a user" is a function that allows the system to receive a question entered by a user.

[1689] "Means for generating multiple answers using a generative AI model" is a function for automatically generating multiple answers to a user's question using a generative AI model.

[1690] The "means for probabilistically evaluating the accuracy of each answer" refers to an algorithm or method for probabilistically evaluating the accuracy of each generated answer.

[1691] "Means for clearly indicating that answers with low accuracy are 'likely incorrect' based on the evaluation results" is a function for displaying "likely incorrect" for answers that are judged to be low in accuracy based on the accuracy evaluation results.

[1692] The "means for displaying organized answers to the user" is a function for displaying answers organized based on the evaluation results to the user.

[1693] A "specific generative algorithm" is a specific algorithm used by a generative AI model to generate an answer to a user's question.

[1694] A "particular evaluation algorithm" is a particular algorithm used to probabilistically evaluate the accuracy of a generated answer.

[1695] This invention relates to a chatbot system that generates answers to user questions and probabilistically evaluates their accuracy. This system operates among three parties: a server, a terminal, and a user.

[1696] The server includes means for receiving questions from users, means for generating multiple answers using a generative AI model, means for probabilistically evaluating the accuracy of each answer, means for indicating that answers with low accuracy are "likely incorrect" based on the evaluation results, and means for displaying the organized answers to the user.

[1697] Specifically, when the server receives a question from a user, it uses a generative AI model (e.g., GPT-4) to generate multiple possible answers. For each generated answer, the server probabilistically evaluates its accuracy. This evaluation is performed using a specific evaluation algorithm. Based on the evaluation results, answers with low accuracy are marked as "likely incorrect." Finally, the organized answers are displayed to the user.

[1698] For example, if a user asks "What is the diameter of the Earth?" the server might process the question as follows:

[1699] 1. A user types into the chatbot interface, "What is the diameter of the Earth?"

[1700] 2. The device sends this question to the server.

[1701] 3. The server receives the question and inputs it into the generative AI model (GPT-4).

[1702] 4. The generative AI model generates two answers: "The diameter of the Earth is approximately 12,742 kilometers" and "The diameter of the Earth is approximately 10,000 kilometers."

[1703] 5. The server rates the accuracy of each answer, rating answer 1 at 95% and answer 2 at 20%.

[1704] 6. Based on the evaluation results, the server marks answer 2 as "likely incorrect."

[1705] 7. The server sends the organized response to the device.

[1706] 8. The device displays to the user "The diameter of the Earth is approximately 12,742 kilometers" and "The diameter of the Earth is approximately 10,000 kilometers (likely incorrect)."

[1707] Example prompt sentence:

[1708] User: What is the diameter of the Earth?

[1709] Server: The diameter of the Earth is approximately 12,742 kilometers.

[1710] Server: The diameter of the Earth is approximately 10,000 kilometers. (likely incorrect)

[1711] This system allows users to receive information that probabilistically evaluates the accuracy of the answers provided, reducing the risk of believing incorrect information. In addition, by clearly indicating the reliability of the answers, users can more easily evaluate the reliability of the answers.

[1712] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1713] Step 1:

[1714] The user enters a question.

[1715] The user inputs a question into the chatbot's interface, for example, "What is the diameter of the Earth?" The input question is sent to the terminal.

[1716] Step 2:

[1717] The terminal sends a question to the server.

[1718] The terminal sends the question entered by the user to the server, where the question is converted into an appropriate format: the input is the user's question, and the output is the question sent to the server.

[1719] Step 3:

[1720] The server receives the query.

[1721] The server receives questions sent from the device. The received questions are prepared for the next processing step. The input is the question sent from the device, and the output is the question data to be input into the generative AI model.

[1722] Step 4:

[1723] The server inputs questions into the generative AI model.

[1724] The server inputs the received question into a generative AI model (e.g., GPT-4). At this time, the question is converted into a format that the generative AI model can understand. The input is the question data, and the output is the question input to the generative AI model.

[1725] Step 5:

[1726] A generative AI model generates multiple answers.

[1727] A generative AI model generates multiple answers to an input question. For example, it generates answers such as "The diameter of the Earth is approximately 12,742 kilometers" or "The diameter of the Earth is approximately 10,000 kilometers." The input is the question entered into the generative AI model, and the output is the multiple answers generated.

[1728] Step 6:

[1729] The server probabilistically rates the accuracy of each answer.

[1730] The server applies an algorithm to each generated answer to probabilistically evaluate its accuracy. For example, answer 1 may be evaluated as 95% accurate, answer 2 as 20% accurate, etc. The input is the generated answers, and the output is the accuracy evaluation result for each answer.

[1731] Step 7:

[1732] The server organizes the answers based on the evaluation results.

[1733] The server organizes each answer based on the evaluation results. Answers with low accuracy are marked as "likely incorrect." The input is the accuracy evaluation result of each answer, and the output is the organized answer.

[1734] Step 8:

[1735] The server sends the organized response to the terminal.

[1736] The server sends the organized answer to the terminal. The answer is formatted in a way that is easy for the user to understand. The input is the organized answer, and the output is the answer sent to the terminal.

[1737] Step 9:

[1738] The terminal displays the answer to the user.

[1739] The terminal displays the answer sent from the server to the user. For example, it might say, "The diameter of the Earth is approximately 12,742 kilometers." or "The diameter of the Earth is approximately 10,000 kilometers. (This is likely incorrect.)" The input is the answer sent from the server, and the output is the answer displayed to the user.

[1740] (Application example 1)

[1741] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1742] With conventional chatbot systems, it is difficult to guarantee the accuracy of answers to user questions, and there is a risk of providing incorrect information, especially for security-related questions. Furthermore, if an answer is likely to be inaccurate, the system lacks the functionality to notify the user, which can lead to the user believing the incorrect information. This not only undermines user trust, but also increases security risks.

[1743] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1744] In this invention, the server includes a means for probabilistically specifying "incorrect," a means for generating multiple answers to a user's question, and a means for probabilistically evaluating the accuracy of each answer, thereby making it possible to provide the user with the most reliable answer and to notify the user when an answer may be inaccurate.

[1745] "Means to clearly indicate that an answer is probabilistically incorrect" is a function that clearly indicates, based on a probabilistic evaluation, when there is a possibility that the answer provided to the user may not be accurate.

[1746] The "means for indicating that the validity cannot be determined by a human" is a function that indicates that the validity of the answer provided cannot be easily confirmed by human judgment.

[1747] "Means for clearly indicating the resources that led to the answer" is a function that clearly indicates to the user which information sources or databases the generated answer is based on.

[1748] "Means for restricting answers to only paper databases, etc." is a function that limits the information sources used when generating answers to specific databases or papers.

[1749] The "means for generating multiple answers to a question from a user" is a function that generates multiple possible answers to a question entered by a user.

[1750] The "means for probabilistically evaluating the accuracy of each answer" is a function for evaluating the accuracy of each generated answer using a probabilistic method.

[1751] "Means for selecting the most reliable answer and providing it to the user" is a function for selecting the most reliable answer from the evaluated answers and providing it to the user.

[1752] "Means for notifying the user if the answer may not be accurate" is a function that notifies the user if the answer provided may not be accurate.

[1753] As an embodiment of the present invention, a system for installing a security assistant application on a smartphone will be described, which provides probabilistically evaluated answers to security questions posed by a user.

[1754] System configuration

[1755] This system uses the following hardware and software:

[1756] Hardware

[1757] Smartphone

[1758] software

[1759] Generative AI models (e.g., GPT-3)

[1760] Python Program

[1761] Data processing and calculation

[1762] When the server receives a question from a user, it uses a generative AI model to generate multiple answers. The generated answers are then evaluated using a probabilistic method. Specifically, the accuracy of each answer is probabilistically evaluated, and the most reliable answer is selected based on that evaluation.

[1763] If the selected answer may not be accurate, we will notify the user so that they can determine the reliability of the information provided.

[1764] Specific examples

[1765] If a user asks, "How can I tell if my password has been compromised?" the system responds:

[1766] This answer may not be accurate: To check for password leaks, use our data leak checker.

[1767] Prompt Sentence Examples

[1768] An example of a prompt for the user to enter a question is:

[1769] User: How can I check if my password has been compromised?

[1770] Assistant: Use our data leak checker to check for password leaks.

[1771] In this way, users can get reliable answers to their security questions and be notified if the answers may not be accurate, allowing them to determine the reliability of the information provided.

[1772] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1773] Step 1:

[1774] The user enters their security questions using their smartphone.

[1775] Input: User question (e.g., "How can I check if my password has been leaked?")

[1776] Output: The user's question is sent to the server.

[1777] Step 2:

[1778] The server inputs the received user question into a generative AI model.

[1779] Input: User question

[1780] Output: Multiple answers generated by the generative AI model (e.g., 5 answers)

[1781] Step 3:

[1782] The server probabilistically evaluates the generated answers.

[1783] Input: Multiple answers generated by a generative AI model

[1784] Output: A probability value indicating the accuracy of each answer (e.g., a confidence score for each answer)

[1785] Step 4:

[1786] The server selects the most reliable answer from the evaluated answers.

[1787] Input: Probability values ​​indicating the accuracy of each answer

[1788] Output: The most reliable answer (e.g., the answer with the highest confidence score)

[1789] Step 5:

[1790] The server will notify the user if the selected answer may not be accurate.

[1791] Input: The most reliable answer and its reliability score

[1792] Output: The answer provided to the user and a notification that it may not be accurate (e.g., "This answer may not be accurate: Use a data leak checker to check if your password has been leaked.")

[1793] Step 6:

[1794] The user checks the answer provided by the server on their smartphone.

[1795] Input: The answer provided by the server and a notification that it may not be accurate

[1796] Output: Answer and notification for user to review

[1797] In this way, users can get reliable answers to their security questions and be notified if the answers may not be accurate, allowing them to determine the reliability of the information provided.

[1798] Example 2

[1799] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1800] Systems using conventional generative AI models have had the problem that it is difficult for users to judge the validity of the generated answers. Furthermore, they lack the functionality to display the probability of an answer to evaluate its validity, which makes it difficult for users to trust the system's answers. Furthermore, there are insufficient means to clearly indicate the resources that led to the answer, making it difficult for users to confirm the basis for the answer.

[1801] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1802] In this invention, the server includes: a means for a user to input a prompt sentence; a means for a terminal to send the prompt sentence to the server; a means for the server to receive the prompt sentence and pass it to a generative AI model; a means for the generative AI model to generate an answer based on the prompt sentence; a means for the server to receive the generated answer and calculate the probability of evaluating its validity; a means for the server to present the answer and the probability of its validity to the user; a means to clearly indicate that the answer is probabilistically "incorrect"; a means to indicate that the answer is "unvalid for a human to determine its validity"; and a means to clearly indicate the resources that led to the answer or to restrict the answer to be based solely on a database. This allows the user to probabilistically evaluate the validity of the generated answer and confirm the basis for the answer.

[1803] A "user" is an entity that utilizes the system to enter prompts and receive generated answers.

[1804] A "terminal" is a device that a user uses to input a prompt and send it to a server. For example, this could be a computer or a smartphone.

[1805] A "server" is a computer system that receives prompt sentences, passes them to a generative AI model, and presents the generated answers to the user.

[1806] A "prompt sentence" is the text of a question or request that a user enters into the system.

[1807] A "generative AI model" is an artificial intelligence algorithm that generates answers based on prompts, such as a natural language processing model.

[1808] An "answer" is text that a generative AI model generates based on a prompt.

[1809] The "probability of evaluating correctness" is a numerical value indicating the probability that the generated answer will be judged to be correct.

[1810] A "means of probabilistically specifying 'incorrect'" is a method for probabilistically indicating the possibility that the generated answer is incorrect.

[1811] The "means of demonstrating that the validity cannot be determined by a human" is a method of providing information that allows a user to judge for themselves the validity of the generated answer.

[1812] "Resources" are the information sources or databases used to arrive at the generated answer.

[1813] The "means for restricting answers to be given only from a database" is a method for generating answers using only information obtained from a specific database.

[1814] This invention relates to a system that allows a user to input a prompt sentence, generates an answer using a generative AI model, and evaluates the validity of the answer. Specific embodiments of this system are described below.

[1815] First, the user inputs a prompt using a terminal. The terminal is a device such as a computer or smartphone, and is responsible for sending the prompt input by the user to the server. Examples of prompts include "What is the diameter of the Earth?" and "How many planets are there in the solar system?"

[1816] Next, the device sends the prompt to the server. The server receives the prompt and passes it to a generative AI model. A natural language processing model, such as OpenAI's GPT-4, is used as the generative AI model.

[1817] The generative AI model generates an answer based on the received prompt, for example, "The diameter of the Earth is approximately 12,742 kilometers." This generated answer is returned to the server.

[1818] The server receives the generated answer and calculates the probability of evaluating its correctness. This probability is calculated based on the internal algorithm of the generative AI model. For example, if the probability that the generated answer will be judged correct is 85%, the server calculates this probability.

[1819] Finally, the server presents the answer and its probability of correctness to the user. For example, it displays the answer in the form of "The diameter of the Earth is approximately 12,742 kilometers. (Probability of correctness: 85%)." This allows the user to probabilistically evaluate the correctness of the generated answer and check the basis of the answer.

[1820] This system includes a means to clearly indicate that a result is probabilistically "incorrect," a means to show that the "correctness cannot be determined by humans," a means to clearly indicate the resources that led to the answer, or a means to restrict the answer to only the database. This allows users to trust the system's answers and use them appropriately.

[1821] As a concrete example, if a user inputs the prompt "What is the diameter of the Earth?", the server will use the generative AI model to generate the answer "The diameter of the Earth is approximately 12,742 kilometers," evaluate the probability of its correctness as 85%, and present it to the user. In this way, the user can use the system while checking the correctness of the answer.

[1822] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1823] Step 1:

[1824] The user enters a prompt statement.

[1825] The user types a question or request into an input field on the terminal, for example, "What is the diameter of the Earth?" The entered prompt is temporarily stored in the terminal's memory.

[1826] Step 2:

[1827] The terminal sends the prompt to the server.

[1828] The device sends the input prompt text to the server as an HTTP POST request. At this time, the device sends data using an Internet connection. The input data is the prompt text, and the output is a notification to the server that transmission has been completed.

[1829] Step 3:

[1830] The server receives the prompt sentence and passes it to the generative AI model.

[1831] The server receives the prompt sentence sent from the terminal. After receiving it, the server passes this prompt sentence to the generative AI model in an appropriate format (for example, JSON format). The input data is the prompt sentence, and the output is the input data to the generative AI model.

[1832] Step 4:

[1833] A generative AI model generates an answer based on the prompt.

[1834] A generative AI model generates an answer based on the prompt it receives, for example, "The diameter of the Earth is approximately 12,742 kilometers." The input data is the prompt, and the output is the generated answer.

[1835] Step 5:

[1836] The server receives the generated answer and calculates the probability of its correctness.

[1837] The server receives the answer generated by the generative AI model. Then, the server calculates the probability to evaluate the correctness of this answer. This probability is calculated based on the internal algorithm of the generative AI model. The input data is the generated answer, and the output is the probability of correctness.

[1838] Step 6:

[1839] The server presents the answer and its probability of correctness to the user.

[1840] The server presents the generated answer and its probability of correctness to the user. For example, it displays "The diameter of the Earth is approximately 12,742 kilometers. (Probability of correctness: 85%)" on a web page. The input data is the generated answer and its probability of correctness, and the output is the display to the user.

[1841] (Application example 2)

[1842] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1843] Conventional generative AI models do not provide users with information to determine the legitimacy of the generated answers, making it difficult for users to evaluate the reliability of the answers. Furthermore, while they are required to provide appropriate answers to security-related questions, the problem is that users cannot determine the legitimacy of the answers. This creates a risk that users will act based on incorrect information.

[1844] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1845] In this invention, the server includes a means for clearly indicating that an answer is probabilistically "incorrect," a means for indicating that "the validity of the answer cannot be determined by humans," a means for specifying the resources used to arrive at the answer or a means for restricting the answer to only a paper database, etc., a means for presenting to the user the probability that each generated answer is judged to be correct, and a means for the generative AI model to generate an answer to a question entered by the user. This allows the user to confirm the validity of the generated answer as a probability, and to evaluate the reliability of the answer.

[1846] "Means for specifying probabilistically incorrect" is a means for probabilistically indicating the possibility that the generated answer is incorrect.

[1847] The "means for demonstrating that the validity cannot be determined by a human" is a means for demonstrating that the validity of the generated answer cannot be easily determined by a human.

[1848] "Means for clearly indicating the resources that led to the answer" refers to means for clearly indicating what information sources or databases the generated answer is based on.

[1849] The "means for restricting answers to be given only in the paper database, etc." is a means for restricting the generated answers to be given only in a specific database, for example, a paper database.

[1850] "Means for presenting to the user, for each generated answer, the probability that the answer will be judged to be correct" refers to means for calculating the probability that the generated answer will be judged to be correct and presenting that probability to the user.

[1851] "Means by which a generative AI model generates an answer to a question entered by a user" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[1852] In the system for implementing this invention, a generative AI model generates an answer to a question entered by a user and presents the validity of the answer as a probability. A specific embodiment of this system is described below.

[1853] System Configuration

[1854] The system is configured using the following hardware and software.

[1855] Hardware: Smartphone

[1856] Software: Python, OpenAI API

[1857] Data processing and calculation

[1858] 1. User enters question:

[1859] Users enter security questions through a smartphone application, such as "Is this password secure?"

[1860] 2. Generate prompt sentence:

[1861] The smartphone application converts the user's question into a prompt, which looks like this:

[1862] User Question: Is this password secure?

[1863] answer:

[1864] 3. Invoke the generative AI model:

[1865] The smartphone application sends the generated prompt sentences to OpenAI's generative AI model (e.g., GPT-3), which generates an appropriate answer.

[1866] 4. Probability calculation of the correctness of the answer:

[1867] The correctness of the generated answer is calculated as a probability. Using a hypothetical logic, it is determined to be correct with, for example, an 85% probability.

[1868] 5. User Presentation:

[1869] The smartphone application presents the generated answer and its probability of correctness to the user, for example, "Answer: Yes, it's safe. Probability of correctness: 85%."

[1870] Specific examples

[1871] If a user types the question "Is this password secure?", the system will:

[1872] 1. Convert the user's question into a prompt.

[1873] User Question: Is this password secure?

[1874] answer:

[1875] 2. Invoke a generative AI model to generate an answer.

[1876] 3. Calculate the correctness of your answer as a probability.

[1877] 4. Present the user with "Answer: Yes, it's safe. Probability of validity: 85%."

[1878] In this way, the user can check the validity of the generated answer as a probability, and can evaluate the reliability of the answer.

[1879] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1880] Step 1:

[1881] The user launches the application on their smartphone and enters security questions.

[1882] Input: User question (e.g. "Is this password secure?")

[1883] Output: The text of the user's question

[1884] Specific behavior: The user enters a question into the application's input field and presses the submit button.

[1885] Step 2:

[1886] The terminal receives the user's question and generates a prompt sentence.

[1887] Input: User question text

[1888] Output: Prompt text (e.g. "User Question: Is this password secure?\nAnswer:")

[1889] What it does: The application formats the user's question and creates a prompt suitable for the generative AI model.

[1890] Step 3:

[1891] The device sends the generated prompt to OpenAI's generative AI model to generate an answer.

[1892] Input: prompt statement

[1893] Output: The answer from the generative AI model (e.g., "Yes, it's safe")

[1894] What it does: The application calls the OpenAI API to send prompts and get answers.

[1895] Step 4:

[1896] The terminal calculates the correctness of the generated answer as a probability.

[1897] Input: Answer from a generative AI model

[1898] Output: Probability of the answer being correct (e.g. 85%)

[1899] What it does: The application uses hypothetical logic to calculate the probability of the answer being correct.

[1900] Step 5:

[1901] The terminal presents the generated answer and its probability of correctness to the user.

[1902] Input: Answer from generative AI model, probability of answer correctness

[1903] Output: Information presented to the user (e.g., "Answer: Yes, it's safe. Probability of validity: 85%")

[1904] Specific behavior: The application displays the generated answer and its probability of correctness on the screen and presents it to the user.

[1905] Example 3

[1906] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1907] In answer generation systems using conventional generative AI models, it is difficult for users to judge the legitimacy and reliability of the generated answers. In addition, because the information sources that led to the answer are not made clear, users cannot confirm the basis for the answer. Furthermore, because it is not possible to restrict the use of only specific information sources, the reliability of the answer may be reduced. There is a need to solve these problems.

[1908] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1909] In this invention, the server includes means for probabilistically indicating that the answer is "incorrect," means for indicating that the "legitimateness cannot be determined by humans," means for indicating the information sources that led to the answer, means for restricting the answer to only specific information sources, means for receiving and analyzing the user's question, means for generating an answer using a generative AI model, and means for presenting the generated answer and its information source to the user. This allows the user to confirm the legitimacy and reliability of the generated answer and clarify the basis for the answer. Furthermore, restricting the use of only specific information sources can improve the reliability of the answer.

[1910] The "means for clearly indicating that the answer is probabilistically incorrect" is a means for notifying the user that the generated answer may be probabilistically incorrect.

[1911] The "means for indicating that the validity cannot be determined by a human" is a means for indicating to the user that the validity of the generated answer cannot be easily determined by a human.

[1912] The "means for clearly indicating the information source that led to the answer" is a means for clearly indicating to the user which information source the generated answer is based on.

[1913] The "means for restricting the system to generate an answer using only a specific information source" is a means for restricting the system to generate an answer using only a specific information source.

[1914] The "means for receiving and analyzing a user's question" is a means for receiving a question entered by a user and analyzing that question.

[1915] "Means for generating an answer using a generative AI model" refers to means for generating an answer to a user's question using a generative AI model.

[1916] The "means for presenting the generated answer and its information source to the user" is a means for presenting the generated answer and the information source that led to the answer to the user.

[1917] This invention is a system that uses a generative AI model to generate answers to user questions and clearly indicates the information sources that led to the answers. Specific embodiments of this system are described below.

[1918] Hardware and software used

[1919] 1. Hardware

[1920] Server: Use a server with a powerful processor and sufficient memory. For example, use a server with a server-grade processor.

[1921] Terminal: A device that allows users to input questions. For example, a device that can connect to the Internet, such as a PC or smartphone.

[1922] 2. Software

[1923] Natural language processing libraries: used to parse the question. For example, use libraries such as SpaCy or NLTK.

[1924] Generative AI models: used to generate answers, for example, OpenAI's GPT-3.

[1925] Database API: Used to reference information sources, for example, using the API of a scientific literature database or a medical literature database.

[1926] System Operation

[1927] 1. User inputs a question

[1928] The user uses the device to input a question. For example, they open a browser and type "Please tell me about papers on the latest AI technology" into the system interface.

[1929] 2. Receiving and analyzing questions

[1930] The server receives the user's question and analyzes it using natural language processing (NLP) technology. For example, the server receives the question and analyzes the intent of the question using the SpaCy library. Specifically, it extracts the keywords "latest AI technology" and "paper."

[1931] 3. References to sources

[1932] The server refers to a specified information source (e.g., a specific paper database). For example, the server calls the API of a scientific and technical literature database to search for papers on "the latest AI technology."

[1933] 4. Answer generation using generative AI models

[1934] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate an answer based on the referenced information source. For example, the server inputs a prompt to GPT-3, such as "Please tell me papers about the latest AI technology," and obtains the generated answer. Specifically, it generates the answer, "Paper A was published in 2023."

[1935] 5. Source citation

[1936] The server presents the information source that led to the answer to the user. For example, the server may add information such as "This answer is based on a scientific and technical literature database."

[1937] 6. Presenting the Answer

[1938] The server sends the generated answer and its source to the user's device and displays it to the user. For example, the user's device might display a message such as, "Paper A, published in 2023, is a paper on the latest AI technology. This answer is based on a scientific and technical literature database."

[1939] Specific examples

[1940] User question input

[1941] Prompt: "Please tell me some papers about the latest AI technology."

[1942] Answer generation using generative AI models

[1943] Answer: "Paper A, published in 2023, is a paper on the latest AI technology."

[1944] Clarification of sources

[1945] Source: "This answer is based on scientific and technical literature databases."

[1946] In this way, the user can know which information source the system's answer is based on and can confirm the reliability of the answer. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1947] Step 1:

[1948] The user uses the terminal to enter a question.

[1949] Input: The user types a question into the terminal interface.

[1950] Specific operation: The user opens a browser and types "Please tell me papers about the latest AI technology" into the system interface.

[1951] Output: The user's question is sent from the terminal to the server.

[1952] Step 2:

[1953] The server receives the user's question and analyzes it using natural language processing (NLP) techniques.

[1954] Input: User question (e.g., "Please tell me papers about the latest AI technology.")

[1955] Specific operation: The server receives the question and analyzes the intent of the question using the SpaCy library. Specifically, it extracts the keywords "latest AI technology" and "paper."

[1956] Output: Extracted keywords and analysis results are obtained.

[1957] Step 3:

[1958] The server consults the specified information source (e.g., a particular research database).

[1959] Input: Extracted keywords and analysis results (e.g., "latest AI technology," "paper")

[1960] Specific operation: The server calls the API of a scientific and technical literature database and searches for papers on "latest AI technology."

[1961] Output: The search results will give you a list of relevant papers.

[1962] Step 4:

[1963] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate an answer based on the referenced sources.

[1964] Input: A list of papers obtained as search results and a prompt (e.g., "Please tell me papers about the latest AI technologies")

[1965] Specific operation: The server inputs the prompt and search results to GPT-3 and obtains the generated answer. Specifically, it generates the answer "Paper A was published in 2023."

[1966] Output: You get the generated answer.

[1967] Step 5:

[1968] The server presents the user with the information sources that led to the answer.

[1969] Input: The generated answer and its source (e.g., "Paper A was published in 2023. This answer is based on a scientific and technical literature database.")

[1970] Specific behavior: The server appends the source of the generated answer.

[1971] Output: Answers with source citations.

[1972] Step 6:

[1973] The server generates an answer and sends the answer and its source to the user's terminal and displays it to the user.

[1974] Input: Answer with source information (e.g., "Paper A was published in 2023. This answer is based on a scientific and technical literature database.")

[1975] Specific operation: The server sends a response to the user's device, specifying the source of the information.

[1976] Output: The answer is displayed on the user's terminal.

[1977] In this way, the user can see which sources the system's answers are based on and can verify the reliability of the answers.

[1978] (Application example 3)

[1979] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1980] In conventional information provision systems, it was difficult for users to judge the reliability of the information provided, and doubts often arose about the accuracy and reliability of the information, especially when the source of the information, such as a news article, was not clearly stated. In addition, there was a lack of means to probabilistically determine the legitimacy of the information, which created the risk that users would make decisions based on incorrect information. This could undermine user trust.

[1981] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1982] In this invention, the server includes a means for probabilistically indicating that a result is "incorrect," a means for indicating that the "validity cannot be determined by humans," a means for indicating the resources used to arrive at the answer or a means for restricting the answer to only using a paper database, etc., and a means for indicating the source of each news article to increase its reliability. This allows users to easily determine the reliability of the information provided and avoid making decisions based on incorrect information. Furthermore, indicating the source of the information increases user trust.

[1983] "Means of clearly stating that information is probabilistically incorrect" refers to a means of using a specific algorithm to determine the accuracy of information probabilistically, and as a result, clearly stating that there is a possibility that the information is incorrect.

[1984] "Means for demonstrating that the legitimacy cannot be determined by humans" refers to means for demonstrating that the legitimacy of the information provided cannot be easily determined by human judgment.

[1985] "Means for indicating the resources that led to the answer" means a means for showing the user which information sources the system-generated answer is based on.

[1986] "Means for imposing a constraint that answers be given only from a research paper database, etc." means a means for imposing a constraint that the system generate answers by referring only to a specific database (for example, a research paper database).

[1987] "Means of clearly indicating which sources each article is based on in order to increase the reliability of news articles" refers to means of clearly indicating to users which sources each article is based on in order to increase the reliability of news articles.

[1988] As an embodiment of the present invention, a news distribution application will be described as an example. The server includes means for probabilistically indicating that a result is "incorrect," means for indicating that "the validity of the result cannot be determined by humans," means for indicating the resources that led to the answer or means for restricting the answer to only a database of papers, etc., and means for indicating which information source each news article is based on in order to increase the reliability of the news articles.

[1989] Explanation of program processing

[1990] The server uses Python requests and the BeautifulSoup library to retrieve news articles from the website. Specifically, the NewsApp class retrieves news articles, and the NewsArticle class holds information about each article. The fetch_news method retrieves news articles from the specified URL, and the display_articles method displays the retrieved articles.

[1991] Hardware and software used

[1992] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1993] Software: Python, requests library, BeautifulSoup library

[1994] Data processing and calculation

[1995] 1. Data retrieval: The server retrieves the HTML data of the news article from the specified URL.

[1996] 2. Data Analysis: Parse the HTML data using the BeautifulSoup library to extract article titles, content, and sources.

[1997] 3. Save data: Save the extracted data as an instance of the NewsArticle class.

[1998] 4. Data display: Use the display_articles method to display the article title, content, and source on the user's device.

[1999] Specific examples

[2000] For example, if a news distribution app retrieves a news article from the URL "https: / / example.com / news1", it inputs the following prompt sentence into the generative AI model.

[2001] Prompt Sentence Examples

[2002] A news distribution app should retrieve news articles from the URL "https: / / example.com / news1" and display the article title, content, and source.

[2003] Using this prompt, the generative AI model can retrieve news articles from the specified URL and provide reliable information to the user.

[2004] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[2005] Step 1:

[2006] The server receives the URL of the news article specified by the user.

[2007] Input: User-specified news article URL

[2008] Output: The URL of the news article to be retrieved.

[2009] What it does: When a user enters the URL of a news article into an application input field and presses the submit button, the server receives the URL.

[2010] Step 2:

[2011] The server retrieves HTML data from the specified URL using the requests library.

[2012] Input: News article URL

[2013] Output: Retrieved HTML data

[2014] Specific operation: The server executes requests.get(URL) to retrieve HTML data from the specified URL.

[2015] Step 3:

[2016] The server uses the BeautifulSoup library to parse the retrieved HTML data and extract the title, content, and source of the news article.

[2017] Input: Retrieved HTML data

[2018] Output: Title, content, and source of extracted news articles

[2019] Specific operation: The server parses the HTML data using BeautifulSoup, extracts the title with soup.find('h1').text, the content with soup.find('div', class_='article-content').text, and the URL itself as the source of information.

[2020] Step 4:

[2021] The server stores the extracted data as an instance of the NewsArticle class.

[2022] Input: Title, content, and source of extracted news articles

[2023] Output: An instance of the NewsArticle class

[2024] Specific operation: The server calls the constructor of the NewsArticle class and saves the extracted data in the instance.

[2025] Step 5:

[2026] The server transmits the stored instance of the news article to the user's terminal for display.

[2027] Input: An instance of the NewsArticle class

[2028] Output: The title, content, and source of the news article displayed on the user's device

[2029] Specific operation: The server calls the display_articles method to send the title, content, and source of the news article to the user terminal and display it.

[2030] Step 6:

[2031] The user checks the title, content, and source of the displayed news article.

[2032] Input: Title, content, and source of news article displayed on user device

[2033] Output: User confirmation result

[2034] What it does: The user checks the title, content, and source of a news article displayed on the device to determine the reliability of the information.

[2035] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2036] "Example 1"

[2037] The present invention is a system that includes an emotion engine that recognizes a user's emotions. Specifically, it receives a question from a user and generates possible answers to that question. In the answer generation process, a specific algorithm is used to generate possible answers to the question. The correctness of each generated answer is probabilistically evaluated, and the evaluation results are presented to the user. Furthermore, the information source (e.g., a specific paper database) referenced to generate the answer is presented to the user. This allows the user to know which information source the system's answer is based on. It is also possible to set restrictions, such as limiting answers to specific paper databases.

[2038] "Example 2"

[2039] The emotion engine recognizes the user's emotional state and adjusts responses based on that emotional state. For example, if the user feels angry, the system will provide them with more polite responses. If the user feels happy, the system will provide them with more cheerful responses. In this way, the emotion engine provides a user experience that takes the user's emotional state into account.

[2040] "Example 3"

[2041] As a concrete example, suppose a user asks, "What's the weather like today?" The system generates potential answers to this question and probabilistically evaluates the accuracy of each answer. It then presents the results of this evaluation to the user along with the information source (e.g., a specific weather database) that was used to generate the answer. Additionally, the emotion engine recognizes the user's emotional state and adjusts the answer based on that emotional state. For example, if the user feels happy, the system might provide a more cheerful answer such as, "It's sunny today. It's going to be a great day!"

[2042] The processing flow of each embodiment will be described below.

[2043] "Example 1"

[2044] Step 1: The system receives a question from the user.

[2045] Step 2: The system uses a specific algorithm to generate possible answers to the question.

[2046] Step 3: Probabilistically evaluate whether each generated answer is correct.

[2047] Step 4: Present the user with the results of the assessment and the sources consulted to generate the answer (e.g., a specific research database).

[2048] Step 5: It is also possible to set restrictions such as limiting responses to specific paper databases. "Example 2"

[2049] Step 1: The emotion engine recognizes the user's emotional state.

[2050] Step 2: Adjust the response based on the user's emotional state. For example, if the user feels angry, the system will provide a more polite response to the user.

[2051] Step 3: If the system senses that the user is happy, it will provide a more enjoyable response to that user.

[2052] "Example 3"

[2053] Step 1: Suppose a user asks, "What's the weather like today?"

[2054] Step 2: The system generates possible answers to this question and probabilistically evaluates the correctness of each answer.

[2055] Step 3: Present the user with the evaluation results and the information sources (e.g., a specific weather database) that were consulted to generate the answer.

[2056] Step 4: The emotion engine recognizes the user's emotional state and adjusts the answer based on that emotional state. For example, if the user feels happy, the system will provide a more cheerful answer such as "It's sunny today. It looks like it's going to be a great day!"

[2057] Example 1

[2058] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2059] Conventional chatbot systems lacked a means to evaluate the accuracy and reliability of answers to user questions, making it difficult for users to determine the reliability of the information provided. Furthermore, they lacked a function to clearly indicate the source of the answer, meaning users could not confirm the basis for the answer. Furthermore, they lacked a function to probabilistically evaluate the accuracy of the answer, making it difficult for users to understand the reliability of the answer.

[2060] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2061] In this invention, the server includes means for receiving a question from a user, means for inputting the received question as a prompt to a generative AI model, means for generating multiple answers from the generative AI model, means for probabilistically evaluating the accuracy of each generated answer, means for presenting an answer including the evaluation result and a reference information source to the user, means for probabilistically indicating that the answer is "incorrect," means for indicating that the "validity cannot be determined by a human," and means for indicating the resources that led to the answer or for restricting the answer to only using a database, etc. This allows the user to evaluate the reliability of the information provided and confirm the basis for the answer.

[2062] A "user" is an entity that inputs questions to the system and receives answers.

[2063] A "server" is a device or system that receives questions from users, generates answers using a generative AI model, and presents them to the user along with evaluation results.

[2064] A "generative AI model" is an artificial intelligence model used to generate multiple answers to a user question.

[2065] A "prompt" is a question or instruction entered into a generative AI model; it is the input data that the model uses to generate an answer.

[2066] "Probabilistic evaluation" means evaluating the accuracy of each generated answer based on probability theory.

[2067] The "evaluation result" is the result of a probabilistic evaluation of the accuracy of the generated answer.

[2068] A "reference source" is a source such as a database or literature that provides information on which the generated answer is based.

[2069] "Specifying that the answer is probabilistically incorrect" means specifying, based on probability theory, that the generated answer may be incorrect.

[2070] "Demonstrating that the validity cannot be determined by a human" means demonstrating that the validity of the generated answer cannot be easily determined by a human.

[2071] "Resources" are the information sources or databases consulted to generate the answer.

[2072] A "database" is a collection of information that is systematically organized and stored so that it can be searched and used as needed.

[2073] This invention relates to a chatbot system that generates answers to user questions. This system receives questions from users, generates answers using a generative AI model, probabilistically evaluates the accuracy of the answers, and presents them to the user.

[2074] The server provides an interface for receiving questions from users. When a user inputs a question, the device sends the question to the server. The server then inputs the received question as a prompt into a generative AI model (e.g., GPT-4). The generative AI model generates multiple answers to the question.

[2075] The server probabilistically evaluates the accuracy of each generated answer. This evaluation is performed using a specific algorithm. For example, the answer "The latest treatment is monoclonal antibody therapy" is evaluated as being correct 80% of the time. Along with this evaluation result, the server presents the user with a reference source (e.g., a specific paper database). This allows the user to know which source the system's answer is based on.

[2076] As a concrete example, consider the case where a user asks, "What is the latest treatment for COVID-19?" The device sends this question to the server. The server uses a generative AI model (GPT-4) to generate possible answers. For example, answers such ...

Claims

1. means for analyzing the input question; A means for generating answers from only specific sources using a generative AI model; A means for calculating a probability of the correctness of the answer using the generative AI model; means for presenting the generated answer, the calculated probability of the answer's correctness, and the source of the answer; A system including:

2. a means for recognizing an emotional state of a user; and means for adjusting the response based on the recognized emotional state of the user. The system of claim 1 .

3. The answer generating means generates a plurality of answers; The presenting means displays the organized plurality of answers. The system of claim 1 .

Images