System

Abstract

To provide a system.SOLUTION: A system includes: means for collecting log data in which a dialogue performed by a user with an artificial intelligence assistant is recorded; means for deleting unnecessary information from the collected log data, extracting a pair of a question and an answer, and normalizing data; and means for learning a generative artificial intelligence model having ability of generating an answer to a question of the user using the preprocessed log data. The generative artificial intelligence model analyzes the input question to identify a keyword and is trained so as to generate relevant structured data on the basis of the identified keyword.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Because generative AI is specialized in generating strings of characters, it has the problem of not being able to perform operations such as making restaurant reservations, providing up-to-date information such as today's weather in Tokyo, or setting alarms. [Means for solving the problem]

[0005] As a means of solving this problem, we propose a system in which the generating AI is trained from the logs of existing AI assistants to return structured data that is easy to identify, and then programmed to take action based on the structured data, and then combines the results with the question to regenerate a response. [Brief explanation of the drawings]

[0006] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 includes an optical system including a lens, an aperture, and a shutter, and a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or CC

[0021] It is a small digital camera equipped with an imaging element such as a D (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] In one embodiment of the present invention, the generation AI learns from the logs of existing AI assistants and returns structured data that is easy to identify. For example, if a user asks, "What's the weather like tomorrow?", the generation AI will identify the keyword "weather" and generate relevant structured data.

[0030] "Example 2"

[0031] Next, we will explain how the AI ​​receives structured data, programs it uses, and performs actions. For example, if the AI ​​recognizes the keyword "weather," the corresponding program will run and retrieve the latest weather information.

[0032] "Example 3"

[0033] Finally, we will explain how the results are combined with the question to regenerate a response. For example, after obtaining the latest weather information, a response such as "Tomorrow's weather will be sunny" is generated based on that information. This makes it possible to provide an appropriate response to the user.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: The generative AI begins learning from the logs of existing AI assistants. Through this learning, the generative AI acquires the ability to generate appropriate responses to user questions.

[0037] Step 2: When a user asks a question, the Generative AI analyzes the question and generates structured data that is easy to identify. For example, in response to the question "What's the weather like tomorrow?", it identifies the keyword "weather" and generates related structured data.

[0038] "Example 2"

[0039] Step 1: The generative AI receives the identified structured data and runs a corresponding program. This program performs a specific action based on the structured data. Step 2: For example, if the keyword "weather" is identified, the program performs an action to obtain the latest weather information. This action allows the user to obtain the information they are looking for.

[0040] "Example 3"

[0041] Step 1: Combine the acquired information with the original question to generate a new response. In this step, the generative AI uses the knowledge it has learned to generate an appropriate response.

[0042] Step 2: For example, after obtaining the latest weather information, generate a response such as "Tomorrow's weather will be sunny." This makes it possible to provide an appropriate response to the user.

[0043] Example 1

[0044] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] Conventional AI assistants often lack the training data to provide appropriate responses to user questions, resulting in low accuracy and relevance of responses. Furthermore, responses to user questions are not structured, making it difficult to obtain and use information. Furthermore, there is also the issue of difficulty in providing up-to-date information appropriately.

[0046] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0047] In this invention, the server includes means for collecting and preprocessing log data of existing AI assistants, means for training a generative AI model using the preprocessed log data, means for receiving questions from users and analyzing the questions to identify keywords, means for generating relevant structured data based on the identified keywords, and means for returning the generated structured data to the user. This makes it possible to provide highly accurate and relevant responses to user questions. Furthermore, returning structured data makes it easier to obtain and use information, and allows for appropriate provision of up-to-date information.

[0048] "Log data of existing AI assistants" refers to data that records questions asked by users to AI assistants and the AI ​​assistant's responses to those questions.

[0049] "Preprocessing" is the process of removing unnecessary information from the collected log data, extracting question-answer pairs, and normalizing the data.

[0050] A "generative artificial intelligence model" is an artificial intelligence model that has the ability to learn from provided data and generate appropriate responses to user questions.

[0051] A "user question" is an inquiry or request that a user inputs to an artificial intelligence system.

[0052] "Keyword identification" means analyzing a user's question and extracting important words and phrases from it.

[0053] "Structured data" is data that is organized according to a specific format or structure, making the information easy to retrieve and use.

[0054] "Updateable information" refers to information whose content changes over time or with changes in circumstances, and includes, for example, weather information and reservation information.

[0055] "Providing a response" means returning appropriate information or answers to a user's questions.

[0056] This invention is a system that collects log data from existing AI assistants, uses the collected data to train a generative AI model, and returns appropriate structured data in response to user questions. Specific embodiments of this system are described below.

[0057] Hardware and software used

[0058] server

[0059] The server is hardware that collects and preprocesses log data from existing AI assistants. The server uses the following software:

[0060] Data collection script: A script to periodically collect log data from your existing artificial intelligence assistant.

[0061] Data preprocessing scripts: Scripts to remove unnecessary information from the collected log data, extract question-answer pairs, and normalize the data.

[0062] API call: An API to provide pre-processed data to the generative artificial intelligence model.

[0063] Generative AI Models

[0064] A generative AI model is software that uses data provided by a server to learn and generate appropriate responses to user questions. Specifically, it uses the following algorithm:

[0065] Natural language processing algorithms: Algorithms for analyzing user questions and identifying keywords.

[0066] Database query: A query to generate relevant structured data based on the identified keywords.

[0067] User terminal

[0068] The user terminal is the hardware that allows the user to input questions to the generative AI model and receive responses. Specifically, it uses the following interface:

[0069] Input interface: The interface through which the user enters their question.

[0070] Output interface: An interface for displaying the structured data returned by the generative AI model.

[0071] Specific examples

[0072] Prompt Sentence Examples

[0073] Below are examples of prompts that users can use to input questions to the generative AI model.

[0074] "What's the weather like tomorrow?"

[0075] "What events are happening this weekend?"

[0076] "Tell me the latest news"

[0077] For these prompts, the generative AI model identifies appropriate keywords and generates and returns relevant structured data. For example, the prompt "What's the weather like tomorrow?" returns structured data containing weather forecast information.

[0078] Operation flow

[0079] The server collects and preprocesses log data from existing AI assistants. The preprocessed data is provided to the generative AI model, which then uses the data to learn. When a user inputs a question through their device, the generative AI model analyzes the question and identifies keywords. Based on the identified keywords, the generative AI model generates relevant structured data and returns it to the user. The user can then review the returned structured data to obtain the information they need.

[0080] In this way, the present invention can provide accurate and relevant responses to user questions, making information easier to obtain and use.

[0081] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0082] Step 1:

[0083] The server collects log data from existing artificial intelligence assistants.

[0084] Input: Log data from an existing AI assistant

[0085] Specific operation: The server executes the data collection script and periodically collects the log data of the artificial intelligence assistant.

[0086] Output: Collected log data

[0087] Step 2:

[0088] The server pre-processes the collected log data.

[0089] Input: Collected log data

[0090] Specific operations: The server runs data preprocessing scripts to remove unnecessary information, extract question-answer pairs, and normalize the data.

[0091] Output: Preprocessed log data

[0092] Step 3:

[0093] The server provides the preprocessed log data to the generative AI model.

[0094] Input: Preprocessed log data

[0095] Specific operation: The server calls the API and sends the preprocessed data to the generative AI model.

[0096] Output: The data provided to the generative AI model

[0097] Step 4:

[0098] Generative AI models learn using the data they are provided with.

[0099] Input: Data provided to a generative AI model

[0100] How it works: The generative AI model goes through a training process based on the data provided, and learns how to generate appropriate responses to user questions.

[0101] Output: A trained generative AI model

[0102] Step 5:

[0103] The user inputs questions into the generative AI model through the terminal.

[0104] Input: User question (prompt)

[0105] Specific operation: The user inputs a question into the input interface using a smartphone or PC.

[0106] Output: The question entered

[0107] Step 6:

[0108] The generative AI model receives a question from a user and analyzes the question to identify keywords.

[0109] Input: The question you entered

[0110] How it works: The generative AI model uses natural language processing algorithms to analyze the question and identify important keywords.

[0111] Output: Identified keywords

[0112] Step 7:

[0113] The generative AI model generates relevant structured data based on the identified keywords.

[0114] Input: Identified keyword

[0115] How it works: The generative AI model runs database queries to retrieve information related to the identified keywords and generates structured data.

[0116] Output: Generated structured data

[0117] Step 8:

[0118] The generative AI model returns the generated structured data to the user.

[0119] Input: Generated structured data

[0120] Specific operation: The generative AI model returns the generated structured data to the user through the output interface.

[0121] Output: Structured data returned to the user

[0122] Step 9:

[0123] The user checks the returned structured data and obtains the required information.

[0124] Input: Structured data returned to the user

[0125] Specific operation: The user checks the returned structured data through the output interface of the terminal.

[0126] Output: Information obtained by the user

[0127] (Application example 1)

[0128] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] Conventional self-driving vehicles lack the ability to provide real-time traffic information, making it difficult to ensure efficient and safe driving. Furthermore, there was no system that returned appropriate structured data in response to user questions, making it cumbersome to obtain information while driving.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0131] In this invention, the server includes means for training the generating AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for providing traffic information in real time to the autonomous vehicle, means for analyzing the user's question and generating relevant structured data, and means for improving driving efficiency and safety based on the generated structured data, thereby enabling the user to obtain appropriate traffic information in real time while driving and improving driving efficiency and safety.

[0132] "Generative AI" is artificial intelligence that learns from existing data and generates structured data that is easy to identify in response to user questions.

[0133] "Structured data" is data that is organized according to a specific format, making it easy to identify and analyze.

[0134] "Logs of existing AI assistants" are data that record the dialogue history and operation history of AI assistants that have been used in the past.

[0135] "Means for programming and taking action" refers to means for executing specific operations or processes based on the generated structured data.

[0136] The "means for regenerating a response sentence" is a means for creating a new response sentence based on the user's question and the generated structured data.

[0137] An "autonomous vehicle" is a vehicle that drives autonomously using artificial intelligence and sensor technology.

[0138] "Means for providing traffic information in real time" refers to means for instantly providing current traffic conditions, traffic signal information, etc.

[0139] "Means for analyzing user questions" refers to means for understanding questions from users and analyzing their intentions.

[0140] The "means for improving driving efficiency and safety" refers to means for improving driving efficiency and safety based on the generated structured data.

[0141] A system for implementing this invention uses a generative AI model to return appropriate structured data in real time in response to user questions, thereby improving the driving efficiency and safety of autonomous vehicles. Specific embodiments of this system are described below.

[0142] The server first learns from the logs of existing AI assistants. This allows the generation AI to acquire the ability to generate structured data that is easy to identify in response to user questions. For example, if a user asks, "Where's the next traffic light?", the generation AI will identify the keyword "traffic light" and generate structured data such as the distance to the next traffic light and the traffic light status (red, yellow, green).

[0143] This structured data can be programmed by the server and used to perform specific actions, such as sending upcoming traffic lights to the autonomous vehicle's control system to make appropriate speed adjustments.

[0144] Furthermore, the server regenerates a response based on the generated structured data and the user's question. This response is intended to provide the user with appropriate information. For example, a specific response such as "The next traffic light is red. The distance is 100 meters" is generated.

[0145] The system includes a means to provide real-time traffic information for autonomous vehicles, analyzing user questions and generating relevant structured data to improve driving efficiency and safety.

[0146] The hardware used includes an on-board computer of an autonomous vehicle with internet connectivity. The software uses Python and the OpenAI (registered trademark) API. Specific examples of prompts include the following:

[0147] Example prompt sentence:

[0148] User Question: What's the next signal?

[0149] Please return the relevant structured data.

[0150] In this way, real-time traffic information assistants can support autonomous vehicle operation, improving efficiency and safety.

[0151] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0152] Step 1:

[0153] The server learns from the logs of existing AI assistants. Specifically, it analyzes past dialogue history and operation history and provides them as training data to the generative AI model. Through this learning, the generative AI model acquires the ability to generate structured data that is easy to identify in response to user questions. The input is the log data of the AI ​​assistant, and the output is the trained generative AI model.

[0154] Step 2:

[0155] A user inputs a question into an autonomous vehicle, for example, "What's the next traffic light?", either through voice input or a touch panel. The input is the user's question, and the output is the question text.

[0156] Step 3:

[0157] The server inputs the user's question as a prompt to the generative AI model. Specifically, it sends the question text to the generative AI model and instructs it to generate related structured data. The input is the user's question text, and the output is the generated structured data.

[0158] Step 4:

[0159] The server receives the generated structured data and performs programming actions, such as sending information about upcoming traffic lights to the autonomous vehicle's control system to adjust the vehicle's speed appropriately. The input is the generated structured data, and the output is instructions to the control system.

[0160] Step 5:

[0161] The server regenerates a response based on the generated structured data and the user's question. Specifically, the generative AI model creates a response to provide appropriate information to the user. The input is the user's question and the generated structured data, and the output is the regenerated response.

[0162] Step 6:

[0163] The server provides the regenerated response sentence to the user. Specifically, it conveys information to the user through voice output or display. The input is the regenerated response sentence, and the output is information provided to the user.

[0164] In this way, real-time traffic information assistants can support autonomous vehicle operation, improving efficiency and safety.

[0165] Example 2

[0166] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] Conventional AI assistant systems often lacked efficient program generation and execution to provide appropriate responses to user requests. Furthermore, the processing required after receiving structured data was complex, resulting in delayed responses to users. Furthermore, information acquisition from external APIs was not performed smoothly, resulting in insufficient information provided to users.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0169] In this invention, the server includes means for training the generation AI from logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and generating a program, means for executing the generated program to perform a specific action, means for analyzing a prompt using a generation AI model, means for generating a program based on the analysis results, means for executing the generated program to obtain information from an external API, and means for returning the obtained information to the user, thereby enabling a prompt and appropriate response to a user request to be provided.

[0170] "Generative AI" is a system that uses artificial intelligence technology to automatically generate programs and responses for specific tasks.

[0171] "Structured data" is data that is organized according to specific formats and rules, making it easy for generative AI to identify and process.

[0172] An "AI assistant" is a system equipped with artificial intelligence to provide information or perform specific tasks in response to user requests.

[0173] A "log" is data that records the operation of a system and the operation history of a user.

[0174] "Means for generating programs" refers to the ability to automatically create code or scripts to perform specific tasks using a generative AI model.

[0175] A "means for performing an action" is a function that executes a generated program to perform a specific task or operation.

[0176] A "generative AI model" is an algorithm or model that forms the core of generative AI, and is a technology for analyzing prompts and generating appropriate responses or programs.

[0177] A "prompt sentence" is text containing a request or question that a user enters into the system.

[0178] The "means of analysis" is the function that uses a generative AI model to understand the meaning of the prompt and determine the appropriate action.

[0179] An "external API" is an interface for accessing external services and databases to obtain information.

[0180] "Means for obtaining information" is a function that executes the generated program to obtain the necessary data from the external API.

[0181] The "means for regenerating a response sentence" is a function that combines the acquired information with the original question sentence to generate an appropriate response to the user.

[0182] MODE FOR CARRYING OUT THE INVENTION

[0183] This invention is a system that uses a generative AI model to generate an appropriate program based on a prompt sentence entered by a user, and then executes the program to perform a specific action. Specific embodiments of this system are described below.

[0184] System configuration

[0185] This system consists of three main elements: a server, a terminal, and a user. The server is responsible for analyzing prompts using a generative AI model, generating and executing programs. The terminal is responsible for sending prompts from the user to the server and displaying responses from the server to the user. The user inputs requests to the system through the terminal.

[0186] Hardware and software used

[0187] The server is a computer system equipped with a high-performance processor and a large amount of memory, and is equipped with software for running a generative AI model (e.g., GPT-4 (registered trademark)). The terminal is a device operated by a user, and includes a smartphone, tablet, PC, etc. The server and terminal communicate via the Internet.

[0188] Data processing and calculation

[0189] The server inputs the prompt received from the user into a generative AI model for analysis. The generative AI model understands the meaning of the prompt and generates a program to determine the appropriate action. The generated program is written in a programming language such as Python and calls a specific API (for example, the OpenWeatherMap API) to obtain the necessary information. The obtained information is then formatted as a response using the generative AI model again and sent to the device.

[0190] Specific examples

[0191] Let's take the example of a user typing "What's the weather today?" This prompt is sent from the device to the server. The server uses the generative AI model to analyze the prompt and identify the keyword "weather." The server then generates a Python script to obtain weather information and calls the OpenWeatherMap API to obtain the latest weather information. The obtained weather information is then formatted as a response using the generative AI model again and sent to the device. The device then displays the received weather information to the user.

[0192] Example prompt sentence:

[0193] "Tell me about today's weather."

[0194] In this way, the system can provide a fast and appropriate response to a user's request.

[0195] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0196] Step 1:

[0197] The user enters a prompt statement.

[0198] The user types "What's the weather like today?" into the input field of the terminal. This input is sent to the system as a prompt.

[0199] Input: The prompt text entered by the user: "What's the weather like today?"

[0200] Output: Data that the terminal sends to the server as a prompt

[0201] Step 2:

[0202] The terminal sends the prompt to the server.

[0203] The terminal sends the prompt text entered by the user to the server as an HTTP request. This communication takes place over the Internet.

[0204] Input: The prompt text entered by the user

[0205] Output: The prompt received by the server

[0206] Step 3:

[0207] The server analyzes the prompt using a generative AI model.

[0208] The server inputs the received prompt sentence into a generative AI model (e.g., GPT-4) for analysis. The generative AI model understands the meaning of the prompt sentence and determines the appropriate action.

[0209] Input: The prompt received by the server

[0210] Output: Analysis results (including the keyword "weather")

[0211] Step 4:

[0212] The server generates a program based on the analysis results.

[0213] The server generates a program to execute a specific action based on the analysis results of the generative AI model. For example, in response to the prompt "What's the weather today?", it generates a Python script to retrieve weather information.

[0214] Input: Analysis results of the generative AI model

[0215] Output: Generated Python script

[0216] Step 5:

[0217] The server runs the generated program.

[0218] The server runs the generated Python script and calls the OpenWeatherMap API to get the latest weather information.

[0219] Input: Generated Python script

[0220] Output: Weather information obtained from the OpenWeatherMap API

[0221] Step 6:

[0222] The server sends the execution results to the terminal.

[0223] The server then sends the acquired weather information to the terminal as an HTTP response. This communication also takes place over the Internet.

[0224] Input: Weather information obtained from the OpenWeatherMap API

[0225] Output: Weather information received by the device

[0226] Step 7:

[0227] The terminal displays the execution results to the user.

[0228] The terminal displays the weather information received from the server to the user, allowing the user to check the latest weather information.

[0229] Input: Weather information received from the server

[0230] Output: Weather information displayed to the user

[0231] (Application example 2)

[0232] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0233] Conventional AI assistant systems have had difficulty in quickly providing appropriate information in response to user questions. In particular, there is a need to obtain and provide updated information, such as traffic and weather information, in real time to users, but there has been a lack of efficient ways to achieve this.

[0234] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for training the generation AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for identifying specific keywords and acquiring corresponding information, and means for providing the acquired information to the user. This makes it possible to provide prompt and appropriate information in response to a user's question.

[0235] "Generative AI" is artificial intelligence that identifies specific keywords and takes appropriate actions based on them.

[0236] "Structured data" is data that is organized in a format that is easy for generating AI to identify.

[0237] "Existing AI assistant logs" are records of user questions and responses that have been handled by the AI ​​assistant in the past.

[0238] A "means for programming and taking action" is a method for receiving structured data and performing specific actions based thereon.

[0239] The "means for regenerating a response sentence" is a method for creating a new response sentence by combining the acquired information with the user's question.

[0240] "Means for identifying specific keywords" refers to a method for extracting important keywords from user input.

[0241] The "means for obtaining information" is a method for obtaining relevant and up-to-date information based on the identified keywords.

[0242] "Means of providing to users" refers to the method used to convey the acquired information to users.

[0243] The system for implementing this invention uses a generative AI model to identify specific keywords, obtain appropriate information based on the keywords, and provide it to users. A specific embodiment of this system is described below.

[0244] System Configuration

[0245] The system consists of three main elements: a server, a terminal, and a user.

[0246] server

[0247] The server includes the following means:

[0248] 1. A way to train the generative AI to return structured data that is easy to identify from the logs of existing AI assistants: A generative AI model is trained using records of past user questions and responses.

[0249] 2. Programmatic actionable use of structured data: Perform specific actions (e.g., API calls) based on identified keywords.

[0250] 3. A means of regenerating a response sentence by combining the results with the question: The acquired information is combined with the user's question to generate a new response sentence.

[0251] 4. Identifying specific keywords and retrieving corresponding information: Extracting important keywords from the user's input and retrieving related information based on them.

[0252] 5. Means of providing acquired information to users: Communicate acquired information to users.

[0253] Terminal

[0254] A terminal is a device such as a smartphone or a head-mounted display that provides an interface for users to interact with the system. The terminal sends user input to the server and displays responses from the server.

[0255] user

[0256] The user is the entity that utilizes the system to obtain information. The user inputs questions by voice or text and receives responses from the system.

[0257] Hardware and software used

[0258] Hardware: Smartphone, head-mounted display

[0259] Software: Python 3.x, requests library, JSON library

[0260] Processing flow

[0261] 1. The user inputs or speaks a specific keyword (e.g., "weather") through the device.

[0262] 2. The device sends the user's input to the server.

[0263] 3. The server uses the generative AI model to identify keywords and retrieve corresponding information.

[0264] 4. The server combines the information it has obtained with the user's question to regenerate a response.

[0265] 5. The server sends the regenerated response to the terminal.

[0266] 6. The terminal displays the response to the user.

[0267] Specific examples

[0268] When the user says "weather," the server retrieves the latest weather information for the user's current location and sends it to the device. The device then displays, "The current weather in Tokyo is sunny, with a temperature of 25 degrees."

[0269] When the user says "traffic information," the server retrieves the latest traffic conditions for the current location and sends them to the device. The device then displays the message, "Current traffic conditions in Tokyo are congested. An alternative route will be suggested."

[0270] Prompt Sentence Examples

[0271] User: "Weather"

[0272] Generative AI: "Getting current weather information..."

[0273] Application: "The current weather in Tokyo is sunny and the temperature is 25 degrees."

[0274] User: "Traffic Information"

[0275] Generator: "Getting current traffic information..."

[0276] Application: "Current traffic conditions in Tokyo are congested. We will suggest an alternative route."

[0277] In this way, users can quickly obtain the information they need in real time.

[0278] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0279] Step 1:

[0280] The user inputs or speaks a specific keyword (e.g., "weather") through the terminal. The input is in text or voice format, and the terminal sends it to the server. The input data is the user's question.

[0281] Step 2:

[0282] The terminal sends the user's input to the server. The input data is the user's question, and the output data is the question sent to the server. The terminal sends the data to the server via the network.

[0283] Step 3:

[0284] The server uses a generative AI model to identify keywords. The input data is the user's question, and the output data is the identified keywords. The server uses the generative AI model to extract important keywords from the user's question.

[0285] Step 4:

[0286] The server retrieves the corresponding information based on the identified keywords. The input data is the identified keywords, and the output data is the retrieved information. The server calls the API to retrieve the latest weather information, traffic information, etc.

[0287] Step 5:

[0288] The server combines the acquired information with the user's question to regenerate a response. The input data is the acquired information and the user's question, and the output data is the regenerated response. The server generates an appropriate response for the user based on the acquired information.

[0289] Step 6:

[0290] The server sends the regenerated response sentence to the terminal. The input data is the regenerated response sentence, and the output data is the response sentence sent to the terminal. The server sends the data to the terminal via the network.

[0291] Step 7:

[0292] The terminal displays the response to the user. The input data is the response sent from the server, and the output data is the response displayed to the user. The terminal displays the response on the screen and provides information to the user.

[0293] Example 3

[0294] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0295] Conventional AI assistant systems have a complex process for generating appropriate responses to user questions, and they face problems such as low efficiency, particularly when gathering information from external data sources and generating responses. It is also difficult to quickly obtain the appropriate data based on the user's question and provide an accurate response. This can lead to a poor user experience and a loss of system reliability.

[0296] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0297] In this invention, the server includes means for receiving a user's question, means for analyzing the question and extracting important keywords and intent, means for collecting necessary data from external data sources, means for generating a response sentence based on the collected data, and means for sending the generated response sentence to the user. This makes it possible to generate a response to the user's question quickly and accurately, improving the user experience.

[0298] The "means for receiving a user's question" is a function for transmitting a question input by a user through a terminal to a server.

[0299] "Means of analyzing questions and extracting important keywords and intent" is a function that uses a generative AI model to analyze the content of a user's question and identify the subject and purpose of the question.

[0300] "Means of collecting necessary data from external data sources" refers to a function that uses an API client to obtain necessary information from external data sources, such as weather information and reservation information.

[0301] "Means for generating responses based on collected data" refers to a function that uses a generative AI model based on acquired data to create appropriate responses to user questions.

[0302] The "means for transmitting the generated response sentence to the user" is a function for transmitting the generated response sentence from the server to the terminal and displaying it to the user.

[0303] "Weather information" is information that includes meteorological data and indicates the weather conditions in a particular area and at a particular time.

[0304] "Reservation information" refers to information relating to a reservation for a particular service or facility, including the date, time, location, and user details.

[0305] "Alarm setting information" is information about an alarm set by a user, including the alarm time and message content.

[0306] A "generative AI model" is a model that uses artificial intelligence to generate and analyze text, and is used to generate responses to user questions.

[0307] A "prompt" is text that is input to a generative AI model and contains instructions or questions that enable the model to generate an appropriate response.

[0308] The present invention relates to a system for generating a response to a user's question quickly and accurately. A specific embodiment of this system will be described below.

[0309] System configuration

[0310] This system mainly consists of three elements: a server, a terminal, and a user. The server is a computer equipped with a high-performance processor and is installed with software to run the generative AI model and database management system. The terminal is a device used by the user, such as a smartphone or PC.

[0311] Hardware and software used

[0312] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[0313] Software: Generative AI models (e.g., OpenAI's GPT-3®), database management systems (e.g., MySQL®), API clients (e.g., Python's requests library)

[0314] Program processing

[0315] When the server receives a user's question, it uses the generative AI model to analyze the question and extract important keywords and intent. Next, the server uses the API client to collect necessary data from external data sources (e.g., weather information APIs). Based on the collected data, the generative AI model generates an appropriate response. Finally, the generated response is sent to the device and displayed to the user.

[0316] Specific examples

[0317] For example, the following process occurs when a user asks, "What's the weather like tomorrow?"

[0318] 1. Receive a user's question: The user types "What's the weather going to be like tomorrow?" into their smartphone and presses the send button. The device then sends this question to the server.

[0319] 2. Question analysis: The server uses a generative AI model to extract the keyword "tomorrow's weather."

[0320] 3. Data collection: The server sends a request to the weather information API to get tomorrow's weather data. The API returns the data "Sunny."

[0321] 4. Generate a response: The server generates a response saying "Tomorrow's weather will be sunny."

[0322] 5. Sending the response: The server sends the generated response to the terminal, which displays it to the user.

[0323] Prompt Sentence Examples

[0324] An example of a prompt sentence to input to the generative AI model is as follows:

[0325] User asks: "What's the weather like tomorrow?"

[0326] Prompt the generative AI model: "Get the latest weather information and tell me what the weather will be like tomorrow."

[0327] This system allows users to obtain information quickly and accurately, improving the user experience. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0328] Step 1:

[0329] A user uses a terminal to input a question. For example, the user types "What's the weather like tomorrow?" The terminal sends this question to the server as an HTTP request. The input is the user's question, and the output is an HTTP request to the server.

[0330] Step 2:

[0331] The server inputs the received question into the generative AI model. The generative AI model analyzes the question and extracts important keywords and intent. Specifically, the keyword "tomorrow's weather" is extracted. The input is the user's question, and the output is the extracted keywords.

[0332] Step 3:

[0333] The server uses an API client to collect the required data from an external data source (e.g., a weather information API). The server requests information about "tomorrow's weather" from the weather information API and receives weather data as a response. The input is the extracted keywords, and the output is the retrieved weather data.

[0334] Step 4:

[0335] The server uses a generative AI model to generate an appropriate response based on the acquired weather data. For example, if the weather data is "sunny," it generates a response such as "Tomorrow's weather will be sunny." The input is the acquired weather data, and the output is the generated response.

[0336] Step 5:

[0337] The server sends the generated response to the terminal as an HTTP response. The terminal displays this response to the user. The user sees the response "Tomorrow's weather will be sunny" on the terminal screen. The input is the generated response, and the output is the response displayed to the user.

[0338] (Application example 3)

[0339] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional food delivery systems did not optimize delivery times and methods in response to weather changes, and failed to provide users with appropriate information. As a result, users often became dissatisfied with delivery delays and uncertainty. In addition, response messages generated using generative AI models were not sufficiently utilized, resulting in low quality information provided to users.

[0341] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0342] In this invention, the server includes: means for training the generating AI from the logs of existing AI assistants to return easily identifiable structured data; means for receiving the structured data and programming it to perform an action; means for combining the result with a question to regenerate a response; means for acquiring the latest weather information and optimizing delivery time and delivery method based on that information; and means for inputting a prompt using the generating AI model and providing an appropriate response to the user. This makes it possible to optimize delivery time and delivery method in response to changes in the weather, and to provide appropriate information to the user.

[0343] "Generative AI" is a technology that uses artificial intelligence to generate new information and responses from data.

[0344] "Structured data" is data that is organized according to specific formats and rules, making it easy for machines to understand and process.

[0345] An "AI assistant" is software or a device equipped with artificial intelligence to respond to user instructions and questions.

[0346] A "log" is data that records the operation history of a system or application.

[0347] "Programming" is the act of writing instructions for a computer to execute.

[0348] An "action" is an operation or process that a system performs based on specific conditions or instructions.

[0349] A "response" is a sentence that the system returns in response to a user's question or instruction.

[0350] "Weather information" refers to data related to the weather, such as the current weather, temperature, and precipitation.

[0351] "Delivery time" is the time it takes for a product or service to be delivered to the user.

[0352] "Delivery method" refers to the means or procedure for delivering a product or service to a user.

[0353] A "prompt" is a sentence of instructions or questions that is input into a generative AI model.

[0354] A system for implementing this invention includes a server, a user terminal, and a generative AI model. The server has a means for training the generative AI from logs of existing AI assistants so that it returns structured data that is easy to identify. This allows the server to obtain basic data for generating appropriate responses to user instructions and questions.

[0355] The server then has the means to program and take action on the structured data, allowing the server to perform specific actions or processes based on the structured data it has received.

[0356] Furthermore, the server has a means to regenerate a response by combining the result with the question, allowing the server to provide an appropriate response to the user.

[0357] The server also has a means of obtaining the latest weather information and optimizing delivery times and methods based on that information. This allows the server to adjust delivery times and methods according to changes in the weather, and provide accurate information to users.

[0358] Finally, the server has a means to input prompts using the generative AI model to provide appropriate responses to the user, allowing the server to generate responses in natural language to the user's questions and instructions.

[0359] As a specific example, when a user orders food delivery, the server uses the Weather API to obtain the latest weather information for the specified location. It then uses a generative AI model (OpenAI's GPT-3) to generate a response for the user. For example, if it is raining, the server generates a response such as, "The current weather is rainy. Delivery time may be longer than usual."

[0360] Example prompt sentence:

[0361] The current weather is rainy. Delivery times may be longer than usual.

[0362] The hardware used is a smartphone or tablet connected to the internet, and the software uses Python, the requests library, WeatherAPI, and OpenAI's GPT-3, allowing users to receive accurate information in real time and reducing uncertainty about delivery times and methods.

[0363] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0364] Step 1:

[0365] The server learns from the logs of existing AI assistants and returns structured data that is easy to identify to the generating AI.

[0366] Input: AI assistant log data

[0367] Output: Structured data

[0368] How it works: The server analyzes the log data and extracts patterns to respond to user questions and instructions, which then translates the data into a format that is easy for the generative AI to understand.

[0369] Step 2:

[0370] The server receives the structured data and performs programming actions based on it.

[0371] Input: Structured data

[0372] Output: Action result

[0373] Specific Action: The server performs a specific action or process based on the structured data, such as retrieving information from a database or sending a request to an external API as instructed by the user.

[0374] Step 3:

[0375] The server regenerates a response sentence by combining the action result and the question sentence.

[0376] Input: Action result, question

[0377] Output: Response

[0378] Specific operation: The server combines the action result with the user's question and uses a generative AI model to generate a response in natural language.

[0379] Step 4:

[0380] The server obtains the latest weather information and optimizes delivery times and methods based on that information.

[0381] Input: User's location

[0382] Output: Weather information, optimized delivery time and delivery method

[0383] Specific operation: The server uses the Weather API to obtain the latest weather information for the specified location and adjusts the delivery time and method based on that information.

[0384] Step 5:

[0385] The server uses a generative AI model to input prompt sentences and provide appropriate responses to the user.

[0386] Input: prompt statement

[0387] Output: Response to the user

[0388] Specific operation: The server inputs a prompt into a generative AI model (e.g., OpenAI's GPT-3) and generates a response in natural language for the user, such as "The current weather is rainy. Delivery times may be longer than usual."

[0389] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0390] "Example 1"

[0391] In one embodiment of the present invention, the generative AI learns from the logs of existing AI assistants and generates structured data that is easy to identify. This generative AI includes an emotion engine that recognizes the user's emotions. The emotion engine reads emotions from the user's tone of voice, facial expressions, and choice of words. For example, if the user feels angry, the emotion engine conveys that information to the generative AI.

[0392] "Example 2"

[0393] The generative AI receives information from the emotion engine and adjusts the generation of structured data or the execution of actions based on that information. For example, if the user feels angry, the generative AI will take that information into account and generate a more appropriate response. Alternatively, if the user feels happy, the generative AI will use that information to generate a more positive response.

[0394] "Example 3"

[0395] The generative AI then combines the results with the question to regenerate a response. This regenerated response is adjusted appropriately based on the user's emotions. For example, if the user feels angry, the generative AI will take that information into account to generate a more polite response. Alternatively, if the user feels happy, the generative AI will use that information to generate a more positive response.

[0396] The processing flow of each embodiment will be described below.

[0397] "Example 1"

[0398] Step 1: The generative AI starts learning from the logs of existing AI assistants.

[0399] Step 2: Generation The AI ​​generates structured data that is easy to identify based on the results of its learning.

[0400] Step 3: The emotion engine reads the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0401] Step 4: The emotion engine conveys the emotional information it has read to the generation AI.

[0402] "Example 2"

[0403] Step 1: The generative AI receives information from the emotion engine.

[0404] Step 2: The generation AI adjusts the generation of structured data and execution of actions based on the information it receives.

[0405] Step 3: The generative AI generates an appropriate response based on the adjusted results.

[0406] "Example 3"

[0407] Step 1: The generative AI matches the structured data with the question.

[0408] Step 2: The generation AI regenerates the response sentence based on this information.

[0409] Step 3: The generation AI provides the regenerated response to the user.

[0410] Example 1

[0411] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0412] Conventional AI assistant systems sometimes lacked consistency in their responses to user questions, making it particularly difficult to respond in a way that took the user's emotions into account. Furthermore, the generation and use of structured data was inefficient, which sometimes resulted in delays in providing appropriate information to users. This resulted in a poor user experience, which was an issue.

[0413] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0414] In this invention, the server includes means for training the generation AI from logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for combining the result with a question to regenerate a response, means for transmitting a user's question from a terminal to the server, means for the server to analyze the question and pass it to a generation AI model, means for the generation AI model to generate structured data based on the question, means for the server to return the generated data to the terminal, means for the terminal to display the data to the user, means for recognizing the user's emotions, and means for generating a response based on the emotions. This enables a quick and appropriate response to a user's question and further enables the provision of a response that takes the user's emotions into consideration.

[0415] "Generative AI" is artificial intelligence that learns from the logs of existing AI assistants and generates structured data that is easy to identify in response to user questions.

[0416] "Structured data" is data that is organized according to a specific format or structure, and is generated by generative AI based on user questions.

[0417] The "server" is a computer system whose role is to receive the user's question, analyze it, and pass it on to the generative AI model.

[0418] A "terminal" is a device through which a user inputs a question and receives a response from the server, and includes smartphones, PCs, etc.

[0419] The "emotion engine" is software that reads emotions from the user's tone of voice, facial expressions, choice of words, etc., and conveys that information to the generative AI.

[0420] "Question analysis" is the process in which the server receives a user's question and extracts keywords and intent.

[0421] "Response regeneration" is the process in which a generative AI generates a new response based on structured data and the user's question.

[0422] "Emotion recognition" is the process by which the emotion engine analyzes the user's emotions and provides that information to the generative AI.

[0423] An "action" is a specific operation or process that the generative AI performs based on structured data.

[0424] "Data display" is the process by which the terminal visually or audibly presents data received from the server to the user.

[0425] The present invention is a system that uses a generative AI model to provide appropriate responses to user questions. This system is realized using hardware and software such as a server, a terminal, a generative AI model, and an emotion engine.

[0426] Hardware and Software Configuration

[0427] Server: A high-performance server (e.g., a virtual machine on a cloud service) is used to host the generative AI model. The server analyzes the user's question and passes it to the generative AI model.

[0428] Terminal: A device used by a user, such as a smartphone or PC. The terminal sends the user's questions to the server, receives responses from the server, and displays them.

[0429] Generative AI models: For example, using advanced natural language processing models such as GPT-4, generative AI models generate structured data based on questions passed to them by the server.

[0430] Emotion engine: For example, using emotion recognition software such as the Affectiva SDK, the emotion engine analyzes the user's tone of voice, facial expressions, and word choice to identify emotions.

[0431] Data processing and calculation

[0432] 1. Handling user questions

[0433] The user asks through the terminal, "What's the weather going to be like tomorrow?"

[0434] The terminal sends this question to the server.

[0435] The server analyzes the question and extracts the keyword "weather."

[0436] The analysis results are passed to the generative AI model.

[0437] 2. Generating structured data

[0438] The generative AI model generates related structured data based on the analysis results passed from the server, such as weather forecast data.

[0439] The generated data is returned to the server.

[0440] 3. Return and Display of Data

[0441] The server returns the generated structured data to the terminal.

[0442] The terminal displays the data to the user, for example, "Tomorrow will be sunny."

[0443] 4. Emotion recognition and response generation

[0444] When a user asks a question or makes a comment, the emotion engine analyzes the user's tone of voice, facial expressions, and word choice.

[0445] The emotion engine communicates the analysis results to the generative AI model.

[0446] The generative AI model generates a response that takes into account the user's emotions and returns it to the device via the server. For example, if the user is angry, it will generate a response such as "Please calm down."

[0447] Specific examples

[0448] Example question: A user asks, "What's the weather like tomorrow?"

[0449] The server identifies the keyword "weather" and generates and returns weather forecast data.

[0450] Emotion example: When a user is angry

[0451] The emotion engine recognizes the user's anger and conveys that information to the generative AI.

[0452] The generative AI generates a response that takes into consideration the user's emotions and returns it to the device.

[0453] Prompt Sentence Examples

[0454] "If a user asks, 'What's the weather going to be like tomorrow?' how does a generative AI respond?"

[0455] "If the emotion engine recognizes that the user is angry, how will the generative AI respond?"

[0456] The above is an embodiment of the present invention. This system enables quick and appropriate responses to user questions, and also provides responses that take into account the user's emotions.

[0457] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0458] Step 1:

[0459] The user enters a question.

[0460] A user uses a device (smartphone or PC) to input a question such as "What's the weather going to be like tomorrow?" The input is done through voice recognition or text input. The input data is voice data or text data.

[0461] Step 2:

[0462] The terminal sends a question to the server.

[0463] The terminal receives the user's question and sends it to the server via the Internet. At this time, the question is converted into an appropriate format (e.g., JSON). The input is the user's question data, and the output is the JSON-formatted data sent to the server.

[0464] Step 3:

[0465] The server analyzes the question and passes it to a generative AI model.

[0466] The server analyzes the received question and extracts keywords (e.g., "weather"). The analysis results are passed to the generative AI model. The input is the question data in JSON format, and the output is the analyzed keyword data.

[0467] Step 4:

[0468] A generative AI model generates structured data based on the question.

[0469] The generative AI model generates related structured data based on the analysis results passed from the server. For example, in response to the question "What's the weather like tomorrow?", it generates weather forecast data. The input is analyzed keyword data, and the output is structured weather forecast data.

[0470] Step 5:

[0471] The server returns the generated data to the terminal.

[0472] The server returns the structured data received from the generative AI model to the device, converting the data into a format that is easy for users to understand. The input is structured weather forecast data, and the output is user-friendly data sent to the device.

[0473] Step 6:

[0474] The terminal displays the data to the user.

[0475] The terminal displays data received from the server to the user, for example, displaying weather forecast information on a screen. The input is the user-friendly data received from the server, and the output is the information displayed to the user.

[0476] Step 7:

[0477] Recognize user emotions.

[0478] When a user asks a question or makes a comment, the device's built-in emotion engine analyzes the user's tone of voice, facial expressions, and word choice to recognize their emotions. The input is the user's voice and video data, and the output is analyzed emotional data.

[0479] Step 8:

[0480] Generate emotion-based responses.

[0481] The emotion engine passes the recognized emotion information to the generative AI model. The generative AI model generates a response that takes the user's emotions into consideration and returns it to the device via the server. For example, if the user is angry, it generates a response such as "Please calm down." The input is analyzed emotion data, and the output is emotion-conscious response data.

[0482] (Application example 1)

[0483] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0484] Conventional navigation systems for autonomous vehicles have difficulty providing appropriate responses to user questions, and in particular, have been unable to respond in a way that takes into account the user's emotions. This can lead to reduced user satisfaction and increased stress while driving.

[0485] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for training the generation AI from the log of an existing AI assistant so as to return easily identifiable structured data; means for receiving the structured data and programming it to perform an action; means for regenerating a response by combining the result with a question; means including an emotion engine that recognizes the user's emotions; and means for adjusting the response based on information from the emotion engine. This makes it possible to provide an appropriate response to a user's question and further respond in accordance with the user's emotions.

[0486] "Generative AI" is an artificial intelligence that learns from the logs of existing AI assistants and returns structured data that is easy to identify in response to user questions.

[0487] "Structured data" is data organized in a specific format that is generated by generative AI based on user questions.

[0488] The "emotion engine" is an engine that has the ability to read emotions from the user's tone of voice, facial expressions, choice of words, etc.

[0489] A "response sentence" is a sentence that the generation AI generates to respond to a user's question.

[0490] "Means for programming and taking action" refers to means for executing specific operations or processes based on the generated structured data.

[0491] The "means of regeneration" refers to the means by which the generative AI generates a new response sentence based on the structured data and the question sentence.

[0492] The "adjustment means" is a means for appropriately changing the response sentence in accordance with the user's emotions based on information from the emotion engine.

[0493] A system for implementing the present invention has the following configuration.

[0494] System Program

[0495] The system includes a generative AI, an emotion engine, a voice recognition module, a camera module, and a navigation module. The generative AI learns from the logs of existing AI assistants and returns structured data that is easy to identify in response to user questions. The emotion engine reads emotions from the user's tone of voice, facial expressions, and choice of words.

[0496] Hardware and Software

[0497] Hardware:

[0498] Microphone: Used to capture audio input.

[0499] Camera: Used to capture the user's facial expressions.

[0500] software:

[0501] speech_recognition library: Used to convert speech to text.

[0502] Transformers library pipeline: Used to perform emotion recognition.

[0503] cv2 (OpenCV): Used to acquire images from the camera.

[0504] Data processing and calculation

[0505] The server converts the voice data acquired from the microphone into text using a speech recognition module. The generation AI then analyzes this text and generates structured data that is easy to identify. The emotion engine reads emotions from the user's tone of voice and facial expressions and conveys this information to the generation AI. The generation AI then adjusts the response sentence based on the information from the emotion engine and provides an appropriate response to the user.

[0506] Specific examples

[0507] For example, if a user asks, "Where is the next gas station?", the speech recognition module converts this question into text. The generation AI identifies the keyword "gas station" and generates information about the nearest gas station as structured data. If the emotion engine determines the user's emotion as "neutral," the generation AI responds, "The nearest gas station is 5 km away."

[0508] Prompt Sentence Examples

[0509] If the user asks, "Where is the next gas station?" and the emotion recognition model determines the answer as "neutral," the system will respond, "The nearest gas station is 5 km away."

[0510] In this way, it is possible to provide appropriate responses to the user's questions and further to respond in accordance with the user's emotions.

[0511] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0512] Step 1:

[0513] The user asks a question into the microphone. The input is the user's voice data, and the output is the voice data passed to the speech recognition module.

[0514] Step 2:

[0515] The server's speech recognition module converts the speech data into text. The input is speech data and the output is text data. Specifically, the speech_recognition library is used to convert speech to text.

[0516] Step 3:

[0517] The server's generation AI analyzes the text data and generates structured data that is easy to identify. The input is text data, and the output is structured data. Specifically, the generation AI model extracts keywords and generates related information as structured data.

[0518] Step 4:

[0519] The server's emotion engine reads emotions from the user's tone of voice and facial expressions. The input is audio data and camera video data, and the output is emotional information. Specifically, emotion recognition uses a pipeline from the transformers library.

[0520] Step 5:

[0521] The server's generation AI adjusts the response sentence based on the emotional information from the emotion engine. The input is structured data and emotional information, and the output is the adjusted response sentence. Specifically, the generation AI generates the response sentence taking into account the emotional information.

[0522] Step 6:

[0523] The server returns the tailored response to the user. The input is the tailored response, and the output is a voice response to the user. Specifically, the server converts the text into speech and responds to the user through a speaker.

[0524] Example 2

[0525] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0526] Conventional AI assistant systems have difficulty generating responses that take the user's emotional state into account, making it difficult to provide appropriate responses to users. Furthermore, they have problems with not being able to efficiently execute actions using structured data or obtain up-to-date information. This has resulted in a poor user experience and limited the usefulness of the system.

[0527] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0528] In this invention, the server includes a means for training the generation AI to return easily identifiable structured data from the logs of existing AI assistants, a means for receiving the structured data and programming it to perform an action, and a means for receiving information from the emotion engine and adjusting the generation of structured data and the execution of actions based on that information. This makes it possible to generate appropriate responses that take the user's emotional state into consideration and efficiently obtain up-to-date information.

[0529] "Generative AI" is an artificial intelligence technology that analyzes input data from users and generates appropriate responses.

[0530] "Structured data" is data that is organized according to a specific format, making it easy for generative AI to identify and process.

[0531] An "AI assistant" is an artificial intelligence system that generates responses to user input and automates various tasks.

[0532] A "log" is data that records the operation of a system and the operation history of a user.

[0533] "Programming" is the process of giving a computer instructions to perform a particular task.

[0534] An "action" is a specific operation or process that a system performs in response to a specific input.

[0535] An "emotion engine" is a technology that analyzes a user's emotional state and provides that information.

[0536] A "response sentence" is a reply message generated by the generation AI in response to user input.

[0537] "Updateable information" is data that changes over time and with the situation, and is information that needs to reflect the latest state.

[0538] This invention is a system for analyzing user input data and generating appropriate responses. The system is realized using a generative AI model, an emotion engine, structured data, and an AI assistant log.

[0539] The server first generates a program using the generative AI model. This program receives input data from the user and executes appropriate actions. Specifically, the server uses the following hardware and software:

[0540] Hardware:

[0541] A server (a computer with a powerful processor and sufficient memory)

[0542] User device (smartphone, PC, etc.)

[0543] software:

[0544] Generative AI models (AI models for natural language processing)

[0545] Emotion engine (software for analyzing the user's emotional state)

[0546] External APIs (e.g., APIs for obtaining weather information)

[0547] The server receives and analyzes input data from the user. During the analysis, the server extracts keywords from the input data. For example, if the user inputs "What's the weather like today?", the server identifies the keyword "weather."

[0548] The server then determines the corresponding action based on the identified keywords, for example, if the keyword "weather" is identified, the server will call an external weather API to get the latest weather information.

[0549] Additionally, the server receives information from the emotion engine. This information includes the user's emotional state (e.g., anger, joy). The server uses the generative AI model to generate a response based on the information from the emotion engine. For example, if the user feels angry, the generative AI takes that information into account to generate a more appropriate response. Alternatively, if the user feels happy, the generative AI uses that information to generate a more positive response.

[0550] Finally, the server returns the generated response to the user, for example, "Today's weather is sunny. The maximum temperature is 25 degrees and the minimum temperature is 15 degrees."

[0551] Examples:

[0552] The specific behavior when a user inputs "What's the weather like today?" is explained below.

[0553] 1. A user uses a smartphone to type "What's the weather like today?" and presses the send button.

[0554] 2. The server receives the text data "What's the weather like today?" sent by the user.

[0555] 3. The server analyzes the received text data and extracts the keyword "weather."

[0556] 4. The server calls an external weather API based on the keyword "weather" to obtain the latest weather information.

[0557] 5. The server receives the information from the emotion engine and confirms the user's emotional state.

[0558] 6. Based on the information from the emotion engine, the server uses a generative AI model to generate a response: "Today's weather is sunny. The maximum temperature is 25 degrees and the minimum temperature is 15 degrees."

[0559] 7. The server returns the generated response to the user. The user's smartphone displays a message saying, "Today's weather is sunny. The maximum temperature is 25 degrees, and the minimum temperature is 15 degrees."

[0560] In this way, the server analyzes the input data from the user, executes appropriate actions, generates a response based on the information in the emotion engine, and sends it back to the user.

[0561] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0562] Step 1:

[0563] The user submits the input data.

[0564] A user uses a terminal (e.g., a smartphone or PC) to send input data in text format to the system. For example, the user enters "What's the weather like today?" and presses the send button. The input data is sent to the server in text format.

[0565] Step 2:

[0566] The server receives the input data.

[0567] The server receives the input data sent by the user. The received data is saved in text format. The input data is "What's the weather like today?" and is saved on the server.

[0568] Step 3:

[0569] The server analyzes the input data and extracts keywords.

[0570] The server analyzes the received input data and extracts keywords. Specifically, it analyzes the text data using natural language processing technology and identifies the keyword "weather." The input is the text data "What's the weather like today?" and the output is the keyword "weather."

[0571] Step 4:

[0572] The server determines the corresponding action based on the extracted keywords.

[0573] The server determines the corresponding action based on the identified keyword. For example, if the keyword "weather" is identified, the server calls an external weather API to obtain the latest weather information. The input is the keyword "weather" and the output is the latest weather information obtained from the weather API.

[0574] Step 5:

[0575] The server receives the information from the emotion engine.

[0576] The server receives information from the emotion engine, which includes the user's emotional state (e.g., anger, joy). The input is data about the user's emotional state, and the output is the analysis result from the emotion engine.

[0577] Step 6:

[0578] The server generates a response using a generative AI model based on information from the emotion engine.

[0579] The server generates a response using a generative AI model based on information from the emotion engine. For example, if the user feels angry, the generative AI takes that information into account and generates a more appropriate response. If the user feels happy, the generative AI uses that information to generate a more positive response. The input is the emotion engine's analysis results and weather information, and the output is the generated response.

[0580] Step 7:

[0581] The server generates a response and sends it back to the user.

[0582] The server returns the generated response to the user. For example, it might reply, "Today's weather is sunny. The maximum temperature is 25 degrees and the minimum temperature is 15 degrees." The input is the generated response sentence, and the output is the message displayed on the user's terminal.

[0583] (Application example 2)

[0584] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0585] Conventional AI assistant systems have difficulty generating responses that reflect the user's emotions, resulting in a poor user experience. Furthermore, when providing information such as news articles, the system was unable to adjust the content or tone to reflect the user's emotions, making it difficult to increase user satisfaction. This can lead to stress and dissatisfaction with the way information is received.

[0586] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0587] In this invention, the server includes a means for training the generation AI to return easily identifiable structured data from the logs of existing AI assistants, a means for receiving the structured data and programming it to perform an action, and a means for receiving information from the emotion analysis engine and adjusting the response based on the user's emotions, thereby making it possible to adjust the appropriate response and the content and tone of the news article display according to the user's emotions.

[0588] "Generative AI" is artificial intelligence that analyzes input from users and generates appropriate responses and actions.

[0589] "Structured data" is data that is organized in a format that is easy for generating AI to identify.

[0590] An "emotion analysis engine" is a system that analyzes emotions from user input and behavior and provides the results.

[0591] "News article information" is data about the latest news and articles.

[0592] A "response sentence" is a reply sentence that the generation AI generates in response to a user's question or request.

[0593] "Tone" refers to the style and atmosphere of news articles and responses.

[0594] "User emotion" is the emotional state that a user expresses through their input and actions.

[0595] The system for implementing the present invention uses the following hardware and software to adjust responses and news article display content according to the user's emotions.

[0596] Hardware and software used

[0597] Hardware: Smartphone

[0598] Software: Python, EmotionEngine (emotion analysis library), NewsAPI (news acquisition API)

[0599] System configuration

[0600] 1. Generative AI: This is artificial intelligence that analyzes user input and generates appropriate responses and actions.

[0601] 2. Structured data: Data that is organized in a format that is easy for generative AI to identify.

[0602] 3. Sentiment analysis engine: A system that analyzes emotions from user input and behavior and provides the results.

[0603] 4. News Article Information: Data about the latest news and articles.

[0604] 5. Response sentence: A response sentence generated by the generative AI in response to a user's question or request.

[0605] 6. Tone: This refers to the tone and atmosphere of a news article or response.

[0606] 7. User Emotion: The emotional state that a user displays through their input and actions.

[0607] System Operation

[0608] The server first trains the generating AI to return structured data that is easy to identify from the logs of existing AI assistants. It then programs and takes action based on the structured data. It also receives information from a sentiment analysis engine and adjusts responses based on the user's emotions. This makes it possible to tailor appropriate responses and the content and tone of news articles to reflect the user's emotions.

[0609] Specific examples

[0610] For example, if a user types "I'm so tired today," the sentiment analysis engine will interpret the emotion as "tired" and display relaxing news (for example, articles about the beauty of nature and relaxation).

[0611] Prompt Sentence Examples

[0612] If a user types "I'm so tired today," the sentiment engine should parse the emotion "tired" and display news that will help them relax.

[0613] In this way, news can be delivered according to the user's emotions, providing a more personalized experience.

[0614] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0615] Step 1:

[0616] A user uses a smartphone to send input text containing emotions to an application.

[0617] Input: User input text (e.g. "I'm very tired today")

[0618] Output: The user's input text is sent to the server

[0619] Step 2:

[0620] The server sends the received user input text to an emotion analysis engine for emotion analysis.

[0621] Input: User-entered text

[0622] Data processing: A sentiment analysis engine analyzes the text to identify the emotional state (e.g., "tired")

[0623] Output: Sentiment analysis result (e.g. "Tired")

[0624] Step 3:

[0625] Based on the emotion analysis results, the server generates an appropriate prompt sentence for the generation AI and sends it as structured data.

[0626] Input: Sentiment analysis result (e.g. "Tired")

[0627] Data processing: Generative AI generates prompts based on the results of sentiment analysis (e.g., "Show me some relaxing news").

[0628] Output: Prompt sentence (e.g., "Show me some relaxing news.")

[0629] Step 4:

[0630] The server uses the generated prompt sentence to send a request to the news API to retrieve the appropriate news article.

[0631] Input: Prompt sentence (e.g., "Show me some relaxing news")

[0632] Data processing: News API searches and retrieves news articles based on prompts

[0633] Output: News article information (e.g., a list of relaxing news articles)

[0634] Step 5:

[0635] The server reconstructs the acquired news article information in a tone that corresponds to the user's emotions and generates a response sentence.

[0636] Input: News article information, sentiment analysis results

[0637] Data processing: Reconstructing news article information based on the results of sentiment analysis and generating a response (e.g., "A list of relaxing news articles")

[0638] Output: Response (e.g., "A list of relaxing news articles")

[0639] Step 6:

[0640] The server sends the generated response text to the user's smartphone and displays it.

[0641] Input: A response sentence (e.g., "A list of relaxing news articles")

[0642] Output: The response is displayed on the user's smartphone.

[0643] In this way, news articles can be provided that correspond to the user's emotions, providing a more personalized experience.

[0644] Example 3

[0645] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0646] Conventional AI assistant systems have had difficulty generating appropriate responses to user questions, especially providing responses that reflect the user's emotions. Acquiring up-to-date information in real time and generating responses based on that information has also been a challenge.

[0647] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0648] In this invention, the server includes means for receiving a question from a user, means for analyzing the question and acquiring necessary information, means for combining the acquired information and the question to input a prompt sentence to the generative AI model, means for the generative AI model to generate a response sentence, means for displaying the generated response sentence to the user, and means for adjusting the response sentence according to the user's emotions, thereby making it possible to provide an appropriate response to the user's question that is appropriate for the user's emotions.

[0649] "Means for receiving questions from users" refers to an interface or function that allows the system to receive questions entered by users.

[0650] "Means for analyzing questions and obtaining necessary information" refers to the function for analyzing received questions and obtaining necessary information based on their content from external data sources or APIs.

[0651] "Means for inputting a prompt sentence into a generative AI model" refers to a function for combining acquired information with a user's question to create a prompt sentence to be input into a generative AI model, and inputting that prompt sentence into the generative AI model.

[0652] "Means by which the generative AI model generates a response sentence" refers to the function by which the generative AI model generates a response sentence based on a prompt sentence.

[0653] "Means for displaying the generated response sentence to the user" refers to an interface or function for displaying the response sentence generated by the generative AI model to the user.

[0654] "Means for adjusting a response sentence according to the user's emotions" refers to a function for analyzing the user's emotions and appropriately adjusting the generated response sentence according to those emotions.

[0655] "Updateable information" refers to information that changes depending on the time and situation, and includes, for example, weather information and reservation information.

[0656] This invention is a system for generating appropriate responses to user questions. The system includes a series of processes: receiving a question from a user, analyzing it, acquiring necessary information, generating a response using a generative AI model, and displaying it to the user.

[0657] Hardware and software used

[0658] Device: The device through which the user enters their question, such as a smartphone, computer, or tablet.

[0659] Server: A central processing unit that analyzes questions, obtains necessary information, and generates responses using generative AI models.

[0660] Generative AI models: For example, natural language processing models such as GPT-3.

[0661] Weather API: External data sources, for example OpenWeatherMap.

[0662] Data processing and calculation

[0663] 1. Receive user questions

[0664] A user types a question into an input field on the device, for example, "What's the weather like tomorrow?"

[0665] 2. Send the question to the server

[0666] The terminal receives the user's input and sends the data to the server.

[0667] 3. Analyze the question and get the necessary information

[0668] The server analyzes the received question and extracts the keyword "weather." It then sends a request to the weather information API to retrieve the data "Tomorrow's weather will be sunny."

[0669] 4. Enter a prompt into the generative AI model

[0670] The server combines the information it has acquired with the user's question to create a prompt. For example, the following prompt is input into the generative AI model: "The user asked, 'What's the weather going to be like tomorrow?' The weather will be sunny tomorrow. Please generate an appropriate response."

[0671] 5. The generative AI model generates a response sentence

[0672] The generative AI model generates a response based on the prompt, for example, "The weather will be fine tomorrow. We apologize for any inconvenience."

[0673] 6. Display the generated response to the user

[0674] The server sends the generated response to the terminal, which displays it to the user.

[0675] Specific examples

[0676] User asks: "What's the weather like tomorrow?"

[0677] Example prompt: "The user asks, 'What's the weather going to be like tomorrow?' The weather will be sunny tomorrow. Generate an appropriate response."

[0678] Generated response: "The weather will be fine tomorrow. We apologize for the inconvenience."

[0679] This system allows users to receive appropriate responses based on information updated in real time. Furthermore, by using a generative AI model, it is possible to generate responses that correspond to the user's emotions. The flow of the identification process in Example 3 will be described with reference to FIG. 21.

[0680] Step 1:

[0681] The user enters a question.

[0682] The user inputs a question into the input field of the device. For example, "What's the weather going to be like tomorrow?" The input question is saved as text data on the device.

[0683] Step 2:

[0684] The terminal sends a question to the server.

[0685] The terminal receives the user's input and sends the text data to the server. The data sent is the text "What's the weather like tomorrow?" The server receives this data.

[0686] Step 3:

[0687] The server analyzes the question and retrieves the required information.

[0688] The server analyzes the received text data and extracts the keyword "weather." It then sends a request to the weather information API and retrieves the data "Tomorrow's weather will be sunny." The input is the question "What's the weather like tomorrow?" and the output is the information "Tomorrow's weather will be sunny."

[0689] Step 4:

[0690] The server inputs a prompt sentence into the generative AI model.

[0691] The server creates a prompt by combining the information it has acquired with the user's question. For example, it generates a prompt like this: "The user asked, 'What's the weather going to be tomorrow?' The weather will be sunny tomorrow. Please generate an appropriate response." The input is the question "What's the weather going to be tomorrow?" and the information "The weather will be sunny tomorrow," and the output is the prompt.

[0692] Step 5:

[0693] The generative AI model generates a response sentence.

[0694] The server inputs a prompt sentence into the generative AI model, which then generates a response sentence. For example, it generates a response sentence such as, "The weather will be sunny tomorrow. We apologize for any inconvenience." The input is the prompt sentence, and the output is the response sentence.

[0695] Step 6:

[0696] The server sends the generated response to the terminal.

[0697] The server sends the generated response to the terminal. The data sent is the text "Tomorrow's weather will be fine. We apologize for any inconvenience." The terminal receives this data.

[0698] Step 7:

[0699] The terminal displays the response to the user.

[0700] The terminal displays the received response to the user. The displayed text is "Tomorrow's weather will be fine. We apologize for the inconvenience." The user confirms this.

[0701] (Application example 3)

[0702] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0703] Conventional response systems using generative AI have difficulty generating appropriate responses based on the user's emotions. In addition, responses to user questions are uniform, making it difficult to improve user satisfaction. In particular, online shopping sites require customer support that responds to the user's emotions, and technology to achieve this is needed.

[0704] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0705] In this invention, the server includes means for training the generation AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for analyzing the user's emotions, and means for adjusting the generated response in accordance with the user's emotions. This makes it possible to generate an appropriate response in accordance with the user's emotions and improve user satisfaction.

[0706] "Generative AI" is artificial intelligence that generates natural language responses based on user input.

[0707] "Structured data" is data that is organized in a specific format that can be easily identified by the generating AI.

[0708] "Programming" is the act of writing instructions for a computer to perform specific actions.

[0709] An "action" is a specific operation or process that is executed by a program.

[0710] A "question" is a query that a user enters into the generating AI.

[0711] A "response sentence" is a sentence that the generation AI generates in response to a user's question.

[0712] A "means for analyzing emotions" is a technique for identifying emotions from user input and evaluating those emotions.

[0713] The "adjustment means" is a technique for appropriately changing the generated response sentence according to the user's emotions.

[0714] A system for implementing this invention uses a generative AI model to generate responses to user questions and adjusts the responses according to the user's emotions. Specific embodiments of this system are described below.

[0715] System configuration

[0716] The system consists of the following main components:

[0717] 1. Server: Hosts the generative AI model and processes user input.

[0718] 2. User device: A device such as a smartphone or computer through which the user enters their question.

[0719] 3. Sentiment Analysis Module: Software for analyzing emotions from user input.

[0720] 4. Response adjustment module: Software for adjusting the generated response sentences according to the user's emotions.

[0721] Program processing

[0722] The server performs the process in the following procedure.

[0723] 1. Receiving user input: Receives a question from the user's device.

[0724] 2. Sentiment Analysis: Use the sentiment analysis module to analyze the sentiment of the user's question. For sentiment analysis, use libraries such as NLTK or TextBlob.

[0725] 3. Prompt generation: Generate a prompt that includes the user's question and sentiment.

[0726] 4. Response generation: Use a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on the prompt.

[0727] 5. Response Adjustment: The response adjustment module is used to adjust the generated response according to the user's emotions.

[0728] 6. Send response: Send the adjusted response to the user terminal.

[0729] Hardware and software used

[0730] Generative AI models: Natural language generation models such as OpenAI's GPT-3.

[0731] Sentiment analysis libraries: NLTK, TextBlob, etc.

[0732] User device: smartphone, computer, etc.

[0733] Server: Cloud server (e.g., AWS (registered trademark), Google (registered trademark) Cloud, etc.).

[0734] Specific examples

[0735] Below are some examples of specific prompt sentences.

[0736] Example prompt sentence:

[0737] User Question: When will this item arrive?

[0738] Emotion: Anger

[0739] Generate an appropriate response.

[0740] Based on this prompt, the generative AI model generates a response like this:

[0741] Response: We apologize for the wait. Your order will usually arrive within 3-5 business days. If you have any other questions, please let us know.

[0742] In this way, an appropriate response can be provided depending on the user's emotions.

[0743] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0744] Step 1:

[0745] Receiving user input

[0746] The user inputs a question using a device (smartphone or PC), which then sends the question to the server.

[0747] Input: User's question

[0748] Output: The question sent to the server

[0749] Step 2:

[0750] sentiment analysis

[0751] The server passes the received question to a sentiment analysis module, which uses libraries such as NLTK and TextBlob to analyze the user's sentiment from the question.

[0752] Input: User's question

[0753] Output: User's emotion (e.g. anger, joy, sadness, etc.)

[0754] Step 3:

[0755] Generate prompt statement

[0756] The server combines the user's question and emotion to generate a prompt, which is then passed to a generative AI model.

[0757] Input: User's question, user's sentiment

[0758] Output: prompt statement

[0759] Step 4:

[0760] Generate a response

[0761] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on the prompt. The generative AI model receives the prompt as input and outputs an appropriate response.

[0762] Input: prompt statement

[0763] Output: Response

[0764] Step 5:

[0765] Adjusting the response

[0766] The server then passes the generated response to the response adjustment module, which then adjusts the response appropriately based on the user's emotions. For example, if the user is angry, the response may be changed to a more polite expression.

[0767] Input: Response sentence, user sentiment

[0768] Output: Adjusted response

[0769] Step 6:

[0770] Sending a response

[0771] The server sends the adjusted response to the user's terminal, where the user can check the response.

[0772] Input: Adjusted response

[0773] Output: Response text displayed on the user's terminal

[0774] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0775] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0776] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0777] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0778] [Second embodiment]

[0779] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0780] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0781] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0782] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0783] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0784] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0785] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0786] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0787] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0788] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0789] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0790] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0791] "Example 1"

[0792] In one embodiment of the present invention, the generation AI learns from the logs of existing AI assistants and returns structured data that is easy to identify. For example, if a user asks, "What's the weather like tomorrow?", the generation AI will identify the keyword "weather" and generate relevant structured data.

[0793] "Example 2"

[0794] Next, we will explain how the AI ​​receives structured data, programs it uses, and performs actions. For example, if the AI ​​recognizes the keyword "weather," the corresponding program will run and retrieve the latest weather information.

[0795] "Example 3"

[0796] Finally, we will explain how the results are combined with the question to regenerate a response. For example, after obtaining the latest weather information, a response such as "Tomorrow's weather will be sunny" is generated based on that information. This makes it possible to provide an appropriate response to the user.

[0797] The processing flow of each embodiment will be described below.

[0798] "Example 1"

[0799] Step 1: The generative AI begins learning from the logs of existing AI assistants. Through this learning, the generative AI acquires the ability to generate appropriate responses to user questions.

[0800] Step 2: When a user asks a question, the Generative AI analyzes the question and generates structured data that is easy to identify. For example, in response to the question "What's the weather like tomorrow?", it identifies the keyword "weather" and generates related structured data.

[0801] "Example 2"

[0802] Step 1: The generative AI receives the identified structured data and runs a corresponding program. This program performs a specific action based on the structured data. Step 2: For example, if the keyword "weather" is identified, the program performs an action to obtain the latest weather information. This action allows the user to obtain the information they are looking for.

[0803] "Example 3"

[0804] Step 1: Combine the acquired information with the original question to generate a new response. In this step, the generative AI uses the knowledge it has learned to generate an appropriate response.

[0805] Step 2: For example, after obtaining the latest weather information, generate a response such as "Tomorrow's weather will be sunny." This makes it possible to provide an appropriate response to the user.

[0806] Example 1

[0807] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0808] Conventional AI assistants often lack the training data to provide appropriate responses to user questions, resulting in low accuracy and relevance of responses. Furthermore, responses to user questions are not structured, making it difficult to obtain and use information. Furthermore, there is also the issue of difficulty in providing up-to-date information appropriately.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0810] In this invention, the server includes means for collecting and preprocessing log data of existing AI assistants, means for training a generative AI model using the preprocessed log data, means for receiving questions from users and analyzing the questions to identify keywords, means for generating relevant structured data based on the identified keywords, and means for returning the generated structured data to the user. This makes it possible to provide highly accurate and relevant responses to user questions. Furthermore, returning structured data makes it easier to obtain and use information, and allows for appropriate provision of up-to-date information.

[0811] "Log data of existing AI assistants" refers to data that records questions asked by users to AI assistants and the AI ​​assistant's responses to those questions.

[0812] "Preprocessing" is the process of removing unnecessary information from the collected log data, extracting question-answer pairs, and normalizing the data.

[0813] A "generative artificial intelligence model" is an artificial intelligence model that has the ability to learn from provided data and generate appropriate responses to user questions.

[0814] A "user question" is an inquiry or request that a user inputs to an artificial intelligence system.

[0815] "Keyword identification" means analyzing a user's question and extracting important words and phrases from it.

[0816] "Structured data" is data that is organized according to a specific format or structure, making the information easy to retrieve and use.

[0817] "Updateable information" refers to information whose content changes over time or with changes in circumstances, and includes, for example, weather information and reservation information.

[0818] "Providing a response" means returning appropriate information or answers to a user's questions.

[0819] This invention is a system that collects log data from existing AI assistants, uses the collected data to train a generative AI model, and returns appropriate structured data in response to user questions. Specific embodiments of this system are described below.

[0820] Hardware and software used

[0821] server

[0822] The server is hardware that collects and preprocesses log data from existing AI assistants. The server uses the following software:

[0823] Data collection script: A script to periodically collect log data from your existing artificial intelligence assistant.

[0824] Data preprocessing scripts: Scripts to remove unnecessary information from the collected log data, extract question-answer pairs, and normalize the data.

[0825] API call: An API to provide pre-processed data to the generative artificial intelligence model.

[0826] Generative AI Models

[0827] A generative AI model is software that uses data provided by a server to learn and generate appropriate responses to user questions. Specifically, it uses the following algorithm:

[0828] Natural language processing algorithms: Algorithms for analyzing user questions and identifying keywords.

[0829] Database query: A query to generate relevant structured data based on the identified keywords.

[0830] User terminal

[0831] The user terminal is the hardware that allows the user to input questions to the generative AI model and receive responses. Specifically, it uses the following interface:

[0832] Input interface: The interface through which the user enters their question.

[0833] Output interface: An interface for displaying the structured data returned by the generative AI model.

[0834] Specific examples

[0835] Prompt Sentence Examples

[0836] Below are examples of prompts that users can use to input questions to the generative AI model.

[0837] "What's the weather like tomorrow?"

[0838] "What events are happening this weekend?"

[0839] "Tell me the latest news"

[0840] For these prompts, the generative AI model identifies appropriate keywords and generates and returns relevant structured data. For example, the prompt "What's the weather like tomorrow?" returns structured data containing weather forecast information.

[0841] Operation flow

[0842] The server collects and preprocesses log data from existing AI assistants. The preprocessed data is provided to the generative AI model, which then uses the data to learn. When a user inputs a question through their device, the generative AI model analyzes the question and identifies keywords. Based on the identified keywords, the generative AI model generates relevant structured data and returns it to the user. The user can then review the returned structured data to obtain the information they need.

[0843] In this way, the present invention can provide accurate and relevant responses to user questions, making information easier to obtain and use.

[0844] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0845] Step 1:

[0846] The server collects log data from existing artificial intelligence assistants.

[0847] Input: Log data from an existing AI assistant

[0848] Specific operation: The server executes the data collection script and periodically collects the log data of the artificial intelligence assistant.

[0849] Output: Collected log data

[0850] Step 2:

[0851] The server pre-processes the collected log data.

[0852] Input: Collected log data

[0853] Specific operations: The server runs data preprocessing scripts to remove unnecessary information, extract question-answer pairs, and normalize the data.

[0854] Output: Preprocessed log data

[0855] Step 3:

[0856] The server provides the preprocessed log data to the generative AI model.

[0857] Input: Preprocessed log data

[0858] Specific operation: The server calls the API and sends the preprocessed data to the generative AI model.

[0859] Output: The data provided to the generative AI model

[0860] Step 4:

[0861] Generative AI models learn using the data they are provided with.

[0862] Input: Data provided to a generative AI model

[0863] How it works: The generative AI model goes through a training process based on the data provided, and learns how to generate appropriate responses to user questions.

[0864] Output: A trained generative AI model

[0865] Step 5:

[0866] The user inputs questions into the generative AI model through the terminal.

[0867] Input: User question (prompt)

[0868] Specific operation: The user inputs a question into the input interface using a smartphone or PC.

[0869] Output: The question entered

[0870] Step 6:

[0871] The generative AI model receives a question from a user and analyzes the question to identify keywords.

[0872] Input: The question you entered

[0873] How it works: The generative AI model uses natural language processing algorithms to analyze the question and identify important keywords.

[0874] Output: Identified keywords

[0875] Step 7:

[0876] The generative AI model generates relevant structured data based on the identified keywords.

[0877] Input: Identified keyword

[0878] How it works: The generative AI model runs database queries to retrieve information related to the identified keywords and generates structured data.

[0879] Output: Generated structured data

[0880] Step 8:

[0881] The generative AI model returns the generated structured data to the user.

[0882] Input: Generated structured data

[0883] Specific operation: The generative AI model returns the generated structured data to the user through the output interface.

[0884] Output: Structured data returned to the user

[0885] Step 9:

[0886] The user checks the returned structured data and obtains the required information.

[0887] Input: Structured data returned to the user

[0888] Specific operation: The user checks the returned structured data through the output interface of the terminal.

[0889] Output: Information obtained by the user

[0890] (Application example 1)

[0891] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0892] Conventional self-driving vehicles lack the ability to provide real-time traffic information, making it difficult to ensure efficient and safe driving. Furthermore, there was no system that returned appropriate structured data in response to user questions, making it cumbersome to obtain information while driving.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0894] In this invention, the server includes means for training the generating AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for providing traffic information in real time to the autonomous vehicle, means for analyzing the user's question and generating relevant structured data, and means for improving driving efficiency and safety based on the generated structured data, thereby enabling the user to obtain appropriate traffic information in real time while driving and improving driving efficiency and safety.

[0895] "Generative AI" is artificial intelligence that learns from existing data and generates structured data that is easy to identify in response to user questions.

[0896] "Structured data" is data that is organized according to a specific format, making it easy to identify and analyze.

[0897] "Logs of existing AI assistants" are data that record the dialogue history and operation history of AI assistants that have been used in the past.

[0898] "Means for programming and taking action" refers to means for executing specific operations or processes based on the generated structured data.

[0899] The "means for regenerating a response sentence" is a means for creating a new response sentence based on the user's question and the generated structured data.

[0900] An "autonomous vehicle" is a vehicle that drives autonomously using artificial intelligence and sensor technology.

[0901] "Means for providing traffic information in real time" refers to means for instantly providing current traffic conditions, traffic signal information, etc.

[0902] "Means for analyzing user questions" refers to means for understanding questions from users and analyzing their intentions.

[0903] The "means for improving driving efficiency and safety" refers to means for improving driving efficiency and safety based on the generated structured data.

[0904] A system for implementing this invention uses a generative AI model to return appropriate structured data in real time in response to user questions, thereby improving the driving efficiency and safety of autonomous vehicles. Specific embodiments of this system are described below.

[0905] The server first learns from the logs of existing AI assistants. This allows the generation AI to acquire the ability to generate structured data that is easy to identify in response to user questions. For example, if a user asks, "Where's the next traffic light?", the generation AI will identify the keyword "traffic light" and generate structured data such as the distance to the next traffic light and the traffic light status (red, yellow, green).

[0906] This structured data can be programmed by the server and used to perform specific actions, such as sending upcoming traffic lights to the autonomous vehicle's control system to make appropriate speed adjustments.

[0907] Furthermore, the server regenerates a response based on the generated structured data and the user's question. This response is intended to provide the user with appropriate information. For example, a specific response such as "The next traffic light is red. The distance is 100 meters" is generated.

[0908] The system includes a means to provide real-time traffic information for autonomous vehicles, analyzing user questions and generating relevant structured data to improve driving efficiency and safety.

[0909] The hardware used includes an on-board computer of an autonomous vehicle with internet connectivity. The software uses Python and the OpenAI API. Specific examples of prompts include:

[0910] Example prompt sentence:

[0911] User Question: What's the next signal?

[0912] Please return the relevant structured data.

[0913] In this way, real-time traffic information assistants can support autonomous vehicle operation, improving efficiency and safety.

[0914] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0915] Step 1:

[0916] The server learns from the logs of existing AI assistants. Specifically, it analyzes past dialogue history and operation history and provides them as training data to the generative AI model. Through this learning, the generative AI model acquires the ability to generate structured data that is easy to identify in response to user questions. The input is the log data of the AI ​​assistant, and the output is the trained generative AI model.

[0917] Step 2:

[0918] A user inputs a question into an autonomous vehicle, for example, "What's the next traffic light?", either through voice input or a touch panel. The input is the user's question, and the output is the question text.

[0919] Step 3:

[0920] The server inputs the user's question as a prompt to the generative AI model. Specifically, it sends the question text to the generative AI model and instructs it to generate related structured data. The input is the user's question text, and the output is the generated structured data.

[0921] Step 4:

[0922] The server receives the generated structured data and performs programming actions, such as sending information about upcoming traffic lights to the autonomous vehicle's control system to adjust the vehicle's speed appropriately. The input is the generated structured data, and the output is instructions to the control system.

[0923] Step 5:

[0924] The server regenerates a response based on the generated structured data and the user's question. Specifically, it creates a response to provide appropriate information to the user. The input is the user's question and the generated structured data, and the output is the regenerated response.

[0925] Step 6:

[0926] The server provides the regenerated response sentence to the user. Specifically, it conveys information to the user through voice output or display. The input is the regenerated response sentence, and the output is information provided to the user.

[0927] In this way, real-time traffic information assistants can support autonomous vehicle operation, improving efficiency and safety.

[0928] Example 2

[0929] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0930] Conventional AI assistant systems often lacked efficient program generation and execution to provide appropriate responses to user requests. Furthermore, the processing required after receiving structured data was complex, resulting in delayed responses to users. Furthermore, information acquisition from external APIs was not performed smoothly, resulting in insufficient information provided to users.

[0931] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0932] In this invention, the server includes means for training the generation AI from logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and generating a program, means for executing the generated program to perform a specific action, means for analyzing a prompt using a generation AI model, means for generating a program based on the analysis results, means for executing the generated program to obtain information from an external API, and means for returning the obtained information to the user, thereby enabling a prompt and appropriate response to a user request to be provided.

[0933] "Generative AI" is a system that uses artificial intelligence technology to automatically generate programs and responses for specific tasks.

[0934] "Structured data" is data that is organized according to specific formats and rules, making it easy for generative AI to identify and process.

[0935] An "AI assistant" is a system equipped with artificial intelligence to provide information or perform specific tasks in response to user requests.

[0936] A "log" is data that records the operation of a system and the operation history of a user.

[0937] "Means for generating programs" refers to the ability to automatically create code or scripts to perform specific tasks using a generative AI model.

[0938] A "means for performing an action" is a function that executes a generated program to perform a specific task or operation.

[0939] A "generative AI model" is an algorithm or model that forms the core of generative AI, and is a technology for analyzing prompts and generating appropriate responses or programs.

[0940] A "prompt sentence" is text containing a request or question that a user enters into the system.

[0941] The "means of analysis" is the function that uses a generative AI model to understand the meaning of the prompt and determine the appropriate action.

[0942] An "external API" is an interface for accessing external services and databases to obtain information.

[0943] "Means for obtaining information" is a function that executes the generated program to obtain the necessary data from the external API.

[0944] The "means for regenerating a response sentence" is a function that combines the acquired information with the original question sentence to generate an appropriate response to the user.

[0945] MODE FOR CARRYING OUT THE INVENTION

[0946] This invention is a system that uses a generative AI model to generate an appropriate program based on a prompt sentence entered by a user, and then executes the program to perform a specific action. Specific embodiments of this system are described below.

[0947] System configuration

[0948] This system consists of three main elements: a server, a terminal, and a user. The server is responsible for analyzing prompts using a generative AI model, generating and executing programs. The terminal is responsible for sending prompts from the user to the server and displaying responses from the server to the user. The user inputs requests to the system through the terminal.

[0949] Hardware and software used

[0950] The server is a computer system equipped with a high-performance processor and large memory capacity, and is equipped with software for running a generative AI model (e.g., GPT-4). The terminal is a device operated by a user, such as a smartphone, tablet, or PC. The server and terminal communicate via the Internet.

[0951] Data processing and calculation

[0952] The server inputs the prompt received from the user into a generative AI model for analysis. The generative AI model understands the meaning of the prompt and generates a program to determine the appropriate action. The generated program is written in a programming language such as Python and calls a specific API (for example, the OpenWeatherMap API) to obtain the necessary information. The obtained information is then formatted as a response using the generative AI model again and sent to the device.

[0953] Specific examples

[0954] Let's take the example of a user typing "What's the weather today?" This prompt is sent from the device to the server. The server uses the generative AI model to analyze the prompt and identify the keyword "weather." The server then generates a Python script to obtain weather information and calls the OpenWeatherMap API to obtain the latest weather information. The obtained weather information is then formatted as a response using the generative AI model again and sent to the device. The device then displays the received weather information to the user.

[0955] Example prompt sentence:

[0956] "Tell me about today's weather."

[0957] In this way, the system can provide a fast and appropriate response to a user's request.

[0958] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0959] Step 1:

[0960] The user enters a prompt statement.

[0961] The user types "What's the weather like today?" into the input field of the terminal. This input is sent to the system as a prompt.

[0962] Input: The prompt text entered by the user: "What's the weather like today?"

[0963] Output: Data that the terminal sends to the server as a prompt

[0964] Step 2:

[0965] The terminal sends the prompt to the server.

[0966] The terminal sends the prompt text entered by the user to the server as an HTTP request. This communication takes place over the Internet.

[0967] Input: The prompt text entered by the user

[0968] Output: The prompt received by the server

[0969] Step 3:

[0970] The server analyzes the prompt using a generative AI model.

[0971] The server inputs the received prompt sentence into a generative AI model (e.g., GPT-4) for analysis. The generative AI model understands the meaning of the prompt sentence and determines the appropriate action.

[0972] Input: The prompt received by the server

[0973] Output: Analysis results (including the keyword "weather")

[0974] Step 4:

[0975] The server generates a program based on the analysis results.

[0976] The server generates a program to execute a specific action based on the analysis results of the generative AI model. For example, in response to the prompt "What's the weather today?", it generates a Python script to retrieve weather information.

[0977] Input: Analysis results of the generative AI model

[0978] Output: Generated Python script

[0979] Step 5:

[0980] The server runs the generated program.

[0981] The server runs the generated Python script and calls the OpenWeatherMap API to get the latest weather information.

[0982] Input: Generated Python script

[0983] Output: Weather information obtained from the OpenWeatherMap API

[0984] Step 6:

[0985] The server sends the execution results to the terminal.

[0986] The server then sends the acquired weather information to the terminal as an HTTP response. This communication also takes place over the Internet.

[0987] Input: Weather information obtained from the OpenWeatherMap API

[0988] Output: Weather information received by the device

[0989] Step 7:

[0990] The terminal displays the execution results to the user.

[0991] The terminal displays the weather information received from the server to the user, allowing the user to check the latest weather information.

[0992] Input: Weather information received from the server

[0993] Output: Weather information displayed to the user

[0994] (Application example 2)

[0995] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0996] Conventional AI assistant systems have had difficulty in quickly providing appropriate information in response to user questions. In particular, there is a need to obtain and provide updated information, such as traffic and weather information, in real time to users, but there has been a lack of efficient ways to achieve this.

[0997] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for training the generation AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for identifying specific keywords and acquiring corresponding information, and means for providing the acquired information to the user. This makes it possible to provide prompt and appropriate information in response to a user's question.

[0998] "Generative AI" is artificial intelligence that identifies specific keywords and takes appropriate actions based on them.

[0999] "Structured data" is data that is organized in a format that is easy for generating AI to identify.

[1000] "Existing AI assistant logs" are records of user questions and responses that have been handled by the AI ​​assistant in the past.

[1001] A "means for programming and taking action" is a method for receiving structured data and performing specific actions based thereon.

[1002] The "means for regenerating a response sentence" is a method for creating a new response sentence by combining the acquired information with the user's question.

[1003] "Means for identifying specific keywords" refers to a method for extracting important keywords from user input.

[1004] The "means for obtaining information" is a method for obtaining relevant and up-to-date information based on the identified keywords.

[1005] "Means of providing to users" refers to the method used to convey the acquired information to users.

[1006] The system for implementing this invention uses a generative AI model to identify specific keywords, obtain appropriate information based on the keywords, and provide it to users. A specific embodiment of this system is described below.

[1007] System Configuration

[1008] The system consists of three main elements: a server, a terminal, and a user.

[1009] server

[1010] The server includes the following means:

[1011] 1. A way to train the generative AI to return structured data that is easy to identify from the logs of existing AI assistants: A generative AI model is trained using records of past user questions and responses.

[1012] 2. Programmatic actionable use of structured data: Perform specific actions (e.g., API calls) based on identified keywords.

[1013] 3. A means of regenerating a response sentence by combining the results with the question: The acquired information is combined with the user's question to generate a new response sentence.

[1014] 4. Identifying specific keywords and retrieving corresponding information: Extracting important keywords from the user's input and retrieving related information based on them.

[1015] 5. Means of providing acquired information to users: Communicate acquired information to users.

[1016] Terminal

[1017] A terminal is a device such as a smartphone or a head-mounted display that provides an interface for users to interact with the system. The terminal sends user input to the server and displays responses from the server.

[1018] user

[1019] The user is the entity that utilizes the system to obtain information. The user inputs questions by voice or text and receives responses from the system.

[1020] Hardware and software used

[1021] Hardware: Smartphone, head-mounted display

[1022] Software: Python 3.x, requests library, JSON library

[1023] Processing flow

[1024] 1. The user inputs or speaks a specific keyword (e.g., "weather") through the device.

[1025] 2. The device sends the user's input to the server.

[1026] 3. The server uses the generative AI model to identify keywords and retrieve corresponding information.

[1027] 4. The server combines the information it has obtained with the user's question to regenerate a response.

[1028] 5. The server sends the regenerated response to the terminal.

[1029] 6. The terminal displays the response to the user.

[1030] Specific examples

[1031] When the user says "weather," the server retrieves the latest weather information for the user's current location and sends it to the device. The device then displays, "The current weather in Tokyo is sunny, with a temperature of 25 degrees."

[1032] When the user says "traffic information," the server retrieves the latest traffic conditions for the current location and sends them to the device. The device then displays the message, "Current traffic conditions in Tokyo are congested. An alternative route will be suggested."

[1033] Prompt Sentence Examples

[1034] User: "Weather"

[1035] Generative AI: "Getting current weather information..."

[1036] Application: "The current weather in Tokyo is sunny and the temperature is 25 degrees."

[1037] User: "Traffic Information"

[1038] Generator: "Getting current traffic information..."

[1039] Application: "Current traffic conditions in Tokyo are congested. We will suggest an alternative route."

[1040] In this way, users can quickly obtain the information they need in real time.

[1041] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1042] Step 1:

[1043] The user inputs or speaks a specific keyword (e.g., "weather") through the terminal. The input is in text or voice format, and the terminal sends it to the server. The input data is the user's question.

[1044] Step 2:

[1045] The terminal sends the user's input to the server. The input data is the user's question, and the output data is the question sent to the server. The terminal sends the data to the server via the network.

[1046] Step 3:

[1047] The server uses a generative AI model to identify keywords. The input data is the user's question, and the output data is the identified keywords. The server uses the generative AI model to extract important keywords from the user's question.

[1048] Step 4:

[1049] The server retrieves the corresponding information based on the identified keywords. The input data is the identified keywords, and the output data is the retrieved information. The server calls the API to retrieve the latest weather information, traffic information, etc.

[1050] Step 5:

[1051] The server combines the acquired information with the user's question to regenerate a response. The input data is the acquired information and the user's question, and the output data is the regenerated response. The server generates an appropriate response for the user based on the acquired information.

[1052] Step 6:

[1053] The server sends the regenerated response sentence to the terminal. The input data is the regenerated response sentence, and the output data is the response sentence sent to the terminal. The server sends the data to the terminal via the network.

[1054] Step 7:

[1055] The terminal displays the response to the user. The input data is the response sent from the server, and the output data is the response displayed to the user. The terminal displays the response on the screen and provides information to the user.

[1056] Example 3

[1057] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1058] Conventional AI assistant systems have a complex process for generating appropriate responses to user questions, and they face problems such as low efficiency, particularly when gathering information from external data sources and generating responses. It is also difficult to quickly obtain the appropriate data based on the user's question and provide an accurate response. This can lead to a poor user experience and a loss of system reliability.

[1059] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1060] In this invention, the server includes means for receiving a user's question, means for analyzing the question and extracting important keywords and intent, means for collecting necessary data from external data sources, means for generating a response sentence based on the collected data, and means for sending the generated response sentence to the user. This makes it possible to generate a response to the user's question quickly and accurately, improving the user experience.

[1061] The "means for receiving a user's question" is a function for transmitting a question input by a user through a terminal to a server.

[1062] "Means of analyzing questions and extracting important keywords and intent" is a function that uses a generative AI model to analyze the content of a user's question and identify the subject and purpose of the question.

[1063] "Means of collecting necessary data from external data sources" refers to a function that uses an API client to obtain necessary information from external data sources, such as weather information and reservation information.

[1064] "Means for generating responses based on collected data" refers to a function that uses a generative AI model based on acquired data to create appropriate responses to user questions.

[1065] The "means for transmitting the generated response sentence to the user" is a function for transmitting the generated response sentence from the server to the terminal and displaying it to the user.

[1066] "Weather information" is information that includes meteorological data and indicates the weather conditions in a particular area and at a particular time.

[1067] "Reservation information" refers to information relating to a reservation for a particular service or facility, including the date, time, location, and user details.

[1068] "Alarm setting information" is information about an alarm set by a user, including the alarm time and message content.

[1069] A "generative AI model" is a model that uses artificial intelligence to generate and analyze text, and is used to generate responses to user questions.

[1070] A "prompt" is text that is input to a generative AI model and contains instructions or questions that enable the model to generate an appropriate response.

[1071] The present invention relates to a system for generating a response to a user's question quickly and accurately. A specific embodiment of this system will be described below.

[1072] System configuration

[1073] This system mainly consists of three elements: a server, a terminal, and a user. The server is a computer equipped with a high-performance processor and is installed with software to run the generative AI model and database management system. The terminal is a device used by the user, such as a smartphone or PC.

[1074] Hardware and software used

[1075] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[1076] Software: Generative AI models (e.g., OpenAI's GPT-3), database management systems (e.g., MySQL), API clients (e.g., Python's requests library)

[1077] Program processing

[1078] When the server receives a user's question, it uses the generative AI model to analyze the question and extract important keywords and intent. Next, the server uses the API client to collect necessary data from external data sources (e.g., weather information APIs). Based on the collected data, the generative AI model generates an appropriate response. Finally, the generated response is sent to the device and displayed to the user.

[1079] Specific examples

[1080] For example, the following process occurs when a user asks, "What's the weather like tomorrow?"

[1081] 1. Receive a user's question: The user types "What's the weather going to be like tomorrow?" into their smartphone and presses the send button. The device then sends this question to the server.

[1082] 2. Question analysis: The server uses a generative AI model to extract the keyword "tomorrow's weather."

[1083] 3. Data collection: The server sends a request to the weather information API to get tomorrow's weather data. The API returns the data "Sunny."

[1084] 4. Generate a response: The server generates a response saying "Tomorrow's weather will be sunny."

[1085] 5. Sending the response: The server sends the generated response to the terminal, which displays it to the user.

[1086] Prompt Sentence Examples

[1087] An example of a prompt sentence to input to the generative AI model is as follows:

[1088] User asks: "What's the weather like tomorrow?"

[1089] Prompt the generative AI model: "Get the latest weather information and tell me what the weather will be like tomorrow."

[1090] This system allows users to obtain information quickly and accurately, improving the user experience. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1091] Step 1:

[1092] A user uses a terminal to input a question. For example, the user types "What's the weather like tomorrow?" The terminal sends this question to the server as an HTTP request. The input is the user's question, and the output is an HTTP request to the server.

[1093] Step 2:

[1094] The server inputs the received question into the generative AI model. The generative AI model analyzes the question and extracts important keywords and intent. Specifically, the keyword "tomorrow's weather" is extracted. The input is the user's question, and the output is the extracted keywords.

[1095] Step 3:

[1096] The server uses an API client to collect the required data from an external data source (e.g., a weather information API). The server requests information about "tomorrow's weather" from the weather information API and receives weather data as a response. The input is the extracted keywords, and the output is the retrieved weather data.

[1097] Step 4:

[1098] The server uses a generative AI model to generate an appropriate response based on the acquired weather data. For example, if the weather data is "sunny," it generates a response such as "Tomorrow's weather will be sunny." The input is the acquired weather data, and the output is the generated response.

[1099] Step 5:

[1100] The server sends the generated response to the terminal as an HTTP response. The terminal displays this response to the user. The user sees the response "Tomorrow's weather will be sunny" on the terminal screen. The input is the generated response, and the output is the response displayed to the user.

[1101] (Application example 3)

[1102] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1103] Conventional food delivery systems did not optimize delivery times and methods in response to weather changes, and failed to provide users with appropriate information. As a result, users often became dissatisfied with delivery delays and uncertainty. In addition, response messages generated using generative AI models were not sufficiently utilized, resulting in low quality information provided to users.

[1104] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1105] In this invention, the server includes: means for training the generating AI from the logs of existing AI assistants to return easily identifiable structured data; means for receiving the structured data and programming it to perform an action; means for combining the result with a question to regenerate a response; means for acquiring the latest weather information and optimizing delivery time and delivery method based on that information; and means for inputting a prompt using the generating AI model and providing an appropriate response to the user. This makes it possible to optimize delivery time and delivery method in response to changes in the weather, and to provide appropriate information to the user.

[1106] "Generative AI" is a technology that uses artificial intelligence to generate new information and responses from data.

[1107] "Structured data" is data that is organized according to specific formats and rules, making it easy for machines to understand and process.

[1108] An "AI assistant" is software or a device equipped with artificial intelligence to respond to user instructions and questions.

[1109] A "log" is data that records the operation history of a system or application.

[1110] "Programming" is the act of writing instructions for a computer to execute.

[1111] An "action" is an operation or process that a system performs based on specific conditions or instructions.

[1112] A "response" is a sentence that the system returns in response to a user's question or instruction.

[1113] "Weather information" refers to data related to the weather, such as the current weather, temperature, and precipitation.

[1114] "Delivery time" is the time it takes for a product or service to be delivered to the user.

[1115] "Delivery method" refers to the means or procedure for delivering a product or service to a user.

[1116] A "prompt" is a sentence of instructions or questions that is input into a generative AI model.

[1117] A system for implementing this invention includes a server, a user terminal, and a generative AI model. The server has a means for training the generative AI from logs of existing AI assistants so that it returns structured data that is easy to identify. This allows the server to obtain basic data for generating appropriate responses to user instructions and questions.

[1118] The server then has the means to program and take action on the structured data, allowing the server to perform specific actions or processes based on the structured data it has received.

[1119] Furthermore, the server has a means to regenerate a response by combining the result with the question, allowing the server to provide an appropriate response to the user.

[1120] The server also has a means of obtaining the latest weather information and optimizing delivery times and methods based on that information. This allows the server to adjust delivery times and methods according to changes in the weather, and provide accurate information to users.

[1121] Finally, the server has a means to input prompts using the generative AI model to provide appropriate responses to the user, allowing the server to generate responses in natural language to the user's questions and instructions.

[1122] As a specific example, when a user orders food delivery, the server uses the Weather API to obtain the latest weather information for the specified location. It then uses a generative AI model (OpenAI's GPT-3) to generate a response for the user. For example, if it is raining, the server generates a response such as, "The current weather is rainy. Delivery time may be longer than usual."

[1123] Example prompt sentence:

[1124] The current weather is rainy. Delivery times may be longer than usual.

[1125] The hardware used is a smartphone or tablet connected to the internet, and the software uses Python, the requests library, WeatherAPI, and OpenAI's GPT-3, allowing users to receive accurate information in real time and reducing uncertainty about delivery times and methods.

[1126] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1127] Step 1:

[1128] The server learns from the logs of existing AI assistants and returns structured data that is easy to identify to the generating AI.

[1129] Input: AI assistant log data

[1130] Output: Structured data

[1131] How it works: The server analyzes the log data and extracts patterns to respond to user questions and instructions, which then translates the data into a format that is easy for the generative AI to understand.

[1132] Step 2:

[1133] The server receives the structured data and performs programming actions based on it.

[1134] Input: Structured data

[1135] Output: Action result

[1136] Specific Action: The server performs a specific action or process based on the structured data, such as retrieving information from a database or sending a request to an external API as instructed by the user.

[1137] Step 3:

[1138] The server regenerates a response sentence by combining the action result and the question sentence.

[1139] Input: Action result, question

[1140] Output: Response

[1141] Specific operation: The server combines the action result with the user's question and uses a generative AI model to generate a response in natural language.

[1142] Step 4:

[1143] The server obtains the latest weather information and optimizes delivery times and methods based on that information.

[1144] Input: User's location

[1145] Output: Weather information, optimized delivery time and delivery method

[1146] Specific operation: The server uses the Weather API to obtain the latest weather information for the specified location and adjusts the delivery time and method based on that information.

[1147] Step 5:

[1148] The server uses a generative AI model to input prompt sentences and provide appropriate responses to the user.

[1149] Input: prompt statement

[1150] Output: Response to the user

[1151] Specific operation: The server inputs a prompt into a generative AI model (e.g., OpenAI's GPT-3) and generates a response in natural language for the user, such as "The current weather is rainy. Delivery times may be longer than usual."

[1152] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1153] "Example 1"

[1154] In one embodiment of the present invention, the generative AI learns from the logs of existing AI assistants and generates structured data that is easy to identify. This generative AI includes an emotion engine that recognizes the user's emotions. The emotion engine reads emotions from the user's tone of voice, facial expressions, and choice of words. For example, if the user feels angry, the emotion engine conveys that information to the generative AI.

[1155] "Example 2"

[1156] The generative AI receives information from the emotion engine and adjusts the generation of structured data and the execution of actions based on that information. For example, if the user feels angry, the generative AI takes that information into account to generate a more appropriate response.

[1157] In this case, the generative AI will use that information to generate a more positive response.

[1158] "Example 3"

[1159] The generative AI then combines the results with the question to regenerate a response. This regenerated response is adjusted appropriately based on the user's emotions. For example, if the user feels angry, the generative AI will take that information into account to generate a more polite response. Alternatively, if the user feels happy, the generative AI will use that information to generate a more positive response.

[1160] The processing flow of each embodiment will be described below.

[1161] "Example 1"

[1162] Step 1: The generative AI starts learning from the logs of existing AI assistants.

[1163] Step 2: Generation The AI ​​generates structured data that is easy to identify based on the results of its learning.

[1164] Step 3: The emotion engine reads the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1165] Step 4: The emotion engine conveys the emotional information it has read to the generation AI.

[1166] "Example 2"

[1167] Step 1: The generative AI receives information from the emotion engine.

[1168] Step 2: The generation AI adjusts the generation of structured data and execution of actions based on the information it receives.

[1169] Step 3: The generative AI generates an appropriate response based on the adjusted results.

[1170] "Example 3"

[1171] Step 1: The generative AI matches the structured data with the question.

[1172] Step 2: The generation AI regenerates the response sentence based on this information.

[1173] Step 3: The generation AI provides the regenerated response to the user.

[1174] Example 1

[1175] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1176] Conventional AI assistant systems sometimes lacked consistency in their responses to user questions, making it particularly difficult to respond in a way that took the user's emotions into account. Furthermore, the generation and use of structured data was inefficient, which sometimes resulted in delays in providing appropriate information to users. This resulted in a poor user experience, which was an issue.

[1177] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1178] In this invention, the server includes means for training the generation AI from logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for combining the result with a question to regenerate a response, means for transmitting a user's question from a terminal to the server, means for the server to analyze the question and pass it to a generation AI model, means for the generation AI model to generate structured data based on the question, means for the server to return the generated data to the terminal, means for the terminal to display the data to the user, means for recognizing the user's emotions, and means for generating a response based on the emotions. This enables a quick and appropriate response to a user's question and further enables the provision of a response that takes the user's emotions into consideration.

[1179] "Generative AI" is artificial intelligence that learns from the logs of existing AI assistants and generates structured data that is easy to identify in response to user questions.

[1180] "Structured data" is data that is organized according to a specific format or structure, and is generated by generative AI based on user questions.

[1181] The "server" is a computer system whose role is to receive the user's question, analyze it, and pass it on to the generative AI model.

[1182] A "terminal" is a device through which a user inputs a question and receives a response from the server, and includes smartphones, PCs, etc.

[1183] The "emotion engine" is software that reads emotions from the user's tone of voice, facial expressions, choice of words, etc., and conveys that information to the generative AI.

[1184] "Question analysis" is the process in which the server receives a user's question and extracts keywords and intent.

[1185] "Response regeneration" is the process in which a generative AI generates a new response based on structured data and the user's question.

[1186] "Emotion recognition" is the process by which the emotion engine analyzes the user's emotions and provides that information to the generative AI.

[1187] An "action" is a specific operation or process that the generative AI performs based on structured data.

[1188] "Data display" is the process by which the terminal visually or audibly presents data received from the server to the user.

[1189] The present invention is a system that uses a generative AI model to provide appropriate responses to user questions. This system is realized using hardware and software such as a server, a terminal, a generative AI model, and an emotion engine.

[1190] Hardware and Software Configuration

[1191] Server: A high-performance server (e.g., a virtual machine on a cloud service) is used to host the generative AI model. The server analyzes the user's question and passes it to the generative AI model.

[1192] Terminal: A device used by a user, such as a smartphone or PC. The terminal sends the user's questions to the server, receives responses from the server, and displays them.

[1193] Generative AI models: For example, using advanced natural language processing models such as GPT-4, generative AI models generate structured data based on questions passed to them by the server.

[1194] Emotion engine: For example, using emotion recognition software such as the Affectiva SDK, the emotion engine analyzes the user's tone of voice, facial expressions, and word choice to identify emotions.

[1195] Data processing and calculation

[1196] 1. Handling user questions

[1197] The user asks through the terminal, "What's the weather going to be like tomorrow?"

[1198] The terminal sends this question to the server.

[1199] The server analyzes the question and extracts the keyword "weather."

[1200] The analysis results are passed to the generative AI model.

[1201] 2. Generating structured data

[1202] The generative AI model generates related structured data based on the analysis results passed from the server, such as weather forecast data.

[1203] The generated data is returned to the server.

[1204] 3. Return and Display of Data

[1205] The server returns the generated structured data to the terminal.

[1206] The terminal displays the data to the user, for example, "Tomorrow will be sunny."

[1207] 4. Emotion recognition and response generation

[1208] When a user asks a question or makes a comment, the emotion engine analyzes the user's tone of voice, facial expressions, and word choice.

[1209] The emotion engine communicates the analysis results to the generative AI model.

[1210] The generative AI model generates a response that takes into account the user's emotions and returns it to the device via the server. For example, if the user is angry, it will generate a response such as "Please calm down."

[1211] Specific examples

[1212] Example question: A user asks, "What's the weather like tomorrow?"

[1213] The server identifies the keyword "weather" and generates and returns weather forecast data.

[1214] Emotion example: When a user is angry

[1215] The emotion engine recognizes the user's anger and conveys that information to the generative AI.

[1216] The generative AI generates a response that takes into consideration the user's emotions and returns it to the device.

[1217] Prompt Sentence Examples

[1218] "If a user asks, 'What's the weather going to be like tomorrow?' how does a generative AI respond?"

[1219] "If the emotion engine recognizes that the user is angry, how will the generative AI respond?"

[1220] The above is an embodiment of the present invention. This system enables quick and appropriate responses to user questions, and also provides responses that take into account the user's emotions.

[1221] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1222] Step 1:

[1223] The user enters a question.

[1224] A user uses a device (smartphone or PC) to input a question such as "What's the weather going to be like tomorrow?" The input is done through voice recognition or text input. The input data is voice data or text data.

[1225] Step 2:

[1226] The terminal sends a question to the server.

[1227] The terminal receives the user's question and sends it to the server via the Internet. At this time, the question is converted into an appropriate format (e.g., JSON). The input is the user's question data, and the output is the JSON-formatted data sent to the server.

[1228] Step 3:

[1229] The server analyzes the question and passes it to a generative AI model.

[1230] The server analyzes the received question and extracts keywords (e.g., "weather"). The analysis results are passed to the generative AI model. The input is the question data in JSON format, and the output is the analyzed keyword data.

[1231] Step 4:

[1232] A generative AI model generates structured data based on the question.

[1233] The generative AI model generates related structured data based on the analysis results passed from the server. For example, in response to the question "What's the weather like tomorrow?", it generates weather forecast data. The input is analyzed keyword data, and the output is structured weather forecast data.

[1234] Step 5:

[1235] The server returns the generated data to the terminal.

[1236] The server returns the structured data received from the generative AI model to the device, converting the data into a format that is easy for users to understand. The input is structured weather forecast data, and the output is user-friendly data sent to the device.

[1237] Step 6:

[1238] The terminal displays the data to the user.

[1239] The terminal displays data received from the server to the user, for example, displaying weather forecast information on a screen. The input is the user-friendly data received from the server, and the output is the information displayed to the user.

[1240] Step 7:

[1241] Recognize user emotions.

[1242] When a user asks a question or makes a comment, the device's built-in emotion engine analyzes the user's tone of voice, facial expressions, and word choice to recognize their emotions. The input is the user's voice and video data, and the output is analyzed emotional data.

[1243] Step 8:

[1244] Generate emotion-based responses.

[1245] The emotion engine passes the recognized emotion information to the generative AI model. The generative AI model generates a response that takes the user's emotions into consideration and returns it to the device via the server. For example, if the user is angry, it generates a response such as "Please calm down." The input is analyzed emotion data, and the output is emotion-conscious response data.

[1246] (Application example 1)

[1247] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1248] Conventional navigation systems for autonomous vehicles have difficulty providing appropriate responses to user questions, and in particular, have been unable to respond in a way that takes into account the user's emotions. This can lead to reduced user satisfaction and increased stress while driving.

[1249] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for training the generation AI from the log of an existing AI assistant so as to return easily identifiable structured data; means for receiving the structured data and programming it to perform an action; means for regenerating a response by combining the result with a question; means including an emotion engine that recognizes the user's emotions; and means for adjusting the response based on information from the emotion engine. This makes it possible to provide an appropriate response to a user's question and further respond in accordance with the user's emotions.

[1250] "Generative AI" is an artificial intelligence that learns from the logs of existing AI assistants and returns structured data that is easy to identify in response to user questions.

[1251] "Structured data" is data organized in a specific format that is generated by generative AI based on user questions.

[1252] The "emotion engine" is an engine that has the ability to read emotions from the user's tone of voice, facial expressions, choice of words, etc.

[1253] A "response sentence" is a sentence that the generation AI generates to respond to a user's question.

[1254] "Means for programming and taking action" refers to means for executing specific operations or processes based on the generated structured data.

[1255] The "means of regeneration" refers to the means by which the generative AI generates a new response sentence based on the structured data and the question sentence.

[1256] The "adjustment means" is a means for appropriately changing the response sentence in accordance with the user's emotions based on information from the emotion engine.

[1257] A system for implementing the present invention has the following configuration.

[1258] System Program

[1259] The system includes a generative AI, an emotion engine, a voice recognition module, a camera module, and a navigation module. The generative AI learns from the logs of existing AI assistants and returns structured data that is easy to identify in response to user questions. The emotion engine reads emotions from the user's tone of voice, facial expressions, and choice of words.

[1260] Hardware and Software

[1261] Hardware:

[1262] Microphone: Used to capture audio input.

[1263] Camera: Used to capture the user's facial expressions.

[1264] software:

[1265] speech_recognition library: Used to convert speech to text.

[1266] Transformers library pipeline: Used to perform emotion recognition.

[1267] cv2 (OpenCV): Used to acquire images from the camera.

[1268] Data processing and calculation

[1269] The server converts the voice data acquired from the microphone into text using a speech recognition module. The generation AI then analyzes this text and generates structured data that is easy to identify. The emotion engine reads emotions from the user's tone of voice and facial expressions and conveys this information to the generation AI. The generation AI then adjusts the response sentence based on the information from the emotion engine and provides an appropriate response to the user.

[1270] Specific examples

[1271] For example, if a user asks, "Where is the next gas station?", the speech recognition module converts this question into text. The generation AI identifies the keyword "gas station" and generates information about the nearest gas station as structured data. If the emotion engine determines the user's emotion as "neutral," the generation AI responds, "The nearest gas station is 5 km away."

[1272] Prompt Sentence Examples

[1273] If the user asks, "Where is the next gas station?" and the emotion recognition model determines the answer as "neutral," the system will respond, "The nearest gas station is 5 km away."

[1274] In this way, it is possible to provide appropriate responses to the user's questions and further to respond in accordance with the user's emotions.

[1275] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1276] Step 1:

[1277] The user asks a question into the microphone. The input is the user's voice data, and the output is the voice data passed to the speech recognition module.

[1278] Step 2:

[1279] The server's speech recognition module converts the speech data into text. The input is speech data and the output is text data. Specifically, the speech_recognition library is used to convert speech to text.

[1280] Step 3:

[1281] The server's generation AI analyzes the text data and generates structured data that is easy to identify. The input is text data, and the output is structured data. Specifically, the generation AI model extracts keywords and generates related information as structured data.

[1282] Step 4:

[1283] The server's emotion engine reads emotions from the user's tone of voice and facial expressions. The input is audio data and camera video data, and the output is emotional information. Specifically, emotion recognition uses a pipeline from the transformers library.

[1284] Step 5:

[1285] The server's generation AI adjusts the response sentence based on the emotional information from the emotion engine. The input is structured data and emotional information, and the output is the adjusted response sentence. Specifically, the generation AI generates the response sentence taking into account the emotional information.

[1286] Step 6:

[1287] The server returns the tailored response to the user. The input is the tailored response, and the output is a voice response to the user. Specifically, the server converts the text into speech and responds to the user through a speaker.

[1288] Example 2

[1289] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1290] Conventional AI assistant systems have difficulty generating responses that take the user's emotional state into account, making it difficult to provide appropriate responses to users. Furthermore, they have problems with not being able to efficiently execute actions using structured data or obtain up-to-date information. This has resulted in a poor user experience and limited the usefulness of the system.

[1291] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1292] In this invention, the server includes a means for training the generation AI to return easily identifiable structured data from the logs of existing AI assistants, a means for receiving the structured data and programming it to perform an action, and a means for receiving information from the emotion engine and adjusting the generation of structured data and the execution of actions based on that information. This makes it possible to generate appropriate responses that take the user's emotional state into consideration and efficiently obtain up-to-date information.

[1293] "Generative AI" is an artificial intelligence technology that analyzes input data from users and generates appropriate responses.

[1294] "Structured data" is data that is organized according to a specific format, making it easy for generative AI to identify and process.

[1295] An "AI assistant" is an artificial intelligence system that generates responses to user input and automates various tasks.

[1296] A "log" is data that records the operation of a system and the operation history of a user.

[1297] "Programming" is the process of giving a computer instructions to perform a particular task.

[1298] An "action" is a specific operation or process that a system performs in response to a specific input.

[1299] An "emotion engine" is a technology that analyzes a user's emotional state and provides that information.

[1300] A "response sentence" is a reply message generated by the generation AI in response to user input.

[1301] "Updateable information" is data that changes over time and with the situation, and is information that needs to reflect the latest state.

[1302] This invention is a system for analyzing user input data and generating appropriate responses. The system is realized using a generative AI model, an emotion engine, structured data, and an AI assistant log.

[1303] The server first generates a program using the generative AI model. This program receives input data from the user and executes appropriate actions. Specifically, the server uses the following hardware and software:

[1304] Hardware:

[1305] A server (a computer with a powerful processor and sufficient memory)

[1306] User device (smartphone, PC, etc.)

[1307] software:

[1308] Generative AI models (AI models for natural language processing)

[1309] Emotion engine (software for analyzing the user's emotional state)

[1310] External APIs (e.g., APIs for obtaining weather information)

[1311] The server receives and analyzes input data from the user. During the analysis, the server extracts keywords from the input data. For example, if the user inputs "What's the weather like today?", the server identifies the keyword "weather."

[1312] The server then determines the corresponding action based on the identified keywords, for example, if the keyword "weather" is identified, the server will call an external weather API to get the latest weather information.

[1313] Additionally, the server receives information from the emotion engine. This information includes the user's emotional state (e.g., anger, joy). The server uses the generative AI model to generate a response based on the information from the emotion engine. For example, if the user feels angry, the generative AI takes that information into account to generate a more appropriate response. Alternatively, if the user feels happy, the generative AI uses that information to generate a more positive response.

[1314] Finally, the server returns the generated response to the user, for example, "Today's weather is sunny. The maximum temperature is 25 degrees and the minimum temperature is 15 degrees."

[1315] Examples:

[1316] The specific behavior when a user inputs "What's the weather like today?" is explained below.

[1317] 1. A user uses a smartphone to type "What's the weather like today?" and presses the send button.

[1318] 2. The server receives the text data "What's the weather like today?" sent by the user.

[1319] 3. The server analyzes the received text data and extracts the keyword "weather."

[1320] 4. The server calls an external weather API based on the keyword "weather" to obtain the latest weather information.

[1321] 5. The server receives the information from the emotion engine and confirms the user's emotional state.

[1322] 6. Based on the information from the emotion engine, the server uses a generative AI model to generate a response: "Today's weather is sunny. The maximum temperature is 25 degrees and the minimum temperature is 15 degrees."

[1323] 7. The server returns the generated response to the user. The user's smartphone displays a message saying, "Today's weather is sunny. The maximum temperature is 25 degrees, and the minimum temperature is 15 degrees."

[1324] In this way, the server analyzes the input data from the user, executes appropriate actions, generates a response based on the information in the emotion engine, and sends it back to the user.

[1325] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1326] Step 1:

[1327] The user submits the input data.

[1328] A user uses a terminal (e.g., a smartphone or PC) to send input data in text format to the system. For example, the user enters "What's the weather like today?" and presses the send button. The input data is sent to the server in text format.

[1329] Step 2:

[1330] The server receives the input data.

[1331] The server receives the input data sent by the user. The received data is saved in text format. The input data is "What's the weather like today?" and is saved on the server.

[1332] Step 3:

[1333] The server analyzes the input data and extracts keywords.

[1334] The server analyzes the received input data and extracts keywords. Specifically, it analyzes the text data using natural language processing technology and identifies the keyword "weather." The input is the text data "What's the weather like today?" and the output is the keyword "weather."

[1335] Step 4:

[1336] The server determines the corresponding action based on the extracted keywords.

[1337] The server determines the corresponding action based on the identified keyword. For example, if the keyword "weather" is identified, the server calls an external weather API to obtain the latest weather information. The input is the keyword "weather" and the output is the latest weather information obtained from the weather API.

[1338] Step 5:

[1339] The server receives the information from the emotion engine.

[1340] The server receives information from the emotion engine, which includes the user's emotional state (e.g., anger, joy). The input is data about the user's emotional state, and the output is the analysis result from the emotion engine.

[1341] Step 6:

[1342] The server generates a response using a generative AI model based on information from the emotion engine.

[1343] The server generates a response using a generative AI model based on information from the emotion engine. For example, if the user feels angry, the generative AI takes that information into account and generates a more appropriate response. If the user feels happy, the generative AI uses that information to generate a more positive response. The input is the emotion engine's analysis results and weather information, and the output is the generated response.

[1344] Step 7:

[1345] The server generates a response and sends it back to the user.

[1346] The server returns the generated response to the user. For example, it might reply, "Today's weather is sunny. The maximum temperature is 25 degrees and the minimum temperature is 15 degrees." The input is the generated response sentence, and the output is the message displayed on the user's terminal.

[1347] (Application example 2)

[1348] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1349] Conventional AI assistant systems have difficulty generating responses that reflect the user's emotions, resulting in a poor user experience. Furthermore, when providing information such as news articles, the system was unable to adjust the content or tone to reflect the user's emotions, making it difficult to increase user satisfaction. This can lead to stress and dissatisfaction with the way information is received.

[1350] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1351] In this invention, the server includes a means for training the generation AI to return easily identifiable structured data from the logs of existing AI assistants, a means for receiving the structured data and programming it to perform an action, and a means for receiving information from the emotion analysis engine and adjusting the response based on the user's emotions, thereby making it possible to adjust the appropriate response and the content and tone of the news article display according to the user's emotions.

[1352] "Generative AI" is artificial intelligence that analyzes input from users and generates appropriate responses and actions.

[1353] "Structured data" is data that is organized in a format that is easy for generating AI to identify.

[1354] An "emotion analysis engine" is a system that analyzes emotions from user input and behavior and provides the results.

[1355] "News article information" is data about the latest news and articles.

[1356] A "response sentence" is a reply sentence that the generation AI generates in response to a user's question or request.

[1357] "Tone" refers to the style and atmosphere of news articles and responses.

[1358] "User emotion" is the emotional state that a user expresses through their input and actions.

[1359] The system for implementing the present invention uses the following hardware and software to adjust responses and news article display content according to the user's emotions.

[1360] Hardware and software used

[1361] Hardware: Smartphone

[1362] Software: Python, EmotionEngine (emotion analysis library), NewsAPI (news acquisition API)

[1363] System configuration

[1364] 1. Generative AI: This is artificial intelligence that analyzes user input and generates appropriate responses and actions.

[1365] 2. Structured data: Data that is organized in a format that is easy for generative AI to identify.

[1366] 3. Sentiment analysis engine: A system that analyzes emotions from user input and behavior and provides the results.

[1367] 4. News Article Information: Data about the latest news and articles.

[1368] 5. Response sentence: A response sentence generated by the generative AI in response to a user's question or request.

[1369] 6. Tone: This refers to the tone and atmosphere of a news article or response.

[1370] 7. User Emotion: The emotional state that a user displays through their input and actions.

[1371] System Operation

[1372] The server first trains the generating AI to return structured data that is easy to identify from the logs of existing AI assistants. It then programs and takes action based on the structured data. It also receives information from a sentiment analysis engine and adjusts responses based on the user's emotions. This makes it possible to tailor appropriate responses and the content and tone of news articles to reflect the user's emotions.

[1373] Specific examples

[1374] For example, if a user types "I'm so tired today," the sentiment analysis engine will interpret the emotion as "tired" and display relaxing news (for example, articles about the beauty of nature and relaxation).

[1375] Prompt Sentence Examples

[1376] If a user types "I'm so tired today," the sentiment engine should parse the emotion "tired" and display news that will help them relax.

[1377] In this way, news can be delivered according to the user's emotions, providing a more personalized experience.

[1378] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1379] Step 1:

[1380] A user uses a smartphone to send input text containing emotions to an application.

[1381] Input: User input text (e.g. "I'm very tired today")

[1382] Output: The user's input text is sent to the server

[1383] Step 2:

[1384] The server sends the received user input text to an emotion analysis engine for emotion analysis.

[1385] Input: User-entered text

[1386] Data processing: A sentiment analysis engine analyzes the text to identify the emotional state (e.g., "tired")

[1387] Output: Sentiment analysis result (e.g. "Tired")

[1388] Step 3:

[1389] Based on the emotion analysis results, the server generates an appropriate prompt sentence for the generation AI and sends it as structured data.

[1390] Input: Sentiment analysis result (e.g. "Tired")

[1391] Data processing: Generative AI generates prompts based on the results of sentiment analysis (e.g., "Show me some relaxing news").

[1392] Output: Prompt sentence (e.g., "Show me some relaxing news.")

[1393] Step 4:

[1394] The server uses the generated prompt sentence to send a request to the news API to retrieve the appropriate news article.

[1395] Input: Prompt sentence (e.g., "Show me some relaxing news")

[1396] Data processing: News API searches and retrieves news articles based on prompts

[1397] Output: News article information (e.g., a list of relaxing news articles)

[1398] Step 5:

[1399] The server reconstructs the acquired news article information in a tone that corresponds to the user's emotions and generates a response sentence.

[1400] Input: News article information, sentiment analysis results

[1401] Data processing: Reconstructing news article information based on the results of sentiment analysis and generating a response (e.g., "A list of relaxing news articles")

[1402] Output: Response (e.g., "A list of relaxing news articles")

[1403] Step 6:

[1404] The server sends the generated response text to the user's smartphone and displays it.

[1405] Input: A response sentence (e.g., "A list of relaxing news articles")

[1406] Output: The response is displayed on the user's smartphone.

[1407] In this way, news articles can be provided that correspond to the user's emotions, providing a more personalized experience.

[1408] Example 3

[1409] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1410] Conventional AI assistant systems have had difficulty generating appropriate responses to user questions, especially providing responses that reflect the user's emotions. Acquiring up-to-date information in real time and generating responses based on that information has also been a challenge.

[1411] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1412] In this invention, the server includes means for receiving a question from a user, means for analyzing the question and acquiring necessary information, means for combining the acquired information and the question to input a prompt sentence to the generative AI model, means for the generative AI model to generate a response sentence, means for displaying the generated response sentence to the user, and means for adjusting the response sentence according to the user's emotions, thereby making it possible to provide an appropriate response to the user's question that is appropriate for the user's emotions.

[1413] "Means for receiving questions from users" refers to an interface or function that allows the system to receive questions entered by users.

[1414] "Means for analyzing questions and obtaining necessary information" refers to the function for analyzing received questions and obtaining necessary information based on their content from external data sources or APIs.

[1415] "Means for inputting a prompt sentence into a generative AI model" refers to a function for combining acquired information with a user's question to create a prompt sentence to be input into a generative AI model, and inputting that prompt sentence into the generative AI model.

[1416] "Means by which the generative AI model generates a response sentence" refers to the function by which the generative AI model generates a response sentence based on a prompt sentence.

[1417] "Means for displaying the generated response sentence to the user" refers to an interface or function for displaying the response sentence generated by the generative AI model to the user.

[1418] "Means for adjusting a response sentence according to the user's emotions" refers to a function for analyzing the user's emotions and appropriately adjusting the generated response sentence according to those emotions.

[1419] "Updateable information" refers to information that changes depending on the time and situation, and includes, for example, weather information and reservation information.

[1420] This invention is a system for generating appropriate responses to user questions. The system includes a series of processes: receiving a question from a user, analyzing it, acquiring necessary information, generating a response using a generative AI model, and displaying it to the user.

[1421] Hardware and software used

[1422] Device: The device through which the user enters their question, such as a smartphone, computer, or tablet.

[1423] Server: A central processing unit that analyzes questions, obtains necessary information, and generates responses using generative AI models.

[1424] Generative AI models: For example, natural language processing models such as GPT-3.

[1425] Weather API: External data sources, for example OpenWeatherMap.

[1426] Data processing and calculation

[1427] 1. Receive user questions

[1428] A user types a question into an input field on the device, for example, "What's the weather like tomorrow?"

[1429] 2. Send the question to the server

[1430] The terminal receives the user's input and sends the data to the server.

[1431] 3. Analyze the question and get the necessary information

[1432] The server analyzes the received question and extracts the keyword "weather." It then sends a request to the weather information API to retrieve the data "Tomorrow's weather will be sunny."

[1433] 4. Enter a prompt into the generative AI model

[1434] The server combines the information it has acquired with the user's question to create a prompt. For example, the following prompt is input into the generative AI model: "The user asked, 'What's the weather going to be like tomorrow?' The weather will be sunny tomorrow. Please generate an appropriate response."

[1435] 5. The generative AI model generates a response sentence

[1436] The generative AI model generates a response based on the prompt, for example, "The weather will be fine tomorrow. We apologize for any inconvenience."

[1437] 6. Display the generated response to the user

[1438] The server sends the generated response to the terminal, which displays it to the user.

[1439] Specific examples

[1440] User asks: "What's the weather like tomorrow?"

[1441] Example prompt: "The user asks, 'What's the weather going to be like tomorrow?' The weather will be sunny tomorrow. Generate an appropriate response."

[1442] Generated response: "The weather will be fine tomorrow. We apologize for the inconvenience."

[1443] This system allows users to receive appropriate responses based on information updated in real time. Furthermore, by using a generative AI model, it is possible to generate responses that correspond to the user's emotions. The flow of the identification process in Example 3 will be described with reference to FIG. 21.

[1444] Step 1:

[1445] The user enters a question.

[1446] The user inputs a question into the input field of the device. For example, "What's the weather going to be like tomorrow?" The input question is saved as text data on the device.

[1447] Step 2:

[1448] The terminal sends a question to the server.

[1449] The terminal receives the user's input and sends the text data to the server. The data sent is the text "What's the weather like tomorrow?" The server receives this data.

[1450] Step 3:

[1451] The server analyzes the question and retrieves the required information.

[1452] The server analyzes the received text data and extracts the keyword "weather." It then sends a request to the weather information API and retrieves the data "Tomorrow's weather will be sunny." The input is the question "What's the weather like tomorrow?" and the output is the information "Tomorrow's weather will be sunny."

[1453] Step 4:

[1454] The server inputs a prompt sentence into the generative AI model.

[1455] The server creates a prompt by combining the information it has acquired with the user's question. For example, it generates a prompt like this: "The user asked, 'What's the weather going to be tomorrow?' The weather will be sunny tomorrow. Please generate an appropriate response." The input is the question "What's the weather going to be tomorrow?" and the information "The weather will be sunny tomorrow," and the output is the prompt.

[1456] Step 5:

[1457] The generative AI model generates a response sentence.

[1458] The server inputs a prompt sentence into the generative AI model, which then generates a response sentence. For example, it generates a response sentence such as, "The weather will be sunny tomorrow. We apologize for any inconvenience." The input is the prompt sentence, and the output is the response sentence.

[1459] Step 6:

[1460] The server sends the generated response to the terminal.

[1461] The server sends the generated response to the terminal. The data sent is the text "Tomorrow's weather will be fine. We apologize for any inconvenience." The terminal receives this data.

[1462] Step 7:

[1463] The terminal displays the response to the user.

[1464] The terminal displays the received response to the user. The displayed text is "Tomorrow's weather will be fine. We apologize for the inconvenience." The user confirms this.

[1465] (Application example 3)

[1466] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1467] Conventional response systems using generative AI have difficulty generating appropriate responses based on the user's emotions. In addition, responses to user questions are uniform, making it difficult to improve user satisfaction. In particular, online shopping sites require customer support that responds to the user's emotions, and technology to achieve this is needed.

[1468] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1469] In this invention, the server includes means for training the generation AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for analyzing the user's emotions, and means for adjusting the generated response in accordance with the user's emotions. This makes it possible to generate an appropriate response in accordance with the user's emotions and improve user satisfaction.

[1470] "Generative AI" is artificial intelligence that generates natural language responses based on user input.

[1471] "Structured data" is data that is organized in a specific format that can be easily identified by the generating AI.

[1472] "Programming" is the act of writing instructions for a computer to perform specific actions.

[1473] An "action" is a specific operation or process that is executed by a program.

[1474] A "question" is a query that a user enters into the generating AI.

[1475] A "response sentence" is a sentence that the generation AI generates in response to a user's question.

[1476] A "means for analyzing emotions" is a technique for identifying emotions from user input and evaluating those emotions.

[1477] The "adjustment means" is a technique for appropriately changing the generated response sentence according to the user's emotions.

[1478] A system for implementing this invention uses a generative AI model to generate responses to user questions and adjusts the responses according to the user's emotions. Specific embodiments of this system are described below.

[1479] System configuration

[1480] The system consists of the following main components:

[1481] 1. Server: Hosts the generative AI model and processes user input.

[1482] 2. User device: A device such as a smartphone or computer through which the user enters their question.

[1483] 3. Sentiment Analysis Module: Software for analyzing emotions from user input.

[1484] 4. Response adjustment module: Software for adjusting the generated response sentences according to the user's emotions.

[1485] Program processing

[1486] The server performs the process in the following procedure.

[1487] 1. Receiving user input: Receives a question from the user's device.

[1488] 2. Sentiment Analysis: Use the sentiment analysis module to analyze the sentiment of the user's question. For sentiment analysis, use libraries such as NLTK or TextBlob.

[1489] 3. Prompt generation: Generate a prompt that includes the user's question and sentiment.

[1490] 4. Response generation: Use a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on the prompt.

[1491] 5. Response Adjustment: The response adjustment module is used to adjust the generated response according to the user's emotions.

[1492] 6. Send response: Send the adjusted response to the user terminal.

[1493] Hardware and software used

[1494] Generative AI models: Natural language generation models such as OpenAI's GPT-3.

[1495] Sentiment analysis libraries: NLTK, TextBlob, etc.

[1496] User device: smartphone, computer, etc.

[1497] Server: Cloud server (e.g. AWS, Google Cloud, etc.).

[1498] Specific examples

[1499] Below are some examples of specific prompt sentences.

[1500] Example prompt sentence:

[1501] User Question: When will this item arrive?

[1502] Emotion: Anger

[1503] Generate an appropriate response.

[1504] Based on this prompt, the generative AI model generates a response like this:

[1505] Response: We apologize for the wait. Your order will usually arrive within 3-5 business days. If you have any other questions, please let us know.

[1506] In this way, an appropriate response can be provided depending on the user's emotions.

[1507] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1508] Step 1:

[1509] Receiving user input

[1510] The user inputs a question using a device (smartphone or PC), which then sends the question to the server.

[1511] Input: User's question

[1512] Output: The question sent to the server

[1513] Step 2:

[1514] sentiment analysis

[1515] The server passes the received question to a sentiment analysis module, which uses libraries such as NLTK and TextBlob to analyze the user's sentiment from the question.

[1516] Input: User's question

[1517] Output: User's emotion (e.g. anger, joy, sadness, etc.)

[1518] Step 3:

[1519] Generate prompt statement

[1520] The server combines the user's question and emotion to generate a prompt, which is then passed to a generative AI model.

[1521] Input: User's question, user's sentiment

[1522] Output: prompt statement

[1523] Step 4:

[1524] Generate a response

[1525] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on the prompt. The generative AI model receives the prompt as input and outputs an appropriate response.

[1526] Input: prompt statement

[1527] Output: Response

[1528] Step 5:

[1529] Adjusting the response

[1530] The server then passes the generated response to the response adjustment module, which then adjusts the response appropriately based on the user's emotions. For example, if the user is angry, the response may be changed to a more polite expression.

[1531] Input: Response sentence, user sentiment

[1532] Output: Adjusted response

[1533] Step 6:

[1534] Sending a response

[1535] The server sends the adjusted response to the user's terminal, where the user can check the response.

[1536] Input: Adjusted response

[1537] Output: Response text displayed on the user's terminal

[1538] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1540] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1541] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1542] [Third embodiment]

[1543] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1544] .

[1545] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1546] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1547] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1548] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1549] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1550] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1551] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1552] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1553] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1554] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1555] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1556] "Example 1"

[1557] In one embodiment of the present invention, the generation AI learns from the logs of existing AI assistants and returns structured data that is easy to identify. For example, if a user asks, "What's the weather like tomorrow?", the generation AI will identify the keyword "weather" and generate relevant structured data.

[1558] "Example 2"

[1559] Next, we will explain how the AI ​​receives structured data, programs it uses, and performs actions. For example, if the AI ​​recognizes the keyword "weather," the corresponding program will run and retrieve the latest weather information.

[1560] "Example 3"

[1561] Finally, we will explain how the results are combined with the question to regenerate a response. For example, after obtaining the latest weather information, a response such as "Tomorrow's weather will be sunny" is generated based on that information. This makes it possible to provide an appropriate response to the user.

[1562] The processing flow of each embodiment will be described below.

[1563] "Example 1"

[1564] Step 1: The generative AI begins learning from the logs of existing AI assistants. Through this learning, the generative AI acquires the ability to generate appropriate responses to user questions.

[1565] Step 2: When a user asks a question, the Generative AI analyzes the question and generates structured data that is easy to identify. For example, in response to the question "What's the weather like tomorrow?", it identifies the keyword "weather" and generates related structured data.

[1566] "Example 2"

[1567] Step 1: The generative AI receives the identified structured data and runs a corresponding program. This program performs a specific action based on the structured data. Step 2: For example, if the keyword "weather" is identified, the program performs an action to obtain the latest weather information. This action allows the user to obtain the information they are looking for.

[1568] "Example 3"

[1569] Step 1: Combine the acquired information with the original question to generate a new response. In this step, the generative AI uses the knowledge it has learned to generate an appropriate response.

[1570] Step 2: For example, after obtaining the latest weather information, generate a response such as "Tomorrow's weather will be sunny." This makes it possible to provide an appropriate response to the user.

[1571] Example 1

[1572] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1573] Conventional AI assistants often lack the training data to provide appropriate responses to user questions, resulting in low accuracy and relevance of responses. Furthermore, responses to user questions are not structured, making it difficult to obtain and use information. Furthermore, there is also the issue of difficulty in providing up-to-date information appropriately.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1575] In this invention, the server includes means for collecting and preprocessing log data of existing AI assistants, means for training a generative AI model using the preprocessed log data, means for receiving questions from users and analyzing the questions to identify keywords, means for generating relevant structured data based on the identified keywords, and means for returning the generated structured data to the user. This makes it possible to provide highly accurate and relevant responses to user questions. Furthermore, returning structured data makes it easier to obtain and use information, and allows for appropriate provision of up-to-date information.

[1576] "Log data of existing AI assistants" refers to data that records questions asked by users to AI assistants and the AI ​​assistant's responses to those questions.

[1577] "Preprocessing" is the process of removing unnecessary information from the collected log data, extracting question-answer pairs, and normalizing the data.

[1578] A "generative artificial intelligence model" is an artificial intelligence model that has the ability to learn from provided data and generate appropriate responses to user questions.

[1579] A "user question" is an inquiry or request that a user inputs to an artificial intelligence system.

[1580] "Keyword identification" means analyzing a user's question and extracting important words and phrases from it.

[1581] "Structured data" is data that is organized according to a specific format or structure, making the information easy to retrieve and use.

[1582] "Updateable information" refers to information whose content changes over time or with changes in circumstances, and includes, for example, weather information and reservation information.

[1583] "Providing a response" means returning appropriate information or answers to a user's questions.

[1584] This invention is a system that collects log data from existing AI assistants, uses the collected data to train a generative AI model, and returns appropriate structured data in response to user questions. Specific embodiments of this system are described below.

[1585] Hardware and software used

[1586] server

[1587] The server is hardware that collects and preprocesses log data from existing AI assistants. The server uses the following software:

[1588] Data collection script: A script to periodically collect log data from your existing artificial intelligence assistant.

[1589] Data preprocessing scripts: Scripts to remove unnecessary information from the collected log data, extract question-answer pairs, and normalize the data.

[1590] API call: An API to provide pre-processed data to the generative artificial intelligence model.

[1591] Generative AI Models

[1592] A generative AI model is software that uses data provided by a server to learn and generate appropriate responses to user questions. Specifically, it uses the following algorithm:

[1593] Natural language processing algorithms: Algorithms for analyzing user questions and identifying keywords.

[1594] Database query: A query to generate relevant structured data based on the identified keywords.

[1595] User terminal

[1596] The user terminal is the hardware that allows the user to input questions to the generative AI model and receive responses. Specifically, it uses the following interface:

[1597] Input interface: The interface through which the user enters their question.

[1598] Output interface: An interface for displaying the structured data returned by the generative AI model.

[1599] Specific examples

[1600] Prompt Sentence Examples

[1601] Below are examples of prompts that users can use to input questions to the generative AI model.

[1602] "What's the weather like tomorrow?"

[1603] "What events are happening this weekend?"

[1604] "Tell me the latest news"

[1605] For these prompts, the generative AI model identifies appropriate keywords and generates and returns relevant structured data. For example, the prompt "What's the weather like tomorrow?" returns structured data containing weather forecast information.

[1606] Operation flow

[1607] The server collects and preprocesses log data from existing AI assistants. The preprocessed data is provided to the generative AI model, which then uses the data to learn. When a user inputs a question through their device, the generative AI model analyzes the question and identifies keywords. Based on the identified keywords, the generative AI model generates relevant structured data and returns it to the user. The user can then review the returned structured data to obtain the information they need.

[1608] In this way, the present invention can provide accurate and relevant responses to user questions, making information easier to obtain and use.

[1609] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1610] Step 1:

[1611] The server collects log data from existing artificial intelligence assistants.

[1612] Input: Log data from an existing AI assistant

[1613] Specific operation: The server executes the data collection script and periodically collects the log data of the artificial intelligence assistant.

[1614] Output: Collected log data

[1615] Step 2:

[1616] The server pre-processes the collected log data.

[1617] Input: Collected log data

[1618] Specific operations: The server runs data preprocessing scripts to remove unnecessary information, extract question-answer pairs, and normalize the data.

[1619] Output: Preprocessed log data

[1620] Step 3:

[1621] The server provides the preprocessed log data to the generative AI model.

[1622] Input: Preprocessed log data

[1623] Specific operation: The server calls the API and sends the preprocessed data to the generative AI model.

[1624] Output: The data provided to the generative AI model

[1625] Step 4:

[1626] Generative AI models learn using the data they are provided with.

[1627] Input: Data provided to a generative AI model

[1628] How it works: The generative AI model goes through a training process based on the data provided, and learns how to generate appropriate responses to user questions.

[1629] Output: A trained generative AI model

[1630] Step 5:

[1631] The user inputs questions into the generative AI model through the terminal.

[1632] Input: User question (prompt)

[1633] Specific operation: The user inputs a question into the input interface using a smartphone or PC.

[1634] Output: The question entered

[1635] Step 6:

[1636] The generative AI model receives a question from a user and analyzes the question to identify keywords.

[1637] Input: The question you entered

[1638] How it works: The generative AI model uses natural language processing algorithms to analyze the question and identify important keywords.

[1639] Output: Identified keywords

[1640] Step 7:

[1641] The generative AI model generates relevant structured data based on the identified keywords.

[1642] Input: Identified keyword

[1643] How it works: The generative AI model runs database queries to retrieve information related to the identified keywords and generates structured data.

[1644] Output: Generated structured data

[1645] Step 8:

[1646] The generative AI model returns the generated structured data to the user.

[1647] Input: Generated structured data

[1648] Specific operation: The generative AI model returns the generated structured data to the user through the output interface.

[1649] Output: Structured data returned to the user

[1650] Step 9:

[1651] The user checks the returned structured data and obtains the required information.

[1652] Input: Structured data returned to the user

[1653] Specific operation: The user checks the returned structured data through the output interface of the terminal.

[1654] Output: Information obtained by the user

[1655] (Application example 1)

[1656] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1657] Conventional self-driving vehicles lack the ability to provide real-time traffic information, making it difficult to ensure efficient and safe driving. Furthermore, there was no system that returned appropriate structured data in response to user questions, making it cumbersome to obtain information while driving.

[1658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1659] In this invention, the server includes means for training the generating AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for providing traffic information in real time to the autonomous vehicle, means for analyzing the user's question and generating relevant structured data, and means for improving driving efficiency and safety based on the generated structured data, thereby enabling the user to obtain appropriate traffic information in real time while driving and improving driving efficiency and safety.

[1660] "Generative AI" is artificial intelligence that learns from existing data and generates structured data that is easy to identify in response to user questions.

[1661] "Structured data" is data that is organized according to a specific format, making it easy to identify and analyze.

[1662] "Logs of existing AI assistants" are data that record the dialogue history and operation history of AI assistants that have been used in the past.

[1663] "Means for programming and taking action" refers to means for executing specific operations or processes based on the generated structured data.

[1664] The "means for regenerating a response sentence" is a means for creating a new response sentence based on the user's question and the generated structured data.

[1665] An "autonomous vehicle" is a vehicle that drives autonomously using artificial intelligence and sensor technology.

[1666] "Means for providing traffic information in real time" refers to means for instantly providing current traffic conditions, traffic signal information, etc.

[1667] "Means for analyzing user questions" refers to means for understanding questions from users and analyzing their intentions.

[1668] The "means for improving driving efficiency and safety" refers to means for improving driving efficiency and safety based on the generated structured data.

[1669] A system for implementing this invention uses a generative AI model to return appropriate structured data in real time in response to user questions, thereby improving the driving efficiency and safety of autonomous vehicles. Specific embodiments of this system are described below.

[1670] The server first learns from the logs of existing AI assistants. This allows the generation AI to acquire the ability to generate structured data that is easy to identify in response to user questions. For example, if a user asks, "Where's the next traffic light?", the generation AI will identify the keyword "traffic light" and generate structured data such as the distance to the next traffic light and the traffic light status (red, yellow, green).

[1671] This structured data can be programmed by the server and used to perform specific actions, such as sending upcoming traffic lights to the autonomous vehicle's control system to make appropriate speed adjustments.

[1672] Furthermore, the server regenerates a response based on the generated structured data and the user's question. This response is intended to provide the user with appropriate information. For example, a specific response such as "The next traffic light is red. The distance is 100 meters" is generated.

[1673] The system includes a means to provide real-time traffic information for autonomous vehicles, analyzing user questions and generating relevant structured data to improve driving efficiency and safety.

[1674] The hardware used includes an on-board computer of an autonomous vehicle with internet connectivity. The software uses Python and the OpenAI API. Specific examples of prompts include:

[1675] Example prompt sentence:

[1676] User Question: What's the next signal?

[1677] Please return the relevant structured data.

[1678] In this way, real-time traffic information assistants can support autonomous vehicle operation, improving efficiency and safety.

[1679] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1680] Step 1:

[1681] The server learns from the logs of existing AI assistants. Specifically, it analyzes past dialogue history and operation history and provides them as training data to the generative AI model. Through this learning, the generative AI model acquires the ability to generate structured data that is easy to identify in response to user questions. The input is the log data of the AI ​​assistant, and the output is the trained generative AI model.

[1682] Step 2:

[1683] A user inputs a question into an autonomous vehicle, for example, "What's the next traffic light?", either through voice input or a touch panel. The input is the user's question, and the output is the question text.

[1684] Step 3:

[1685] The server inputs the user's question as a prompt to the generative AI model. Specifically, it sends the question text to the generative AI model and instructs it to generate related structured data. The input is the user's question text, and the output is the generated structured data.

[1686] Step 4:

[1687] The server receives the generated structured data and performs programming actions, such as sending information about upcoming traffic lights to the autonomous vehicle's control system to adjust the vehicle's speed appropriately. The input is the generated structured data, and the output is instructions to the control system.

[1688] Step 5:

[1689] The server regenerates a response based on the generated structured data and the user's question. Specifically, it creates a response to provide appropriate information to the user. The input is the user's question and the generated structured data, and the output is the regenerated response.

[1690] Step 6:

[1691] The server provides the regenerated response sentence to the user. Specifically, it conveys information to the user through voice output or display. The input is the regenerated response sentence, and the output is information provided to the user.

[1692] In this way, real-time traffic information assistants can support autonomous vehicle operation, improving efficiency and safety.

[1693] Example 2

[1694] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1695] Conventional AI assistant systems often lacked efficient program generation and execution to provide appropriate responses to user requests. Furthermore, the processing required after receiving structured data was complex, resulting in delayed responses to users. Furthermore, information acquisition from external APIs was not performed smoothly, resulting in insufficient information provided to users.

[1696] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1697] In this invention, the server includes means for training the generation AI from logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and generating a program, means for executing the generated program to perform a specific action, means for analyzing a prompt using a generation AI model, means for generating a program based on the analysis results, means for executing the generated program to obtain information from an external API, and means for returning the obtained information to the user, thereby enabling a prompt and appropriate response to a user request to be provided.

[1698] "Generative AI" is a system that uses artificial intelligence technology to automatically generate programs and responses for specific tasks.

[1699] "Structured data" is data that is organized according to specific formats and rules, making it easy for generative AI to identify and process.

[1700] An "AI assistant" is a system equipped with artificial intelligence to provide information or perform specific tasks in response to user requests.

[1701] A "log" is data that records the operation of a system and the operation history of a user.

[1702] "Means for generating programs" refers to the ability to automatically create code or scripts to perform specific tasks using a generative AI model.

[1703] A "means for performing an action" is a function that executes a generated program to perform a specific task or operation.

[1704] A "generative AI model" is an algorithm or model that forms the core of generative AI, and is a technology for analyzing prompts and generating appropriate responses or programs.

[1705] A "prompt sentence" is text containing a request or question that a user enters into the system.

[1706] The "means of analysis" is the function that uses a generative AI model to understand the meaning of the prompt and determine the appropriate action.

[1707] An "external API" is an interface for accessing external services and databases to obtain information.

[1708] "Means for obtaining information" is a function that executes the generated program to obtain the necessary data from the external API.

[1709] The "means for regenerating a response sentence" is a function that combines the acquired information with the original question sentence to generate an appropriate response to the user.

[1710] MODE FOR CARRYING OUT THE INVENTION

[1711] This invention is a system that uses a generative AI model to generate an appropriate program based on a prompt sentence entered by a user, and then executes the program to perform a specific action. Specific embodiments of this system are described below.

[1712] System configuration

[1713] This system consists of three main elements: a server, a terminal, and a user. The server is responsible for analyzing prompts using a generative AI model, generating and executing programs. The terminal is responsible for sending prompts from the user to the server and displaying responses from the server to the user. The user inputs requests to the system through the terminal.

[1714] Hardware and software used

[1715] The server is a computer system equipped with a high-performance processor and large memory capacity, and is equipped with software for running a generative AI model (e.g., GPT-4). The terminal is a device operated by a user, such as a smartphone, tablet, or PC. The server and terminal communicate via the Internet.

[1716] Data processing and calculation

[1717] The server inputs the prompt received from the user into a generative AI model for analysis. The generative AI model understands the meaning of the prompt and generates a program to determine the appropriate action. The generated program is written in a programming language such as Python and calls a specific API (for example, the OpenWeatherMap API) to obtain the necessary information. The obtained information is then formatted as a response using the generative AI model again and sent to the device.

[1718] Specific examples

[1719] Let's take the example of a user typing "What's the weather today?" This prompt is sent from the device to the server. The server uses the generative AI model to analyze the prompt and identify the keyword "weather." The server then generates a Python script to obtain weather information and calls the OpenWeatherMap API to obtain the latest weather information. The obtained weather information is then formatted as a response using the generative AI model again and sent to the device. The device then displays the received weather information to the user.

[1720] Example prompt sentence:

[1721] "Tell me about today's weather."

[1722] In this way, the system can provide a fast and appropriate response to a user's request.

[1723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1724] Step 1:

[1725] The user enters a prompt statement.

[1726] The user types "What's the weather like today?" into the input field of the terminal. This input is sent to the system as a prompt.

[1727] Input: The prompt text entered by the user: "What's the weather like today?"

[1728] Output: Data that the terminal sends to the server as a prompt

[1729] Step 2:

[1730] The terminal sends the prompt to the server.

[1731] The terminal sends the prompt text entered by the user to the server as an HTTP request. This communication takes place over the Internet.

[1732] Input: The prompt text entered by the user

[1733] Output: The prompt received by the server

[1734] Step 3:

[1735] The server analyzes the prompt using a generative AI model.

[1736] The server inputs the received prompt sentence into a generative AI model (e.g., GPT-4) for analysis. The generative AI model understands the meaning of the prompt sentence and determines the appropriate action.

[1737] Input: The prompt received by the server

[1738] Output: Analysis results (including the keyword "weather")

[1739] Step 4:

[1740] The server generates a program based on the analysis results.

[1741] The server generates a program to execute a specific action based on the analysis results of the generative AI model. For example, in response to the prompt "What's the weather today?", it generates a Python script to retrieve weather information.

[1742] Input: Analysis results of the generative AI model

[1743] Output: Generated Python script

[1744] Step 5:

[1745] The server runs the generated program.

[1746] The server runs the generated Python script and calls the OpenWeatherMap API to get the latest weather information.

[1747] Input: Generated Python script

[1748] Output: Weather information obtained from the OpenWeatherMap API

[1749] Step 6:

[1750] The server sends the execution results to the terminal.

[1751] The server then sends the acquired weather information to the terminal as an HTTP response. This communication also takes place over the Internet.

[1752] Input: Weather information obtained from the OpenWeatherMap API

[1753] Output: Weather information received by the device

[1754] Step 7:

[1755] The terminal displays the execution results to the user.

[1756] The terminal displays the weather information received from the server to the user, allowing the user to check the latest weather information.

[1757] Input: Weather information received from the server

[1758] Output: Weather information displayed to the user

[1759] (Application example 2)

[1760] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1761] Conventional AI assistant systems have had difficulty in quickly providing appropriate information in response to user questions. In particular, there is a need to obtain and provide updated information, such as traffic and weather information, in real time to users, but there has been a lack of efficient ways to achieve this.

[1762] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for training the generation AI from the logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for regenerating a response by combining the result with a question, means for identifying specific keywords and acquiring corresponding information, and means for providing the acquired information to the user. This makes it possible to provide prompt and appropriate information in response to a user's question.

[1763] "Generative AI" is artificial intelligence that identifies specific keywords and takes appropriate actions based on them.

[1764] "Structured data" is data that is organized in a format that is easy for generating AI to identify.

[1765] "Existing AI assistant logs" are records of user questions and responses that have been handled by the AI ​​assistant in the past.

[1766] A "means for programming and taking action" is a method for receiving structured data and performing specific actions based thereon.

[1767] The "means for regenerating a response sentence" is a method for creating a new response sentence by combining the acquired information with the user's question.

[1768] "Means for identifying specific keywords" refers to a method for extracting important keywords from user input.

[1769] The "means for obtaining information" is a method for obtaining relevant and up-to-date information based on the identified keywords.

[1770] "Means of providing to users" refers to the method used to convey the acquired information to users.

[1771] The system for implementing this invention uses a generative AI model to identify specific keywords, obtain appropriate information based on the keywords, and provide it to users. A specific embodiment of this system is described below.

[1772] System Configuration

[1773] The system consists of three main elements: a server, a terminal, and a user.

[1774] server

[1775] The server includes the following means:

[1776] 1. A way to train the generative AI to return structured data that is easy to identify from the logs of existing AI assistants: A generative AI model is trained using records of past user questions and responses.

[1777] 2. Programmatic actionable use of structured data: Perform specific actions (e.g., API calls) based on identified keywords.

[1778] 3. A means of regenerating a response sentence by combining the results with the question: The acquired information is combined with the user's question to generate a new response sentence.

[1779] 4. Identifying specific keywords and retrieving corresponding information: Extracting important keywords from the user's input and retrieving related information based on them.

[1780] 5. Means of providing acquired information to users: Communicate acquired information to users.

[1781] Terminal

[1782] A terminal is a device such as a smartphone or a head-mounted display that provides an interface for users to interact with the system. The terminal sends user input to the server and displays responses from the server.

[1783] user

[1784] The user is the entity that utilizes the system to obtain information. The user inputs questions by voice or text and receives responses from the system.

[1785] Hardware and software used

[1786] Hardware: Smartphone, head-mounted display

[1787] Software: Python 3.x, requests library, JSON library

[1788] Processing flow

[1789] 1. The user inputs or speaks a specific keyword (e.g., "weather") through the device.

[1790] 2. The device sends the user's input to the server.

[1791] 3. The server uses the generative AI model to identify keywords and retrieve corresponding information.

[1792] 4. The server combines the information it has obtained with the user's question to regenerate a response.

[1793] 5. The server sends the regenerated response to the terminal.

[1794] 6. The terminal displays the response to the user.

[1795] Specific examples

[1796] When the user says "weather," the server retrieves the latest weather information for the user's current location and sends it to the device. The device then displays, "The current weather in Tokyo is sunny, with a temperature of 25 degrees."

[1797] When the user says "traffic information," the server retrieves the latest traffic conditions for the current location and sends them to the device. The device then displays the message, "Current traffic conditions in Tokyo are congested. An alternative route will be suggested."

[1798] Prompt Sentence Examples

[1799] User: "Weather"

[1800] Generative AI: "Getting current weather information..."

[1801] Application: "The current weather in Tokyo is sunny and the temperature is 25 degrees."

[1802] User: "Traffic Information"

[1803] Generator: "Getting current traffic information..."

[1804] Application: "Current traffic conditions in Tokyo are congested. We will suggest an alternative route."

[1805] In this way, users can quickly obtain the information they need in real time.

[1806] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1807] Step 1:

[1808] The user inputs or speaks a specific keyword (e.g., "weather") through the terminal. The input is in text or voice format, and the terminal sends it to the server. The input data is the user's question.

[1809] Step 2:

[1810] The terminal sends the user's input to the server. The input data is the user's question, and the output data is the question sent to the server. The terminal sends the data to the server via the network.

[1811] Step 3:

[1812] The server uses a generative AI model to identify keywords. The input data is the user's question, and the output data is the identified keywords. The server uses the generative AI model to extract important keywords from the user's question.

[1813] Step 4:

[1814] The server retrieves the corresponding information based on the identified keywords. The input data is the identified keywords, and the output data is the retrieved information. The server calls the API to retrieve the latest weather information, traffic information, etc.

[1815] Step 5:

[1816] The server combines the acquired information with the user's question to regenerate a response. The input data is the acquired information and the user's question, and the output data is the regenerated response. The server generates an appropriate response for the user based on the acquired information.

[1817] Step 6:

[1818] The server sends the regenerated response sentence to the terminal. The input data is the regenerated response sentence, and the output data is the response sentence sent to the terminal. The server sends the data to the terminal via the network.

[1819] Step 7:

[1820] The terminal displays the response to the user. The input data is the response sent from the server, and the output data is the response displayed to the user. The terminal displays the response on the screen and provides information to the user.

[1821] Example 3

[1822] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1823] Conventional AI assistant systems have a complex process for generating appropriate responses to user questions, and they face problems such as low efficiency, particularly when gathering information from external data sources and generating responses. It is also difficult to quickly obtain the appropriate data based on the user's question and provide an accurate response. This can lead to a poor user experience and a loss of system reliability.

[1824] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1825] In this invention, the server includes means for receiving a user's question, means for analyzing the question and extracting important keywords and intent, means for collecting necessary data from external data sources, means for generating a response sentence based on the collected data, and means for sending the generated response sentence to the user. This makes it possible to generate a response to the user's question quickly and accurately, improving the user experience.

[1826] The "means for receiving a user's question" is a function for transmitting a question input by a user through a terminal to a server.

[1827] "Means of analyzing questions and extracting important keywords and intent" is a function that uses a generative AI model to analyze the content of a user's question and identify the subject and purpose of the question.

[1828] "Means of collecting necessary data from external data sources" refers to a function that uses an API client to obtain necessary information from external data sources, such as weather information and reservation information.

[1829] "Means for generating responses based on collected data" refers to a function that uses a generative AI model based on acquired data to create appropriate responses to user questions.

[1830] The "means for transmitting the generated response sentence to the user" is a function for transmitting the generated response sentence from the server to the terminal and displaying it to the user.

[1831] "Weather information" is information that includes meteorological data and indicates the weather conditions in a particular area and at a particular time.

[1832] "Reservation information" refers to information relating to a reservation for a particular service or facility, including the date, time, location, and user details.

[1833] "Alarm setting information" is information about an alarm set by a user, including the alarm time and message content.

[1834] A "generative AI model" is a model that uses artificial intelligence to generate and analyze text, and is used to generate responses to user questions.

[1835] A "prompt" is text that is input to a generative AI model and contains instructions or questions that enable the model to generate an appropriate response.

[1836] The present invention relates to a system for generating a response to a user's question quickly and accurately. A specific embodiment of this system will be described below.

[1837] System configuration

[1838] This system mainly consists of three elements: a server, a terminal, and a user. The server is a computer equipped with a high-performance processor and is installed with software to run the generative AI model and database management system. The terminal is a device used by the user, such as a smartphone or PC.

[1839] Hardware and software used

[1840] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[1841] Software: Generative AI models (e.g., OpenAI's GPT-3), database management systems (e.g., MySQL), API clients (e.g., Python's requests library)

[1842] Program processing

[1843] When the server receives a user's question, it uses the generative AI model to analyze the question and extract important keywords and intent. Next, the server uses the API client to collect necessary data from external data sources (e.g., weather information APIs). Based on the collected data, the generative AI model generates an appropriate response. Finally, the generated response is sent to the device and displayed to the user.

[1844] Specific examples

[1845] For example, the following process occurs when a user asks, "What's the weather like tomorrow?"

[1846] 1. Receive a user's question: The user types "What's the weather going to be like tomorrow?" into their smartphone and presses the send button. The device then sends this question to the server.

[1847] 2. Question analysis: The server uses a generative AI model to extract the keyword "tomorrow's weather."

[1848] 3. Data collection: The server sends a request to the weather information API to get tomorrow's weather data. The API returns the data "Sunny."

[1849] 4. Generate a response: The server generates a response saying "Tomorrow's weather will be sunny."

[1850] 5. Sending the response: The server sends the generated response to the terminal, which displays it to the user.

[1851] Prompt Sentence Examples

[1852] An example of a prompt sentence to input to the generative AI model is as follows:

[1853] User asks: "What's the weather like tomorrow?"

[1854] Prompt the generative AI model: "Get the latest weather information and tell me what the weather will be like tomorrow."

[1855] This system allows users to obtain information quickly and accurately, improving the user experience. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1856] Step 1:

[1857] A user uses a terminal to input a question. For example, the user types "What's the weather like tomorrow?" The terminal sends this question to the server as an HTTP request. The input is the user's question, and the output is an HTTP request to the server.

[1858] Step 2:

[1859] The server inputs the received question into the generative AI model. The generative AI model analyzes the question and extracts important keywords and intent. Specifically, the keyword "tomorrow's weather" is extracted. The input is the user's question, and the output is the extracted keywords.

[1860] Step 3:

[1861] The server uses an API client to collect the required data from an external data source (e.g., a weather information API). The server requests information about "tomorrow's weather" from the weather information API and receives weather data as a response. The input is the extracted keywords, and the output is the retrieved weather data.

[1862] Step 4:

[1863] The server uses a generative AI model to generate an appropriate response based on the acquired weather data. For example, if the weather data is "sunny," it generates a response such as "Tomorrow's weather will be sunny." The input is the acquired weather data, and the output is the generated response.

[1864] Step 5:

[1865] The server sends the generated response to the terminal as an HTTP response. The terminal displays this response to the user. The user sees the response "Tomorrow's weather will be sunny" on the terminal screen. The input is the generated response, and the output is the response displayed to the user.

[1866] (Application example 3)

[1867] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1868] Conventional food delivery systems did not optimize delivery times and methods in response to weather changes, and failed to provide users with appropriate information. As a result, users often became dissatisfied with delivery delays and uncertainty. In addition, response messages generated using generative AI models were not sufficiently utilized, resulting in low quality information provided to users.

[1869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1870] In this invention, the server includes: means for training the generating AI from the logs of existing AI assistants to return easily identifiable structured data; means for receiving the structured data and programming it to perform an action; means for combining the result with a question to regenerate a response; means for acquiring the latest weather information and optimizing delivery time and delivery method based on that information; and means for inputting a prompt using the generating AI model and providing an appropriate response to the user. This makes it possible to optimize delivery time and delivery method in response to changes in the weather, and to provide appropriate information to the user.

[1871] "Generative AI" is a technology that uses artificial intelligence to generate new information and responses from data.

[1872] "Structured data" is data that is organized according to specific formats and rules, making it easy for machines to understand and process.

[1873] An "AI assistant" is software or a device equipped with artificial intelligence to respond to user instructions and questions.

[1874] A "log" is data that records the operation history of a system or application.

[1875] "Programming" is the act of writing instructions for a computer to execute.

[1876] An "action" is an operation or process that a system performs based on specific conditions or instructions.

[1877] A "response" is a sentence that the system returns in response to a user's question or instruction.

[1878] "Weather information" refers to data related to the weather, such as the current weather, temperature, and precipitation.

[1879] "Delivery time" is the time it takes for a product or service to be delivered to the user.

[1880] "Delivery method" refers to the means or procedure for delivering a product or service to a user.

[1881] A "prompt" is a sentence of instructions or questions that is input into a generative AI model.

[1882] A system for implementing this invention includes a server, a user terminal, and a generative AI model. The server has a means for training the generative AI from logs of existing AI assistants so that it returns structured data that is easy to identify. This allows the server to obtain basic data for generating appropriate responses to user instructions and questions.

[1883] The server then has the means to program and take action on the structured data, allowing the server to perform specific actions or processes based on the structured data it has received.

[1884] Furthermore, the server has a means to regenerate a response by combining the result with the question, allowing the server to provide an appropriate response to the user.

[1885] The server also has a means of obtaining the latest weather information and optimizing delivery times and methods based on that information. This allows the server to adjust delivery times and methods according to changes in the weather, and provide accurate information to users.

[1886] Finally, the server has a means to input prompts using the generative AI model to provide appropriate responses to the user, allowing the server to generate responses in natural language to the user's questions and instructions.

[1887] As a specific example, when a user orders food delivery, the server uses the Weather API to obtain the latest weather information for the specified location. It then uses a generative AI model (OpenAI's GPT-3) to generate a response for the user. For example, if it is raining, the server generates a response such as, "The current weather is rainy. Delivery time may be longer than usual."

[1888] Example prompt sentence:

[1889] The current weather is rainy. Delivery times may be longer than usual.

[1890] The hardware used is a smartphone or tablet connected to the internet, and the software uses Python, the requests library, WeatherAPI, and OpenAI's GPT-3, allowing users to receive accurate information in real time and reducing uncertainty about delivery times and methods.

[1891] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1892] Step 1:

[1893] The server learns from the logs of existing AI assistants and returns structured data that is easy to identify to the generating AI.

[1894] Input: AI assistant log data

[1895] Output: Structured data

[1896] How it works: The server analyzes the log data and extracts patterns to respond to user questions and instructions, which then translates the data into a format that is easy for the generative AI to understand.

[1897] Step 2:

[1898] The server receives the structured data and performs programming actions based on it.

[1899] Input: Structured data

[1900] Output: Action result

[1901] Specific Action: The server performs a specific action or process based on the structured data, such as retrieving information from a database or sending a request to an external API as instructed by the user.

[1902] Step 3:

[1903] The server regenerates a response sentence by combining the action result and the question sentence.

[1904] Input: Action result, question

[1905] Output: Response

[1906] Specific operation: The server combines the action result with the user's question and uses a generative AI model to generate a response in natural language.

[1907] Step 4:

[1908] The server obtains the latest weather information and optimizes delivery times and methods based on that information.

[1909] Input: User's location

[1910] Output: Weather information, optimized delivery time and delivery method

[1911] Specific operation: The server uses the Weather API to obtain the latest weather information for the specified location and adjusts the delivery time and method based on that information.

[1912] Step 5:

[1913] The server uses a generative AI model to input prompt sentences and provide appropriate responses to the user.

[1914] Input: prompt statement

[1915] Output: Response to the user

[1916] Specific operation: The server inputs a prompt into a generative AI model (e.g., OpenAI's GPT-3) and generates a response in natural language for the user, such as "The current weather is rainy. Delivery times may be longer than usual."

[1917] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1918] "Example 1"

[1919] In one embodiment of the present invention, the generative AI learns from the logs of existing AI assistants and generates structured data that is easy to identify. This generative AI includes an emotion engine that recognizes the user's emotions. The emotion engine reads emotions from the user's tone of voice, facial expressions, and choice of words. For example, if the user feels angry, the emotion engine conveys that information to the generative AI.

[1920] "Example 2"

[1921] The generative AI receives information from the emotion engine and adjusts the generation of structured data or the execution of actions based on that information. For example, if the user feels angry, the generative AI will take that information into account and generate a more appropriate response. Alternatively, if the user feels happy, the generative AI will use that information to generate a more positive response.

[1922] "Example 3"

[1923] The generative AI then combines the results with the question to regenerate a response. This regenerated response is adjusted appropriately based on the user's emotions. For example, if the user feels angry, the generative AI will take that information into account to generate a more polite response. Alternatively, if the user feels happy, the generative AI will use that information to generate a more positive response.

[1924] The processing flow of each embodiment will be described below.

[1925] "Example 1"

[1926] Step 1: The generative AI starts learning from the logs of existing AI assistants.

[1927] Step 2: Generation The AI ​​generates structured data that is easy to identify based on the results of its learning.

[1928] Step 3: The emotion engine reads the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1929] Step 4: The emotion engine conveys the emotional information it has read to the generation AI.

[1930] "Example 2"

[1931] Step 1: The generative AI receives information from the emotion engine.

[1932] Step 2: The generation AI adjusts the generation of structured data and execution of actions based on the information it receives.

[1933] Step 3: The generative AI generates an appropriate response based on the adjusted results.

[1934] "Example 3"

[1935] Step 1: The generative AI matches the structured data with the question.

[1936] Step 2: The generation AI regenerates the response sentence based on this information.

[1937] Step 3: The generation AI provides the regenerated response to the user.

[1938] Example 1

[1939] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1940] Conventional AI assistant systems sometimes lacked consistency in their responses to user questions, making it particularly difficult to respond in a way that took the user's emotions into account. Furthermore, the generation and use of structured data was inefficient, which sometimes resulted in delays in providing appropriate information to users. This resulted in a poor user experience, which was an issue.

[1941] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1942] In this invention, the server includes means for training the generation AI from logs of existing AI assistants to return easily identifiable structured data, means for receiving the structured data and programming it to perform an action, means for combining the result with a question to regenerate a response, means for transmitting a user's question from a terminal to the server, means for the server to analyze the question and pass it to a generation AI model, means for the generation AI model to generate structured data based on the question, means for the server to return the generated data to the terminal, means for the terminal to display the data to the user, means for recognizing the user's emotions, and means for generating a response based on the emotions. This enables a quick and appropriate response to a user's question and further enables the provision of a response that takes the user's emotions into consideration.

[1943] "Generative AI" is artificial intelligence that learns from the logs of existing AI assistants and generates structured data that is easy to identify in response to user questions.

[1944] "Structured data" is data that is organized according to a specific format or structure, and is generated by generative AI based on user questions.

[1945] The "server" is a computer system whose role is to receive the user's question, analyze it, and pass it on to the generative AI model.

[1946] A "terminal" is a device through which a user inputs a question and receives a response from the server, and includes smartphones, PCs, etc.

[1947] The "emotion engine" is software that reads emotions from the user's tone of voice, facial expressions, choice of words, etc., and conveys that information to the generative AI.

[1948] "Question analysis" is the process in which the server receives a user's question and extracts keywords and intent.

[1949] "Response regeneration" is the process in which a generative AI generates a new response based on structured data and the user's question.

[1950] "Emotion recognition" is the process by which the emotion engine analyzes the user's emotions and provides that information to the generative AI.

[1951] An "action" is a specific operation or process that the generative AI performs based on structured data.

[1952] "Data display" is the process by which the terminal visually or audibly presents data received from the server to the user.

[1953] The present invention is a system that uses a generative AI model to provide appropriate responses to user questions. This system is realized using hardware and software such as a server, a terminal, a generative AI model, and an emotion engine.

[1954] Hardware and Software Configuration

[1955] Server: A high-performance server (e.g., a virtual machine on a cloud service) is used to host the generative AI model. The server analyzes the user's question and passes it to the generative AI model.

[1956] Terminal: A device used by a user, such as a smartphone or PC. The terminal sends the user's questions to the server, receives responses from the server, and displays them.

[1957] Generative AI models: For example, using advanced natural language processing models such as GPT-4, generative AI models generate structured data based on questions passed to them by the server.

[1958] Emotion engine: For example, using emotion recognition software such as the Affectiva SDK, the emotion engine analyzes the user's tone of voice, facial expressions, and word choice to identify emotions.

[1959] Data processing and calculation

[1960] 1. Handling user questions

[1961] The user asks through the terminal, "What's the weather going to be like tomorrow?"

[1962] The terminal sends this question to the server.

[1963] The server analyzes the question and extracts the keyword "weather."

[1964] The analysis results are passed to the generative AI model.

[1965] 2. Generating structured data

[1966] The generative AI model generates related structured data based on the analysis results passed from the server, such as weather forecast data.

[1967] The generated data is returned to the server.

[1968] 3. Return and Display of Data

[1969] The server returns the generated structured data to the terminal.

[1970] The terminal displays the data to the user, for example, "Tomorrow will be sunny."

[1971] 4. Emotion recognition and response generation

[1972] When a user asks a question or makes a comment, the emotion engine analyzes the user's tone of voice, facial expressions, and word choice.

[1973] The emotion engine communicates the analysis results to the generative AI model.

[1974] The generative AI model generates a response that takes into account the user's emotions and returns it to the device via the server. For example, if the user is angry, it will generate a response such as "Please calm down."

[1975] Specific examples

[1976] Example question: A user asks, "What's the weather like tomorrow?"

[1977] The server identifies the keyword "weather" and generates and returns weather forecast data.

[1978] Emotion example: When a user is angry

[1979] The emotion engine recognizes the user's anger and conveys that information to the generative AI.

[1980] The generative AI generates a response that takes into consideration the user's emotions and returns it to the device.

[1981] Prompt Sentence Examples

[1982] "If a user asks, 'What's the weather going to be like tomorrow?' how does a generative AI respond?"

[1983] "If the emotion engine recognizes that the user is angry, how will the generative AI respond?"

[1984] The above is an embodiment of the present invention. This system enables quick and appropriate responses to user questions, and also provides responses that take into account the user's emotions.

[1985] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1986] Step 1:

[1987] The user enters a question.

[1988] A user uses a device (smartphone or PC) to input a question such as "What's the weather going to be like tomorrow?" The input is done through voice recognition or text input. The input data is voice data or text data.

[1989] Step 2:

[1990] The terminal sends a question to the server.

[1991] The terminal receives the user's question and sends it to the server via the Internet. At this time, the question is converted into an appropriate format (e.g., JSON). The input is the user's question data, and the output is the JSON-formatted data sent to the server.

[1992] Step 3:

[1993] The server analyzes the question and passes it to a generative AI model.

[1994] The server analyzes the received question and extracts keywords (e.g., "weather"). The analysis results are passed to the generative AI model. The input is the question data in JSON format, and the output is the analyzed keyword data.

[1995] Step 4:

[1996] A generative AI model generates structured data based on the question.

[1997] The generative AI model generates related structured data based on the analysis results passed from the server. For example, in response to the question "What's the weather like tomorrow?", it generates weather forecast data. The input is analyzed keyword data, and the output is structured weather forecast data.

[1998] Step 5:

[1999] The server returns the generated data to the terminal.

[2000] The server returns the structured data received from the generative AI model to the device, converting the data into a format that is easy for users to understand. The input is structured weather forecast data, and the output is user-friendly data sent to the device.

[2001] Step 6:

[2002] The terminal displays the data to the user.

[2003] The terminal displays data received from the server to the user, for example, displaying weather forecast information on a screen. The input is the user-friendly data received from the server, and the output is the information displayed to the user.

[2004] Step 7:

[2005] Recognize user emotions.

[2006] When a user asks a question or makes a comment, the device's built-in emotion engine analyzes the user's tone of voice, facial expressions, and word choice to recognize their emotions. The input is the user's voice and video data, and the output is analyzed emotional data.

[2007] Step 8:

[2008] Generate emotion-based responses.

[2009] The emotion engine passes the recognized emotion information to the generative AI model. The generative AI model generates a response that takes the user's emotions into consideration and returns it to the device via the server. For example, if the user is angry, it generates a response such as "Please calm down." The input is analyzed emotion data, and the output is emotion-conscious response data.

[2010] (Application example 1)

[2011] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2012] Conventional navigation systems for autonomous vehicles have difficulty providing appropriate responses to user questions, and in particular, have been unable to respond in a way that takes into account the user's emotions. This can lead to reduced user satisfaction and increased stress while driving.

[2013] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for training the generation AI from the log of an existing AI assistant so as to return easily identifiable structured data; means for receiving the structured data and programming it to perform an action; means for regenerating a response by combining the result with a question; means including an emotion engine that recognizes the user's emotions; and means for adjusting the response based on information from the emotion engine. This makes it possible to provide an appropriate response to a user's question and further respond in accordance with the user's emotions.

[2014] "Generative AI" is an artificial intelligence that learns from the logs of existing AI assistants and returns structured data that is easy to identify in response to user questions.

[2015] "Structured data" is data organized in a specific format that is generated by generative AI based on user questions.

[2016] The "emotion engine" is an engine that has the ability to read emotions from the user's tone of voice, facial expressions, choice of words, etc.

[2017] A "response sentence" is a sentence that the generation AI generates to respond to a user's question.

[2018] "Means for programming and taking action" refers to means for executing specific operations or processes based on the generated structured data.

[2019] The "means of regeneration" refers to the means by which the generative AI generates a new response sentence based on the structured data and the question sentence.

[2020] The "adjustment means" is a means for appropriately changing the response sentence in accordance with the user's emotions based on information from the emotion engine.

[2021] A system for implementing the present invention has the following configuration.

[2022] System Program

[2023] The system includes a generative AI, an emotion engine, a voice recognition module, a camera module, and a navigation module. The generative AI learns from the logs of existing AI assistants and returns structured data that is easy to identify in response to user questions. The emotion engine reads emotions from the user's tone of voice, facial expressions, and choice of words.

[2024] Hardware and Software

[2025] Hardware:

[2026] Microphone: Used to capture audio input.

[2027] Camera: Used to capture the user's facial expressions.

[2028] software:

[2029] speech_recognition library: Used to convert speech to text.

[2030] Transformers library pipeline: Used to perform emotion recognition.

[2031] cv2 (OpenCV): Used to acquire images from the camera.

[2032] Data processing and calculation

[2033] The server converts the voice data acquired from the microphone into text using a speech recognition module. The generation AI then analyzes this text and generates structured data that is easy to identify. The emotion engine reads emotions from the user's tone of voice and facial expressions and conveys this information to the generation AI. The generation AI then adjusts the response sentence based on the information from the emotion engine and provides an appropriate response to the user.

[2034] Specific examples

[2035] For example, if a user asks, "Where is the next gas station?", the speech recognition module converts this question into text. The generation AI identifies the keyword "gas station" and generates information about the nearest gas station as structured data. If the emotion engine determines the user's emotion as "neutral," the generation AI responds, "The nearest gas station is 5 km away."

[2036] Prompt Sentence Examples

[2037] If the user asks, "Where is the next gas station?" and the emotion recognition model determines the answer as "neutral," the system will respond, "The nearest gas station is 5 km away."

[2038] In this way, it is possible to provide appropriate responses to the user's questions and further to respond in accordance with the user's emotions.

[2039] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2040] Step 1:

[2041] The user asks a question into the microphone. The input is the user's voice data, and the output is the voice data passed to the speech recognition module.

[2042] Step 2:

[2043] The server's speech recognition module converts the speech data into text. The input is speech data and the output is text data. Specifically, the speech_recognition library is used to convert speech to text.

[2044] Step 3:

[2045] The server's generation AI analyzes the text data and generates structured data that is easy to identify. The input is text data, and the output is structured data. Specifically, the generation AI model extracts keywords and generates related information as structured data.

[2046] Step 4:

[2047] The server's emotion engine reads emotions from the user's tone of voice and facial expressions. The input is audio data and camera video data, and the output is emotional information. Specifically, emotion recognition uses a pipeline from the transformers library.

[2048] Step 5:

[2049] The server's generation AI adjusts the response sentence based on the emotional information from the emotion engine. The input is structured data and emotional information, and the output is the adjusted response sentence. Specifically, the generation AI generates the response sentence taking into account the emotional information.

[2050] Step 6:

[2051] The server returns the tailored response to the user. The input is the tailored response, and the output is a voice response to the user. Specifically, the server converts the text into speech and responds to the user through a speaker.

[2052] Example 2

[2053] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2054] Conventional AI assistant systems have difficulty generating responses that take the user's emotional state into account, making it difficult to provide appropriate responses to users. Furthermore, they have problems with not being able to efficiently execute actions using structured data or obtain up-to-date information. This has resulted in a poor user experience and limited the usefulness of the system.

[2055] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2056] In this invention, the server includes a means for training the generation AI to return easily identifiable structured data from the logs of existing AI assistants, a means for receiving the structured data and programming it to perform an action, and a means for receiving information from the emotion engine and adjusting the generation of structured data and the execution of actions based on that information. This makes it possible to generate appropriate responses that take the user's emotional state into consideration and efficiently obtain up-to-date information.

[2057] "Generative AI" is an artificial intelligence technology that analyzes input data from users and generates appropriate responses.

[2058] "Structured data" is data that is organized according to a specific format, making it easy for generative AI to identify and process.

[2059] An "AI assistant" is an artificial intelligence system that generates responses to user input and automates various tasks.

[2060] A "log" is data that records the operation of a system and the operation history of a user.

[2061] "Programming" is the process of giving a computer instructions to perform a particular task.

[2062] An "action" is a specific operation or process that a system performs in response to a specific input.

[2063] An "emotion engine" is a technology that analyzes a user's emotional state and provides that information.

[2064] A "response sentence" is a reply message generated by the generation AI in response to user input. 【206...

Claims

1. A means for collecting log data that records interactions between a user and an artificial intelligence assistant; A means for removing unnecessary information from the collected log data, extracting question and answer pairs, and normalizing the data; means for training a generative artificial intelligence model capable of generating responses to user questions using the preprocessed log data; Including, The generative artificial intelligence model is trained to analyze input questions to identify keywords and generate relevant structured data based on the identified keywords. system.

2. A means for generating structured data related to a user's question using a generative AI model trained using log data that records a conversation between a user and an artificial intelligence assistant as training data; A means for transmitting the structured data to a control system of an autonomous vehicle and causing the control system to control the vehicle; A means for inputting the user's question and the structured data into the generative AI model to obtain a response sentence to the user's question, the response sentence being generated by the generative model; A system including:

3. A means for acquiring voice data including a question from a user; means for converting the voice data into text data; means for generating structured data relevant to the query by analyzing the text data using a generative AI model; means for recognizing an emotion of the user based on the voice data; A means for generating a response sentence to the question based on the recognized emotion and the structured data using the generative AI model; A system including:

Images