system

Abstract

We provide the system. [Solution] A system including means for operation from a smartphone, means for linking with a generative AI, means for creating a chatbot utilizing the generative AI, means for prototyping the created chatbot, and means for allowing others to use the prototype chatbot.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method executed by at least one processor, the method comprising: receiving a user utterance; adding the user utterance to a prompt including an instruction associated with a description of a character of a chatbot; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance responding to the user utterance.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] There is a problem that it is difficult for people who are not digitally literate to easily create, prototype, and have others use chatbots utilizing generative AI.

Means for Solving the Problem

[0005] By providing an operation means from a smartphone, a means for cooperating with generative AI, a means for creating a chatbot utilizing generative AI, a means for prototyping the created chatbot, and a means for allowing others to use the prototyped chatbot, even people who are not digitally literate can easily create, prototype, and have others use a chatbot utilizing generative AI.

Brief Description of Drawings

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16]FIG. 12 is a sequence diagram showing a processing flow of the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 13 is a sequence diagram showing a processing flow of the data processing system in Example 1 of Embodiment 1 when an emotion engine is combined. [Figure 18] FIG. 14 is a sequence diagram showing a processing flow of the data processing system in Application Example 1 of Embodiment 1 when an emotion engine is combined. [Figure 19] FIG. 15 is a sequence diagram showing a processing flow of the data processing system in Example 2 of Embodiment 2 when an emotion engine is combined. [Figure 20] FIG. 16 is a sequence diagram showing a processing flow of the data processing system in Application Example 2 of Embodiment 2 when an emotion engine is combined. [Figure 21] FIG. 17 is a sequence diagram showing a processing flow of the data processing system in Example 3 of Embodiment 3 when an emotion engine is combined. [Figure 22] FIG. 18 is a sequence diagram showing a processing flow of the data processing system in Application Example 3 of Embodiment 3 when an emotion engine is combined. [Figure 23] FIG. 19 is a sequence diagram showing a processing flow of the data processing system in another embodiment. MODE FOR CARRYING OUT THE INVENTION

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, terms used in the following description will be explained.

[0009] In the following embodiments, a labeled processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of a plurality of arithmetic devices. Further, the processor may be one type of arithmetic device or a combination of a plurality of types of arithmetic devices. Examples of arithmetic devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, labeled RAM (Random Access Memory) is a memory that temporarily stores information, and is used as a working memory by the processor.

[0011] In the following embodiments, labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. Communication I / F manages communication between a plurality of computers. Examples of communication standards applied to communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0014] [First Embodiment]

[0015] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0016] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] One embodiment of the present invention uses a dedicated application as a means of operation from a smartphone. This application provides an interface for the user to interact with the generative AI. Specifically, it includes functions for the user to update the generative AI's learning data and functions for creating a chatbot that utilizes the generative AI.

[0029] "Example of form 2"

[0030] One embodiment of the present invention provides a means for creating a chatbot that utilizes generative AI, which includes a function for the user to set the chatbot's response patterns. Specifically, the user sets answers to specific questions, and these answers are learned by the generative AI.

[0031] "Example of form 3"

[0032] One embodiment of the present invention provides a function for users to test the response patterns of a chatbot as a means of prototyping a created chatbot. Specifically, the user asks a question to the chatbot and verifies whether the answer is appropriate.

[0033] "Example of form 4"

[0034] One embodiment of the present invention provides a function that allows a user to publish a chatbot as a means of making a prototype chatbot available for use by others. Specifically, a user publishes a chatbot, making it available for others to use.

[0035] The following describes the processing flow for each example of the form.

[0036] "Example of form 1"

[0037] Step 1: The user installs a dedicated application on their smartphone. Step 2: The user opens the application and operates the interface for interacting with the generating AI.

[0038] Step 3: The user updates the training data for the generated AI using the function to update the training data for the generated AI.

[0039] Step 4: Create a chatbot using the user-generated AI-powered chatbot creation feature.

[0040] "Example of form 2"

[0041] Step 1: Use the feature that allows users to set chatbot response patterns.

[0042] Step 2: The user sets the answer to a specific question.

[0043] Step 3: The set responses are learned by the generating AI, and the chatbot's response patterns are updated.

[0044] "Example of form 3"

[0045] Step 1: Use the feature that allows users to test the chatbot's response patterns.

[0046] Step 2: The user asks a question to the chatbot.

[0047] Step 3: The chatbot's response is displayed, and the user checks if the response is appropriate.

[0048] "Example of form 4"

[0049] Step 1: Use the feature that allows users to publish chatbots.

[0050] Step 2: The user publishes the chatbot, and the published chatbot becomes accessible to others.

[0051] Step 3: Others use a publicly available chatbot and interact with it.

[0052] (Example 1)

[0053] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0054] In modern artificial intelligence systems using information processing equipment, there is a need to provide an environment that allows users to easily interact with AI and create and prototype interactive programs. Furthermore, efficiently updating the AI's training data and generating appropriate responses to user input are key challenges.

[0055] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0056] In this invention, the server includes means for operating from an information processing device, means for coordinating with artificial intelligence, and means for creating interactive programs utilizing artificial intelligence. This allows the user to update the AI's learning data, prototype interactive programs, and allow others to use them.

[0057] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides a means for users to operate it.

[0058] Artificial intelligence is a technology that enables computer systems to mimic human intellectual behavior, possessing the ability to learn from data and perform reasoning and judgment.

[0059] "Means of collaboration" refers to methods and processes that enable different systems or components to exchange information and work together in coordination.

[0060] An "interactive program" is software that provides information and answers questions through dialogue with the user, and it operates using artificial intelligence.

[0061] "Methods for prototyping" refer to methods and processes for experimentally creating new products or systems and evaluating their performance and functionality.

[0062] "Training data" refers to the dataset used by artificial intelligence for learning, and it forms the basis for improving the accuracy and performance of the model.

[0063] "Input text" refers to text data that a user inputs into an information processing device, and it serves as the basis for artificial intelligence to generate a response.

[0064] A description of embodiments for carrying out this invention will be given.

[0065] The user launches a dedicated application on their smartphone, which acts as an information processing device. This application provides an interface for the user to interact with artificial intelligence. The user logs into the application, selects a dataset to update the AI's training data, and uploads it to the server via their device. The server updates the training data of the AI ​​model using the received dataset. This process utilizes machine learning libraries such as TENSORFLOW® and PyTorch.

[0066] The server retrains the artificial intelligence model based on the uploaded data, improving the model's accuracy. Once training is complete, the server notifies the user, who can then use the generated AI to create an interactive program. The user uses the application's features to set the objective and response patterns of the interactive program.

[0067] For example, if a user enters a prompt such as "Tell me the features of the new product," the terminal sends this prompt to the server. The server uses a generative AI model to generate a response to the prompt and sends it back to the terminal. The user can then review the generated response and re-enter the prompt if necessary.

[0068] In this way, users can operate artificial intelligence through an information processing device to update data, create interactive programs, and generate responses using prompt statements.

[0069] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0070] Step 1:

[0071] The user launches a dedicated application on their smartphone, which acts as an information processing device, and logs in. They enter their login information and send it to the server. The server authenticates the received login information and grants the user access. This allows the user to access the application's functions.

[0072] Step 2:

[0073] The user selects a new dataset to update the training data for a generative AI model. The device uploads the selected dataset to the server. The server stores the received dataset and verifies its integrity. It preprocesses the data and converts it into a format suitable for training.

[0074] Step 3:

[0075] The server updates the training data for the generative AI model using the uploaded dataset. It then retrains the model using machine learning libraries such as TensorFlow or PyTorch. The updated model is obtained as output, using preprocessed data as input. Once training is complete, the server notifies the user.

[0076] Step 4:

[0077] Users create interactive programs that utilize generative AI by using the application's features. Users set the purpose and response patterns of the interactive program. The server builds the interactive program based on the settings and verifies its operation.

[0078] Step 5:

[0079] The user inputs a prompt message to the generative AI model. The terminal sends the input prompt message to the server. The server uses the generative AI model to generate a response to the prompt message. The prompt message is used as input, and the generated response is obtained as output. The response is sent back to the terminal, and the user confirms it.

[0080] (Application Example 1)

[0081] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0082] In today's information society, users are required to efficiently acquire useful information from a vast amount of data. However, conventional information provision systems have the problem of not adequately providing personalized information based on users' interests and preferences. Furthermore, even in information generation using generative AI, there is a lack of mechanisms to effectively reflect user feedback, which makes it difficult to improve the accuracy and relevance of the information.

[0083] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0084] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for generating information based on the user's interests, and means for collecting the user's evaluation of the generated information and updating the generative AI's learning data. This makes it possible to provide personalized information to the user and improve the accuracy and relevance of the generative AI.

[0085] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and it is a device that can perform various functions through user operation.

[0086] "Generative AI" refers to a system that uses artificial intelligence technology to generate new information and content from data, providing appropriate output according to user requests.

[0087] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[0088] "Means of generating information based on user interests" refers to methods and technologies for analyzing user input and past behavioral history and generating highly relevant information based on that analysis.

[0089] "Means of collecting user evaluations of generated information and updating the training data of the generating AI" refers to methods and technologies that enable the generation of more accurate information by collecting feedback from users and improving the training data of the generating AI based on that feedback.

[0090] The system for carrying out this invention comprises an information processing device, a generative AI, and a server. The information processing device provides an interface for user operation and generates information in cooperation with the generative AI. The generative AI is responsible for receiving input from the user and generating relevant information based on it. The server provides the generated information to the user and collects user feedback to update the generative AI's learning data.

[0091] Specifically, the information processing device is a device such as a smartphone or tablet, which runs an application for the user to input topics of interest. This application is developed using React Native and provides the user interface. The topics entered by the user are sent as prompts to the generating AI. The generating AI uses OpenAI's GPT model and generates relevant content based on the input prompts.

[0092] The generated content is returned to the information processing device via the server and displayed to the user. The user evaluates the displayed content, and this evaluation is collected by the server. Based on the collected evaluations, the server updates the training data of the generating AI, improving the accuracy of future information generation.

[0093] For example, if a user enters "latest technology news," the AI ​​will generate articles such as "Latest Technology Trends of 2023" or "Articles on the Evolution of AI Technology." An example of a prompt would be, "Please tell me the latest technology news. I'm especially interested in information on AI and robotics." In this way, users can efficiently obtain information that matches their interests.

[0094] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0095] Step 1:

[0096] The user launches an application on the information processing device and enters a topic of interest. The entered topic is then prepared to be sent to the generating AI as a prompt. The input data is in text format and reflects the user's interests.

[0097] Step 2:

[0098] The terminal sends the input prompt to a generating AI model. The generating AI model uses OpenAI's GPT and generates relevant information based on the prompt. As a data processing step, it performs natural language processing to parse the prompt and generate the relevant information. The output is the generated text information.

[0099] Step 3:

[0100] The server sends the information received from the generating AI to the terminal. The terminal displays the received information to the user. The user reviews the displayed information and makes an evaluation. The evaluation is based on the relevance and usefulness of the information.

[0101] Step 4:

[0102] User ratings are sent from the device to the server. The server updates the training data for the generative AI based on the received ratings. As part of the data calculation, the rating data is analyzed and the model parameters of the generative AI are adjusted. This improves the accuracy of information generation in subsequent instances.

[0103] Step 5:

[0104] The server incorporates the updated training data into the generative AI. This allows the generative AI to generate information while incorporating user feedback. The output is the updated generative AI model.

[0105] (Example 2)

[0106] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0107] The challenge in interactive programs utilizing generative AI is to provide a system that allows users to flexibly configure responses to specific inquiries and efficiently train the generative AI model. Furthermore, it is necessary for the generative AI model to accurately understand the user's intent and effectively generate prompt sentences that produce appropriate responses.

[0108] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0109] In this invention, the server includes means for the user to set a response to a specific query, means for training a generating AI model with the set response, and means for generating prompt sentences for the generating AI model. This enables the generating AI model to efficiently learn the response patterns set by the user, allowing the generating AI model to accurately understand the user's intent and generate an appropriate response.

[0110] An "information terminal" is an electronic device used by a user, and includes devices such as computers and smartphones.

[0111] "Generative AI" is an artificial intelligence technology that performs natural language processing, and is particularly used as a model for generating responses in conversational programs.

[0112] An "interactive program" is software designed to interact with users, and includes systems such as chatbots.

[0113] A "user" is an individual or group that operates an interactive program and sets up responses to specific inquiries.

[0114] A "response pattern" is a pre-configured format for answering a specific inquiry, serving as a standard for how an interactive program responds to a user.

[0115] A "generative AI model" is a core algorithm of generative AI, a pre-trained model designed to generate natural language responses based on input data.

[0116] A "prompt" is an instruction given to a generative AI model to generate a specific response, and it serves as a clue for the model to understand the user's intent.

[0117] One embodiment of this invention is a system that constructs an interactive program utilizing generative AI, enabling users to set responses to specific inquiries. This system is implemented using an information terminal, a server, and a generative AI model.

[0118] Users configure response patterns for interactive programs using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions they want to set and their corresponding answers. For example, a user can set the answer to the question "What is the weather forecast?" to "Today's weather is sunny."

[0119] The terminal sends the user's entered question and answer data to the server. The server analyzes the received data and generates prompts to train a generative AI model. These prompts are designed so that the generative AI model accurately understands the user's intent and generates appropriate responses. A concrete example of a prompt would be, "If the user asks, 'What is the weather forecast?', answer, 'Today's weather is sunny.'"

[0120] The server passes the generated prompt text to the generative AI model for training. The generative AI model, for example, uses an algorithm specialized in natural language processing to memorize the response patterns set by the user. This allows the generative AI model to generate an appropriate response and send it to the terminal when a user asks a question to the chatbot.

[0121] In this way, users can freely set and customize the response patterns of conversational programs that utilize generative AI.

[0122] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0123] Step 1:

[0124] The user configures the response patterns for the interactive program using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions and answers they wish to configure. The input data consists of question-and-answer pairs. This data is stored on the terminal as response patterns that reflect the user's intent.

[0125] Step 2:

[0126] The terminal sends the user-entered question and answer data to the server. The transmitted data is structured as an HTTP request. The server parses the received data and prepares it for use in the next step. Specifically, it verifies the data's integrity and converts the format as needed.

[0127] Step 3:

[0128] The server generates prompt sentences to train a generative AI model based on the received questions and answers. The input data is question-and-answer pairs, and the output is the generated prompt sentences. Specifically, the server creates a prompt sentence in the format of "When the user asks 'What is the weather forecast?', please answer 'Today's weather is sunny.'"

[0129] Step 4:

[0130] The server passes the generated prompt sentences to the generative AI model for training. The input is the prompt sentence, and the output is the trained response pattern. The generative AI model uses a natural language processing algorithm to memorize the response patterns set by the user.

[0131] Step 5:

[0132] When a user asks a question to the chatbot, the server invokes a generative AI model to generate an appropriate response. The input is the user's question, and the output is the generated response. The generated response is sent to the terminal and displayed to the user. Specifically, if the user asks, "What's the weather forecast?", the chatbot will respond, "Today's weather is sunny."

[0133] (Application Example 2)

[0134] Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0135] Traditional interactive programs make it difficult for users to set individual response patterns, and the responses provided by the generating AI may not always match the user's needs. Furthermore, because the data used for training the generating AI is fixed, it faces the challenge of not being able to respond flexibly to the user's business needs.

[0136] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0137] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generative AI, and means for creating an interactive program utilizing the generative AI. This allows the user to freely set the response patterns of the interactive program, and the generative AI to learn based on those settings and select the optimal response.

[0138] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones and tablets.

[0139] "Generative AI" is a system that uses artificial intelligence technology to learn from data and perform natural language processing.

[0140] An "interactive program" is software that provides information or answers questions through dialogue with the user.

[0141] A "response pattern" refers to a pre-set format or content of answers to a specific question.

[0142] "Means of learning" refers to the process by which generative AI acquires new knowledge based on data provided by users and improves the accuracy of its responses.

[0143] "The means of selecting the optimal response" refers to a function that automatically selects the most appropriate answer to a user's question based on the data the generation AI has learned.

[0144] A system for carrying out this invention includes an information terminal, a generative AI model, and a server. The information terminal is a device for user operation, such as a smartphone or tablet. The user can use the information terminal to set response patterns for an interactive program.

[0145] The server works in conjunction with a generative AI model, which learns based on response patterns set by the user. The generative AI model is a system for natural language processing, acquiring new knowledge based on data provided by the user and improving the accuracy of its responses.

[0146] Specifically, when a user enters FAQ questions and answers via an information terminal, that data is stored on a server. The server provides the stored data to a generative AI model, which then learns from it. This allows the generative AI model to generate flexible responses tailored to the user's business needs.

[0147] For example, if a user asks "How do I return an item?" and the answer is set to "Returns are possible within 30 days of receiving the item. Please use the return label," the generative AI model will learn from this information and provide appropriate responses to similar questions.

[0148] Examples of prompts used to train a generative AI model include the following:

[0149] "User question: 'How do I return an item?' Set answer: 'Returns are possible within 30 days of receiving the item. Please use the return label.' Prompt to train the generating AI model: 'Generate an appropriate response to the question about how to return an item.'"

[0150] In this way, users can freely customize the responses of the interactive program, and the generating AI model can select the optimal response.

[0151] The flow of the specific processing in Application Example 2 will be explained using Figure 14.

[0152] Step 1:

[0153] The user sets the response patterns for the interactive program using an information terminal. Specifically, the user inputs FAQ questions and their answers. The entered data is sent from the information terminal to the server.

[0154] Step 2:

[0155] The server stores the question and answer data received from the user in a database. This stored data serves as the foundational data for the generative AI model to learn from. The server then prepares the data stored in the database to be provided to the generative AI model.

[0156] Step 3:

[0157] The server provides stored data to the generative AI model, which then learns. Specifically, the server generates prompt sentences and inputs them into the generative AI model. The generative AI model performs natural language processing based on the prompt sentences and learns response patterns set by the user.

[0158] Step 4:

[0159] The generative AI model generates the optimal response to a new question from the user based on the data it has learned. The server provides the user with the response obtained from the generative AI model. Specifically, the server sends the response to the user's information terminal so that the user can confirm it.

[0160] Step 5:

[0161] Users can review the responses generated through their information terminals and readjust the response patterns as needed. This allows users to continuously improve the responses of the interactive program and enhance the accuracy of the generated AI model.

[0162] (Example 3)

[0163] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0164] Traditional dialogue program development has presented challenges in efficient development and operation, as testing and publishing response patterns is difficult, and developers must manually manage many processes. Furthermore, the lack of a process to evaluate the appropriateness of responses using generative AI models has made it difficult to provide dialogue programs that are highly satisfying to users.

[0165] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0166] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI model, means for creating a dialogue program utilizing the generative AI model, means for testing the response patterns of the created dialogue program, and means for making the tested dialogue program public to others. This makes it possible to efficiently develop a dialogue program, evaluate the appropriateness of the responses, and make it public to others.

[0167] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and is equipped with an interface for user operation.

[0168] A "generative AI model" is a model that uses artificial intelligence technology to perform natural language processing and has the ability to generate appropriate responses to user input.

[0169] A "dialogue program" is software designed for interaction with users and has the function of providing responses to user questions using a generative AI model.

[0170] "Means for testing response patterns" refers to a process for evaluating the appropriateness of responses generated by a dialogue program, and a means for verifying responses to user input.

[0171] "Means of publication" refers to the process of placing a dialogue program on the internet and making it accessible to others.

[0172] A description of embodiments for carrying out this invention will be given.

[0173] The user operates the dialogue program using an information processing device. The information processing device receives questions entered by the user and sends that data to the server. The server works in conjunction with a generative AI model, and upon receiving input from the user, passes that data to the generative AI model. The generative AI model uses natural language processing techniques to generate appropriate responses to the entered questions.

[0174] The generated response is returned to the information processing unit via the server. The user can review this response and evaluate whether the response pattern of the dialogue program is appropriate. For example, if the user asks "What's the weather like today?", the generating AI model will generate a response such as "It's sunny today."

[0175] Furthermore, users can upload tested dialogue programs to a server and make them publicly available for others to use. The server places the dialogue programs on the internet, making them accessible to others. This allows others to access and use the publicly available dialogue programs.

[0176] An example of a prompt statement is, "Create a dialogue program that uses a generative AI model to generate appropriate responses to questions entered by the user." By using this prompt statement, the generative AI model generates responses that meet the user's requests, enabling efficient development and operation of the dialogue program. The flow of specific processing in Example 3 will be explained using Figure 15.

[0177] Step 1:

[0178] The user enters the question on their device.

[0179] The user enters a question through the terminal's interface. This input data is sent to the server in text format. For example, the user might enter a question such as, "What's the news today?"

[0180] Step 2:

[0181] The server sends the question to the AI ​​model that generates it.

[0182] The server sends the text data received from the user to the generative AI model. At this time, the server converts the data into an appropriate format so that the generative AI model can process it. The generative AI model uses natural language processing techniques to analyze the question and prepares to generate an appropriate response.

[0183] Step 3:

[0184] The generative AI model generates the response.

[0185] The generative AI model generates responses based on the received question data, utilizing its internal training data. This process involves understanding the intent of the question, extracting relevant information, and constructing a response. For example, it might generate a response such as, "Today's main news is the rise in economic growth."

[0186] Step 4:

[0187] The server returns a response to the user.

[0188] The server receives response data from the generated AI model and sends it back to the user's terminal. At this time, the server converts the response data into a format that is easy for the user to understand. The user reviews this response and evaluates whether the dialogue program's response pattern is appropriate.

[0189] Step 5:

[0190] The user uploads the dialogue program to the server and makes it publicly available.

[0191] The user uploads the completed interactive program to the server. The server places the interactive program on the internet, making it accessible to others. This allows others to access and use the publicly available interactive program.

[0192] (Application Example 3)

[0193] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] In modern information distribution services, it is difficult for users to obtain detailed information about the content they are watching in real time. In particular, there is a lack of means to immediately resolve questions that arise during viewing, and there is a need to improve the user experience.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0196] In this invention, the server includes means for operation from an information terminal, means for coordinating with generative artificial intelligence, and means for creating an interactive program utilizing generative artificial intelligence. This makes it possible for users to obtain information about the content they are viewing in real time and to resolve their questions immediately.

[0197] An "information terminal" is an electronic device used by a user to operate, and includes smartphones, tablets, and other similar devices.

[0198] "Generative artificial intelligence" is an artificial intelligence technology that uses natural language processing and machine learning to generate appropriate responses to user input.

[0199] An "interactive program" is software that provides information through interaction with the user, and includes chatbots and similar applications.

[0200] "Content being viewed" refers to the video or audio media that the user is currently viewing, including movies and television programs.

[0201] "Real-time acquisition" means providing information immediately in response to user requests, and receiving a response without delay.

[0202] One embodiment of the invention provides a system that allows a user to obtain information about the content they are viewing using an information terminal in real time. This system uses an interactive program that utilizes generative artificial intelligence to instantly generate responses to questions from the user.

[0203] The server works in conjunction with generative artificial intelligence to analyze user input. Specifically, it uses natural language processing technology to understand user questions and retrieve relevant information from a database. The server then provides the retrieved information to the user through an interactive program. In this process, the server uses chatbot engines such as Dialogflow or Rasa, and databases such as Firebase or MongoDB.

[0204] Information terminals receive input from users through a user interface and transmit it to a server. The user interface operates on smartphones and tablets, enabling intuitive operation.

[0205] For example, if a user asks "Who directed this movie?" while watching a film, the information terminal sends this question to the server. The server analyzes the question, retrieves the director's information from the database, and returns the answer to the user.

[0206] An example of a prompt would be: "Design a chatbot that provides relevant information in real time when a user enters a question about the content they are watching. For example, if a user asks about the director of a film, include a feature that will quickly provide that information."

[0207] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0208] Step 1:

[0209] The user enters questions about the content they are viewing using an information terminal. The entered questions are sent to the server through the information terminal's user interface. The input data consists of questions written in natural language by the user.

[0210] Step 2:

[0211] The server analyzes the received question using natural language processing technology. Specifically, it utilizes a generative AI model to understand the intent of the question and extract relevant keywords. This process clarifies the content of the question and provides the information necessary for the database search in the next step.

[0212] Step 3:

[0213] The server searches the database based on the analysis results and retrieves relevant information. The database contains detailed information about the content being viewed. The server executes database queries using the extracted keywords and retrieves the corresponding information.

[0214] Step 4:

[0215] The server provides the acquired information to the user through an interactive program. Specifically, it uses a generative AI model to generate information in a format that is easy for the user to understand and sends it to the information terminal. The output data is the answer to the user's question.

[0216] Step 5:

[0217] The information terminal displays the answers received from the server on the user interface. Users can obtain information in real time to resolve questions about the content they are viewing.

[0218] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0219] "Example of form 1"

[0220] One embodiment of the present invention is a system incorporating an emotion engine. This system analyzes emotions from a user's text input. Specifically, when a user inputs text to a chatbot, the emotion engine analyzes the user's emotions from that text. The analyzed emotions are sent to a generative AI, which then generates a response based on those emotions. For example, if a user inputs the text "I'm very sad today," the emotion engine analyzes the emotion "sadness" from that text. The generative AI then generates an appropriate response for the emotion "sadness."

[0221] "Example of form 2"

[0222] Another embodiment of the present invention involves a system in which an emotion engine analyzes emotions from a user's voice input. Specifically, when a user provides voice input to a chatbot, the emotion engine analyzes the user's emotions from that voice. The analyzed emotions are sent to a generating AI, which then generates a response based on those emotions. For example, if a user inputs the voice "I'm very happy today," the emotion engine analyzes the emotion "joy" from that voice. The generating AI then generates an appropriate response to the emotion "joy."

[0223] "Example of form 3"

[0224] Furthermore, in another embodiment of the present invention, there is a system in which an emotion engine analyzes emotions from a user's facial expression. Specifically, when a user makes a facial expression to a chatbot, the emotion engine analyzes the user's emotion from that facial expression. The analyzed emotion is sent to a generating AI, which then generates a response based on that emotion. For example, if a user makes a facial expression of "surprise," the emotion engine analyzes the emotion of "surprise" from that facial expression. The generating AI then generates an appropriate response to the emotion of "surprise."

[0225] "Example of form 4"

[0226] Another embodiment of the present invention involves a system in which an emotion engine analyzes emotions from a user's behavioral patterns. Specifically, when a user exhibits a particular behavioral pattern to a chatbot, the emotion engine analyzes the user's emotions from that behavioral pattern. The analyzed emotions are sent to a generating AI, which then generates a response based on those emotions. For example, if a user exhibits a behavioral pattern such as "suddenly typing faster," the emotion engine analyzes that behavioral pattern to determine an emotion such as "anger." The generating AI then generates an appropriate response to that emotion.

[0227] The following describes the processing flow for each example of the form.

[0228] "Example of form 1"

[0229] Step 1: The user enters text into the chatbot.

[0230] Step 2: The emotion engine analyzes the user's emotions from the text.

[0231] Step 3: The analyzed emotions are sent to the generating AI.

[0232] Step 4: The generating AI generates a response based on that emotion.

[0233] "Example of form 2"

[0234] Step 1: The user provides voice input to the chatbot.

[0235] Step 2: The emotion engine analyzes the user's emotions from the audio.

[0236] Step 3: The analyzed emotions are sent to the generating AI.

[0237] Step 4: The generating AI generates a response based on that emotion.

[0238] "Example of form 3"

[0239] Step 1: The user makes a facial expression to the chatbot.

[0240] Step 2: The emotion engine analyzes the user's emotions from their facial expressions.

[0241] Step 3: The analyzed emotions are sent to the generating AI.

[0242] Step 4: The generating AI generates a response based on that emotion.

[0243] "Example of form 4"

[0244] Step 1: The user demonstrates a specific behavioral pattern to the chatbot.

[0245] Step 2: The emotion engine analyzes the user's emotions based on their behavioral patterns.

[0246] Step 3: The analyzed emotions are sent to the generating AI.

[0247] Step 4: The generating AI generates a response based on that emotion.

[0248] (Example 1)

[0249] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0250] Traditional interactive programs have struggled to accurately analyze user emotions and generate responses based on them. Furthermore, there was a lack of efficient methods for updating the training data of the generative AI and for prototyping and testing interactive programs, creating a need for improved user experience.

[0251] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0252] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for creating an interactive program utilizing the generative AI, means for performing emotion analysis, and means for generating a response based on the analyzed emotion. This makes it possible to analyze the user's emotions and generate a natural response based on them. Furthermore, it enables efficient updating of the generative AI's training data and prototyping and testing of interactive programs, thereby improving the user experience.

[0253] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides an interface for user operation.

[0254] "Generative AI" is a system that uses artificial intelligence technology to learn from data and generate responses based on user input.

[0255] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[0256] "Sentiment analysis" is a process that identifies emotions from user input data and generates appropriate responses based on those emotions.

[0257] "Response generation" is the process of providing users with appropriate information and messages based on analyzed data and emotions.

[0258] This invention is a system for a user to interact with a generated AI and execute an interactive program using an information processing device. The user operates a dedicated application on the information processing device and utilizes an interface for interacting with the generated AI. This application provides functions for the user to update the generated AI's training data and create interactive programs.

[0259] The server is equipped with an emotion engine for sentiment analysis. When a user enters text into an interactive program, the server receives the text and uses the emotion engine to analyze the user's emotions. The analyzed emotions are sent to a generative AI, which then generates an appropriate response based on those emotions.

[0260] For example, if a user types "I'm very sad today," the server analyzes this text using its emotion engine and identifies the emotion "sadness." The generative AI then generates an appropriate response to "sadness" and provides it to the user.

[0261] An example of a prompt is, "If the user enters 'I am very sad today,' explain how the emotion engine analyzes the emotion and what response the generative AI generates." Using this prompt, we can understand the process by which the generative AI model generates a response based on the user's emotions.

[0262] This system allows users to engage in natural, emotion-based conversations with the AI, thereby improving the user experience.

[0263] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0264] Step 1:

[0265] The user launches a dedicated application on the information processing device and authenticates by entering their account information on the login screen. If authentication is successful, the application's main screen is displayed. The input is the user's account information, and the output is the authentication result and the display of the main screen.

[0266] Step 2:

[0267] The user initiates interaction with the interactive program through the application's interface. The user enters a message in the text input field and presses the send button. The input is the user's text message, and the output is the message sent to the server.

[0268] Step 3:

[0269] The server receives text messages sent by users. The emotion engine analyzes these texts to identify the user's emotions. The input is the user's text message, and the output is the analyzed emotion data. Specifically, it extracts emotions using a text analysis algorithm.

[0270] Step 4:

[0271] The server sends the analyzed emotion data to a generative AI model. The generative AI model generates an appropriate response based on the received emotion. The input is the analyzed emotion data, and the output is the generated response message. Specifically, the process involves the generative AI model generating a response that corresponds to the emotion.

[0272] Step 5:

[0273] The device receives a response from the generating AI and displays it on the user's screen. The user can then review this response and continue the conversation. The input is the response message from the generating AI, and the output is the response displayed on the user's screen. Specifically, the device visually presents the received message to the user.

[0274] (Application Example 1)

[0275] Next, Application Example 1 of Embodiment 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0276] In modern information-overloaded society, it is difficult for users to quickly and accurately obtain information that matches their own emotions. In addition, since there is a lack of personalized information provision based on users' emotions, improvement of user experience is required.

[0277] The specifying process performed by the specifying processing unit 290 of the data processing device 12 in Application Example 1 is implemented by the following respective means.

[0278] In the present invention, the server includes: operation means from a smartphone; means for cooperating with generative AI; means for analyzing a user's emotions; and means for recommending information based on the analyzed emotions. This enables provision of personalized information based on the user's emotions.

[0279] The "operation means from a smartphone" is a method for a user to operate the system using a portable information terminal.

[0280] The "means for cooperating with generative AI" is a method for processing information by utilizing artificial intelligence technology and cooperating with other systems and data.

[0281] The "means for creating a chatbot utilizing generative AI" is a method for developing an automatic response program using artificial intelligence technology.

[0282] The "means for prototyping the created chatbot" is a method for testing the developed automatic response program and confirming its functions.

[0283] "Methods for allowing others to use a prototype chatbot" refers to methods for providing a tested automated response program so that other users can utilize it.

[0284] "Methods for analyzing user emotions" refer to methods for analyzing a user's emotional state based on the input information.

[0285] "Methods for recommending information based on analyzed emotions" refer to methods for presenting users with appropriate information and content based on analysis results.

[0286] The system for implementing this invention is primarily composed of a smartphone-based operating mechanism. The user can operate an application that interacts with a generative AI via the smartphone. This application sends the text entered by the user to an emotion analysis engine, which then analyzes the user's emotions. For emotion analysis, an emotion analysis engine such as IBM Watson® Tone Analyzer can be used.

[0287] The analyzed sentiment data is sent to a generative AI. The generative AI can use generative AI models such as OpenAI GPT-3 (registered trademark). Based on the analyzed sentiment, the generative AI recommends information and content suitable for the user. In this recommendation process, the generative AI selects movies, music, articles, etc., that match the user's sentiment and displays them on the smartphone screen.

[0288] For example, if a user enters "I'm very happy today," the emotion analysis engine will analyze this as the emotion "joy." The generative AI will then recommend cheerful movies or upbeat music that match this "joy." In this case, an example of a prompt to input into the generative AI model would be: "The user entered 'I'm very happy today.' Please recommend content that matches this joy."

[0289] In this way, personalized information based on the user's emotions becomes possible, improving the user experience.

[0290] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0291] Step 1:

[0292] The user launches the application on their smartphone and enters a sentence expressing their emotion into the text input field. This entered text becomes the input data for the emotion analysis engine.

[0293] Step 2:

[0294] The device sends the entered text to the sentiment analysis engine. The sentiment analysis engine analyzes the text data and identifies the user's emotions. This analysis process uses natural language processing techniques to classify the emotions in the text into categories such as "joy" and "sadness," and outputs the results.

[0295] Step 3:

[0296] The server sends the sentiment data received from the sentiment analysis engine to the generative AI. The generative AI uses the received sentiment data as input to generate content suitable for the user. In this process, the generative AI model selects content based on sentiment and generates prompt sentences for recommendation.

[0297] Step 4:

[0298] The server receives output from the generating AI and determines which content to recommend to the user. Specifically, it organizes the information such as movies, music, and articles selected by the generating AI and prepares the data to be presented to the user.

[0299] Step 5:

[0300] The terminal displays the recommended content received from the server to the user. The user can check and select personalized content based on emotion on the screen of the smartphone.

[0301] (Example 2)

[0302] Next, Example 2 of Mode 2 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0303] Conventional interactive programs have a problem that it is difficult to generate responses that consider user emotions, which limits user experience. There has also been a demand for a system that can efficiently learn response patterns set by the user and respond appropriately.

[0304] The specifying processing performed by the specifying processing unit 290 of the data processing apparatus 12 in Example 2 is implemented by the following means.

[0305] In the present invention, the server includes means for operating from an information terminal, means for cooperating with generative AI, means for creating an interactive program utilizing generative AI, means for analyzing emotion from voice input, and means for generating a response based on the analyzed emotion. This enables natural dialogue that takes into account the user's emotion.

[0306] An "information terminal" is an electronic device operated by a user, and is an apparatus that inputs and outputs data.

[0307] "Generative AI" is a system that generates and learns data using artificial intelligence technology.

[0308] An "interactive program" is software intended for dialogue with a user, and has a function of generating a response in accordance with the user's input.

[0309] "Voice input" is a method by which a user transmits information to the system through voice.

[0310] "Methods for analyzing emotions" refer to technologies that identify and analyze a user's emotions from voice or text.

[0311] "Means for generating a response" refers to techniques for creating an appropriate response based on analyzed information.

[0312] One embodiment of this invention is a system that uses a conversational program utilizing generative AI to achieve natural dialogue with the user. A specific embodiment is shown below.

[0313] The user uses an information terminal to set response patterns for an interactive program. The user inputs answers to specific questions and sends this information to the server. The server stores these response patterns in a database and uses them to train a generative AI model. This allows the generative AI model to respond appropriately to similar questions in the future.

[0314] When a user provides voice input, the device sends the voice data to a server. The server uses emotion analysis techniques to analyze the user's emotions from the voice data. The analyzed emotion data is sent to a generative AI model, which generates an appropriate response based on those emotions.

[0315] For example, if a user inputs "I'm very happy today" via voice input, the means of analyzing the emotion will analyze it as "joy." Then, the generative AI model will generate a response such as "That's wonderful! Did something special happen today?"

[0316] An example of a prompt is, "If the user says, 'I'm very happy today,' generate a response that expresses joy." By using this prompt, the AI ​​can generate an appropriate response that matches the user's emotions.

[0317] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0318] Step 1:

[0319] The user uses an information terminal to set response patterns for an interactive program. The user inputs answers to specific questions and sends this information to the server. The input is a question-and-answer pair set by the user, and the output is a response pattern stored on the server. The server saves this data to a database to prepare for future learning.

[0320] Step 2:

[0321] The server trains a generative AI model using stored response patterns. The input is the response patterns stored in the database, and the output is the trained generative AI model. The server processes the data using the response patterns and updates the model's parameters.

[0322] Step 3:

[0323] The user provides voice input through an information terminal. The input is the user's voice data, which the terminal converts into a digital format and sends to the server. The output is the digital voice data sent to the server.

[0324] Step 4:

[0325] The server receives audio data and analyzes the user's emotions using emotion analysis techniques. The input is digital audio data, and the output is the analyzed emotion data. The server processes the audio data through an analysis algorithm to identify emotions.

[0326] Step 5:

[0327] The server sends the analyzed sentiment data to a generative AI model, which then generates an appropriate response. The input is sentiment data, and the output is the generated response. The generative AI model generates prompt sentences based on the sentiment data and creates an appropriate response.

[0328] Step 6:

[0329] The terminal displays the response received from the server to the user. The input is the response from the generative AI model, and the output is the response message presented to the user. The terminal communicates the response to the user through the screen or audio output.

[0330] (Application Example 2)

[0331] Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0332] In modern brick-and-mortar stores, there is a growing need to personalize customer interactions and improve customer satisfaction. However, traditional conversational agents have struggled to generate responses that take customer emotions into account, limiting their ability to improve the customer experience.

[0333] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0334] In this invention, the server includes means for operation from a smart device, means for coordinating with a generative AI, and means for analyzing emotions from voice input. This enables the generation of responses based on customer emotions, thereby providing a more personalized customer experience.

[0335] "Means of operation from smart devices" refers to methods for operating a system using mobile information terminals such as smartphones and tablets.

[0336] "Means of collaborating with generative AI" refers to means of exchanging data and instructions with generative AI models and utilizing the functions of AI.

[0337] "Methods for creating conversational agents" refers to methods for designing and building agents that engage in conversations with users using generative AI.

[0338] "Methods for prototyping conversational agents" refer to methods for testing the functions and response patterns of the conversational agents that have been created.

[0339] "Means of allowing others to use it" refers to means of making the prototype conversational agent available for use by other users.

[0340] "Methods for analyzing emotions from voice input" refer to methods for analyzing a user's voice data and identifying the emotions contained within it.

[0341] "Means for generating responses based on analyzed emotions" refers to means for generating appropriate responses based on the results of emotion analysis.

[0342] To implement this invention, it is necessary to build a system that combines a smart device, a server, a generative AI model, a speech recognition API, and an emotion analysis engine. The smart device receives the user's voice input and sends it to the server. The server converts the voice data into text data using the speech recognition API. Specifically, Google® Cloud Speech-to-Text can be used as the speech recognition API.

[0343] Next, the server uses an emotion analysis engine to analyze the user's emotions from the text data. This analysis can utilize emotion analysis engines such as IBM Watson Tone Analyzer. The analyzed emotion data is sent to a generative AI model. The generative AI model uses tools such as OpenAI GPT to generate appropriate responses based on those emotions.

[0344] The generated response is sent to a smart device and presented to the user. This allows the user to experience a dialogue that is tailored to their emotions.

[0345] For example, if a user voice-inputs "I'm very tired today" into a smart device, the server converts this voice into text and uses an emotion analysis engine to analyze the emotion "fatigue." The generative AI model then generates a suggestion in response to "fatigue," such as "You must be tired. Are you looking for something to help you relax?"

[0346] An example of a prompt message for a generative AI model is, "The user's emotion is fatigue. Please generate an appropriate response."

[0347] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0348] Step 1:

[0349] The user uses a smart device to perform voice input. The input voice data is captured through the smart device's microphone and sent to the server.

[0350] Step 2:

[0351] The server converts audio data into text data using a speech recognition API. Specifically, it analyzes the audio signal using Google Cloud Speech-to-Text and generates the corresponding text. The input for this step is audio data, and the output is text data.

[0352] Step 3:

[0353] The server analyzes the user's emotions from text data using an emotion analysis engine. Specifically, it uses IBM Watson Tone Analyzer to evaluate the emotional tone of the text and identify emotions such as "joy" or "fatigue." The input for this step is text data, and the output is emotion data.

[0354] Step 4:

[0355] The server inputs sentiment data into a generative AI model and generates an appropriate response. Specifically, it uses OpenAI GPT to generate natural, sentiment-based responses. The input for this step is sentiment data, and the output is response text.

[0356] Step 5:

[0357] The server sends the generated response text to the smart device. The smart device displays the received response to the user. This allows the user to experience a dialogue that is empathetic to their feelings.

[0358] (Example 3)

[0359] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0360] Traditional dialogue programs have a drawback: they generate responses without considering the user's emotions, resulting in a limited user experience. Furthermore, the process of effectively testing and sharing the response patterns of these dialogue programs with others is cumbersome.

[0361] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0362] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generation AI, and means for analyzing the user's emotions using an emotion analysis engine. This makes it possible to generate responses based on the user's emotions.

[0363] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones, tablets, and personal computers.

[0364] "Generative AI" refers to a system that uses artificial intelligence technology to analyze data and generate responses or information.

[0365] A "dialogue program" is software that mimics a conversation with a user and provides responses to questions.

[0366] An "emotion analysis engine" is a technology that analyzes a user's facial expressions and behavioral patterns to identify their emotions.

[0367] A "response pattern" is a pre-configured format of response that a dialogue program uses to respond to user input.

[0368] "Training data" refers to the dataset that a generative AI uses to generate responses, and is updated to improve the AI's performance.

[0369] One embodiment of this invention is a system in which a user operates a dialogue program using an information terminal and generates a response in cooperation with a generative AI. The user accesses the dialogue program through the information terminal and inputs a question. The terminal sends this input to a server, and the server uses the generative AI to generate an appropriate response.

[0370] The server uses an emotion analysis engine to analyze the user's facial expressions and behavioral patterns. For example, if the user smiles at the camera, the device sends the image to the server. The server uses the emotion analysis engine to identify the emotion as "joy" and sends it to a generative AI model. Based on this emotion, the generative AI model generates a response such as "You look happy!" and sends it back to the device.

[0371] Furthermore, when users test the response patterns of the dialogue program, they input various questions through the terminal and check the responses generated by the server. This allows users to improve the accuracy of the dialogue program.

[0372] For example, if a user asks "What's the weather like today?", the server will generate a response such as "It's sunny today" and display it on the terminal. An example of a prompt would be "Generate a response for when the user makes a surprised face." By inputting this prompt into the AI ​​model, an appropriate response can be obtained. The flow of specific processing in Example 3 will be explained using Figure 21.

[0373] Step 1:

[0374] The user accesses the interactive program using an information terminal and inputs a question. The terminal receives this input and sends it to the server. The input is in text format and is a question that includes the user's intent.

[0375] Step 2:

[0376] The server analyzes the received question and sends it to the generative AI model. The server uses natural language processing techniques to understand the intent of the question and formats the data so that the generative AI model can generate an appropriate response. The output is data in a format that the generative AI model can process.

[0377] Step 3:

[0378] The generative AI model receives data from the server and generates responses to questions. Based on pre-trained data, the generative AI model calculates the optimal answer to the user's question. The output is the response sentence that should be returned to the user.

[0379] Step 4:

[0380] The server receives the response from the generated AI model and sends it to the terminal. The server converts the response into a user-friendly format and prepares it for display on the terminal. The output is the text-based response displayed to the user.

[0381] Step 5:

[0382] The terminal displays the response received from the server to the user. The user can review this response and ask further questions if necessary. The terminal waits for user input and prepares to accept the next input.

[0383] Step 6:

[0384] When a user displays facial expressions or behavioral patterns, the device uses its camera and sensors to detect them and transmit the information to the server. The input consists of image data and behavioral data.

[0385] Step 7:

[0386] The server receives data from the terminal and analyzes the user's emotions using an emotion analysis engine. The server identifies the user's emotions using image processing and behavioral analysis technologies. The output is the analyzed emotion data.

[0387] Step 8:

[0388] The server sends the analyzed emotion data to a generative AI model, which then generates an emotion-based response. The generative AI model considers the emotion data and calculates a response appropriate to the user's emotions. The output is an emotion-based response sentence.

[0389] Step 9:

[0390] The server receives emotion-based responses from the generated AI model and sends them to the device. The device then displays these responses to the user, improving the user experience.

[0391] (Application Example 3)

[0392] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0393] In physical stores, there is a need to understand customer emotions in real time and provide appropriate services accordingly. However, traditional methods have the challenge of making it difficult to accurately analyze customer emotions and respond immediately.

[0394] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0395] In this invention, the server includes means for operation from a smart device, means for coordinating with a generative AI, and means for analyzing the user's facial expressions. This makes it possible to analyze customer emotions in real time and generate appropriate responses.

[0396] A "smart device" is an electronic device that is portable to the user and capable of connecting to the internet and running applications.

[0397] "Generative AI" is a system that uses artificial intelligence technology to learn from data and generate new information or responses.

[0398] A "conversational agent" is a software program that provides information and performs tasks through dialogue with the user.

[0399] "Prototyping" refers to the process of creating an initial model of a product or system to verify its functionality and performance.

[0400] "Methods for analyzing facial expressions" refer to technologies that detect the user's facial movements and expressions and infer their emotions based on that.

[0401] "A means of displaying emotions in real time" refers to a technology that instantly analyzes a user's emotions and displays the results immediately.

[0402] In order to implement this invention, it is necessary to construct a system that combines a smart device, a generative AI, an interactive agent, facial expression analysis technology, and real-time display technology.

[0403] The server receives input from smart devices and works with generative AI to create conversational agents. When a user communicates with the conversational agent through a smart device, the device captures the user's facial expressions with its camera and analyzes their emotions using facial recognition technology. This analysis uses a facial recognition library (e.g., OpenCV). The analyzed emotion data is sent to a generative AI model (e.g., OpenAI GPT) to generate an appropriate response.

[0404] The generated response is displayed in real time on the smart device's screen. This allows users to receive personalized services tailored to their emotions.

[0405] For example, if a user makes a surprised face when looking at a product, the device analyzes the emotion of "surprise" and sends a prompt message to the generative AI model saying, "The customer is showing a surprised expression. Please generate an appropriate response." Based on this prompt, the generative AI model generates the response, "Are you interested in this product?" and displays it to the user.

[0406] In this way, customer service in physical stores can be more personalized, and an improvement in customer satisfaction can be expected.

[0407] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0408] Step 1:

[0409] The device captures the user's facial expressions using a camera. The input is camera footage, and the output is facial feature point data. A facial recognition library (e.g., OpenCV) is used to extract facial feature points from the footage.

[0410] Step 2:

[0411] The server receives facial feature point data transmitted from the terminal and analyzes the user's emotions using facial expression analysis technology. The input is facial feature point data, and the output is the analyzed emotion data. The emotion analysis engine analyzes the feature point data and infers the emotion.

[0412] Step 3:

[0413] The server sends the analyzed sentiment data to a generative AI model. The input is sentiment data, and the output is the generated response. The generative AI model (e.g., OpenAI GPT) generates an appropriate response based on the sentiment data.

[0414] Step 4:

[0415] The device displays responses received from the generative AI model to the user in real time. The input is the generated response, and the output is the display on the screen. The device displays personalized messages to the user to facilitate interaction.

[0416] (Other examples)

[0417] Next, other embodiments will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0418] In recent years, dialogue systems utilizing generative AI models have become widespread, but mechanisms for appropriately analyzing user intent and generating responses based on that intent are not yet fully established. In particular, generating responses that take user emotions into account, applying custom responses, and appropriately updating training data remain challenges. Conventional systems can only provide uniform responses to user input, making it difficult to conduct dialogues tailored to the individual user's situation. Furthermore, because the training data of generative AI models is not properly updated, there is a problem in that the accuracy of dialogues cannot be sufficiently improved by reflecting user feedback.

[0419] The identification process performed by the identification processing unit 290 of the data processing device 12 in other embodiments is realized by the following means.

[0420] In this invention, the server includes a prompt generation means that receives user input data, analyzes the user's intent, and creates a prompt to instruct the generative AI model to generate a response; a response optimization means that evaluates the appropriateness of the response received from the generative AI model and optimizes it to suit the user's emotions and intentions; and a learning data update means that collects user feedback, analyzes the response data, and updates the learning data of the generative AI model. This enables the provision of appropriate dialogue according to the user's intentions and emotions, the application of custom responses, and the continuous learning of the generative AI model.

[0421] A "prompt" is an instruction or question given to a generative AI model to generate a response, and is text data optimized to obtain appropriate output according to the user's intent and context. A "generative AI model" is an artificial intelligence system based on deep learning that generates natural language text based on a given prompt. Specifically, it refers to AI models that utilize natural language processing technologies such as GPT-4, Claude, and Gemini. A "prompt generation means" is a means of analyzing user input data, creating an appropriate prompt based on its content and intent, and sending it to a generation AI model. A "response acquisition means" is a means of receiving a response from a generative AI model that sent a prompt, and processing that response by converting it into an appropriate format. A "response optimization method" is a means of providing the optimal response by evaluating the appropriateness of the response obtained from a generated AI model and making adjustments according to the user's emotions and input content. A "training data update method" is a means of improving response accuracy by collecting and analyzing user feedback and dialogue history data, and appropriately updating the training data of the generated AI model. "Emotion analysis methods" are means of analyzing user input text, voice data, or camera footage to estimate the user's emotions, using machine learning models to classify and determine the intensity of emotions. "Chatbot creation methods" refer to methods for building conversational programs using generative AI models to achieve natural dialogue with users. "Chatbot publishing methods" refer to methods for making a created chatbot available to other users online or locally. "Information recommendation methods" are means of analyzing a user's conversation history and interests, having an AI model generate optimal information, and then providing that information to the user. A "content information provision method" is a means of identifying the content a user is viewing, obtaining information related to that content, and presenting it to the user.

[0422] This invention provides a system that utilizes a generative AI model to provide responses by analyzing text, voice, and camera images input by the user and generating appropriate prompt sentences. This system links the user's terminal (smartphone, PC, tablet, etc.) with the generative AI model on a cloud server to realize dialogue based on the user's intent. The following components will be used in this system. User devices: Smartphones, tablets, PCs (OS: iOS, Android, Windows, macOS) Cloud servers: AWS Lambda, GCP Cloud Functions, Azure Functions Databases: PostgreSQL, MongoDB (for managing user interaction history and custom response settings) Generative AI models: GPT-4, Claude, Gemini (response generation using natural language processing) Emotion analysis engine: OpenAI Whisper (speech-to-text conversion), Microsoft Face API (facial expression analysis) The server receives input data sent from the user terminal, analyzes the user's intent, and creates an appropriate prompt. The created prompt is sent to a generative AI model, which generates a natural language response based on the prompt. The server then evaluates the appropriateness of the response, optimizes it as needed, and sends the response to the user terminal. Furthermore, this system is equipped with an emotion analysis engine to provide responses that take user emotions into consideration. The user's terminal acquires audio and camera video and sends them to the server, which performs emotion analysis and adjusts the prompt text based on the results. This makes it possible to provide appropriate responses according to the user's situation. Furthermore, this system has a function to improve the accuracy of the generated AI model by collecting user feedback and updating the training data. Specifically, the server accumulates user evaluation data and adjusts the prompt sentence generation logic based on the evaluation results, thereby continuously improving the overall system performance. Specific example This system generates prompt messages like the following and sends them to the AI ​​model. (1) Obtaining weather forecasts User input: "Tell me the weather for tomorrow." Prompt: "Please describe the weather in Tokyo tomorrow." Response from the generative AI model: "Tomorrow's weather in Tokyo will be sunny, with a high of 27°C and a low of 18°C." (2) Generation of emotion-based responses User input: "Tired" + (negative facial expression) Prompt: "The user is feeling tired; please create an encouraging message." Response from the generative AI model: "You did a great job today. Take a break and refresh yourself!" (3) Custom response settings User setting: "Please summarize the news every morning at 7 AM." Prompt: "Select five recent news articles and summarize each article in 50 characters or less." Response from the generative AI model: - "The Nikkei average has surpassed 30,000 yen." - "Typhoon No. 10 is approaching Okinawa" - "Apple announces new iPhone" - "Demand for overseas travel is recovering." - "AI will play an active role in the medical field" (4) Providing information about the content being viewed Video currently being watched by the user: ** "Historical Documentary: Sengoku Period" Prompt: "Please provide additional information about the Sengoku period." Response from the generative AI model: "The Sengoku period is a significant part of Japanese history..." As described above, this system provides optimal responses by analyzing user input and generating appropriate prompt sentences. Furthermore, it is possible to improve dialogue accuracy by combining sentiment analysis and updating the learning data.

[0423] The flow of specific processing in other embodiments will be explained using Figure 23.

[0424] Step 1: The device retrieves the user's input data. Input: The user enters text, voice, or camera footage. process: The terminal receives user input and processes it according to the data format. For text input, the data entered via the keyboard is retrieved directly. For voice input, the device uses the Google Speech-to-Text API to convert the voice data into text. For camera footage, the Microsoft Face API is used to analyze the user's facial expression data. output: - Text data (e.g., "What's the weather like tomorrow?") - Text data converted from speech - User facial expression data (e.g., "positive," "negative") Step 2: The terminal sends data to the server. Input: Text data obtained in Step 1, or converted audio data, facial expression data. process: The terminal converts user input data into JSON format and sends an HTTPS request to the cloud server. The transmitted data includes the user's ID, speech content, sentiment data, and timestamp. output: - JSON data sent to the cloud server Step 3: The server analyzes the user's intent and generates a prompt message. Input: User text data, facial expression data process: The server uses an NLP (Natural Language Processing) model to analyze user input. Based on the analysis results, an appropriate prompt message is generated. If sentiment data is included, the prompt message will be adjusted to reflect that sentiment. Specific actions: Example 1 (Obtaining weather forecasts) Input: "Tell me the weather for tomorrow." Prompt: "Please describe the weather in Tokyo tomorrow." Example 2 (Response that takes emotions into consideration) Input: "tired" + "negative facial expression" Prompt: "The user is feeling tired; please create an encouraging message." output: Prompt message to send to the generative AI model Step 4: The server sends a prompt message to the AI ​​model and retrieves a response. Input: The prompt generated in Step 3 process: The server sends prompt messages to the AI ​​model that generates them (GPT-4, Claude, Gemini, etc.). The generative AI model analyzes the prompt text and generates the optimal response. Specific actions: Example 1 (Weather forecast) Prompt: "Please describe the weather in Tokyo tomorrow." AI-generated response: "Tomorrow's weather in Tokyo will be sunny, with a high of 27°C and a low of 18°C." Example 2 (An appropriate response for the emotion) Prompt: "The user is feeling tired; please create an encouraging message." AI-generated response: "You did a great job today. Take a break and refresh yourself!" output: Response data generated by the generative AI model Step 5: The server evaluates the appropriateness of the response and performs optimization. Input: Response data from a generative AI model process: Evaluate the appropriateness of the response (naturalness of language, accuracy, and emotional consistency). Adjust the response as needed. Specific actions: Filtering out incorrect information: **Regenerate weather forecast responses if they do not match the latest data.** Emotional adjustment: If an overly cold response is generated to a negative user, modify it to a gentler expression. output: Optimized response to send to the user terminal Step 6: The server sends a response to the user terminal, and the terminal outputs the response. Input: Optimized response data process: The server sends the response data to the user's terminal. The device displays the received response in an appropriate format (text, voice, notification). Specific actions: Text display: "Tomorrow's weather in Tokyo will be sunny, with a high of 27°C and a low of 18°C." will be displayed on the screen. Text-to-speech: Uses Google TTS (Text-to-Speech API) to deliver responses via voice. Push notification: Sends a notification saying, "We have summarized the latest news articles." output: Response data displayed and communicated to the user Step 7: The server collects user feedback and updates the training data. Input: User feedback (e.g., "Helpful," "Not helpful") process: The device allows users to send feedback on the response. The server stores the feedback in a database and uses it to improve response patterns that receive low ratings. If necessary, change the method for generating prompts and update the training data for the generating AI model. Specific actions: If a user rates it as "unhelpful," then adjust the prompt text accordingly. If a particular response consistently receives low ratings, train the generative AI model with additional data. output: Updated prompt generation logic, training data In this way, this system analyzes user input data, creates and sends prompt messages, obtains the optimal response from a generated AI model, and provides it to the user. Furthermore, continuous improvement can be achieved by utilizing feedback.

[0425] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0426] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0427] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0428] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0429] [Second Embodiment]

[0430] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0431] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0436] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0437] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0438] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0441] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0442] "Example of form 1"

[0443] One embodiment of the present invention uses a dedicated application as a means of operation from a smartphone. This application provides an interface for the user to interact with the generative AI. Specifically, it includes functions for the user to update the generative AI's learning data and functions for creating a chatbot that utilizes the generative AI.

[0444] "Example of form 2"

[0445] One embodiment of the present invention provides a means for creating a chatbot that utilizes generative AI, which includes a function for the user to set the chatbot's response patterns. Specifically, the user sets answers to specific questions, and these answers are learned by the generative AI.

[0446] "Example of form 3"

[0447] One embodiment of the present invention provides a function for users to test the response patterns of a chatbot as a means of prototyping a created chatbot. Specifically, the user asks a question to the chatbot and verifies whether the answer is appropriate.

[0448] "Example of form 4"

[0449] One embodiment of the present invention provides a function that allows a user to publish a chatbot as a means of making a prototype chatbot available for use by others. Specifically, a user publishes a chatbot, making it available for others to use.

[0450] The following describes the processing flow for each example of the form.

[0451] "Example of form 1"

[0452] Step 1: The user installs a dedicated application on their smartphone. Step 2: The user opens the application and operates the interface for interacting with the generating AI.

[0453] Step 3: The user updates the training data for the generated AI using the function to update the training data for the generated AI.

[0454] Step 4: Create a chatbot using the user-generated AI-powered chatbot creation feature.

[0455] "Example of form 2"

[0456] Step 1: Use the feature that allows users to set chatbot response patterns.

[0457] Step 2: The user sets the answer to a specific question.

[0458] Step 3: The set responses are learned by the generating AI, and the chatbot's response patterns are updated.

[0459] "Example of form 3"

[0460] Step 1: Use the feature that allows users to test the chatbot's response patterns.

[0461] Step 2: The user asks a question to the chatbot.

[0462] Step 3: The chatbot's response is displayed, and the user checks if the response is appropriate.

[0463] "Example of form 4"

[0464] Step 1: Use the feature that allows users to publish chatbots.

[0465] Step 2: The user publishes the chatbot, and the published chatbot becomes accessible to others.

[0466] Step 3: Others use a publicly available chatbot and interact with it.

[0467] (Example 1)

[0468] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0469] In modern artificial intelligence systems using information processing equipment, there is a need to provide an environment that allows users to easily interact with AI and create and prototype interactive programs. Furthermore, efficiently updating the AI's training data and generating appropriate responses to user input are key challenges.

[0470] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0471] In this invention, the server includes means for operating from an information processing device, means for coordinating with artificial intelligence, and means for creating interactive programs utilizing artificial intelligence. This allows the user to update the AI's learning data, prototype interactive programs, and allow others to use them.

[0472] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides a means for users to operate it.

[0473] Artificial intelligence is a technology that enables computer systems to mimic human intellectual behavior, possessing the ability to learn from data and perform reasoning and judgment.

[0474] "Means of collaboration" refers to methods and processes that enable different systems or components to exchange information and work together in coordination.

[0475] An "interactive program" is software that provides information and answers questions through dialogue with the user, and it operates using artificial intelligence.

[0476] "Methods for prototyping" refer to methods and processes for experimentally creating new products or systems and evaluating their performance and functionality.

[0477] "Training data" refers to the dataset used by artificial intelligence for learning, and it forms the basis for improving the accuracy and performance of the model.

[0478] "Input text" refers to text data that a user inputs into an information processing device, and it serves as the basis for artificial intelligence to generate a response.

[0479] A description of embodiments for carrying out this invention will be given.

[0480] The user launches a dedicated application on their smartphone, which acts as an information processing device. This application provides an interface for the user to interact with artificial intelligence. The user logs into the application, selects a dataset to update the AI's training data, and uploads it to the server via their device. The server uses the received dataset to update the training data for the AI ​​model. This process utilizes machine learning libraries such as TensorFlow and PyTorch.

[0481] The server retrains the artificial intelligence model based on the uploaded data, improving the model's accuracy. Once training is complete, the server notifies the user, who can then use the generated AI to create an interactive program. The user uses the application's features to set the objective and response patterns of the interactive program.

[0482] For example, if a user enters a prompt such as "Tell me the features of the new product," the terminal sends this prompt to the server. The server uses a generative AI model to generate a response to the prompt and sends it back to the terminal. The user can then review the generated response and re-enter the prompt if necessary.

[0483] In this way, users can operate artificial intelligence through an information processing device to update data, create interactive programs, and generate responses using prompt statements.

[0484] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0485] Step 1:

[0486] The user launches a dedicated application on their smartphone, which acts as an information processing device, and logs in. They enter their login information and send it to the server. The server authenticates the received login information and grants the user access. This allows the user to access the application's functions.

[0487] Step 2:

[0488] The user selects a new dataset to update the training data for a generative AI model. The device uploads the selected dataset to the server. The server stores the received dataset and verifies its integrity. It preprocesses the data and converts it into a format suitable for training.

[0489] Step 3:

[0490] The server updates the training data for the generative AI model using the uploaded dataset. It then retrains the model using machine learning libraries such as TensorFlow or PyTorch. The updated model is obtained as output, using preprocessed data as input. Once training is complete, the server notifies the user.

[0491] Step 4:

[0492] Users create interactive programs that utilize generative AI by using the application's features. Users set the purpose and response patterns of the interactive program. The server builds the interactive program based on the settings and verifies its operation.

[0493] Step 5:

[0494] The user inputs a prompt message to the generative AI model. The terminal sends the input prompt message to the server. The server uses the generative AI model to generate a response to the prompt message. The prompt message is used as input, and the generated response is obtained as output. The response is sent back to the terminal, and the user confirms it.

[0495] (Application Example 1)

[0496] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0497] In today's information society, users are required to efficiently acquire useful information from a vast amount of data. However, conventional information provision systems have the problem of not adequately providing personalized information based on users' interests and preferences. Furthermore, even in information generation using generative AI, there is a lack of mechanisms to effectively reflect user feedback, which makes it difficult to improve the accuracy and relevance of the information.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for generating information based on the user's interests, and means for collecting the user's evaluation of the generated information and updating the generative AI's learning data. This makes it possible to provide personalized information to the user and improve the accuracy and relevance of the generative AI.

[0500] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and it is a device that can perform various functions through user operation.

[0501] "Generative AI" refers to a system that uses artificial intelligence technology to generate new information and content from data, providing appropriate output according to user requests.

[0502] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[0503] "Means of generating information based on user interests" refers to methods and technologies for analyzing user input and past behavioral history and generating highly relevant information based on that analysis.

[0504] "Means of collecting user evaluations of generated information and updating the training data of the generating AI" refers to methods and technologies that enable the generation of more accurate information by collecting feedback from users and improving the training data of the generating AI based on that feedback.

[0505] The system for carrying out this invention comprises an information processing device, a generative AI, and a server. The information processing device provides an interface for user operation and generates information in cooperation with the generative AI. The generative AI is responsible for receiving input from the user and generating relevant information based on it. The server provides the generated information to the user and collects user feedback to update the generative AI's learning data.

[0506] Specifically, the information processing device is a device such as a smartphone or tablet, which runs an application for the user to input topics of interest. This application is developed using React Native and provides the user interface. The topics entered by the user are sent as prompts to the generative AI. The generative AI uses OpenAI's GPT model and generates relevant content based on the input prompts.

[0507] The generated content is returned to the information processing device via the server and displayed to the user. The user evaluates the displayed content, and this evaluation is collected by the server. Based on the collected evaluations, the server updates the training data of the generating AI, improving the accuracy of future information generation.

[0508] For example, if a user enters "latest technology news," the AI ​​will generate articles such as "Latest Technology Trends of 2023" or "Articles on the Evolution of AI Technology." An example of a prompt would be, "Please tell me the latest technology news. I'm especially interested in information on AI and robotics." In this way, users can efficiently obtain information that matches their interests.

[0509] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0510] Step 1:

[0511] The user launches an application on the information processing device and enters a topic of interest. The entered topic is then prepared to be sent to the generating AI as a prompt. The input data is in text format and reflects the user's interests.

[0512] Step 2:

[0513] The terminal sends the input prompt to a generating AI model. The generating AI model uses OpenAI's GPT and generates relevant information based on the prompt. As a data processing step, it performs natural language processing to parse the prompt and generate the relevant information. The output is the generated text information.

[0514] Step 3:

[0515] The server sends the information received from the generating AI to the terminal. The terminal displays the received information to the user. The user reviews the displayed information and makes an evaluation. The evaluation is based on the relevance and usefulness of the information.

[0516] Step 4:

[0517] User ratings are sent from the device to the server. The server updates the training data for the generative AI based on the received ratings. As part of the data calculation, the rating data is analyzed and the model parameters of the generative AI are adjusted. This improves the accuracy of information generation in subsequent instances.

[0518] Step 5:

[0519] The server incorporates the updated training data into the generative AI. This allows the generative AI to generate information while incorporating user feedback. The output is the updated generative AI model.

[0520] (Example 2)

[0521] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0522] The challenge in interactive programs utilizing generative AI is to provide a system that allows users to flexibly configure responses to specific inquiries and efficiently train the generative AI model. Furthermore, it is necessary for the generative AI model to accurately understand the user's intent and effectively generate prompt sentences that produce appropriate responses.

[0523] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0524] In this invention, the server includes means for the user to set a response to a specific query, means for training a generating AI model with the set response, and means for generating prompt sentences for the generating AI model. This enables the generating AI model to efficiently learn the response patterns set by the user, allowing the generating AI model to accurately understand the user's intent and generate an appropriate response.

[0525] An "information terminal" is an electronic device used by a user, and includes devices such as computers and smartphones.

[0526] "Generative AI" is an artificial intelligence technology that performs natural language processing, and is particularly used as a model for generating responses in conversational programs.

[0527] An "interactive program" is software designed to interact with users, and includes systems such as chatbots.

[0528] A "user" is an individual or group that operates an interactive program and sets up responses to specific inquiries.

[0529] A "response pattern" is a pre-configured format for answering a specific inquiry, serving as a standard for how an interactive program responds to a user.

[0530] A "generative AI model" is a core algorithm of generative AI, a pre-trained model designed to generate natural language responses based on input data.

[0531] A "prompt" is an instruction given to a generative AI model to generate a specific response, and it serves as a clue for the model to understand the user's intent.

[0532] One embodiment of this invention is a system that constructs an interactive program utilizing generative AI, enabling users to set responses to specific inquiries. This system is implemented using an information terminal, a server, and a generative AI model.

[0533] Users configure response patterns for interactive programs using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions they want to set and their corresponding answers. For example, a user can set the answer to the question "What is the weather forecast?" to "Today's weather is sunny."

[0534] The terminal sends the user's entered question and answer data to the server. The server analyzes the received data and generates prompts to train a generative AI model. These prompts are designed so that the generative AI model accurately understands the user's intent and generates appropriate responses. A concrete example of a prompt would be, "If the user asks, 'What is the weather forecast?', answer, 'Today's weather is sunny.'"

[0535] The server passes the generated prompt text to the generative AI model for training. The generative AI model, for example, uses an algorithm specialized in natural language processing to memorize the response patterns set by the user. This allows the generative AI model to generate an appropriate response and send it to the terminal when a user asks a question to the chatbot.

[0536] In this way, users can freely set and customize the response patterns of conversational programs that utilize generative AI.

[0537] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0538] Step 1:

[0539] The user configures the response patterns for the interactive program using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions and answers they wish to configure. The input data consists of question-and-answer pairs. This data is stored on the terminal as response patterns that reflect the user's intent.

[0540] Step 2:

[0541] The terminal sends the user-entered question and answer data to the server. The transmitted data is structured as an HTTP request. The server parses the received data and prepares it for use in the next step. Specifically, it verifies the data's integrity and converts the format as needed.

[0542] Step 3:

[0543] The server generates prompt sentences to train a generative AI model based on the received questions and answers. The input data is question-and-answer pairs, and the output is the generated prompt sentences. Specifically, the server creates a prompt sentence in the format of "When the user asks 'What is the weather forecast?', please answer 'Today's weather is sunny.'"

[0544] Step 4:

[0545] The server passes the generated prompt sentences to the generative AI model for training. The input is the prompt sentence, and the output is the trained response pattern. The generative AI model uses a natural language processing algorithm to memorize the response patterns set by the user.

[0546] Step 5:

[0547] When a user asks a question to the chatbot, the server invokes a generative AI model to generate an appropriate response. The input is the user's question, and the output is the generated response. The generated response is sent to the terminal and displayed to the user. Specifically, if the user asks, "What's the weather forecast?", the chatbot will respond, "Today's weather is sunny."

[0548] (Application Example 2)

[0549] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0550] Traditional interactive programs make it difficult for users to set individual response patterns, and the responses provided by the generating AI may not always match the user's needs. Furthermore, because the data used for training the generating AI is fixed, it faces the challenge of not being able to respond flexibly to the user's business needs.

[0551] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0552] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generative AI, and means for creating an interactive program utilizing the generative AI. This allows the user to freely set the response patterns of the interactive program, and the generative AI to learn based on those settings and select the optimal response.

[0553] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones and tablets.

[0554] "Generative AI" is a system that uses artificial intelligence technology to learn from data and perform natural language processing.

[0555] An "interactive program" is software that provides information or answers questions through dialogue with the user.

[0556] A "response pattern" refers to a pre-set format or content of answers to a specific question.

[0557] "Means of learning" refers to the process by which generative AI acquires new knowledge based on data provided by users and improves the accuracy of its responses.

[0558] "The means of selecting the optimal response" refers to a function that automatically selects the most appropriate answer to a user's question based on the data the generation AI has learned.

[0559] A system for carrying out this invention includes an information terminal, a generative AI model, and a server. The information terminal is a device for user operation, such as a smartphone or tablet. The user can use the information terminal to set response patterns for an interactive program.

[0560] The server works in conjunction with a generative AI model, which learns based on response patterns set by the user. The generative AI model is a system for natural language processing, acquiring new knowledge based on data provided by the user and improving the accuracy of its responses.

[0561] Specifically, when a user enters FAQ questions and answers via an information terminal, that data is stored on a server. The server provides the stored data to a generative AI model, which then learns from it. This allows the generative AI model to generate flexible responses tailored to the user's business needs.

[0562] For example, if a user asks "How do I return an item?" and the answer is set to "Returns are possible within 30 days of receiving the item. Please use the return label," the generative AI model will learn from this information and provide appropriate responses to similar questions.

[0563] Examples of prompts used to train a generative AI model include the following:

[0564] "User question: 'How do I return an item?' Set answer: 'Returns are possible within 30 days of receiving the item. Please use the return label.' Prompt to train the generating AI model: 'Generate an appropriate response to the question about how to return an item.'"

[0565] In this way, users can freely customize the responses of the interactive program, and the generating AI model can select the optimal response.

[0566] The flow of the specific processing in Application Example 2 will be explained using Figure 14.

[0567] Step 1:

[0568] The user sets the response patterns for the interactive program using an information terminal. Specifically, the user inputs FAQ questions and their answers. The entered data is sent from the information terminal to the server.

[0569] Step 2:

[0570] The server stores the question and answer data received from the user in a database. This stored data serves as the foundational data for the generative AI model to learn from. The server then prepares the data stored in the database to be provided to the generative AI model.

[0571] Step 3:

[0572] The server provides stored data to the generative AI model, which then learns. Specifically, the server generates prompt sentences and inputs them into the generative AI model. The generative AI model performs natural language processing based on the prompt sentences and learns response patterns set by the user.

[0573] Step 4:

[0574] The generative AI model generates the optimal response to a new question from the user based on the data it has learned. The server provides the user with the response obtained from the generative AI model. Specifically, the server sends the response to the user's information terminal so that the user can confirm it.

[0575] Step 5:

[0576] Users can review the responses generated through their information terminals and readjust the response patterns as needed. This allows users to continuously improve the responses of the interactive program and enhance the accuracy of the generated AI model.

[0577] (Example 3)

[0578] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0579] Traditional dialogue program development has presented challenges in efficient development and operation, as testing and publishing response patterns is difficult, and developers must manually manage many processes. Furthermore, the lack of a process to evaluate the appropriateness of responses using generative AI models has made it difficult to provide dialogue programs that are highly satisfying to users.

[0580] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0581] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI model, means for creating a dialogue program utilizing the generative AI model, means for testing the response patterns of the created dialogue program, and means for making the tested dialogue program public to others. This makes it possible to efficiently develop a dialogue program, evaluate the appropriateness of the responses, and make it public to others.

[0582] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and is equipped with an interface for user operation.

[0583] A "generative AI model" is a model that uses artificial intelligence technology to perform natural language processing and has the ability to generate appropriate responses to user input.

[0584] A "dialogue program" is software designed for interaction with users and has the function of providing responses to user questions using a generative AI model.

[0585] "Means for testing response patterns" refers to a process for evaluating the appropriateness of responses generated by a dialogue program, and a means for verifying responses to user input.

[0586] "Means of publication" refers to the process of placing a dialogue program on the internet and making it accessible to others.

[0587] A description of embodiments for carrying out this invention will be given.

[0588] The user operates the dialogue program using an information processing device. The information processing device receives questions entered by the user and sends that data to the server. The server works in conjunction with a generative AI model, and upon receiving input from the user, passes that data to the generative AI model. The generative AI model uses natural language processing techniques to generate appropriate responses to the entered questions.

[0589] The generated response is returned to the information processing unit via the server. The user can review this response and evaluate whether the response pattern of the dialogue program is appropriate. For example, if the user asks "What's the weather like today?", the generating AI model will generate a response such as "It's sunny today."

[0590] Furthermore, users can upload tested dialogue programs to a server and make them publicly available for others to use. The server places the dialogue programs on the internet, making them accessible to others. This allows others to access and use the publicly available dialogue programs.

[0591] An example of a prompt statement is, "Create a dialogue program that uses a generative AI model to generate appropriate responses to questions entered by the user." By using this prompt statement, the generative AI model generates responses that meet the user's requests, enabling efficient development and operation of the dialogue program. The flow of specific processing in Example 3 will be explained using Figure 15.

[0592] Step 1:

[0593] The user enters the question on their device.

[0594] The user enters a question through the terminal's interface. This input data is sent to the server in text format. For example, the user might enter a question such as, "What's the news today?"

[0595] Step 2:

[0596] The server sends the question to the AI ​​model that generates it.

[0597] The server sends the text data received from the user to the generative AI model. At this time, the server converts the data into an appropriate format so that the generative AI model can process it. The generative AI model uses natural language processing techniques to analyze the question and prepares to generate an appropriate response.

[0598] Step 3:

[0599] The generative AI model generates the response.

[0600] The generative AI model generates responses based on the received question data, utilizing its internal training data. This process involves understanding the intent of the question, extracting relevant information, and constructing a response. For example, it might generate a response such as, "Today's main news is the rise in economic growth."

[0601] Step 4:

[0602] The server returns a response to the user.

[0603] The server receives response data from the generated AI model and sends it back to the user's terminal. At this time, the server converts the response data into a format that is easy for the user to understand. The user reviews this response and evaluates whether the dialogue program's response pattern is appropriate.

[0604] Step 5:

[0605] The user uploads the dialogue program to the server and makes it publicly available.

[0606] The user uploads the completed interactive program to the server. The server places the interactive program on the internet, making it accessible to others. This allows others to access and use the publicly available interactive program.

[0607] (Application Example 3)

[0608] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 as a "terminal".

[0609] In modern information distribution services, it is difficult for users to obtain detailed information about the content they are watching in real time. In particular, there is a lack of means to immediately resolve questions that arise during viewing, and there is a need to improve the user experience.

[0610] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0611] In this invention, the server includes means for operation from an information terminal, means for coordinating with generative artificial intelligence, and means for creating an interactive program utilizing generative artificial intelligence. This makes it possible for users to obtain information about the content they are viewing in real time and to resolve their questions immediately.

[0612] An "information terminal" is an electronic device used by a user to operate, and includes smartphones, tablets, and other similar devices.

[0613] "Generative artificial intelligence" is an artificial intelligence technology that uses natural language processing and machine learning to generate appropriate responses to user input.

[0614] An "interactive program" is software that provides information through interaction with the user, and includes chatbots and similar applications.

[0615] "Content being viewed" refers to the video or audio media that the user is currently viewing, including movies and television programs.

[0616] "Real-time acquisition" means providing information immediately in response to user requests, and receiving a response without delay.

[0617] One embodiment of the invention provides a system that allows a user to obtain information about the content they are viewing using an information terminal in real time. This system uses an interactive program that utilizes generative artificial intelligence to instantly generate responses to questions from the user.

[0618] The server works in conjunction with generative artificial intelligence to analyze user input. Specifically, it uses natural language processing technology to understand user questions and retrieve relevant information from a database. The server then provides the retrieved information to the user through an interactive program. In this process, the server uses chatbot engines such as Dialogflow or Rasa, and databases such as Firebase or MongoDB.

[0619] Information terminals receive input from users through a user interface and transmit it to a server. The user interface operates on smartphones and tablets, enabling intuitive operation.

[0620] For example, if a user asks "Who directed this movie?" while watching a film, the information terminal sends this question to the server. The server analyzes the question, retrieves the director's information from the database, and returns the answer to the user.

[0621] An example of a prompt would be: "Design a chatbot that provides relevant information in real time when a user enters a question about the content they are watching. For example, if a user asks about the director of a film, include a feature that will quickly provide that information."

[0622] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0623] Step 1:

[0624] The user enters questions about the content they are viewing using an information terminal. The entered questions are sent to the server through the information terminal's user interface. The input data consists of questions written in natural language by the user.

[0625] Step 2:

[0626] The server analyzes the received question using natural language processing technology. Specifically, it utilizes a generative AI model to understand the intent of the question and extract relevant keywords. This process clarifies the content of the question and provides the information necessary for the database search in the next step.

[0627] Step 3:

[0628] The server searches the database based on the analysis results and retrieves relevant information. The database contains detailed information about the content being viewed. The server executes database queries using the extracted keywords and retrieves the corresponding information.

[0629] Step 4:

[0630] The server provides the acquired information to the user through an interactive program. Specifically, it uses a generative AI model to generate information in a format that is easy for the user to understand and sends it to the information terminal. The output data is the answer to the user's question.

[0631] Step 5:

[0632] The information terminal displays the answers received from the server on the user interface. Users can obtain information in real time to resolve questions about the content they are viewing.

[0633] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0634] "Example of form 1"

[0635] One embodiment of the present invention is a system incorporating an emotion engine. This system analyzes emotions from a user's text input. Specifically, when a user inputs text to a chatbot, the emotion engine analyzes the user's emotions from that text. The analyzed emotions are sent to a generative AI, which then generates a response based on those emotions. For example, if a user inputs the text "I'm very sad today," the emotion engine analyzes the emotion "sadness" from that text. The generative AI then generates an appropriate response for the emotion "sadness."

[0636] "Example of form 2"

[0637] Another embodiment of the present invention involves a system in which an emotion engine analyzes emotions from a user's voice input. Specifically, when a user provides voice input to a chatbot, the emotion engine analyzes the user's emotions from that voice. The analyzed emotions are sent to a generating AI, which then generates a response based on those emotions. For example, if a user inputs the voice "I'm very happy today," the emotion engine analyzes the emotion "joy" from that voice. The generating AI then generates an appropriate response to the emotion "joy."

[0638] "Example of form 3"

[0639] Furthermore, in another embodiment of the present invention, there is a system in which an emotion engine analyzes emotions from a user's facial expression. Specifically, when a user makes a facial expression to a chatbot, the emotion engine analyzes the user's emotion from that facial expression. The analyzed emotion is sent to a generating AI, which generates a response based on that emotion. For example, if the user expresses "surprise"

[0640] When a facial expression is displayed, the emotion engine analyzes that expression to determine the emotion of "surprise." Then, the generative AI generates an appropriate response to the emotion of "surprise."

[0641] "Example of form 4"

[0642] Another embodiment of the present invention involves a system in which an emotion engine analyzes emotions from a user's behavioral patterns. Specifically, when a user exhibits a particular behavioral pattern to a chatbot, the emotion engine analyzes the user's emotions from that behavioral pattern. The analyzed emotions are sent to a generating AI, which then generates a response based on those emotions. For example, if a user exhibits a behavioral pattern such as "suddenly typing faster," the emotion engine analyzes that behavioral pattern to determine an emotion such as "anger." The generating AI then generates an appropriate response to that emotion.

[0643] The following describes the processing flow for each example of the form.

[0644] "Example of form 1"

[0645] Step 1: The user enters text into the chatbot.

[0646] Step 2: The emotion engine analyzes the user's emotions from the text.

[0647] Step 3: The analyzed emotions are sent to the generating AI.

[0648] Step 4: The generating AI generates a response based on that emotion.

[0649] "Example of form 2"

[0650] Step 1: The user provides voice input to the chatbot.

[0651] Step 2: The emotion engine analyzes the user's emotions from the audio.

[0652] Step 3: The analyzed emotions are sent to the generating AI.

[0653] Step 4: The generating AI generates a response based on that emotion.

[0654] "Example of form 3"

[0655] Step 1: The user makes a facial expression to the chatbot.

[0656] Step 2: The emotion engine analyzes the user's emotions from their facial expressions.

[0657] Step 3: The analyzed emotions are sent to the generating AI.

[0658] Step 4: The generating AI generates a response based on that emotion.

[0659] "Example of form 4"

[0660] Step 1: The user demonstrates a specific behavioral pattern to the chatbot.

[0661] Step 2: The emotion engine analyzes the user's emotions based on their behavioral patterns.

[0662] Step 3: The analyzed emotions are sent to the generating AI.

[0663] Step 4: The generating AI generates a response based on that emotion.

[0664] (Example 1)

[0665] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0666] Traditional interactive programs have struggled to accurately analyze user emotions and generate responses based on them. Furthermore, there was a lack of efficient methods for updating the training data of the generative AI and for prototyping and testing interactive programs, creating a need for improved user experience.

[0667] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0668] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for creating an interactive program utilizing the generative AI, means for performing emotion analysis, and means for generating a response based on the analyzed emotion. This makes it possible to analyze the user's emotions and generate a natural response based on them. Furthermore, it enables efficient updating of the generative AI's training data and prototyping and testing of interactive programs, thereby improving the user experience.

[0669] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides an interface for user operation.

[0670] "Generative AI" is a system that uses artificial intelligence technology to learn from data and generate responses based on user input.

[0671] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[0672] "Sentiment analysis" is a process that identifies emotions from user input data and generates appropriate responses based on those emotions.

[0673] "Response generation" is the process of providing users with appropriate information and messages based on analyzed data and emotions.

[0674] This invention is a system for a user to interact with a generated AI and execute an interactive program using an information processing device. The user operates a dedicated application on the information processing device and utilizes an interface for interacting with the generated AI. This application provides functions for the user to update the generated AI's training data and create interactive programs.

[0675] The server is equipped with an emotion engine for sentiment analysis. When a user enters text into an interactive program, the server receives the text and uses the emotion engine to analyze the user's emotions. The analyzed emotions are sent to a generative AI, which then generates an appropriate response based on those emotions.

[0676] For example, if a user types "I'm very sad today," the server analyzes this text using its emotion engine and identifies the emotion "sadness." The generative AI then generates an appropriate response to "sadness" and provides it to the user.

[0677] An example of a prompt is, "If the user enters 'I am very sad today,' explain how the emotion engine analyzes the emotion and what response the generative AI generates." Using this prompt, we can understand the process by which the generative AI model generates a response based on the user's emotions.

[0678] This system allows users to engage in natural, emotion-based conversations with the AI, thereby improving the user experience.

[0679] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0680] Step 1:

[0681] The user launches a dedicated application on the information processing device and authenticates by entering their account information on the login screen. If authentication is successful, the application's main screen is displayed. The input is the user's account information, and the output is the authentication result and the display of the main screen.

[0682] Step 2:

[0683] The user initiates interaction with the interactive program through the application's interface. The user enters a message in the text input field and presses the send button. The input is the user's text message, and the output is the message sent to the server.

[0684] Step 3:

[0685] The server receives text messages sent by users. The emotion engine analyzes these texts to identify the user's emotions. The input is the user's text message, and the output is the analyzed emotion data. Specifically, it extracts emotions using a text analysis algorithm.

[0686] Step 4:

[0687] The server sends the analyzed emotion data to a generative AI model. The generative AI model generates an appropriate response based on the received emotion. The input is the analyzed emotion data, and the output is the generated response message. Specifically, the process involves the generative AI model generating a response that corresponds to the emotion.

[0688] Step 5:

[0689] The device receives a response from the generating AI and displays it on the user's screen. The user can then review this response and continue the conversation. The input is the response message from the generating AI, and the output is the response displayed on the user's screen. Specifically, the device visually presents the received message to the user.

[0690] (Application Example 1)

[0691] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0692] In today's information-saturated society, it is difficult for users to quickly and accurately obtain information that resonates with their emotions. Furthermore, the lack of personalized information based on user emotions highlights the need for improved user experience.

[0693] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0694] In this invention, the server includes means for operation from a smartphone, means for coordinating with a generative AI, means for analyzing the user's emotions, and means for recommending information based on the analyzed emotions. This enables the provision of personalized information based on the user's emotions.

[0695] "Operation methods from a smartphone" refers to methods by which users operate a system using a mobile information terminal.

[0696] "Methods for collaborating with generative AI" refers to methods for processing information using artificial intelligence technology and collaborating with other systems and data.

[0697] "Methods for creating chatbots using generative AI" refers to methods for developing automated response programs using artificial intelligence technology.

[0698] "Methods for prototyping a created chatbot" refers to methods for testing the developed automated response program and verifying its functionality.

[0699] "Methods for allowing others to use a prototype chatbot" refers to methods for providing a tested automated response program so that other users can utilize it.

[0700] "Methods for analyzing user emotions" refer to methods for analyzing a user's emotional state based on the input information.

[0701] "Methods for recommending information based on analyzed emotions" refer to methods for presenting users with appropriate information and content based on analysis results.

[0702] The system for carrying out this invention is primarily composed of a smartphone-based operating mechanism. The user can operate an application that interacts with a generative AI via the smartphone. This application sends the text entered by the user to an emotion analysis engine, which then analyzes the user's emotions. An emotion analysis engine such as IBM Watson Tone Analyzer can be used for this analysis.

[0703] The analyzed sentiment data is sent to a generative AI. The generative AI can use generative AI models such as OpenAI GPT-3. Based on the analyzed sentiment, the generative AI recommends information and content suitable for the user. In this recommendation process, the generative AI selects movies, music, articles, etc., that match the user's sentiment and displays them on the smartphone screen.

[0704] For example, if a user enters "I'm very happy today," the emotion analysis engine will analyze this as the emotion "joy." The generative AI will then recommend cheerful movies or upbeat music that match this "joy." In this case, an example of a prompt to input into the generative AI model would be: "The user entered 'I'm very happy today.' Please recommend content that matches this joy."

[0705] In this way, personalized information based on the user's emotions becomes possible, improving the user experience.

[0706] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0707] Step 1:

[0708] The user launches the application on their smartphone and enters a sentence expressing their emotion into the text input field. This entered text becomes the input data for the emotion analysis engine.

[0709] Step 2:

[0710] The device sends the entered text to the sentiment analysis engine. The sentiment analysis engine analyzes the text data and identifies the user's emotions. This analysis process uses natural language processing techniques to classify the emotions in the text into categories such as "joy" and "sadness," and outputs the results.

[0711] Step 3:

[0712] The server sends the sentiment data received from the sentiment analysis engine to the generative AI. The generative AI uses the received sentiment data as input to generate content suitable for the user. In this process, the generative AI model selects content based on sentiment and generates prompt sentences for recommendation.

[0713] Step 4:

[0714] The server receives output from the generating AI and determines which content to recommend to the user. Specifically, it organizes the information such as movies, music, and articles selected by the generating AI and prepares the data to be presented to the user.

[0715] Step 5:

[0716] The device displays recommended content received from the server to the user. The user can view and select personalized content based on their emotions on their smartphone screen.

[0717] (Example 2)

[0718] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0719] Traditional interactive programs have struggled to generate responses that take user emotions into account, resulting in a limited user experience. Furthermore, there was a need for a system that could efficiently learn user-defined response patterns and respond appropriately.

[0720] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0721] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generative AI, means for creating an interactive program utilizing the generative AI, means for analyzing emotions from voice input, and means for generating a response based on the analyzed emotions. This enables natural dialogue that takes the user's emotions into consideration.

[0722] An "information terminal" is an electronic device operated by a user, and is a device that inputs and outputs data.

[0723] "Generative AI" is a system that uses artificial intelligence technology to generate and learn from data.

[0724] An "interactive program" is software designed for interaction with users and has the function of generating responses in response to user input.

[0725] "Voice input" is a method of transmitting information to a system through voice.

[0726] "Methods for analyzing emotions" refer to technologies that identify and analyze a user's emotions from voice or text.

[0727] "Means for generating a response" refers to techniques for creating an appropriate response based on analyzed information.

[0728] One embodiment of this invention is a system that uses a conversational program utilizing generative AI to achieve natural dialogue with the user. A specific embodiment is shown below.

[0729] The user uses an information terminal to set response patterns for an interactive program. The user inputs answers to specific questions and sends this information to the server. The server stores these response patterns in a database and uses them to train a generative AI model. This allows the generative AI model to respond appropriately to similar questions in the future.

[0730] When a user provides voice input, the device sends the voice data to a server. The server uses emotion analysis techniques to analyze the user's emotions from the voice data. The analyzed emotion data is sent to a generative AI model, which generates an appropriate response based on those emotions.

[0731] For example, if a user inputs "I'm very happy today" via voice input, the means of analyzing the emotion will analyze it as "joy." Then, the generative AI model will generate a response such as "That's wonderful! Did something special happen today?"

[0732] An example of a prompt is, "If the user says, 'I'm very happy today,' generate a response that expresses joy." By using this prompt, the AI ​​can generate an appropriate response that matches the user's emotions.

[0733] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0734] Step 1:

[0735] The user uses an information terminal to set response patterns for an interactive program. The user inputs answers to specific questions and sends this information to the server. The input is a question-and-answer pair set by the user, and the output is a response pattern stored on the server. The server saves this data to a database to prepare for future learning.

[0736] Step 2:

[0737] The server trains a generative AI model using stored response patterns. The input is the response patterns stored in the database, and the output is the trained generative AI model. The server processes the data using the response patterns and updates the model's parameters.

[0738] Step 3:

[0739] The user provides voice input through an information terminal. The input is the user's voice data, which the terminal converts into a digital format and sends to the server. The output is the digital voice data sent to the server.

[0740] Step 4:

[0741] The server receives audio data and analyzes the user's emotions using emotion analysis techniques. The input is digital audio data, and the output is the analyzed emotion data. The server processes the audio data through an analysis algorithm to identify emotions.

[0742] Step 5:

[0743] The server sends the analyzed sentiment data to a generative AI model, which then generates an appropriate response. The input is sentiment data, and the output is the generated response. The generative AI model generates prompt sentences based on the sentiment data and creates an appropriate response.

[0744] Step 6:

[0745] The terminal displays the response received from the server to the user. The input is the response from the generative AI model, and the output is the response message presented to the user. The terminal communicates the response to the user through the screen or audio output.

[0746] (Application Example 2)

[0747] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0748] In modern brick-and-mortar stores, there is a growing need to personalize customer interactions and improve customer satisfaction. However, traditional conversational agents have struggled to generate responses that take customer emotions into account, limiting their ability to improve the customer experience.

[0749] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0750] In this invention, the server includes means for operation from a smart device, means for coordinating with a generative AI, and means for analyzing emotions from voice input. This enables the generation of responses based on customer emotions, thereby providing a more personalized customer experience.

[0751] "Means of operation from smart devices" refers to methods for operating a system using mobile information terminals such as smartphones and tablets.

[0752] "Means of collaborating with generative AI" refers to means of exchanging data and instructions with generative AI models and utilizing the functions of AI.

[0753] "Methods for creating conversational agents" refers to methods for designing and building agents that engage in conversations with users using generative AI.

[0754] "Methods for prototyping conversational agents" refer to methods for testing the functions and response patterns of the conversational agents that have been created.

[0755] "Means of allowing others to use it" refers to means of making the prototype conversational agent available for use by other users.

[0756] "Methods for analyzing emotions from voice input" refer to methods for analyzing a user's voice data and identifying the emotions contained within it.

[0757] "Means for generating responses based on analyzed emotions" refers to means for generating appropriate responses based on the results of emotion analysis.

[0758] To implement this invention, it is necessary to build a system that combines a smart device, a server, a generative AI model, a speech recognition API, and an emotion analysis engine. The smart device receives the user's voice input and sends it to the server. The server converts the voice data into text data using the speech recognition API. Specifically, Google Cloud Speech-to-Text can be used as the speech recognition API.

[0759] Next, the server uses an emotion analysis engine to analyze the user's emotions from the text data. This analysis can utilize emotion analysis engines such as IBM Watson Tone Analyzer. The analyzed emotion data is sent to a generative AI model. The generative AI model uses tools such as OpenAI GPT to generate appropriate responses based on those emotions.

[0760] The generated response is sent to a smart device and presented to the user. This allows the user to experience a dialogue that is tailored to their emotions.

[0761] For example, if a user voice-inputs "I'm very tired today" into a smart device, the server converts this voice into text and uses an emotion analysis engine to analyze the emotion "fatigue." The generative AI model then generates a suggestion in response to "fatigue," such as "You must be tired. Are you looking for something to help you relax?"

[0762] An example of a prompt message for a generative AI model is, "The user's emotion is fatigue. Please generate an appropriate response."

[0763] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0764] Step 1:

[0765] The user uses a smart device to perform voice input. The input voice data is captured through the smart device's microphone and sent to the server.

[0766] Step 2:

[0767] The server converts audio data into text data using a speech recognition API. Specifically, it analyzes the audio signal using Google Cloud Speech-to-Text and generates the corresponding text. The input for this step is audio data, and the output is text data.

[0768] Step 3:

[0769] The server analyzes the user's emotions from text data using an emotion analysis engine. Specifically, it uses IBM Watson Tone Analyzer to evaluate the emotional tone of the text and identify emotions such as "joy" or "fatigue." The input for this step is text data, and the output is emotion data.

[0770] Step 4:

[0771] The server inputs sentiment data into a generative AI model and generates an appropriate response. Specifically, it uses OpenAI GPT to generate natural, sentiment-based responses. The input for this step is sentiment data, and the output is response text.

[0772] Step 5:

[0773] The server sends the generated response text to the smart device. The smart device displays the received response to the user. This allows the user to experience a dialogue that is empathetic to their feelings.

[0774] (Example 3)

[0775] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0776] Traditional dialogue programs have a drawback: they generate responses without considering the user's emotions, resulting in a limited user experience. Furthermore, the process of effectively testing and sharing the response patterns of these dialogue programs with others is cumbersome.

[0777] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0778] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generation AI, and means for analyzing the user's emotions using an emotion analysis engine. This makes it possible to generate responses based on the user's emotions.

[0779] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones, tablets, and personal computers.

[0780] "Generative AI" refers to a system that uses artificial intelligence technology to analyze data and generate responses or information.

[0781] A "dialogue program" is software that mimics a conversation with a user and provides responses to questions.

[0782] An "emotion analysis engine" is a technology that analyzes a user's facial expressions and behavioral patterns to identify their emotions.

[0783] A "response pattern" is a pre-configured format of response that a dialogue program uses to respond to user input.

[0784] "Training data" refers to the dataset that a generative AI uses to generate responses, and is updated to improve the AI's performance.

[0785] One embodiment of this invention is a system in which a user operates a dialogue program using an information terminal and generates a response in cooperation with a generative AI. The user accesses the dialogue program through the information terminal and inputs a question. The terminal sends this input to a server, and the server uses the generative AI to generate an appropriate response.

[0786] The server uses an emotion analysis engine to analyze the user's facial expressions and behavioral patterns. For example, if the user smiles at the camera, the device sends the image to the server. The server uses the emotion analysis engine to identify the emotion as "joy" and sends it to a generative AI model. Based on this emotion, the generative AI model generates a response such as "You look happy!" and sends it back to the device.

[0787] Furthermore, when users test the response patterns of the dialogue program, they input various questions through the terminal and check the responses generated by the server. This allows users to improve the accuracy of the dialogue program.

[0788] For example, if a user asks "What's the weather like today?", the server will generate a response such as "It's sunny today" and display it on the terminal. An example of a prompt would be "Generate a response for when the user makes a surprised face." By inputting this prompt into the AI ​​model, an appropriate response can be obtained. The flow of specific processing in Example 3 will be explained using Figure 21.

[0789] Step 1:

[0790] The user accesses the interactive program using an information terminal and inputs a question. The terminal receives this input and sends it to the server. The input is in text format and is a question that includes the user's intent.

[0791] Step 2:

[0792] The server analyzes the received question and sends it to the generative AI model. The server uses natural language processing techniques to understand the intent of the question and formats the data so that the generative AI model can generate an appropriate response. The output is data in a format that the generative AI model can process.

[0793] Step 3:

[0794] The generative AI model receives data from the server and generates responses to questions. Based on pre-trained data, the generative AI model calculates the optimal answer to the user's question. The output is the response sentence that should be returned to the user.

[0795] Step 4:

[0796] The server receives the response from the generated AI model and sends it to the terminal. The server converts the response into a user-friendly format and prepares it for display on the terminal. The output is the text-based response displayed to the user.

[0797] Step 5:

[0798] The terminal displays the response received from the server to the user. The user can review this response and ask further questions if necessary. The terminal waits for user input and prepares to accept the next input.

[0799] Step 6:

[0800] When a user displays facial expressions or behavioral patterns, the device uses its camera and sensors to detect them and transmit the information to the server. The input consists of image data and behavioral data.

[0801] Step 7:

[0802] The server receives data from the terminal and analyzes the user's emotions using an emotion analysis engine. The server identifies the user's emotions using image processing and behavioral analysis technologies. The output is the analyzed emotion data.

[0803] Step 8:

[0804] The server sends the analyzed emotion data to a generative AI model, which then generates an emotion-based response. The generative AI model considers the emotion data and calculates a response appropriate to the user's emotions. The output is an emotion-based response sentence.

[0805] Step 9:

[0806] The server receives emotion-based responses from the generated AI model and sends them to the device. The device then displays these responses to the user, improving the user experience.

[0807] (Application Example 3)

[0808] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 as a "terminal".

[0809] In physical stores, there is a need to understand customer emotions in real time and provide appropriate services accordingly. However, traditional methods have the challenge of making it difficult to accurately analyze customer emotions and respond immediately.

[0810] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0811] In this invention, the server includes means for operation from a smart device, means for coordinating with a generative AI, and means for analyzing the user's facial expressions. This makes it possible to analyze customer emotions in real time and generate appropriate responses.

[0812] A "smart device" is an electronic device that is portable to the user and capable of connecting to the internet and running applications.

[0813] "Generative AI" is a system that uses artificial intelligence technology to learn from data and generate new information or responses.

[0814] A "conversational agent" is a software program that provides information and performs tasks through dialogue with the user.

[0815] "Prototyping" refers to the process of creating an initial model of a product or system to verify its functionality and performance.

[0816] "Methods for analyzing facial expressions" refer to technologies that detect the user's facial movements and expressions and infer their emotions based on that.

[0817] "A means of displaying emotions in real time" refers to a technology that instantly analyzes a user's emotions and displays the results immediately.

[0818] In order to implement this invention, it is necessary to construct a system that combines a smart device, a generative AI, an interactive agent, facial expression analysis technology, and real-time display technology.

[0819] The server receives input from smart devices and works with generative AI to create conversational agents. When a user communicates with the conversational agent through a smart device, the device captures the user's facial expressions with its camera and analyzes their emotions using facial recognition technology. This analysis uses a facial recognition library (e.g., OpenCV). The analyzed emotion data is sent to a generative AI model (e.g., OpenAI GPT) to generate an appropriate response.

[0820] The generated response is displayed in real time on the smart device's screen. This allows users to receive personalized services tailored to their emotions.

[0821] For example, if a user makes a surprised face when looking at a product, the device analyzes the emotion of "surprise" and sends a prompt message to the generative AI model saying, "The customer is showing a surprised expression. Please generate an appropriate response." Based on this prompt, the generative AI model generates the response, "Are you interested in this product?" and displays it to the user.

[0822] In this way, customer service in physical stores can be more personalized, and an improvement in customer satisfaction can be expected.

[0823] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0824] Step 1:

[0825] The device captures the user's facial expressions using a camera. The input is camera footage, and the output is facial feature point data. A facial recognition library (e.g., OpenCV) is used to extract facial feature points from the footage.

[0826] Step 2:

[0827] The server receives facial feature point data transmitted from the terminal and analyzes the user's emotions using facial expression analysis technology. The input is facial feature point data, and the output is the analyzed emotion data. The emotion analysis engine analyzes the feature point data and infers the emotion.

[0828] Step 3:

[0829] The server sends the analyzed sentiment data to a generative AI model. The input is sentiment data, and the output is the generated response. The generative AI model (e.g., OpenAI GPT) generates an appropriate response based on the sentiment data.

[0830] Step 4:

[0831] The device displays responses received from the generative AI model to the user in real time. The input is the generated response, and the output is the display on the screen. The device displays personalized messages to the user to facilitate interaction.

[0832] (Other examples)

[0833] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[0834] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0835] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0836] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[0837] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0838] [Third Embodiment]

[0839] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0840] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0841] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0842] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0843] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0844] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0845] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0846] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0847] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0848] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0849] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0850] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0851] "Example of form 1"

[0852] One embodiment of the present invention uses a dedicated application as a means of operation from a smartphone. This application provides an interface for the user to interact with the generative AI. Specifically, it includes functions for the user to update the generative AI's learning data and functions for creating a chatbot that utilizes the generative AI.

[0853] "Example of form 2"

[0854] One embodiment of the present invention provides a means for creating a chatbot that utilizes generative AI, which includes a function for the user to set the chatbot's response patterns. Specifically, the user sets answers to specific questions, and these answers are learned by the generative AI.

[0855] "Example of form 3"

[0856] One embodiment of the present invention provides a function for users to test the response patterns of a chatbot as a means of prototyping a created chatbot. Specifically, the user asks a question to the chatbot and verifies whether the answer is appropriate.

[0857] "Example of form 4"

[0858] One embodiment of the present invention provides a function that allows a user to publish a chatbot as a means of making a prototype chatbot available for use by others. Specifically, a user publishes a chatbot, making it available for others to use.

[0859] The following describes the processing flow for each example of the form.

[0860] "Example of form 1"

[0861] Step 1: The user installs a dedicated application on their smartphone. Step 2: The user opens the application and operates the interface for interacting with the generating AI.

[0862] Step 3: The user updates the training data for the generated AI using the function to update the training data for the generated AI.

[0863] Step 4: Create a chatbot using the user-generated AI-powered chatbot creation feature.

[0864] "Example of form 2"

[0865] Step 1: Use the feature that allows users to set chatbot response patterns.

[0866] Step 2: The user sets the answer to a specific question.

[0867] Step 3: The set responses are learned by the generating AI, and the chatbot's response patterns are updated.

[0868] "Example of form 3"

[0869] Step 1: Use the feature that allows users to test the chatbot's response patterns.

[0870] Step 2: The user asks a question to the chatbot.

[0871] Step 3: The chatbot's response is displayed, and the user checks if the response is appropriate.

[0872] "Example of form 4"

[0873] Step 1: Use the feature that allows users to publish chatbots.

[0874] Step 2: The user publishes the chatbot, and the published chatbot becomes accessible to others.

[0875] Step 3: Others use a publicly available chatbot and interact with it.

[0876] (Example 1)

[0877] Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0878] In modern artificial intelligence systems using information processing equipment, there is a need to provide an environment that allows users to easily interact with AI and create and prototype interactive programs. Furthermore, efficiently updating the AI's training data and generating appropriate responses to user input are key challenges.

[0879] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0880] In this invention, the server includes means for operating from an information processing device, means for coordinating with artificial intelligence, and means for creating interactive programs utilizing artificial intelligence. This allows the user to update the AI's learning data, prototype interactive programs, and allow others to use them.

[0881] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides a means for users to operate it.

[0882] Artificial intelligence is a technology that enables computer systems to mimic human intellectual behavior, possessing the ability to learn from data and perform reasoning and judgment.

[0883] "Means of collaboration" refers to methods and processes that enable different systems or components to exchange information and work together in coordination.

[0884] An "interactive program" is software that provides information and answers questions through dialogue with the user, and it operates using artificial intelligence.

[0885] "Methods for prototyping" refer to methods and processes for experimentally creating new products or systems and evaluating their performance and functionality.

[0886] "Training data" refers to the dataset used by artificial intelligence for learning, and it forms the basis for improving the accuracy and performance of the model.

[0887] "Input text" refers to text data that a user inputs into an information processing device, and it serves as the basis for artificial intelligence to generate a response.

[0888] A description of embodiments for carrying out this invention will be given.

[0889] The user launches a dedicated application on their smartphone, which acts as an information processing device. This application provides an interface for the user to interact with artificial intelligence. The user logs into the application, selects a dataset to update the AI's training data, and uploads it to the server via their device. The server uses the received dataset to update the training data for the AI ​​model. This process utilizes machine learning libraries such as TensorFlow and PyTorch.

[0890] The server retrains the artificial intelligence model based on the uploaded data, improving the model's accuracy. Once training is complete, the server notifies the user, who can then use the generated AI to create an interactive program. The user uses the application's features to set the objective and response patterns of the interactive program.

[0891] For example, if a user enters a prompt such as "Tell me the features of the new product," the terminal sends this prompt to the server. The server uses a generative AI model to generate a response to the prompt and sends it back to the terminal. The user can then review the generated response and re-enter the prompt if necessary.

[0892] In this way, users can operate artificial intelligence through an information processing device to update data, create interactive programs, and generate responses using prompt statements.

[0893] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0894] Step 1:

[0895] The user launches a dedicated application on their smartphone, which acts as an information processing device, and logs in. They enter their login information and send it to the server. The server authenticates the received login information and grants the user access. This allows the user to access the application's functions.

[0896] Step 2:

[0897] The user selects a new dataset to update the training data for a generative AI model. The device uploads the selected dataset to the server. The server stores the received dataset and verifies its integrity. It preprocesses the data and converts it into a format suitable for training.

[0898] Step 3:

[0899] The server updates the training data for the generative AI model using the uploaded dataset. It then retrains the model using machine learning libraries such as TensorFlow or PyTorch. The updated model is obtained as output, using preprocessed data as input. Once training is complete, the server notifies the user.

[0900] Step 4:

[0901] Users create interactive programs that utilize generative AI by using the application's features. Users set the purpose and response patterns of the interactive program. The server builds the interactive program based on the settings and verifies its operation.

[0902] Step 5:

[0903] The user inputs a prompt message to the generative AI model. The terminal sends the input prompt message to the server. The server uses the generative AI model to generate a response to the prompt message. The prompt message is used as input, and the generated response is obtained as output. The response is sent back to the terminal, and the user confirms it.

[0904] (Application Example 1)

[0905] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0906] In today's information society, users are required to efficiently acquire useful information from a vast amount of data. However, conventional information provision systems have the problem of not adequately providing personalized information based on users' interests and preferences. Furthermore, even in information generation using generative AI, there is a lack of mechanisms to effectively reflect user feedback, which makes it difficult to improve the accuracy and relevance of the information.

[0907] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0908] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for generating information based on the user's interests, and means for collecting the user's evaluation of the generated information and updating the generative AI's learning data. This makes it possible to provide personalized information to the user and improve the accuracy and relevance of the generative AI.

[0909] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and it is a device that can perform various functions through user operation.

[0910] "Generative AI" refers to a system that uses artificial intelligence technology to generate new information and content from data, providing appropriate output according to user requests.

[0911] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[0912] "Means of generating information based on user interests" refers to methods and technologies for analyzing user input and past behavioral history and generating highly relevant information based on that analysis.

[0913] "Means of collecting user evaluations of generated information and updating the training data of the generating AI" refers to methods and technologies that enable the generation of more accurate information by collecting feedback from users and improving the training data of the generating AI based on that feedback.

[0914] The system for carrying out this invention comprises an information processing device, a generative AI, and a server. The information processing device provides an interface for user operation and generates information in cooperation with the generative AI. The generative AI is responsible for receiving input from the user and generating relevant information based on it. The server provides the generated information to the user and collects user feedback to update the generative AI's learning data.

[0915] Specifically, the information processing device is a device such as a smartphone or tablet, which runs an application for the user to input topics of interest. This application is developed using React Native and provides the user interface. The topics entered by the user are sent as prompts to the generative AI. The generative AI uses OpenAI's GPT model and generates relevant content based on the input prompts.

[0916] The generated content is returned to the information processing device via the server and displayed to the user. The user evaluates the displayed content, and this evaluation is collected by the server. Based on the collected evaluations, the server updates the training data of the generating AI, improving the accuracy of future information generation.

[0917] For example, if a user enters "latest technology news," the AI ​​will generate articles such as "Latest Technology Trends of 2023" or "Articles on the Evolution of AI Technology." An example of a prompt would be, "Please tell me the latest technology news. I'm especially interested in information on AI and robotics." In this way, users can efficiently obtain information that matches their interests.

[0918] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0919] Step 1:

[0920] The user launches an application on the information processing device and enters a topic of interest. The entered topic is then prepared to be sent to the generating AI as a prompt. The input data is in text format and reflects the user's interests.

[0921] Step 2:

[0922] The terminal sends the input prompt to a generating AI model. The generating AI model uses OpenAI's GPT and generates relevant information based on the prompt. As a data processing step, it performs natural language processing to parse the prompt and generate the relevant information. The output is the generated text information.

[0923] Step 3:

[0924] The server sends the information received from the generating AI to the terminal. The terminal displays the received information to the user. The user reviews the displayed information and makes an evaluation. The evaluation is based on the relevance and usefulness of the information.

[0925] Step 4:

[0926] User ratings are sent from the device to the server. The server updates the training data for the generative AI based on the received ratings. As part of the data calculation, the rating data is analyzed and the model parameters of the generative AI are adjusted. This improves the accuracy of information generation in subsequent instances.

[0927] Step 5:

[0928] The server incorporates the updated training data into the generative AI. This allows the generative AI to generate information while incorporating user feedback. The output is the updated generative AI model.

[0929] (Example 2)

[0930] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0931] The challenge in interactive programs utilizing generative AI is to provide a system that allows users to flexibly configure responses to specific inquiries and efficiently train the generative AI model. Furthermore, it is necessary for the generative AI model to accurately understand the user's intent and effectively generate prompt sentences that produce appropriate responses.

[0932] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0933] In this invention, the server includes means for the user to set a response to a specific query, means for training a generating AI model with the set response, and means for generating prompt sentences for the generating AI model. This enables the generating AI model to efficiently learn the response patterns set by the user, allowing the generating AI model to accurately understand the user's intent and generate an appropriate response.

[0934] An "information terminal" is an electronic device used by a user, and includes devices such as computers and smartphones.

[0935] "Generative AI" is an artificial intelligence technology that performs natural language processing, and is particularly used as a model for generating responses in conversational programs.

[0936] An "interactive program" is software designed to interact with users, and includes systems such as chatbots.

[0937] A "user" is an individual or group that operates an interactive program and sets up responses to specific inquiries.

[0938] A "response pattern" is a pre-configured format for answering a specific inquiry, serving as a standard for how an interactive program responds to a user.

[0939] A "generative AI model" is a core algorithm of generative AI, a pre-trained model designed to generate natural language responses based on input data.

[0940] A "prompt" is an instruction given to a generative AI model to generate a specific response, and it serves as a clue for the model to understand the user's intent.

[0941] One embodiment of this invention is a system that constructs an interactive program utilizing generative AI, enabling users to set responses to specific inquiries. This system is implemented using an information terminal, a server, and a generative AI model.

[0942] Users configure response patterns for interactive programs using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions they want to set and their corresponding answers. For example, a user can set the answer to the question "What is the weather forecast?" to "Today's weather is sunny."

[0943] The terminal sends the user's entered question and answer data to the server. The server analyzes the received data and generates prompts to train a generative AI model. These prompts are designed so that the generative AI model accurately understands the user's intent and generates appropriate responses. A concrete example of a prompt would be, "If the user asks, 'What is the weather forecast?', answer, 'Today's weather is sunny.'"

[0944] The server passes the generated prompt text to the generative AI model for training. The generative AI model, for example, uses an algorithm specialized in natural language processing to memorize the response patterns set by the user. This allows the generative AI model to generate an appropriate response and send it to the terminal when a user asks a question to the chatbot.

[0945] In this way, users can freely set and customize the response patterns of conversational programs that utilize generative AI.

[0946] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0947] Step 1:

[0948] The user configures the response patterns for the interactive program using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions and answers they wish to configure. The input data consists of question-and-answer pairs. This data is stored on the terminal as response patterns that reflect the user's intent.

[0949] Step 2:

[0950] The terminal sends the user-entered question and answer data to the server. The transmitted data is structured as an HTTP request. The server parses the received data and prepares it for use in the next step. Specifically, it verifies the data's integrity and converts the format as needed.

[0951] Step 3:

[0952] The server generates prompt sentences to train a generative AI model based on the received questions and answers. The input data is question-and-answer pairs, and the output is the generated prompt sentences. Specifically, the server creates a prompt sentence in the format of "When the user asks 'What is the weather forecast?', please answer 'Today's weather is sunny.'"

[0953] Step 4:

[0954] The server passes the generated prompt sentences to the generative AI model for training. The input is the prompt sentence, and the output is the trained response pattern. The generative AI model uses a natural language processing algorithm to memorize the response patterns set by the user.

[0955] Step 5:

[0956] When a user asks a question to the chatbot, the server invokes a generative AI model to generate an appropriate response. The input is the user's question, and the output is the generated response. The generated response is sent to the terminal and displayed to the user. Specifically, if the user asks, "What's the weather forecast?", the chatbot will respond, "Today's weather is sunny."

[0957] (Application Example 2)

[0958] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0959] Traditional interactive programs make it difficult for users to set individual response patterns, and the responses provided by the generating AI may not always match the user's needs. Furthermore, because the data used for training the generating AI is fixed, it faces the challenge of not being able to respond flexibly to the user's business needs.

[0960] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0961] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generative AI, and means for creating an interactive program utilizing the generative AI. This allows the user to freely set the response patterns of the interactive program, and the generative AI to learn based on those settings and select the optimal response.

[0962] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones and tablets.

[0963] "Generative AI" is a system that uses artificial intelligence technology to learn from data and perform natural language processing.

[0964] An "interactive program" is software that provides information or answers questions through dialogue with the user.

[0965] A "response pattern" refers to a pre-set format or content of answers to a specific question.

[0966] "Means of learning" refers to the process by which generative AI acquires new knowledge based on data provided by users and improves the accuracy of its responses.

[0967] "The means of selecting the optimal response" refers to a function that automatically selects the most appropriate answer to a user's question based on the data the generation AI has learned.

[0968] A system for carrying out this invention includes an information terminal, a generative AI model, and a server. The information terminal is a device for user operation, such as a smartphone or tablet. The user can use the information terminal to set response patterns for an interactive program.

[0969] The server works in conjunction with a generative AI model, which learns based on response patterns set by the user. The generative AI model is a system for natural language processing, acquiring new knowledge based on data provided by the user and improving the accuracy of its responses.

[0970] Specifically, when a user enters FAQ questions and answers via an information terminal, that data is stored on a server. The server provides the stored data to a generative AI model, which then learns from it. This allows the generative AI model to generate flexible responses tailored to the user's business needs.

[0971] For example, if a user asks "How do I return an item?" and the answer is set to "Returns are possible within 30 days of receiving the item. Please use the return label," the generative AI model will learn from this information and provide appropriate responses to similar questions.

[0972] Examples of prompts used to train a generative AI model include the following:

[0973] "User question: 'How do I return an item?' Set answer: 'Returns are possible within 30 days of receiving the item. Please use the return label.' Prompt to train the generating AI model: 'Generate an appropriate response to the question about how to return an item.'"

[0974] In this way, users can freely customize the responses of the interactive program, and the generating AI model can select the optimal response.

[0975] The flow of the specific processing in Application Example 2 will be explained using Figure 14.

[0976] Step 1:

[0977] The user sets the response patterns for the interactive program using an information terminal. Specifically, the user inputs FAQ questions and their answers. The entered data is sent from the information terminal to the server.

[0978] Step 2:

[0979] The server stores the question and answer data received from the user in a database. This stored data serves as the foundational data for the generative AI model to learn from. The server then prepares the data stored in the database to be provided to the generative AI model.

[0980] Step 3:

[0981] The server provides stored data to the generative AI model, which then learns. Specifically, the server generates prompt sentences and inputs them into the generative AI model. The generative AI model performs natural language processing based on the prompt sentences and learns response patterns set by the user.

[0982] Step 4:

[0983] The generative AI model generates the optimal response to a new question from the user based on the data it has learned. The server provides the user with the response obtained from the generative AI model. Specifically, the server sends the response to the user's information terminal so that the user can confirm it.

[0984] Step 5:

[0985] Users can review the responses generated through their information terminals and readjust the response patterns as needed. This allows users to continuously improve the responses of the interactive program and enhance the accuracy of the generated AI model.

[0986] (Example 3)

[0987] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0988] Traditional dialogue program development has presented challenges in efficient development and operation, as testing and publishing response patterns is difficult, and developers must manually manage many processes. Furthermore, the lack of a process to evaluate the appropriateness of responses using generative AI models has made it difficult to provide dialogue programs that are highly satisfying to users.

[0989] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0990] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI model, means for creating a dialogue program utilizing the generative AI model, means for testing the response patterns of the created dialogue program, and means for making the tested dialogue program public to others. This makes it possible to efficiently develop a dialogue program, evaluate the appropriateness of the responses, and make it public to others.

[0991] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and is equipped with an interface for user operation.

[0992] A "generative AI model" is a model that uses artificial intelligence technology to perform natural language processing and has the ability to generate appropriate responses to user input.

[0993] A "dialogue program" is software designed for interaction with users and has the function of providing responses to user questions using a generative AI model.

[0994] "Means for testing response patterns" refers to a process for evaluating the appropriateness of responses generated by a dialogue program, and a means for verifying responses to user input.

[0995] "Means of publication" refers to the process of placing a dialogue program on the internet and making it accessible to others.

[0996] A description of embodiments for carrying out this invention will be given.

[0997] The user operates the dialogue program using an information processing device. The information processing device receives questions entered by the user and sends that data to the server. The server works in conjunction with a generative AI model, and upon receiving input from the user, passes that data to the generative AI model. The generative AI model uses natural language processing techniques to generate appropriate responses to the entered questions.

[0998] The generated response is returned to the information processing unit via the server. The user can review this response and evaluate whether the response pattern of the dialogue program is appropriate. For example, if the user asks "What's the weather like today?", the generating AI model will generate a response such as "It's sunny today."

[0999] Furthermore, users can upload tested dialogue programs to a server and make them publicly available for others to use. The server places the dialogue programs on the internet, making them accessible to others. This allows others to access and use the publicly available dialogue programs.

[1000] An example of a prompt statement is, "Create a dialogue program that uses a generative AI model to generate appropriate responses to questions entered by the user." By using this prompt statement, the generative AI model generates responses that meet the user's requests, enabling efficient development and operation of the dialogue program. The flow of specific processing in Example 3 will be explained using Figure 15.

[1001] Step 1:

[1002] The user enters the question on their device.

[1003] The user enters a question through the terminal's interface. This input data is sent to the server in text format. For example, the user might enter a question such as, "What's the news today?"

[1004] Step 2:

[1005] The server sends the question to the AI ​​model that generates it.

[1006] The server sends the text data received from the user to the generative AI model. At this time, the server converts the data into an appropriate format so that the generative AI model can process it. The generative AI model uses natural language processing techniques to analyze the question and prepares to generate an appropriate response.

[1007] Step 3:

[1008] The generative AI model generates the response.

[1009] The generative AI model generates responses based on the received question data, utilizing its internal training data. This process involves understanding the intent of the question, extracting relevant information, and constructing a response. For example, it might generate a response such as, "Today's main news is the rise in economic growth."

[1010] Step 4:

[1011] The server returns a response to the user.

[1012] The server receives response data from the generated AI model and sends it back to the user's terminal. At this time, the server converts the response data into a format that is easy for the user to understand. The user reviews this response and evaluates whether the dialogue program's response pattern is appropriate.

[1013] Step 5:

[1014] The user uploads the dialogue program to the server and makes it publicly available.

[1015] The user uploads the completed interactive program to the server. The server places the interactive program on the internet, making it accessible to others. This allows others to access and use the publicly available interactive program.

[1016] (Application Example 3)

[1017] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1018] In modern information distribution services, it is difficult for users to obtain detailed information about the content they are watching in real time. In particular, there is a lack of means to immediately resolve questions that arise during viewing, and there is a need to improve the user experience.

[1019] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1020] In this invention, the server includes means for operation from an information terminal, means for coordinating with generative artificial intelligence, and means for creating an interactive program utilizing generative artificial intelligence. This makes it possible for users to obtain information about the content they are viewing in real time and to resolve their questions immediately.

[1021] An "information terminal" is an electronic device used by a user to operate, and includes smartphones, tablets, and other similar devices.

[1022] "Generative artificial intelligence" is an artificial intelligence technology that uses natural language processing and machine learning to generate appropriate responses to user input.

[1023] An "interactive program" is software that provides information through interaction with the user, and includes chatbots and similar applications.

[1024] "Content being viewed" refers to the video or audio media that the user is currently viewing, including movies and television programs.

[1025] "Real-time acquisition" means providing information immediately in response to user requests, and receiving a response without delay.

[1026] One embodiment of the invention provides a system that allows a user to obtain information about the content they are viewing using an information terminal in real time. This system uses an interactive program that utilizes generative artificial intelligence to instantly generate responses to questions from the user.

[1027] The server works in conjunction with generative artificial intelligence to analyze user input. Specifically, it uses natural language processing technology to understand user questions and retrieve relevant information from a database. The server then provides the retrieved information to the user through an interactive program. In this process, the server uses chatbot engines such as Dialogflow or Rasa, and databases such as Firebase or MongoDB.

[1028] Information terminals receive input from users through a user interface and transmit it to a server. The user interface operates on smartphones and tablets, enabling intuitive operation.

[1029] For example, if a user asks "Who directed this movie?" while watching a film, the information terminal sends this question to the server. The server analyzes the question, retrieves the director's information from the database, and returns the answer to the user.

[1030] An example of a prompt would be: "Design a chatbot that provides relevant information in real time when a user enters a question about the content they are watching. For example, if a user asks about the director of a film, include a feature that will quickly provide that information."

[1031] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[1032] Step 1:

[1033] The user enters questions about the content they are viewing using an information terminal. The entered questions are sent to the server through the information terminal's user interface. The input data consists of questions written in natural language by the user.

[1034] Step 2:

[1035] The server analyzes the received question using natural language processing technology. Specifically, it utilizes a generative AI model to understand the intent of the question and extract relevant keywords. This process clarifies the content of the question and provides the information necessary for the database search in the next step.

[1036] Step 3:

[1037] The server searches the database based on the analysis results and retrieves relevant information. The database contains detailed information about the content being viewed. The server executes database queries using the extracted keywords and retrieves the corresponding information.

[1038] Step 4:

[1039] The server provides the acquired information to the user through an interactive program. Specifically, it uses a generative AI model to generate information in a format that is easy for the user to understand and sends it to the information terminal. The output data is the answer to the user's question.

[1040] Step 5:

[1041] The information terminal displays the answers received from the server on the user interface. Users can obtain information in real time to resolve questions about the content they are viewing.

[1042] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1043] "Example of form 1"

[1044] One embodiment of the present invention is a system incorporating an emotion engine. This system analyzes emotions from a user's text input. Specifically, when a user inputs text to a chatbot, the emotion engine analyzes the user's emotions from that text. The analyzed emotions are sent to a generative AI, which then generates a response based on those emotions. For example, if a user inputs the text "I'm very sad today," the emotion engine analyzes the emotion "sadness" from that text. The generative AI then generates an appropriate response for the emotion "sadness."

[1045] "Example of form 2"

[1046] Another embodiment of the present invention involves a system in which an emotion engine analyzes emotions from a user's voice input. Specifically, when a user provides voice input to a chatbot, the emotion engine analyzes the user's emotions from that voice. The analyzed emotions are sent to a generating AI, which then generates a response based on those emotions. For example, if a user inputs the voice "I'm very happy today," the emotion engine analyzes the emotion "joy" from that voice. The generating AI then generates an appropriate response to the emotion "joy."

[1047] "Example of form 3"

[1048] Furthermore, in another embodiment of the present invention, there is a system in which an emotion engine analyzes emotions from a user's facial expression. Specifically, when a user makes a facial expression to a chatbot, the emotion engine analyzes the user's emotion from that facial expression. The analyzed emotion is sent to a generating AI, which then generates a response based on that emotion. For example, if a user makes a facial expression of "surprise," the emotion engine analyzes the emotion of "surprise" from that facial expression. The generating AI then generates an appropriate response to the emotion of "surprise."

[1049] "Example of form 4"

[1050] Another embodiment of the present invention involves a system in which an emotion engine analyzes emotions from a user's behavioral patterns. Specifically, when a user exhibits a particular behavioral pattern to a chatbot, the emotion engine analyzes the user's emotions from that behavioral pattern. The analyzed emotions are sent to a generating AI, which then generates a response based on those emotions. For example, if a user exhibits a behavioral pattern such as "suddenly typing faster," the emotion engine analyzes that behavioral pattern to determine an emotion such as "anger." The generating AI then generates an appropriate response to that emotion.

[1051] The following describes the processing flow for each example of the form.

[1052] "Example of form 1"

[1053] Step 1: The user enters text into the chatbot.

[1054] Step 2: The emotion engine analyzes the user's emotions from the text.

[1055] Step 3: The analyzed emotions are sent to the generating AI.

[1056] Step 4: The generating AI generates a response based on that emotion.

[1057] "Example of form 2"

[1058] Step 1: The user provides voice input to the chatbot.

[1059] Step 2: The emotion engine analyzes the user's emotions from the audio.

[1060] Step 3: The analyzed emotions are sent to the generating AI.

[1061] Step 4: The generating AI generates a response based on that emotion.

[1062] "Example of form 3"

[1063] Step 1: The user makes a facial expression to the chatbot.

[1064] Step 2: The emotion engine analyzes the user's emotions from their facial expressions.

[1065] Step 3: The analyzed emotions are sent to the generating AI.

[1066] Step 4: The generating AI generates a response based on that emotion.

[1067] "Example of form 4"

[1068] Step 1: The user demonstrates a specific behavioral pattern to the chatbot.

[1069] Step 2: The emotion engine analyzes the user's emotions based on their behavioral patterns.

[1070] Step 3: The analyzed emotions are sent to the generating AI.

[1071] Step 4: The generating AI generates a response based on that emotion.

[1072] (Example 1)

[1073] Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1074] Traditional interactive programs have struggled to accurately analyze user emotions and generate responses based on them. Furthermore, there was a lack of efficient methods for updating the training data of the generative AI and for prototyping and testing interactive programs, creating a need for improved user experience.

[1075] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1076] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for creating an interactive program utilizing the generative AI, means for performing emotion analysis, and means for generating a response based on the analyzed emotion. This makes it possible to analyze the user's emotions and generate a natural response based on them. Furthermore, it enables efficient updating of the generative AI's training data and prototyping and testing of interactive programs, thereby improving the user experience.

[1077] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides an interface for user operation.

[1078] "Generative AI" is a system that uses artificial intelligence technology to learn from data and generate responses based on user input.

[1079] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[1080] "Sentiment analysis" is a process that identifies emotions from user input data and generates appropriate responses based on those emotions.

[1081] "Response generation" is the process of providing users with appropriate information and messages based on analyzed data and emotions.

[1082] This invention is a system for a user to interact with a generated AI and execute an interactive program using an information processing device. The user operates a dedicated application on the information processing device and utilizes an interface for interacting with the generated AI. This application provides functions for the user to update the generated AI's training data and create interactive programs.

[1083] The server is equipped with an emotion engine for sentiment analysis. When a user enters text into an interactive program, the server receives the text and uses the emotion engine to analyze the user's emotions. The analyzed emotions are sent to a generative AI, which then generates an appropriate response based on those emotions.

[1084] For example, if a user types "I'm very sad today," the server analyzes this text using its emotion engine and identifies the emotion "sadness." The generative AI then generates an appropriate response to "sadness" and provides it to the user.

[1085] An example of a prompt is, "If the user enters 'I am very sad today,' explain how the emotion engine analyzes the emotion and what response the generative AI generates." Using this prompt, we can understand the process by which the generative AI model generates a response based on the user's emotions.

[1086] This system allows users to engage in natural, emotion-based conversations with the AI, thereby improving the user experience.

[1087] The flow of the specific processing in Example 1 will be explained using Figure 17.

[1088] Step 1:

[1089] The user launches a dedicated application on the information processing device and authenticates by entering their account information on the login screen. If authentication is successful, the application's main screen is displayed. The input is the user's account information, and the output is the authentication result and the display of the main screen.

[1090] Step 2:

[1091] The user initiates interaction with the interactive program through the application's interface. The user enters a message in the text input field and presses the send button. The input is the user's text message, and the output is the message sent to the server.

[1092] Step 3:

[1093] The server receives text messages sent by users. The emotion engine analyzes these texts to identify the user's emotions. The input is the user's text message, and the output is the analyzed emotion data. Specifically, it extracts emotions using a text analysis algorithm.

[1094] Step 4:

[1095] The server sends the analyzed emotion data to a generative AI model. The generative AI model generates an appropriate response based on the received emotion. The input is the analyzed emotion data, and the output is the generated response message. Specifically, the process involves the generative AI model generating a response that corresponds to the emotion.

[1096] Step 5:

[1097] The device receives a response from the generating AI and displays it on the user's screen. The user can then review this response and continue the conversation. The input is the response message from the generating AI, and the output is the response displayed on the user's screen. Specifically, the device visually presents the received message to the user.

[1098] (Application Example 1)

[1099] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1100] In today's information-saturated society, it is difficult for users to quickly and accurately obtain information that resonates with their emotions. Furthermore, the lack of personalized information based on user emotions highlights the need for improved user experience.

[1101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1102] In this invention, the server includes means for operation from a smartphone, means for coordinating with a generative AI, means for analyzing the user's emotions, and means for recommending information based on the analyzed emotions. This enables the provision of personalized information based on the user's emotions.

[1103] "Operation methods from a smartphone" refers to methods by which users operate a system using a mobile information terminal.

[1104] "Methods for collaborating with generative AI" refers to methods for processing information using artificial intelligence technology and collaborating with other systems and data.

[1105] "Methods for creating chatbots using generative AI" refers to methods for developing automated response programs using artificial intelligence technology.

[1106] "Methods for prototyping a created chatbot" refers to methods for testing the developed automated response program and verifying its functionality.

[1107] "Methods for allowing others to use a prototype chatbot" refers to methods for providing a tested automated response program so that other users can utilize it.

[1108] "Methods for analyzing user emotions" refer to methods for analyzing a user's emotional state based on the input information.

[1109] "Methods for recommending information based on analyzed emotions" refer to methods for presenting users with appropriate information and content based on analysis results.

[1110] The system for carrying out this invention is primarily composed of a smartphone-based operating mechanism. The user can operate an application that interacts with a generative AI via the smartphone. This application sends the text entered by the user to an emotion analysis engine, which then analyzes the user's emotions. An emotion analysis engine such as IBM Watson Tone Analyzer can be used for this analysis.

[1111] The analyzed sentiment data is sent to a generative AI. The generative AI can use generative AI models such as OpenAI GPT-3. Based on the analyzed sentiment, the generative AI recommends information and content suitable for the user. In this recommendation process, the generative AI selects movies, music, articles, etc., that match the user's sentiment and displays them on the smartphone screen.

[1112] For example, if a user enters "I'm very happy today," the emotion analysis engine will analyze this as the emotion "joy." The generative AI will then recommend cheerful movies or upbeat music that match this "joy." In this case, an example of a prompt to input into the generative AI model would be: "The user entered 'I'm very happy today.' Please recommend content that matches this joy."

[1113] In this way, personalized information based on the user's emotions becomes possible, improving the user experience.

[1114] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[1115] Step 1:

[1116] The user launches the application on their smartphone and enters a sentence expressing their emotion into the text input field. This entered text becomes the input data for the emotion analysis engine.

[1117] Step 2:

[1118] The device sends the entered text to the sentiment analysis engine. The sentiment analysis engine analyzes the text data and identifies the user's emotions. This analysis process uses natural language processing techniques to classify the emotions in the text into categories such as "joy" and "sadness," and outputs the results.

[1119] Step 3:

[1120] The server sends the sentiment data received from the sentiment analysis engine to the generative AI. The generative AI uses the received sentiment data as input to generate content suitable for the user. In this process, the generative AI model selects content based on sentiment and generates prompt sentences for recommendation.

[1121] Step 4:

[1122] The server receives output from the generating AI and determines which content to recommend to the user. Specifically, it organizes the information such as movies, music, and articles selected by the generating AI and prepares the data to be presented to the user.

[1123] Step 5:

[1124] The device displays recommended content received from the server to the user. The user can view and select personalized content based on their emotions on their smartphone screen.

[1125] (Example 2)

[1126] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1127] Traditional interactive programs have struggled to generate responses that take user emotions into account, resulting in a limited user experience. Furthermore, there was a need for a system that could efficiently learn user-defined response patterns and respond appropriately.

[1128] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1129] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generative AI, means for creating an interactive program utilizing the generative AI, means for analyzing emotions from voice input, and means for generating a response based on the analyzed emotions. This enables natural dialogue that takes the user's emotions into consideration.

[1130] An "information terminal" is an electronic device operated by a user, and is a device that inputs and outputs data.

[1131] "Generative AI" is a system that uses artificial intelligence technology to generate and learn from data.

[1132] An "interactive program" is software designed for interaction with users and has the function of generating responses in response to user input.

[1133] "Voice input" is a method of transmitting information to a system through voice.

[1134] "Methods for analyzing emotions" refer to technologies that identify and analyze a user's emotions from voice or text.

[1135] "Means for generating a response" refers to techniques for creating an appropriate response based on analyzed information.

[1136] One embodiment of this invention is a system that uses a conversational program utilizing generative AI to achieve natural dialogue with the user. A specific embodiment is shown below.

[1137] The user uses an information terminal to set response patterns for an interactive program. The user inputs answers to specific questions and sends this information to the server. The server stores these response patterns in a database and uses them to train a generative AI model. This allows the generative AI model to respond appropriately to similar questions in the future.

[1138] When a user provides voice input, the device sends the voice data to a server. The server uses emotion analysis techniques to analyze the user's emotions from the voice data. The analyzed emotion data is sent to a generative AI model, which generates an appropriate response based on those emotions.

[1139] For example, if a user inputs "I'm very happy today" via voice input, the means of analyzing the emotion will analyze it as "joy." Then, the generative AI model will generate a response such as "That's wonderful! Did something special happen today?"

[1140] An example of a prompt is, "If the user says, 'I'm very happy today,' generate a response that expresses joy." By using this prompt, the AI ​​can generate an appropriate response that matches the user's emotions.

[1141] The flow of the specific processing in Example 2 will be explained using Figure 19.

[1142] Step 1:

[1143] The user uses an information terminal to set response patterns for an interactive program. The user inputs answers to specific questions and sends this information to the server. The input is a question-and-answer pair set by the user, and the output is a response pattern stored on the server. The server saves this data to a database to prepare for future learning.

[1144] Step 2:

[1145] The server trains a generative AI model using stored response patterns. The input is the response patterns stored in the database, and the output is the trained generative AI model. The server processes the data using the response patterns and updates the model's parameters.

[1146] Step 3:

[1147] The user provides voice input through an information terminal. The input is the user's voice data, which the terminal converts into a digital format and sends to the server. The output is the digital voice data sent to the server.

[1148] Step 4:

[1149] The server receives audio data and analyzes the user's emotions using emotion analysis techniques. The input is digital audio data, and the output is the analyzed emotion data. The server processes the audio data through an analysis algorithm to identify emotions.

[1150] Step 5:

[1151] The server sends the analyzed sentiment data to a generative AI model, which then generates an appropriate response. The input is sentiment data, and the output is the generated response. The generative AI model generates prompt sentences based on the sentiment data and creates an appropriate response.

[1152] Step 6:

[1153] The terminal displays the response received from the server to the user. The input is the response from the generative AI model, and the output is the response message presented to the user. The terminal communicates the response to the user through the screen or audio output.

[1154] (Application Example 2)

[1155] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1156] In modern brick-and-mortar stores, there is a growing need to personalize customer interactions and improve customer satisfaction. However, traditional conversational agents have struggled to generate responses that take customer emotions into account, limiting their ability to improve the customer experience.

[1157] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1158] In this invention, the server includes means for operation from a smart device, means for coordinating with a generative AI, and means for analyzing emotions from voice input. This enables the generation of responses based on customer emotions, thereby providing a more personalized customer experience.

[1159] "Means of operation from smart devices" refers to methods for operating a system using mobile information terminals such as smartphones and tablets.

[1160] "Means of collaborating with generative AI" refers to means of exchanging data and instructions with generative AI models and utilizing the functions of AI.

[1161] "Methods for creating conversational agents" refers to methods for designing and building agents that engage in conversations with users using generative AI.

[1162] "Methods for prototyping conversational agents" refer to methods for testing the functions and response patterns of the conversational agents that have been created.

[1163] "Means of allowing others to use it" refers to means of making the prototype conversational agent available for use by other users.

[1164] "Methods for analyzing emotions from voice input" refer to methods for analyzing a user's voice data and identifying the emotions contained within it.

[1165] "Means for generating responses based on analyzed emotions" refers to means for generating appropriate responses based on the results of emotion analysis.

[1166] To implement this invention, it is necessary to build a system that combines a smart device, a server, a generative AI model, a speech recognition API, and an emotion analysis engine. The smart device receives the user's voice input and sends it to the server. The server converts the voice data into text data using the speech recognition API. Specifically, Google Cloud Speech-to-Text can be used as the speech recognition API.

[1167] Next, the server uses an emotion analysis engine to analyze the user's emotions from the text data. This analysis can utilize emotion analysis engines such as IBM Watson Tone Analyzer. The analyzed emotion data is sent to a generative AI model. The generative AI model uses tools such as OpenAI GPT to generate appropriate responses based on those emotions.

[1168] The generated response is sent to a smart device and presented to the user. This allows the user to experience a dialogue that is tailored to their emotions.

[1169] For example, if a user voice-inputs "I'm very tired today" into a smart device, the server converts this voice into text and uses an emotion analysis engine to analyze the emotion "fatigue." The generative AI model then generates a suggestion in response to "fatigue," such as "You must be tired. Are you looking for something to help you relax?"

[1170] An example of a prompt message for a generative AI model is, "The user's emotion is fatigue. Please generate an appropriate response."

[1171] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[1172] Step 1:

[1173] The user uses a smart device to perform voice input. The input voice data is captured through the smart device's microphone and sent to the server.

[1174] Step 2:

[1175] The server converts audio data into text data using a speech recognition API. Specifically, it analyzes the audio signal using Google Cloud Speech-to-Text and generates the corresponding text. The input for this step is audio data, and the output is text data.

[1176] Step 3:

[1177] The server analyzes the user's emotions from text data using an emotion analysis engine. Specifically, it uses IBM Watson Tone Analyzer to evaluate the emotional tone of the text and identify emotions such as "joy" or "fatigue." The input for this step is text data, and the output is emotion data.

[1178] Step 4:

[1179] The server inputs sentiment data into a generative AI model and generates an appropriate response. Specifically, it uses OpenAI GPT to generate natural, sentiment-based responses. The input for this step is sentiment data, and the output is response text.

[1180] Step 5:

[1181] The server sends the generated response text to the smart device. The smart device displays the received response to the user. This allows the user to experience a dialogue that is empathetic to their feelings.

[1182] (Example 3)

[1183] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1184] Traditional dialogue programs have a drawback: they generate responses without considering the user's emotions, resulting in a limited user experience. Furthermore, the process of effectively testing and sharing the response patterns of these dialogue programs with others is cumbersome.

[1185] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1186] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generation AI, and means for analyzing the user's emotions using an emotion analysis engine. This makes it possible to generate responses based on the user's emotions.

[1187] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones, tablets, and personal computers.

[1188] "Generative AI" refers to a system that uses artificial intelligence technology to analyze data and generate responses or information.

[1189] A "dialogue program" is software that mimics a conversation with a user and provides responses to questions.

[1190] An "emotion analysis engine" is a technology that analyzes a user's facial expressions and behavioral patterns to identify their emotions.

[1191] A "response pattern" is a pre-configured format of response that a dialogue program uses to respond to user input.

[1192] "Training data" refers to the dataset that a generative AI uses to generate responses, and is updated to improve the AI's performance.

[1193] One embodiment of this invention is a system in which a user operates a dialogue program using an information terminal and generates a response in cooperation with a generative AI. The user accesses the dialogue program through the information terminal and inputs a question. The terminal sends this input to a server, and the server uses the generative AI to generate an appropriate response.

[1194] The server uses an emotion analysis engine to analyze the user's facial expressions and behavioral patterns. For example, if the user smiles at the camera, the device sends the image to the server. The server uses the emotion analysis engine to identify the emotion as "joy" and sends it to a generative AI model. Based on this emotion, the generative AI model generates a response such as "You look happy!" and sends it back to the device.

[1195] Furthermore, when users test the response patterns of the dialogue program, they input various questions through the terminal and check the responses generated by the server. This allows users to improve the accuracy of the dialogue program.

[1196] For example, if a user asks "What's the weather like today?", the server will generate a response such as "It's sunny today" and display it on the terminal. An example of a prompt would be "Generate a response for when the user makes a surprised face." By inputting this prompt into the AI ​​model, an appropriate response can be obtained. The flow of specific processing in Example 3 will be explained using Figure 21.

[1197] Step 1:

[1198] The user accesses the interactive program using an information terminal and inputs a question. The terminal receives this input and sends it to the server. The input is in text format and is a question that includes the user's intent.

[1199] Step 2:

[1200] The server analyzes the received question and sends it to the generative AI model. The server uses natural language processing techniques to understand the intent of the question and formats the data so that the generative AI model can generate an appropriate response. The output is data in a format that the generative AI model can process.

[1201] Step 3:

[1202] The generative AI model receives data from the server and generates responses to questions. Based on pre-trained data, the generative AI model calculates the optimal answer to the user's question. The output is the response sentence that should be returned to the user.

[1203] Step 4:

[1204] The server receives the response from the generated AI model and sends it to the terminal. The server converts the response into a user-friendly format and prepares it for display on the terminal. The output is the text-based response displayed to the user.

[1205] Step 5:

[1206] The terminal displays the response received from the server to the user. The user can review this response and ask further questions if necessary. The terminal waits for user input and prepares to accept the next input.

[1207] Step 6:

[1208] When a user displays facial expressions or behavioral patterns, the device uses its camera and sensors to detect them and transmit the information to the server. The input consists of image data and behavioral data.

[1209] Step 7:

[1210] The server receives data from the terminal and analyzes the user's emotions using an emotion analysis engine. The server identifies the user's emotions using image processing and behavioral analysis technologies. The output is the analyzed emotion data.

[1211] Step 8:

[1212] The server sends the analyzed emotion data to a generative AI model, which then generates an emotion-based response. The generative AI model considers the emotion data and calculates a response appropriate to the user's emotions. The output is an emotion-based response sentence.

[1213] Step 9:

[1214] The server receives emotion-based responses from the generated AI model and sends them to the device. The device then displays these responses to the user, improving the user experience.

[1215] (Application Example 3)

[1216] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1217] In physical stores, there is a need to understand customer emotions in real time and provide appropriate services accordingly. However, traditional methods have the challenge of making it difficult to accurately analyze customer emotions and respond immediately.

[1218] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1219] In this invention, the server includes means for operation from a smart device, means for coordinating with a generative AI, and means for analyzing the user's facial expressions. This makes it possible to analyze customer emotions in real time and generate appropriate responses.

[1220] A "smart device" is an electronic device that is portable to the user and capable of connecting to the internet and running applications.

[1221] "Generative AI" is a system that uses artificial intelligence technology to learn from data and generate new information or responses.

[1222] A "conversational agent" is a software program that provides information and performs tasks through dialogue with the user.

[1223] "Prototyping" refers to the process of creating an initial model of a product or system to verify its functionality and performance.

[1224] "Methods for analyzing facial expressions" refer to technologies that detect the user's facial movements and expressions and infer their emotions based on that.

[1225] "A means of displaying emotions in real time" refers to a technology that instantly analyzes a user's emotions and displays the results immediately.

[1226] In order to implement this invention, it is necessary to construct a system that combines a smart device, a generative AI, an interactive agent, facial expression analysis technology, and real-time display technology.

[1227] The server receives input from smart devices and works with generative AI to create conversational agents. When a user communicates with the conversational agent through a smart device, the device captures the user's facial expressions with its camera and analyzes their emotions using facial recognition technology. This analysis uses a facial recognition library (e.g., OpenCV). The analyzed emotion data is sent to a generative AI model (e.g., OpenAI GPT) to generate an appropriate response.

[1228] The generated response is displayed in real time on the smart device's screen. This allows users to receive personalized services tailored to their emotions.

[1229] For example, if a user makes a surprised face when looking at a product, the device analyzes the emotion of "surprise" and sends a prompt message to the generative AI model saying, "The customer is showing a surprised expression. Please generate an appropriate response." Based on this prompt, the generative AI model generates the response, "Are you interested in this product?" and displays it to the user.

[1230] In this way, customer service in physical stores can be more personalized, and an improvement in customer satisfaction can be expected.

[1231] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[1232] Step 1:

[1233] The device captures the user's facial expressions using a camera. The input is camera footage, and the output is facial feature point data. A facial recognition library (e.g., OpenCV) is used to extract facial feature points from the footage.

[1234] Step 2:

[1235] The server receives facial feature point data transmitted from the terminal and analyzes the user's emotions using facial expression analysis technology. The input is facial feature point data, and the output is the analyzed emotion data. The emotion analysis engine analyzes the feature point data and infers the emotion.

[1236] Step 3:

[1237] The server sends the analyzed sentiment data to a generative AI model. The input is sentiment data, and the output is the generated response. The generative AI model (e.g., OpenAI GPT) generates an appropriate response based on the sentiment data.

[1238] Step 4:

[1239] The device displays responses received from the generative AI model to the user in real time. The input is the generated response, and the output is the display on the screen. The device displays personalized messages to the user to facilitate interaction.

[1240] (Other examples)

[1241] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[1242] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1243] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1244] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1245] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1246] [Fourth Embodiment]

[1247] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1248] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1249] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1250] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1251] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1252] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1253] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1254] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1255] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1256] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1257] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1258] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1259] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1260] "Example of form 1"

[1261] One embodiment of the present invention uses a dedicated application as a means of operation from a smartphone. This application provides an interface for the user to interact with the generative AI. Specifically, it includes functions for the user to update the generative AI's learning data and functions for creating a chatbot that utilizes the generative AI.

[1262] "Example of form 2"

[1263] One embodiment of the present invention provides a means for creating a chatbot that utilizes generative AI, which includes a function for the user to set the chatbot's response patterns. Specifically, the user sets answers to specific questions, and these answers are learned by the generative AI.

[1264] "Example of form 3"

[1265] One embodiment of the present invention provides a function for users to test the response patterns of a chatbot as a means of prototyping a created chatbot. Specifically, the user asks a question to the chatbot and verifies whether the answer is appropriate.

[1266] "Example of form 4"

[1267] One embodiment of the present invention provides a function that allows a user to publish a chatbot as a means of making a prototype chatbot available for use by others. Specifically, a user publishes a chatbot, making it available for others to use.

[1268] The following describes the processing flow for each example of the form.

[1269] "Example of form 1"

[1270] Step 1: The user installs a dedicated application on their smartphone. Step 2: The user opens the application and operates the interface for interacting with the generating AI.

[1271] Step 3: The user updates the training data for the generated AI using the function to update the training data for the generated AI.

[1272] Step 4: Create a chatbot using the user-generated AI-powered chatbot creation feature.

[1273] "Example of form 2"

[1274] Step 1: Use the feature that allows users to set chatbot response patterns.

[1275] Step 2: The user sets the answer to a specific question.

[1276] Step 3: The set responses are learned by the generating AI, and the chatbot's response patterns are updated.

[1277] "Example of form 3"

[1278] Step 1: Use the feature that allows users to test the chatbot's response patterns.

[1279] Step 2: The user asks a question to the chatbot.

[1280] Step 3: The chatbot's response is displayed, and the user checks if the response is appropriate.

[1281] "Example of form 4"

[1282] Step 1: Use the feature that allows users to publish chatbots.

[1283] Step 2: The user publishes the chatbot, and the published chatbot becomes accessible to others.

[1284] Step 3: Others use a publicly available chatbot and interact with it.

[1285] (Example 1)

[1286] Next, we will describe Embodiment 1 of Example Form 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1287] In modern artificial intelligence systems using information processing equipment, there is a need to provide an environment that allows users to easily interact with AI and create and prototype interactive programs. Furthermore, efficiently updating the AI's training data and generating appropriate responses to user input are key challenges.

[1288] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1289] In this invention, the server includes means for operating from an information processing device, means for coordinating with artificial intelligence, and means for creating interactive programs utilizing artificial intelligence. This allows the user to update the AI's learning data, prototype interactive programs, and allow others to use them.

[1290] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and provides a means for users to operate it.

[1291] Artificial intelligence is a technology that enables computer systems to mimic human intellectual behavior, possessing the ability to learn from data and perform reasoning and judgment.

[1292] "Means of collaboration" refers to methods and processes that enable different systems or components to exchange information and work together in coordination.

[1293] An "interactive program" is software that provides information and answers questions through dialogue with the user, and it operates using artificial intelligence.

[1294] "Methods for prototyping" refer to methods and processes for experimentally creating new products or systems and evaluating their performance and functionality.

[1295] "Training data" refers to the dataset used by artificial intelligence for learning, and it forms the basis for improving the accuracy and performance of the model.

[1296] "Input text" refers to text data that a user inputs into an information processing device, and it serves as the basis for artificial intelligence to generate a response.

[1297] A description of embodiments for carrying out this invention will be given.

[1298] The user launches a dedicated application on their smartphone, which acts as an information processing device. This application provides an interface for the user to interact with artificial intelligence. The user logs into the application, selects a dataset to update the AI's training data, and uploads it to the server via their device. The server uses the received dataset to update the training data for the AI ​​model. This process utilizes machine learning libraries such as TensorFlow and PyTorch.

[1299] The server retrains the artificial intelligence model based on the uploaded data, improving the model's accuracy. Once training is complete, the server notifies the user, who can then use the generated AI to create an interactive program. The user uses the application's features to set the objective and response patterns of the interactive program.

[1300] For example, if a user enters a prompt such as "Tell me the features of the new product," the terminal sends this prompt to the server. The server uses a generative AI model to generate a response to the prompt and sends it back to the terminal. The user can then review the generated response and re-enter the prompt if necessary.

[1301] In this way, users can operate artificial intelligence through an information processing device to update data, create interactive programs, and generate responses using prompt statements.

[1302] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1303] Step 1:

[1304] The user launches a dedicated application on their smartphone, which acts as an information processing device, and logs in. They enter their login information and send it to the server. The server authenticates the received login information and grants the user access. This allows the user to access the application's functions.

[1305] Step 2:

[1306] The user selects a new dataset to update the training data for a generative AI model. The device uploads the selected dataset to the server. The server stores the received dataset and verifies its integrity. It preprocesses the data and converts it into a format suitable for training.

[1307] Step 3:

[1308] The server updates the training data for the generative AI model using the uploaded dataset. It then retrains the model using machine learning libraries such as TensorFlow or PyTorch. The updated model is obtained as output, using preprocessed data as input. Once training is complete, the server notifies the user.

[1309] Step 4:

[1310] Users create interactive programs that utilize generative AI by using the application's features. Users set the purpose and response patterns of the interactive program. The server builds the interactive program based on the settings and verifies its operation.

[1311] Step 5:

[1312] The user inputs a prompt message to the generative AI model. The terminal sends the input prompt message to the server. The server uses the generative AI model to generate a response to the prompt message. The prompt message is used as input, and the generated response is obtained as output. The response is sent back to the terminal, and the user confirms it.

[1313] (Application Example 1)

[1314] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1315] In today's information society, users are required to efficiently acquire useful information from a vast amount of data. However, conventional information provision systems have the problem of not adequately providing personalized information based on users' interests and preferences. Furthermore, even in information generation using generative AI, there is a lack of mechanisms to effectively reflect user feedback, which makes it difficult to improve the accuracy and relevance of the information.

[1316] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1317] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI, means for generating information based on the user's interests, and means for collecting the user's evaluation of the generated information and updating the generative AI's learning data. This makes it possible to provide personalized information to the user and improve the accuracy and relevance of the generative AI.

[1318] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and it is a device that can perform various functions through user operation.

[1319] "Generative AI" refers to a system that uses artificial intelligence technology to generate new information and content from data, providing appropriate output according to user requests.

[1320] An "interactive program" is software that provides information or generates responses in response to user requests through dialogue with the user.

[1321] "Means of generating information based on user interests" refers to methods and technologies for analyzing user input and past behavioral history and generating highly relevant information based on that analysis.

[1322] "Means of collecting user evaluations of generated information and updating the training data of the generating AI" refers to methods and technologies that enable the generation of more accurate information by collecting feedback from users and improving the training data of the generating AI based on that feedback.

[1323] The system for carrying out this invention comprises an information processing device, a generative AI, and a server. The information processing device provides an interface for user operation and generates information in cooperation with the generative AI. The generative AI is responsible for receiving input from the user and generating relevant information based on it. The server provides the generated information to the user and collects user feedback to update the generative AI's learning data.

[1324] Specifically, the information processing device is a device such as a smartphone or tablet, which runs an application for the user to input topics of interest. This application is developed using React Native and provides the user interface. The topics entered by the user are sent as prompts to the generative AI. The generative AI uses OpenAI's GPT model and generates relevant content based on the input prompts.

[1325] The generated content is returned to the information processing device via the server and displayed to the user. The user evaluates the displayed content, and this evaluation is collected by the server. Based on the collected evaluations, the server updates the training data of the generating AI, improving the accuracy of future information generation.

[1326] For example, if a user enters "latest technology news," the AI ​​will generate articles such as "Latest Technology Trends of 2023" or "Articles on the Evolution of AI Technology." An example of a prompt would be, "Please tell me the latest technology news. I'm especially interested in information on AI and robotics." In this way, users can efficiently obtain information that matches their interests.

[1327] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1328] Step 1:

[1329] The user launches an application on the information processing device and enters a topic of interest. The entered topic is then prepared to be sent to the generating AI as a prompt. The input data is in text format and reflects the user's interests.

[1330] Step 2:

[1331] The terminal sends the input prompt to a generating AI model. The generating AI model uses OpenAI's GPT and generates relevant information based on the prompt. As a data processing step, it performs natural language processing to parse the prompt and generate the relevant information. The output is the generated text information.

[1332] Step 3:

[1333] The server sends the information received from the generating AI to the terminal. The terminal displays the received information to the user. The user reviews the displayed information and makes an evaluation. The evaluation is based on the relevance and usefulness of the information.

[1334] Step 4:

[1335] User ratings are sent from the device to the server. The server updates the training data for the generative AI based on the received ratings. As part of the data calculation, the rating data is analyzed and the model parameters of the generative AI are adjusted. This improves the accuracy of information generation in subsequent instances.

[1336] Step 5:

[1337] The server incorporates the updated training data into the generative AI. This allows the generative AI to generate information while incorporating user feedback. The output is the updated generative AI model.

[1338] (Example 2)

[1339] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1340] The challenge in interactive programs utilizing generative AI is to provide a system that allows users to flexibly configure responses to specific inquiries and efficiently train the generative AI model. Furthermore, it is necessary for the generative AI model to accurately understand the user's intent and effectively generate prompt sentences that produce appropriate responses.

[1341] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1342] In this invention, the server includes means for the user to set a response to a specific query, means for training a generating AI model with the set response, and means for generating prompt sentences for the generating AI model. This enables the generating AI model to efficiently learn the response patterns set by the user, allowing the generating AI model to accurately understand the user's intent and generate an appropriate response.

[1343] An "information terminal" is an electronic device used by a user, and includes devices such as computers and smartphones.

[1344] "Generative AI" is an artificial intelligence technology that performs natural language processing, and is particularly used as a model for generating responses in conversational programs.

[1345] An "interactive program" is software designed to interact with users, and includes systems such as chatbots.

[1346] A "user" is an individual or group that operates an interactive program and sets up responses to specific inquiries.

[1347] A "response pattern" is a pre-configured format for answering a specific inquiry, serving as a standard for how an interactive program responds to a user.

[1348] A "generative AI model" is a core algorithm of generative AI, a pre-trained model designed to generate natural language responses based on input data.

[1349] A "prompt" is an instruction given to a generative AI model to generate a specific response, and it serves as a clue for the model to understand the user's intent.

[1350] One embodiment of this invention is a system that constructs an interactive program utilizing generative AI, enabling users to set responses to specific inquiries. This system is implemented using an information terminal, a server, and a generative AI model.

[1351] Users configure response patterns for interactive programs using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions they want to set and their corresponding answers. For example, a user can set the answer to the question "What is the weather forecast?" to "Today's weather is sunny."

[1352] The terminal sends the user's entered question and answer data to the server. The server analyzes the received data and generates prompts to train a generative AI model. These prompts are designed so that the generative AI model accurately understands the user's intent and generates appropriate responses. A concrete example of a prompt would be, "If the user asks, 'What is the weather forecast?', answer, 'Today's weather is sunny.'"

[1353] The server passes the generated prompt text to the generative AI model for training. The generative AI model, for example, uses an algorithm specialized in natural language processing to memorize the response patterns set by the user. This allows the generative AI model to generate an appropriate response and send it to the terminal when a user asks a question to the chatbot.

[1354] In this way, users can freely set and customize the response patterns of conversational programs that utilize generative AI.

[1355] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1356] Step 1:

[1357] The user configures the response patterns for the interactive program using an information terminal. Specifically, they access a dedicated interface via a web browser and input the questions and answers they wish to configure. The input data consists of question-and-answer pairs. This data is stored on the terminal as response patterns that reflect the user's intent.

[1358] Step 2:

[1359] The terminal sends the user-entered question and answer data to the server. The transmitted data is structured as an HTTP request. The server parses the received data and prepares it for use in the next step. Specifically, it verifies the data's integrity and converts the format as needed.

[1360] Step 3:

[1361] The server generates prompt sentences to train a generative AI model based on the received questions and answers. The input data is question-and-answer pairs, and the output is the generated prompt sentences. Specifically, the server creates a prompt sentence in the format of "When the user asks 'What is the weather forecast?', please answer 'Today's weather is sunny.'"

[1362] Step 4:

[1363] The server passes the generated prompt sentences to the generative AI model for training. The input is the prompt sentence, and the output is the trained response pattern. The generative AI model uses a natural language processing algorithm to memorize the response patterns set by the user.

[1364] Step 5:

[1365] When a user asks a question to the chatbot, the server invokes a generative AI model to generate an appropriate response. The input is the user's question, and the output is the generated response. The generated response is sent to the terminal and displayed to the user. Specifically, if the user asks, "What's the weather forecast?", the chatbot will respond, "Today's weather is sunny."

[1366] (Application Example 2)

[1367] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1368] Traditional interactive programs make it difficult for users to set individual response patterns, and the responses provided by the generating AI may not always match the user's needs. Furthermore, because the data used for training the generating AI is fixed, it faces the challenge of not being able to respond flexibly to the user's business needs.

[1369] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1370] In this invention, the server includes means for operation from an information terminal, means for coordinating with a generative AI, and means for creating an interactive program utilizing the generative AI. This allows the user to freely set the response patterns of the interactive program, and the generative AI to learn based on those settings and select the optimal response.

[1371] An "information terminal" is an electronic device used by a user, and includes devices such as smartphones and tablets.

[1372] "Generative AI" is a system that uses artificial intelligence technology to learn from data and perform natural language processing.

[1373] An "interactive program" is software that provides information or answers questions through dialogue with the user.

[1374] A "response pattern" refers to a pre-set format or content of answers to a specific question.

[1375] "Means of learning" refers to the process by which generative AI acquires new knowledge based on data provided by users and improves the accuracy of its responses.

[1376] "The means of selecting the optimal response" refers to a function that automatically selects the most appropriate answer to a user's question based on the data the generation AI has learned.

[1377] A system for carrying out this invention includes an information terminal, a generative AI model, and a server. The information terminal is a device for user operation, such as a smartphone or tablet. The user can use the information terminal to set response patterns for an interactive program.

[1378] The server works in conjunction with a generative AI model, which learns based on response patterns set by the user. The generative AI model is a system for natural language processing, acquiring new knowledge based on data provided by the user and improving the accuracy of its responses.

[1379] Specifically, when a user enters FAQ questions and answers via an information terminal, that data is stored on a server. The server provides the stored data to a generative AI model, which then learns from it. This allows the generative AI model to generate flexible responses tailored to the user's business needs.

[1380] For example, if a user asks "How do I return an item?" and the answer is set to "Returns are possible within 30 days of receiving the item. Please use the return label," the generative AI model will learn from this information and provide appropriate responses to similar questions.

[1381] Examples of prompts used to train a generative AI model include the following:

[1382] "User question: 'How do I return an item?' Set answer: 'Returns are possible within 30 days of receiving the item. Please use the return label.' Prompt to train the generating AI model: 'Generate an appropriate response to the question about how to return an item.'"

[1383] In this way, users can freely customize the responses of the interactive program, and the generating AI model can select the optimal response.

[1384] The flow of the specific processing in Application Example 2 will be explained using Figure 14.

[1385] Step 1:

[1386] The user sets the response patterns for the interactive program using an information terminal. Specifically, the user inputs FAQ questions and their answers. The entered data is sent from the information terminal to the server.

[1387] Step 2:

[1388] The server stores the question and answer data received from the user in a database. This stored data serves as the foundational data for the generative AI model to learn from. The server then prepares the data stored in the database to be provided to the generative AI model.

[1389] Step 3:

[1390] The server provides stored data to the generative AI model, which then learns. Specifically, the server generates prompt sentences and inputs them into the generative AI model. The generative AI model performs natural language processing based on the prompt sentences and learns response patterns set by the user.

[1391] Step 4:

[1392] The generative AI model generates the optimal response to a new question from the user based on the data it has learned. The server provides the user with the response obtained from the generative AI model. Specifically, the server sends the response to the user's information terminal so that the user can confirm it.

[1393] Step 5:

[1394] Users can review the responses generated through their information terminals and readjust the response patterns as needed. This allows users to continuously improve the responses of the interactive program and enhance the accuracy of the generated AI model.

[1395] (Example 3)

[1396] Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1397] Traditional dialogue program development has presented challenges in efficient development and operation, as testing and publishing response patterns is difficult, and developers must manually manage many processes. Furthermore, the lack of a process to evaluate the appropriateness of responses using generative AI models has made it difficult to provide dialogue programs that are highly satisfying to users.

[1398] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1399] In this invention, the server includes means for operating from an information processing device, means for coordinating with a generative AI model, means for creating a dialogue program utilizing the generative AI model, means for testing the response patterns of the created dialogue program, and means for making the tested dialogue program public to others. This makes it possible to efficiently develop a dialogue program, evaluate the appropriateness of the responses, and make it public to others.

[1400] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and is equipped with an interface for user operation.

[1401] A "generative AI model" is a model that uses artificial intelligence technology to perform natural language processing and has the ability to generate appropriate responses to user input.

[1402] A "dialogue program" is software designed for interaction with users and has the function of providing responses to user questions using a generative AI model.

[1403] "Means for testing response patterns" refers to a process for evaluating the appropriateness of responses generated by a dialogue program, and a means for verifying responses to user input.

[1404] "Means of publication" refers to the process of placing a dialogue program on the internet and making it accessible to others.

[1405] A description of embodiments for carrying out this invention will be given.

[1406] The user operates the dialogue program using an information processing device. The information processing device receives questions entered by the user and sends that data to the server. The server works in conjunction with a generative AI model, and upon receiving input from the user, passes that data to the generative AI model. The generative AI model uses natural language processing techniques to generate appropriate responses to the entered questions.

[1407] The generated response is returned to the information processing unit via the server. The user can review this response and evaluate whether the response pattern of the dialogue program is appropriate. For example, if the user asks "What's the weather like today?", the generating AI model will generate a response such as "It's sunny today."

[1408] Furthermore, users can upload tested dialogue programs to a server and make them publicly available for others to use. The server places the dialogue programs on the internet, making them accessible to others. This allows others to access and use the publicly available dialogue programs.

[1409] An example of a prompt statement is, "Create a dialogue program that uses a generative AI model to generate appropriate responses to questions entered by the user." By using this prompt statement, the generative AI model generates responses that meet the user's requests, enabling efficient development and operation of the dialogue program. The flow of specific processing in Example 3 will be explained using Figure 15.

[1410] Step 1:

[1411] The user enters the question on their device.

[1412] The user enters a question through the terminal's interface. This input data is sent to the server in text format. For example, the user might enter a question such as, "What's the news today?"

[1413] Step 2:

[1414] The server sends the question to the AI ​​model that generates it.

[1415] The server sends the text data received from the user to the generative AI model. At this time, the server converts the data into an appropriate format so that the generative AI model can process it. The generative AI model uses natural language processing techniques to analyze the question and prepares to generate an appropriate response.

[1416] Step 3:

[1417] The generative AI model generates the response.

[1418] The generative AI model generates responses based on the received question data, utilizing its internal training data. This process involves understanding the intent of the question, extracting relevant information, and constructing a response. For example, it might generate a response such as, "Today's main news is the rise in economic growth."

[1419] Step 4:

[1420] The server returns a response to the user.

[1421] The server receives response data from the generated AI model and sends it back to the user's terminal. At this time, the server converts the response data into a format that is easy for the user to understand. The user reviews this response and evaluates whether the dialogue program's response pattern is appropriate.

[1422] Step 5:

[1423] The user uploads the dialogue program to the server and makes it publicly available.

[1424] The user uploads the completed interactive program to the server. The server places the interactive program on the internet, making it accessible to others. This allows others to access and use the publicly available interactive program.

[1425] (Application Example 3)

[1426] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1427] In modern information distribution services, it is difficult for users to obtain detailed information about the content they are watching in real time. In particular, there is a lack of means to immediately resolve questions that arise during viewing, and there is a need to improve the user experience.

[1428] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1429] In this invention, the server includes means for operation from an information terminal, means for coordinating with generative artificial intelligence, and means for creating an interactive program utilizing generative artificial intelligence. This makes it possible for users to obtain information about the content they are viewing in real time and to resolve their questions immediately.

[1430] An "information terminal" is an electronic device used by a user to operate, and includes smartphones, tablets, and other similar devices.

[1431] "Generative artificial intelligence" is an artificial intelligence technology that uses natural language processing and machine learning to generate appropriate responses to user input.

[1432] An "interactive program" is software that provides information through interaction with the user, and includes chatbots and similar applications.

[1433] "Content being viewed" refers to the video or audio media that the user is currently viewing, including movies and television programs.

[1434] "Real-time acquisition" means providing information immediately in response to user requests, and receiving a response without delay.

[1435] One embodiment of the invention provides a system that allows a user to obtain information about the content they are viewing using an information terminal in real time. This system uses an interactive program that utilizes generative artificial intelligence to instantly generate responses to questions from the user.

[1436] The server works in conjunction with generative artificial intelligence to analyze user input. Specifically, it uses natural language processing technology to understand user questions and retrieve relevant information from a database. The server then provides the retrieved information to the user through an interactive program. In this process, the server uses chatbot engines such as Dialogflow or Rasa, and databases such as Firebase or MongoDB.

[1437] Information terminals receive input from users through a user interface and transmit it to a server. The user interface operates on smartphones and tablets, enabling intuitive operation.

[1438] For example, if a user asks "Who directed this movie?" while watching a film, the information terminal sends this question to the server. The server analyzes the question, retrieves the director's information from the database, and returns the answer to the user.

[1439] An example of a prompt would be: "Design a chatbot that provides relevant information i...

Claims

[Claim 1] Equipped with a processor, The aforementioned processor, The system accepts requests to make the generated AI model available through an application or web interface installed on an information processing device such as a smartphone, tablet, or personal computer. The information processing device analyzes the user's request input and creates a prompt to instruct the generating AI model to generate a response. The prompt is sent to the generation AI model, the response generated by the generation AI model is obtained, and the obtained response is sent back to the information processing device. The system accepts data input including question and answer pairs set by the user via the information processing device, generates prompt sentences for the generating AI model to learn based on the data, passes the generated prompt sentences to the generating AI model, and causes the generating AI model to store the response patterns set by the user, thereby creating a chatbot that provides interactive responses. In order to evaluate the appropriateness of the responses of the created chatbot, the system receives a question entered by the user, returns the response generated by the chatbot to the information processing device, and allows the user to evaluate whether the response is appropriate. The system acquires user input text, voice data, or camera footage showing the user's facial expressions, and analyzes the acquired data using an emotion engine to identify the user's emotions from the user's facial expressions in the case of camera footage. Based on the identified emotion, a prompt is created to instruct the generative AI model to generate a response appropriate to the emotion, and the response is optimized using the prompt. The information processing device receives input from the user for a specific question and its answer, and creates a prompt to instruct the generating AI model to generate a specific response using the specific question and its answer. The aforementioned chatbot is made public online or locally so that other users can use it. The chatbot identifies the user's interests by obtaining data on topics of interest entered in text format by the user, sends a prompt containing the topics of interest to the generating AI model, and causes the generating AI model to generate information related to the prompt. The AI ​​model generates the relevant information and has the user evaluate it. The system receives evaluation data, which is the user's evaluation, analyzes the evaluation data, and updates the training data of the generated AI model based on the evaluation data. The information processing device receives a natural language question from the user regarding the content being viewed, analyzes the question using natural language processing technology to extract relevant keywords, and uses the extracted keywords to obtain detailed information about the content being viewed from a database. system.

Images