system

Abstract

We provide the system. [Solution] A means for the user to input questions or inquiries in text format via a terminal, A means for sending the entered question data to the server, A means for the server to transfer question data to a generative artificial intelligence engine, A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer, A means by which the server sends the generated response text back to the terminal, A means of displaying the returned response to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In many modern families, especially young children have many questions and inquiries every day, but the problem is that they have few opportunities to solve them immediately. In such an environment, the expansion and growth of children's knowledge are likely to be hindered. Also, for parents, it is difficult to effectively solve children's questions because they do not know how to use generative artificial intelligence (AI) or have concerns about it. As a result, there is a problem that children's question-solving and learning opportunities are limited.

Means for Solving the Problems

[0005] This invention solves the above problem by providing a system in which a user inputs questions in text format via a terminal and sends the input question data to a server. In this system, the server transfers the question data to a generative artificial intelligence engine, which analyzes the question and generates an appropriate answer. The server then sends the generated answer text back to the terminal, and the terminal displays the returned answer to the user. Furthermore, after the generated answer text is displayed on the terminal, the user can confirm the displayed answer. The invention also provides content for parent users that explains how to use the generative artificial intelligence engine and its advantages. In this way, the system can quickly resolve children's questions and effectively support children's learning while parents learn how to use generative artificial intelligence.

[0006] "Users" refer to children and their parents who use the system to try to resolve their questions and concerns.

[0007] A "terminal" refers to a device that a user uses to access a system and input questions or inquiries. Specifically, this includes personal computers, smartphones, and tablets.

[0008] "Questions and doubts" refer to doubts and problems that children have in their daily lives or while learning, but which they find difficult to solve on their own.

[0009] "Text format" refers to a writing method that uses characters such as the alphabet, kanji, hiragana, and katakana.

[0010] "Question data" refers to questions or inquiries in text format that are entered by the user and sent from the terminal to the server.

[0011] A "server" refers to a device and system that receives question data sent from a terminal and transfers it to a generative artificial intelligence engine.

[0012] A "generative artificial intelligence engine" refers to an algorithm and program that analyzes user questions and concerns and generates appropriate answers.

[0013] "Analysis" refers to the process by which a generative artificial intelligence engine understands the input question text and derives an answer.

[0014] "Answer" refers to the response generated by a generative artificial intelligence engine in response to a question.

[0015] "Answer text" refers to the textual representation of a response to a user's question or inquiry, generated by a generative artificial intelligence engine.

[0016] "Return" refers to the process by which the server sends back the response text obtained from the generative artificial intelligence engine to the terminal.

[0017] "Display" refers to the visual presentation of the response text received by the device to the user.

[0018] "Content for parent users" refers to information that explains to parents how to use generative artificial intelligence engines, their advantages, and how to apply them at home.

[0019] "Content" refers to explanatory texts and guides provided in a format that is easy for parental users to read and understand. [Brief explanation of the drawing]

[0020] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4]It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0024] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention describes how to implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, the invention describes a mechanism for providing content to parent users explaining how to use the generative artificial intelligence engine and its benefits.

[0042] Program processing

[0043] 1. User enters question

[0044] Users access the website using their devices and enter their questions or inquiries in text format.

[0045] For example, a user (a child) might input the question, "Why is the sky blue?"

[0046] 2. Submitting Question Data

[0047] The entered question data is sent from the terminal to the server. The server receives this data and stores it temporarily.

[0048] 3. Transfer to a generative artificial intelligence engine

[0049] The server transfers the received question data to the generative artificial intelligence engine.

[0050] The generative artificial intelligence engine analyzes the question data and generates appropriate answers to it.

[0051] 4. Generating the answer

[0052] A generative artificial intelligence engine analyzes the question and generates an answer in natural language. For example, it might generate an answer like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0053] 5. Return your response

[0054] The server sends the generated response text back to the terminal. The terminal displays the received response on its screen.

[0055] The user (child) checks the answer displayed on the screen, and their question is resolved.

[0056] 6. Providing content for parent users

[0057] The device also displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines.

[0058] For example, parents can obtain information such as, "By using generative AI, you can instantly answer your child's questions and improve their learning efficiency."

[0059] Specific example

[0060] As a concrete example, let's explain what happens when a child types "Why is the sky blue?" into a device.

[0061] 1. The user (child) enters "Why is the sky blue?" into the input field on the device and clicks the send button.

[0062] 2. The question data is sent to the server immediately.

[0063] 3. The server receives the question data and passes it to the generative artificial intelligence engine.

[0064] 4. The generative artificial intelligence engine analyzes the question and generates the answer, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0065] 5. The server sends the generated response back to the terminal, and the terminal displays the response on its screen.

[0066] 6. We will also provide content for parent users that explains the convenience of generative artificial intelligence engines and how to use them at home.

[0067] In this way, the system of the present invention is designed to quickly resolve children's questions, and parents can learn about the use and benefits of generative artificial intelligence engines, and apply them to their daily lives.

[0068] The following describes the processing flow.

[0069] Step 1:

[0070] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[0071] Step 2:

[0072] When the user clicks the submit button, the device sends the entered question data to the server. An HTTP POST request is used for submission.

[0073] Step 3:

[0074] The server analyzes the received question data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[0075] Step 4:

[0076] The server transfers the transformed question data to the generative artificial intelligence engine. The question data is sent to the generative artificial intelligence engine using an API request.

[0077] Step 5:

[0078] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. It then generates an appropriate answer to the question.

[0079] Step 6:

[0080] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[0081] Step 7:

[0082] The server sends the received response text back to the user's device. The server embeds the response data into an HTML template and sends it to the device as an HTTP response.

[0083] Step 8:

[0084] The device receives a response from the server and displays the answer on the screen. Specifically, an answer such as, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight," is displayed.

[0085] Step 9:

[0086] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0087] Step 10:

[0088] The device also displays content explaining how to use and the convenience of the generative artificial intelligence engine for parent users. Parent users read the explanations and understand how to use it at home.

[0089] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions and promoting the understanding and application of generative artificial intelligence technology.

[0090] (Example 1)

[0091] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Conventional question-answering systems suffer from an inefficient process, where users input questions in text format and then receive appropriate answers, resulting in slow response times. Furthermore, they lacked user-friendliness and ease of use when utilizing generative artificial intelligence engines. There is a need to address these issues and provide a more efficient and immediate question-answering system.

[0093] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0094] In this invention, the server includes means for sending input question data to the server using an HTTP POST request, means for the server to temporarily store the received question data in a database, and means for the server to send the question data to the API of a generative artificial intelligence engine. This makes it possible to quickly and efficiently generate an appropriate answer after a user inputs and submits a question, and to display that answer to the user. By providing explanatory content for parent users, it is also possible to deepen their understanding of how to use and the convenience of the generative artificial intelligence engine.

[0095] A "user" is an individual or group that uses a system or service.

[0096] A "terminal" refers to an electronic device used by a user to access a system, such as a smartphone, tablet, or personal computer.

[0097] "Questions and doubts" refer to information that users want to know, problems they want to solve, or things they want to confirm.

[0098] "Text format" refers to the representation format of data entered as a string of characters.

[0099] "Question data" refers to data that includes questions and doubts entered by users.

[0100] A "server" is a computer system that provides services over a network, and is a device that receives, processes, and stores question data.

[0101] A "generative artificial intelligence engine" is a program that possesses artificial intelligence technology to analyze input question data and generate appropriate answers.

[0102] "Answer text" refers to the content of the answer to the user's question, generated by a generative artificial intelligence engine.

[0103] "Means of display" refers to a method or function for visually displaying the answer text on the device screen.

[0104] An "HTTP POST request" is a type of HTTP protocol used to send data from a user's device to a server.

[0105] A "database" refers to a system for systematically storing, managing, and retrieving information, and specifically includes relational database management systems (RDBMS).

[0106] An "API" is an interface that allows a specific program or service to communicate with other programs.

[0107] "Content for parent users" refers to content that provides information explaining how to use and the benefits of generative artificial intelligence engines.

[0108] This invention relates to a system that allows users to input questions or doubts via a terminal and quickly generates and provides appropriate answers to those questions. This system has a series of processes that analyze questions using a generative artificial intelligence engine and generate appropriate answers.

[0109] The specific hardware configuration of this system includes a terminal that receives user input, a server that receives, stores, transfers, and returns question data, and a generative artificial intelligence engine that analyzes the question data and generates answers. The software used includes a web browser, a server API for processing HTTP POST requests, a database management system, and the generative artificial intelligence engine.

[0110] Hardware and software

[0111] 1. Terminal

[0112] The user enters the question by opening a web browser on a device (smartphone, tablet, PC, etc.) and accessing a specific website. The device communicates with the server via an internet connection.

[0113] 2. Server

[0114] The server receives the input question data and transfers it to the generative artificial intelligence engine. The server temporarily stores the question data using a database (e.g., MySQL®). Then, it sends the question data using the generative artificial intelligence engine's API and receives the generated answer.

[0115] 3. Generative Artificial Intelligence Engine

[0116] A generative artificial intelligence engine (for example, OpenAI's GPT-4®) analyzes question data sent from a server and generates the optimal answer to that question. Based on the analysis results, the engine creates an answer in natural language and sends that answer back to the server.

[0117] Specific examples of data processing

[0118] For example, if a user (a child) enters the question "Why is the sky blue?" into the device, the process will proceed as follows:

[0119] 1. The user enters "Why is the sky blue?" into the input field on their device and clicks the send button.

[0120] 2. The terminal sends the entered question data to the server using an HTTP POST request.

[0121] 3. The server receives the question data and temporarily stores it in the database.

[0122] 4. The server sends the question data to the API of the generative artificial intelligence engine.

[0123] Prompt example: "Why is the sky blue?"

[0124] 5. A generative artificial intelligence engine analyzes the question and generates the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0125] 6. The server sends the generated response text back to the terminal, and the terminal displays the response on its screen.

[0126] 7. The user (child) checks the answer displayed on the device screen.

[0127] 8. The device displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. For example, it might display content such as, "By using generative AI, you can instantly answer your child's questions and improve learning efficiency."

[0128] This system makes it possible to provide users with quick and appropriate answers to their questions, and also helps parents understand the convenience of generative artificial intelligence engines.

[0129] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0130] Step 1:

[0131] The user opens a web browser on their device. The user enters a question or doubt in text format into an input field. For example, they might enter the question, "Why is the sky blue?"

[0132] Input: Questions or inquiries entered by the user in text format.

[0133] Output: Question text displayed in the input field

[0134] Step 2:

[0135] When the user clicks the submit button, the device sends the entered question data to the server using an HTTP POST request.

[0136] Input: Data of the questions or inquiries that were entered.

[0137] Output: HTTP POST request sent to the server

[0138] Step 3:

[0139] The server receives the HTTP POST request and verifies the query data. The server temporarily stores the received query data in a database (e.g., MySQL).

[0140] Input: Question data sent in an HTTP POST request

[0141] Output: Question data temporarily stored in the database

[0142] Step 4:

[0143] The server retrieves the question data from the database and sends it to the API of the generative artificial intelligence engine.

[0144] Input: Question data stored in the database

[0145] Output: Question data sent to the API of a generative artificial intelligence engine.

[0146] Step 5:

[0147] A generative artificial intelligence engine receives the question data and analyzes it. Based on the analysis results, the generative artificial intelligence engine generates an answer in natural language.

[0148] Input: Question data sent to a generative artificial intelligence engine

[0149] Output: Generated answer text

[0150] Specific operation: For example, if the prompt is "Why is the sky blue?", the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight" will be generated.

[0151] Step 6:

[0152] The generative artificial intelligence engine sends the generated response back to the server. The server checks the received response text and sends the response back to the terminal using an HTTP response.

[0153] Input: Response text returned by the generative artificial intelligence engine

[0154] Output: The HTTP response sent to the terminal will include the response text.

[0155] Step 7:

[0156] The terminal receives an HTTP response from the server, parses the received response text, and formats it for display. The terminal displays the response on the screen, and the user confirms it.

[0157] Input: HTTP response sent from the server

[0158] Output: Answer text displayed on the screen

[0159] Specific action: For example, the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight," will be displayed on the device screen.

[0160] Step 8:

[0161] The device further displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. Parent users will understand the educational benefits that can be gained by utilizing generative artificial intelligence engines.

[0162] Input: Details on the advantages and usage of a specific generative artificial intelligence engine.

[0163] Output: Explanatory content displayed for the parent user

[0164] Specific actions: For example, information such as "By using generative AI, children's questions can be answered instantly, improving learning efficiency" will be displayed.

[0165] (Application Example 1)

[0166] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0167] In current virtual stores, it is difficult for users to get immediate and accurate answers to their questions. Furthermore, the lack of mechanisms for personalized suggestions based on customer purchase history makes it difficult to provide optimal product selection and detailed product information. Additionally, systems utilizing generative artificial intelligence engines are difficult for the average user to understand, and their advantages and usage are not clearly communicated, which is another challenge.

[0168] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0169] In this invention, the server includes means for a user to input questions or inquiries in text format via an information terminal; means for transmitting the input question data to an information processing device; means for the information processing device to transfer the question data to a generative artificial intelligence engine; means for the generative artificial intelligence engine to analyze the questions and generate appropriate answers; means for the information processing device to return the generated answer text to the information terminal; means for the information terminal to display the returned answers to the user; means for providing immediate answers to customer inquiries in a virtual space; and means for providing personalized suggestions based on the customer's purchase history. As a result, users can resolve their questions in real time and enjoy a more efficient and satisfying shopping experience by receiving personalized suggestions. Furthermore, by providing parents with information on the advantages and usage of the generative artificial intelligence engine, it is expected that its use within the home will be promoted.

[0170] A "user" refers to an individual who inputs questions or inquiries via an information terminal.

[0171] "Information terminal" is a general term for electronic devices used by users to input questions or inquiries, and includes smartphones, smart glasses, and head-mounted displays.

[0172] "Questions and doubts" refer to things that users want to know or things they are unsure about, expressed in text format.

[0173] An "information processing device" refers to a device that receives question data sent by a user and transfers it to a generative artificial intelligence engine.

[0174] A "generative artificial intelligence engine" refers to artificial intelligence that analyzes user questions and generates appropriate answers.

[0175] "Answer text" refers to the response generated by a generative artificial intelligence engine in response to a question, and is written in natural language.

[0176] "Virtual space" refers to a general term for three-dimensional spaces virtually constructed on a computer system, providing an environment where users can act as if they were actually present.

[0177] "Customer purchase history" refers to a record of products that a customer has purchased in the past, and this information is used to provide personalized recommendations.

[0178] "Personalized recommendations" refer to recommendations for products and services tailored to individual preferences, based on a customer's purchase history.

[0179] This invention is a system that provides immediate answers to user questions and offers personalized suggestions. Specific embodiments are described below.

[0180] First, the user inputs questions or inquiries via an information terminal. These terminals include, for example, smartphones, smart glasses, and head-mounted displays. These terminals operate within a virtual space and provide an interface that allows the user to input questions in text format.

[0181] The input question data is sent to the information processing device. The information processing device receives this question data and stores it temporarily. Next, the information processing device transfers the received question data to a generative artificial intelligence engine. This generative artificial intelligence engine uses GPT-4 or an equivalent AI model. The generative artificial intelligence engine analyzes the question data and generates an appropriate answer to it.

[0182] The generated response text is sent back to the information terminal by the information processing device. The information terminal displays the received response to the user. This allows the user to obtain answers to their questions in real time.

[0183] Furthermore, this includes means of providing instant answers to customer questions in a virtual space, as well as means of making personalized suggestions based on customer purchase history. Customer purchase history data is stored in an information processing device, and an AI engine analyzes it to suggest the most suitable products and services for each individual user.

[0184] The system's operation is illustrated with a concrete example. For instance, if a user enters "Please tell me the features of the new smartphone released at this store," the question data is sent to the information processing device and passed to a generative artificial intelligence engine. The AI ​​engine performs analysis and generates an answer such as "The new model has a longer battery life and improved camera performance," which is then sent back and displayed on the information terminal.

[0185] An example of a prompt would be, "I am a 40-year-old man. Please advise me on what kind of clothes would suit me." In response to this prompt, the generative artificial intelligence engine will generate an answer that includes advice on fashion styles suitable for a 40-year-old man and provide it to the user.

[0186] In this way, the present invention quickly resolves user questions and improves the purchasing experience. By providing parents with information on the advantages and usage of the generative artificial intelligence engine, it can also promote its use within the home.

[0187] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0188] Step 1:

[0189] The user uses an information terminal to input a question or inquiry in text format and presses the send button. The entered question is captured by the information terminal as text data. This input data is then formatted for subsequent processing. Specifically, the user inputs "Please tell me about the features of the new smartphone."

[0190] Step 2:

[0191] The entered question data is sent from the information terminal to the information processing device (server). Here, the information terminal sends the data using an HTTP POST request. The server receives this and temporarily stores the question data. This step verifies that the input data has been successfully delivered to the server.

[0192] Step 3:

[0193] The server forwards the received question data to the generative artificial intelligence engine. Specifically, the server sends the question data again to the AI ​​engine's API endpoint using an HTTP POST request. At this time, the data is converted to JSON format. After sending, the server waits for a response from the generative artificial intelligence engine.

[0194] Step 4:

[0195] The generative artificial intelligence engine analyzes the received question data and generates an appropriate answer. An AI model (e.g., GPT-4) is used in this process. Specifically, it receives the question text as an input prompt, performs natural language processing, and generates the most appropriate answer. For example, it might generate the answer, "The new model has a longer battery life and improved camera performance."

[0196] Step 5:

[0197] The generated response text is sent back from the generative artificial intelligence engine to the server. The server receives it and temporarily stores it again. The returned data is in JSON format, and after receiving this data, the server prepares to convert it to an appropriate format.

[0198] Step 6:

[0199] The server sends the generated response text back to the information terminal. It then uses an HTTP POST request again to send the data to the appropriate endpoint. At this point, the data is converted to a text format suitable for display. After sending, the server verifies the response.

[0200] Step 7:

[0201] The information terminal receives the returned response and displays it to the user. Specifically, a display area is created on the interface the user is using, and the response text is displayed there. For example, in response to the question, "What are the features of the new smartphone?", the answer displayed might be, "The new model has a longer battery life and improved camera performance."

[0202] Step 8:

[0203] The information terminal also displays personalized suggestions to the user. Based on past purchase history data, the server suggests the most suitable products and services for the user. This allows users to see recommended products based on their purchase history. For example, if a user has purchased many camera-related products in the past, new cameras and accessories will be suggested.

[0204] Example of a prompt:

[0205] "I'm a 40-year-old man. Please give me some advice on what kind of clothes would suit me."

[0206] This entire process allows users to enjoy a real-time and personalized shopping experience.

[0207] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0208] This invention describes how to specifically implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, by incorporating an emotion engine, the system includes means to recognize the user's emotions and provide more effective answers. The specific operation is described below.

[0209] Program processing

[0210] 1. User enters question

[0211] A user accesses a website using their device and enters a question or inquiry in text format. For example, they might enter the question, "Why is the sky blue?"

[0212] 2. Collection of emotional data

[0213] As the user enters a question, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[0214] 3. Submitting Question Data

[0215] The device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for transmission.

[0216] 4. Processing by the server

[0217] The server analyzes the received question data and sentiment data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[0218] 5. Transfer to a generative artificial intelligence engine

[0219] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[0220] 6. Generating the answer

[0221] A generative artificial intelligence engine analyzes question data and understands the question using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle response such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0222] 7. Return your response

[0223] The generative artificial intelligence engine sends the generated response text back to the server. The server then sends the received response data back to the terminal.

[0224] 8. Display on the device

[0225] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0226] 9. User Verification

[0227] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0228] 10. Providing content for parent users

[0229] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[0230] Specific example

[0231] For example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures the user's facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine. The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) confirms it. Additionally, content explaining the benefits and usage of the emotion engine is displayed for the parent user.

[0232] In this way, the system of the present invention not only solves children's questions but also utilizes an emotion engine to provide appropriate answers that correspond to the user's emotions, thereby promoting parents' understanding and use of generative artificial intelligence technology.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[0236] Step 2:

[0237] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[0238] Step 3:

[0239] When the user clicks the submit button, the device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for submission.

[0240] Step 4:

[0241] The server analyzes the received question data and sentiment data and temporarily stores them in a database. The server then converts the question data and sentiment data into a format suitable for a generative artificial intelligence engine.

[0242] Step 5:

[0243] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[0244] Step 6:

[0245] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate answer that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle-toned answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0246] Step 7:

[0247] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[0248] Step 8:

[0249] The server sends the received response text back to the terminal. The server embeds the response data into an HTML template and sends it to the terminal as an HTTP response.

[0250] Step 9:

[0251] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0252] Step 10:

[0253] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0254] Step 11:

[0255] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[0256] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions, and utilizing the emotion engine to provide appropriate answers that respond to the user's emotions, thereby promoting the parent's understanding and use of generative artificial intelligence technology.

[0257] (Example 2)

[0258] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0259] Traditional question-answering systems often generate answers without considering the user's emotions, resulting in low levels of user understanding and satisfaction. Furthermore, the lack of content explaining the system's usefulness and usage for parental users made effective use of the system difficult.

[0260] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the terminal to collect the user's facial expressions and tone of voice in real time and generate emotion data, means for transmitting the collected emotion data to the server, and means for adjusting the generated response text based on the emotion data. This makes it possible to provide an appropriate response that corresponds to the user's emotions and to improve the usefulness of the system and user satisfaction.

[0261] "User" refers to an individual or their representative who uses the system.

[0262] A "terminal" refers to an electronic device used by a user to operate (for example, a smartphone, tablet, or personal computer).

[0263] "Questions and inquiries" refer to text-based input from users regarding information they want to know or problems they want to solve.

[0264] "Text format" refers to a form of writing that uses characters and symbols.

[0265] "Question data" refers to the text information of questions and doubts entered by users.

[0266] A "server" refers to a central computer that is connected to by multiple terminals via a network.

[0267] "Emotional data" refers to emotional information analyzed based on the user's facial expressions and tone of voice.

[0268] A "generative artificial intelligence engine" refers to artificial intelligence technology that analyzes user questions and generates appropriate answers.

[0269] "Answer text" refers to the content of the answers to user questions and inquiries generated by a generative artificial intelligence engine.

[0270] An "API request" refers to a request for data made through an application programming interface.

[0271] A "database" refers to a system for organizing and managing data.

[0272] "Natural language processing technology" refers to the technology used to process natural language used by humans using computers.

[0273] An "HTTP POST request" refers to a request method for sending data to a server using the HTTP protocol.

[0274] This invention relates to a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. In this system, the server transfers the question data to a generative artificial intelligence engine, which analyzes the input content, generates an appropriate answer, and sends it back to the user. Furthermore, by combining it with an emotion engine, the system also includes means to recognize the user's emotions and provide answers more effectively.

[0275] First, the user accesses the website using their device and enters a question or inquiry in text format. For example, they might type the question "Why is the sky blue?" into the text box and click the submit button. At this stage, the device retrieves the user's input.

[0276] Next, as the user enters a question, the camera and microphone on the device collect facial expressions and voice tone in real time. This emotion data is analyzed by an emotion engine to identify the user's emotions (joy, confusion, anger, etc.). This emotion data is sent to the server along with the question data using an HTTP POST request.

[0277] The server analyzes the received question data and sentiment data and temporarily stores this data in a database. The server converts the question data into a format suitable for a generative artificial intelligence engine, and the sentiment data is tagged and formatted appropriately. The server then transfers the converted question data and sentiment data to the generative artificial intelligence engine using an API request.

[0278] The generative artificial intelligence engine analyzes the question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, the engine might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0279] The generated response text is adjusted based on sentiment data and sent back to the server. The server sends the received response text back to the device, which displays the response on the screen. The user can review the displayed response, resolve their questions, and gain new knowledge. Content explaining the usage and benefits of generative artificial intelligence engines is also provided for parent users.

[0280] To give a concrete example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. The device's camera captures the user's confused expression and collects it as emotion data. This data is sent to a server and transferred to a generative artificial intelligence engine. The generative AI engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) has their question answered. In addition, content explaining the benefits and usage of the emotion engine is displayed for parent users. This allows parents to understand the usefulness of the system and promotes its use at home.

[0281] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0282] Step 1: The user enters questions or inquiries in text form via the terminal. An input form on the web browser is used for the input. The user enters a question such as "Why is the sky blue?" into the input form and clicks the send button.

[0283] Step 2: The terminal obtains the entered text and simultaneously uses the camera and microphone to collect the user's expression and voice tone. These data are analyzed by the emotion engine to identify the user's emotion. The input data is the question text, and the output data is the analyzed emotion data.

[0284] Step 3: The terminal packages the question text and emotion data in JSON format and sends it to the server using an HTTP POST request. The input data is the question text and emotion data, and the output is the transmission to the server.

[0285] Step 4: The server analyzes the received question text and emotion data and temporarily saves them in the database. The input data is the received text and emotion data, and the output is the saving to the database. The server converts the question text into a form suitable for the generative artificial intelligence engine. The converted data is output.

[0286] Step 5: The server transfers the converted question text and emotion data to the generative artificial intelligence engine using a REST API. The input data is the converted text and emotion data. The output is the transfer to the generative artificial intelligence engine.

[0287] Step 6: The generative artificial intelligence engine analyzes the question text and uses natural language processing technology to understand the question content. Considering the emotion data, it generates an appropriate answer corresponding to the user's emotion. For example, an answer such as "The sky is blue because small molecules contained in the Earth's atmosphere scatter sunlight. Did that make sense?" is generated. The input data is the question text and emotion data, and the output is the generated answer text.

[0288] Step 7: The generative AI engine sends the generated response text back to the server. This is also done using a REST API. The input is the generated response text, and the output is the text sent back to the server.

[0289] Step 8: The server sends the received answer text back to the terminal, and the terminal displays this answer on its screen. The input data is the answer text, and the output is the display on the device. For example, the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" will be displayed on the screen.

[0290] Step 9: The user confirms the displayed answer, and the question is resolved. The input is the answer displayed on the screen, and the output is the user's understanding.

[0291] Step 10: The device displays content for the parent user explaining how to use the generative artificial intelligence engine and its benefits. There is no input, and the output is explanatory content for the parent user. For example, it might display something like, "By using this system, you can answer your child's questions quickly and accurately."

[0292] (Application Example 2)

[0293] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0294] Traditional question-solving systems have resulted in low user satisfaction because they lack a system that can appropriately respond to users' emotions when they input questions. Furthermore, answers are simply displayed as text information, lacking features to deepen user understanding. Additionally, the lack of educational content for parents hinders the adoption of generative artificial intelligence engines in the home.

[0295] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0296] In this invention, the server includes means for collecting emotional data while the user inputs a question, means for transferring the collected emotional data to a generative artificial intelligence engine and reflecting it in the generated answer, and means for the user to confirm the displayed answer after the generated answer text is displayed on the terminal. This makes it possible to provide an appropriate answer that matches the user's emotions and deepen the user's understanding. Furthermore, by providing content that explains how to use the generative artificial intelligence engine and its advantages for parent users, its use in the home can be promoted.

[0297] definition statement

[0298] A "user" is someone who uses a system to input questions or inquiries and receive answers.

[0299] A "terminal" is an electronic device used by users to input questions or inquiries in text format and display answers.

[0300] "Question data" refers to information about questions and inquiries entered by users in text format.

[0301] A "server" is a central device that receives question data, transfers it to a generative artificial intelligence engine, and sends the generated answer back to the terminal.

[0302] A "generative artificial intelligence engine" is an artificial intelligence system that analyzes question data and generates appropriate answers.

[0303] "Answer text" refers to the information generated by a generative artificial intelligence engine in response to a user's question.

[0304] "Emotional data" refers to information about facial expressions and tone of voice collected while the user is entering questions.

[0305] The "Emotion Engine" is a device or software that analyzes emotion data and identifies the user's emotional state.

[0306] The "parent user" is an adult user related to the child user and is a person who understands and utilizes the usage and advantages of the generative artificial intelligence engine.

[0307] "Educational content" is information that explains the usage and advantages of the generative artificial intelligence engine for parent users.

[0308] Mode for Implementing the Invention

[0309] This invention is a system in which a user inputs questions or inquiries in text form via a terminal, and based on that information, a generative artificial intelligence engine generates an appropriate answer, and further improves the answer using emotion data. The server plays a role in processing question data and emotion data and transferring them to the generative artificial intelligence engine. The answer corresponding to the user's question is displayed on the terminal. In addition, content explaining the usage and convenience of the generative artificial intelligence engine for parent users is provided.

[0310] Hardware and Software to be Used

[0311] 1. Hardware

[0312] Terminal: An electronic device such as a smartphone or tablet. One equipped with a camera and a microphone.

[0313] Server: A network-connected computer system that receives, processes, and transfers data.

[0314] 2. Software

[0315] Flask: A Python (registered trademark)-based web framework. It processes requests from users and communicates with the server.

[0316] OpenAI API: Functions as a generative artificial intelligence engine. It analyzes question data and generates appropriate answers.

[0317] Emotion_recognition library: Analyzes emotional data. It evaluates the user's facial expressions and voice tone in real time.

[0318] System operation

[0319] 1. User enters question

[0320] The user uses their device to input questions or inquiries in text format. For example, they might input the question, "Why is the sky blue?"

[0321] 2. Collection of emotional data

[0322] While the user is entering a question, the emotion engine (Emotion_recognition library) uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time, and collects this as emotion data.

[0323] 3. Submitting Question Data

[0324] The entered question data and collected sentiment data are sent to the server via an HTTP POST request.

[0325] 4. Data processing by the server

[0326] The server analyzes the received question data and sentiment data and temporarily stores them in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API).

[0327] 5. Data transfer to the generative artificial intelligence engine

[0328] The server transfers the converted question data and sentiment data to the generative artificial intelligence engine and sends a request to generate an appropriate answer.

[0329] 6. Generating the answer

[0330] The generative artificial intelligence engine analyzes question data and sentiment data, and uses natural language processing techniques to generate appropriate answers that correspond to the user's emotions. For example, if the question is "Why is the sky blue?" and it is presumed that the user is confused, it will generate an answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0331] 7. Returning and displaying responses

[0332] The generated response text is sent back to the terminal via the server and displayed on the user's terminal.

[0333] Specific example

[0334] For example, a user types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures their facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine (OpenAI). The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user confirms it.

[0335] Example of a prompt

[0336] User question: Why is the sky blue?

[0337] User emotion:

[0338] AI response:

[0339] This system allows users to resolve their questions while simultaneously obtaining answers in a more understandable format using sentiment data. It also provides content explaining the convenience and applications of generative artificial intelligence engines for parental users.

[0340] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0341] Processing steps

[0342] Step 1:

[0343] The user enters questions or inquiries in text format via their device. The entered text is stored on the user's device. A concrete example is the user entering the question, "Why is the sky blue?"

[0344] Step 2:

[0345] The device's camera and microphone capture the user's facial expressions and voice tone in real time. An emotion engine (Emotion_recognition library) is used to analyze the emotional data and recognize the user's emotional state. This data is collected as emotion data. The input is the video and audio data captured by the device, and the output is the analyzed emotion data.

[0346] Step 3:

[0347] The terminal sends the entered question data and collected sentiment data to the server. The data is sent using an HTTP POST request. The input consists of the question data and sentiment data, and the output is a notification that the data transfer to the server is complete.

[0348] Step 4:

[0349] The server temporarily stores the received question data and sentiment data in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API). The input is the received question data and sentiment data, and the output is the converted data format.

[0350] Step 5:

[0351] The server sends the transformed question data and sentiment data to the generative artificial intelligence engine as an API request. The API request includes a prompt. The input is the transformed question data and sentiment data, and the output is the sending of a request to the generative artificial intelligence engine.

[0352] Step 6:

[0353] A generative artificial intelligence engine analyzes question data and sentiment data to generate appropriate answers. For example, it might generate the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight." The input is question data and sentiment data, and the output is the generated answer text.

[0354] Step 7:

[0355] The generated response text is sent back to the server. The server receives the response text and forwards it back to the terminal. The input is the generated response text, and the output is the transmission of the response to the terminal.

[0356] Step 8:

[0357] The device displays the answer text received from the server on the screen. The user checks the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input is the answer text, and the output is the displayed answer text.

[0358] Step 9:

[0359] The device further displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. This promotes the use of the engine at home. The input is educational content, and the output is the displayed educational content.

[0360] These steps provide users with appropriate responses that respond to their emotions, deepen their understanding, and allow parent users to learn how to utilize generative artificial intelligence engines.

[0361] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0362] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0363] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0364] [Second Embodiment]

[0365] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0366] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0367] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0368] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0369] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0370] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0371] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0372] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0373] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0374] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0375] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0376] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0377] This invention describes how to implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, the invention describes a mechanism for providing content to parent users explaining how to use the generative artificial intelligence engine and its benefits.

[0378] Program processing

[0379] 1. User enters question

[0380] Users access the website using their devices and enter their questions or inquiries in text format.

[0381] For example, a user (a child) might input the question, "Why is the sky blue?"

[0382] 2. Submitting Question Data

[0383] The entered question data is sent from the terminal to the server. The server receives this data and stores it temporarily.

[0384] 3. Transfer to a generative artificial intelligence engine

[0385] The server transfers the received question data to the generative artificial intelligence engine.

[0386] The generative artificial intelligence engine analyzes the question data and generates appropriate answers to it.

[0387] 4. Generating the answer

[0388] A generative artificial intelligence engine analyzes the question and generates an answer in natural language. For example, it might generate an answer like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0389] 5. Return your response

[0390] The server sends the generated response text back to the terminal. The terminal displays the received response on its screen.

[0391] The user (child) checks the answer displayed on the screen, and their question is resolved.

[0392] 6. Providing content for parent users

[0393] The device also displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines.

[0394] For example, parents can obtain information such as, "By using generative AI, you can instantly answer your child's questions and improve their learning efficiency."

[0395] Specific example

[0396] As a concrete example, let's explain what happens when a child types "Why is the sky blue?" into a device.

[0397] 1. The user (child) enters "Why is the sky blue?" into the input field on the device and clicks the send button.

[0398] 2. The question data is sent to the server immediately.

[0399] 3. The server receives the question data and passes it to the generative artificial intelligence engine.

[0400] 4. The generative artificial intelligence engine analyzes the question and generates the answer, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0401] 5. The server sends the generated response back to the terminal, and the terminal displays the response on its screen.

[0402] 6. We will also provide content for parent users that explains the convenience of generative artificial intelligence engines and how to use them at home.

[0403] In this way, the system of the present invention is designed to quickly resolve children's questions, and parents can learn about the use and benefits of generative artificial intelligence engines, and apply them to their daily lives.

[0404] The following describes the processing flow.

[0405] Step 1:

[0406] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[0407] Step 2:

[0408] When the user clicks the submit button, the device sends the entered question data to the server. An HTTP POST request is used for submission.

[0409] Step 3:

[0410] The server analyzes the received question data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[0411] Step 4:

[0412] The server transfers the transformed question data to the generative artificial intelligence engine. The question data is sent to the generative artificial intelligence engine using an API request.

[0413] Step 5:

[0414] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. It then generates an appropriate answer to the question.

[0415] Step 6:

[0416] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[0417] Step 7:

[0418] The server sends the received response text back to the user's device. The server embeds the response data into an HTML template and sends it to the device as an HTTP response.

[0419] Step 8:

[0420] The device receives a response from the server and displays the answer on the screen. Specifically, an answer such as, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight," is displayed.

[0421] Step 9:

[0422] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0423] Step 10:

[0424] The device also displays content explaining how to use and the convenience of the generative artificial intelligence engine for parent users. Parent users read the explanations and understand how to use it at home.

[0425] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions and promoting the understanding and application of generative artificial intelligence technology.

[0426] (Example 1)

[0427] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0428] Conventional question-answering systems suffer from an inefficient process, where users input questions in text format and then receive appropriate answers, resulting in slow response times. Furthermore, they lacked user-friendliness and ease of use when utilizing generative artificial intelligence engines. There is a need to address these issues and provide a more efficient and immediate question-answering system.

[0429] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0430] In this invention, the server includes means for sending input question data to the server using an HTTP POST request, means for the server to temporarily store the received question data in a database, and means for the server to send the question data to the API of a generative artificial intelligence engine. This makes it possible to quickly and efficiently generate an appropriate answer after a user inputs and submits a question, and to display that answer to the user. By providing explanatory content for parent users, it is also possible to deepen their understanding of how to use and the convenience of the generative artificial intelligence engine.

[0431] A "user" is an individual or group that uses a system or service.

[0432] A "terminal" refers to an electronic device used by a user to access a system, such as a smartphone, tablet, or personal computer.

[0433] "Questions and doubts" refer to information that users want to know, problems they want to solve, or things they want to confirm.

[0434] "Text format" refers to the representation format of data entered as a string of characters.

[0435] "Question data" refers to data that includes questions and doubts entered by users.

[0436] A "server" is a computer system that provides services over a network, and is a device that receives, processes, and stores question data.

[0437] A "generative artificial intelligence engine" is a program that possesses artificial intelligence technology to analyze input question data and generate appropriate answers.

[0438] "Answer text" refers to the content of the answer to the user's question, generated by a generative artificial intelligence engine.

[0439] "Means of display" refers to a method or function for visually displaying the answer text on the device screen.

[0440] An "HTTP POST request" is a type of HTTP protocol used to send data from a user's device to a server.

[0441] A "database" refers to a system for systematically storing, managing, and retrieving information, and specifically includes relational database management systems (RDBMS).

[0442] An "API" is an interface that allows a specific program or service to communicate with other programs.

[0443] "Content for parent users" refers to content that provides information explaining how to use and the benefits of generative artificial intelligence engines.

[0444] This invention relates to a system that allows users to input questions or doubts via a terminal and quickly generates and provides appropriate answers to those questions. This system has a series of processes that analyze questions using a generative artificial intelligence engine and generate appropriate answers.

[0445] The specific hardware configuration of this system includes a terminal that receives user input, a server that receives, stores, transfers, and returns question data, and a generative artificial intelligence engine that analyzes the question data and generates answers. The software used includes a web browser, a server API for processing HTTP POST requests, a database management system, and the generative artificial intelligence engine.

[0446] Hardware and software

[0447] 1. Terminal

[0448] The user enters the question by opening a web browser on a device (smartphone, tablet, PC, etc.) and accessing a specific website. The device communicates with the server via an internet connection.

[0449] 2. Server

[0450] The server receives the input question data and forwards it to the generative artificial intelligence engine. The server temporarily stores the question data using a database (e.g., MySQL). Then, it sends the question data using the generative artificial intelligence engine's API and receives the generated answer.

[0451] 3. Generative Artificial Intelligence Engine

[0452] A generative artificial intelligence engine (for example, OpenAI's GPT-4) analyzes question data sent from a server and generates the optimal answer to that question. Based on the analysis results, the engine creates an answer in natural language and sends that answer back to the server.

[0453] Specific examples of data processing

[0454] For example, if a user (a child) enters the question "Why is the sky blue?" into the device, the process will proceed as follows:

[0455] 1. The user enters "Why is the sky blue?" into the input field on their device and clicks the send button.

[0456] 2. The terminal sends the entered question data to the server using an HTTP POST request.

[0457] 3. The server receives the question data and temporarily stores it in the database.

[0458] 4. The server sends the question data to the API of the generative artificial intelligence engine.

[0459] Prompt example: "Why is the sky blue?"

[0460] 5. A generative artificial intelligence engine analyzes the question and generates the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0461] 6. The server sends the generated response text back to the terminal, and the terminal displays the response on its screen.

[0462] 7. The user (child) checks the answer displayed on the device screen.

[0463] 8. The device displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. For example, it might display content such as, "By using generative AI, you can instantly answer your child's questions and improve learning efficiency."

[0464] This system makes it possible to provide users with quick and appropriate answers to their questions, and also helps parents understand the convenience of generative artificial intelligence engines.

[0465] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0466] Step 1:

[0467] The user opens a web browser on their device. The user enters a question or doubt in text format into an input field. For example, they might enter the question, "Why is the sky blue?"

[0468] Input: Questions or inquiries entered by the user in text format.

[0469] Output: Question text displayed in the input field

[0470] Step 2:

[0471] When the user clicks the submit button, the device sends the entered question data to the server using an HTTP POST request.

[0472] Input: Data of the questions or inquiries that were entered.

[0473] Output: HTTP POST request sent to the server

[0474] Step 3:

[0475] The server receives the HTTP POST request and verifies the query data. The server temporarily stores the received query data in a database (e.g., MySQL).

[0476] Input: Question data sent in an HTTP POST request

[0477] Output: Question data temporarily stored in the database

[0478] Step 4:

[0479] The server retrieves the question data from the database and sends it to the API of the generative artificial intelligence engine.

[0480] Input: Question data stored in the database

[0481] Output: Question data sent to the API of a generative artificial intelligence engine.

[0482] Step 5:

[0483] A generative artificial intelligence engine receives the question data and analyzes it. Based on the analysis results, the generative artificial intelligence engine generates an answer in natural language.

[0484] Input: Question data sent to a generative artificial intelligence engine

[0485] Output: Generated answer text

[0486] Specific operation: For example, if the prompt is "Why is the sky blue?", the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight" will be generated.

[0487] Step 6:

[0488] The generative artificial intelligence engine sends the generated response back to the server. The server checks the received response text and sends the response back to the terminal using an HTTP response.

[0489] Input: Response text returned by the generative artificial intelligence engine

[0490] Output: The HTTP response sent to the terminal will include the response text.

[0491] Step 7:

[0492] The terminal receives an HTTP response from the server, parses the received response text, and formats it for display. The terminal displays the response on the screen, and the user confirms it.

[0493] Input: HTTP response sent from the server

[0494] Output: Answer text displayed on the screen

[0495] Specific action: For example, the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight," will be displayed on the device screen.

[0496] Step 8:

[0497] The device further displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. Parent users will understand the educational benefits that can be gained by utilizing generative artificial intelligence engines.

[0498] Input: Details on the advantages and usage of a specific generative artificial intelligence engine.

[0499] Output: Explanatory content displayed for the parent user

[0500] Specific actions: For example, information such as "By using generative AI, children's questions can be answered instantly, improving learning efficiency" will be displayed.

[0501] (Application Example 1)

[0502] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0503] In current virtual stores, it is difficult for users to get immediate and accurate answers to their questions. Furthermore, the lack of mechanisms for personalized suggestions based on customer purchase history makes it difficult to provide optimal product selection and detailed product information. Additionally, systems utilizing generative artificial intelligence engines are difficult for the average user to understand, and their advantages and usage are not clearly communicated, which is another challenge.

[0504] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0505] In this invention, the server includes means for a user to input questions or inquiries in text format via an information terminal; means for transmitting the input question data to an information processing device; means for the information processing device to transfer the question data to a generative artificial intelligence engine; means for the generative artificial intelligence engine to analyze the questions and generate appropriate answers; means for the information processing device to return the generated answer text to the information terminal; means for the information terminal to display the returned answers to the user; means for providing immediate answers to customer inquiries in a virtual space; and means for providing personalized suggestions based on the customer's purchase history. As a result, users can resolve their questions in real time and enjoy a more efficient and satisfying shopping experience by receiving personalized suggestions. Furthermore, by providing parents with information on the advantages and usage of the generative artificial intelligence engine, it is expected that its use within the home will be promoted.

[0506] A "user" refers to an individual who inputs questions or inquiries via an information terminal.

[0507] "Information terminal" is a general term for electronic devices used by users to input questions or inquiries, and includes smartphones, smart glasses, and head-mounted displays.

[0508] "Questions and doubts" refer to things that users want to know or things they are unsure about, expressed in text format.

[0509] An "information processing device" refers to a device that receives question data sent by a user and transfers it to a generative artificial intelligence engine.

[0510] A "generative artificial intelligence engine" refers to artificial intelligence that analyzes user questions and generates appropriate answers.

[0511] "Answer text" refers to the response generated by a generative artificial intelligence engine in response to a question, and is written in natural language.

[0512] "Virtual space" refers to a general term for three-dimensional spaces virtually constructed on a computer system, providing an environment where users can act as if they were actually present.

[0513] "Customer purchase history" refers to a record of products that a customer has purchased in the past, and this information is used to provide personalized recommendations.

[0514] "Personalized recommendations" refer to recommendations for products and services tailored to individual preferences, based on a customer's purchase history.

[0515] This invention is a system that provides immediate answers to user questions and offers personalized suggestions. Specific embodiments are described below.

[0516] First, the user inputs questions or inquiries via an information terminal. These terminals include, for example, smartphones, smart glasses, and head-mounted displays. These terminals operate within a virtual space and provide an interface that allows the user to input questions in text format.

[0517] The input question data is sent to the information processing device. The information processing device receives this question data and stores it temporarily. Next, the information processing device transfers the received question data to a generative artificial intelligence engine. This generative artificial intelligence engine uses GPT-4 or an equivalent AI model. The generative artificial intelligence engine analyzes the question data and generates an appropriate answer to it.

[0518] The generated response text is sent back to the information terminal by the information processing device. The information terminal displays the received response to the user. This allows the user to obtain answers to their questions in real time.

[0519] Furthermore, this includes means of providing instant answers to customer questions in a virtual space, as well as means of making personalized suggestions based on customer purchase history. Customer purchase history data is stored in an information processing device, and an AI engine analyzes it to suggest the most suitable products and services for each individual user.

[0520] The system's operation is illustrated with a concrete example. For instance, if a user enters "Please tell me the features of the new smartphone released at this store," the question data is sent to the information processing device and passed to a generative artificial intelligence engine. The AI ​​engine performs analysis and generates an answer such as "The new model has a longer battery life and improved camera performance," which is then sent back and displayed on the information terminal.

[0521] An example of a prompt would be, "I am a 40-year-old man. Please advise me on what kind of clothes would suit me." In response to this prompt, the generative artificial intelligence engine will generate an answer that includes advice on fashion styles suitable for a 40-year-old man and provide it to the user.

[0522] In this way, the present invention quickly resolves user questions and improves the purchasing experience. By providing parents with information on the advantages and usage of the generative artificial intelligence engine, it can also promote its use within the home.

[0523] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0524] Step 1:

[0525] The user uses an information terminal to input a question or inquiry in text format and presses the send button. The entered question is captured by the information terminal as text data. This input data is then formatted for subsequent processing. Specifically, the user inputs "Please tell me about the features of the new smartphone."

[0526] Step 2:

[0527] The entered question data is sent from the information terminal to the information processing device (server). Here, the information terminal sends the data using an HTTP POST request. The server receives this and temporarily stores the question data. This step verifies that the input data has been successfully delivered to the server.

[0528] Step 3:

[0529] The server forwards the received question data to the generative artificial intelligence engine. Specifically, the server sends the question data again to the AI ​​engine's API endpoint using an HTTP POST request. At this time, the data is converted to JSON format. After sending, the server waits for a response from the generative artificial intelligence engine.

[0530] Step 4:

[0531] The generative artificial intelligence engine analyzes the received question data and generates an appropriate answer. An AI model (e.g., GPT-4) is used in this process. Specifically, it receives the question text as an input prompt, performs natural language processing, and generates the most appropriate answer. For example, it might generate the answer, "The new model has a longer battery life and improved camera performance."

[0532] Step 5:

[0533] The generated response text is sent back from the generative artificial intelligence engine to the server. The server receives it and temporarily stores it again. The returned data is in JSON format, and after receiving this data, the server prepares to convert it to an appropriate format.

[0534] Step 6:

[0535] The server sends the generated response text back to the information terminal. It then uses an HTTP POST request again to send the data to the appropriate endpoint. At this point, the data is converted to a text format suitable for display. After sending, the server verifies the response.

[0536] Step 7:

[0537] The information terminal receives the returned response and displays it to the user. Specifically, a display area is created on the interface the user is using, and the response text is displayed there. For example, in response to the question, "What are the features of the new smartphone?", the answer displayed might be, "The new model has a longer battery life and improved camera performance."

[0538] Step 8:

[0539] The information terminal also displays personalized suggestions to the user. Based on past purchase history data, the server suggests the most suitable products and services for the user. This allows users to see recommended products based on their purchase history. For example, if a user has purchased many camera-related products in the past, new cameras and accessories will be suggested.

[0540] Example of a prompt:

[0541] "I'm a 40-year-old man. Please give me some advice on what kind of clothes would suit me."

[0542] This entire process allows users to enjoy a real-time and personalized shopping experience.

[0543] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0544] This invention describes how to specifically implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, by incorporating an emotion engine, the system includes means to recognize the user's emotions and provide more effective answers. The specific operation is described below.

[0545] Program processing

[0546] 1. User enters question

[0547] A user accesses a website using their device and enters a question or inquiry in text format. For example, they might enter the question, "Why is the sky blue?"

[0548] 2. Collection of emotional data

[0549] As the user enters a question, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[0550] 3. Submitting Question Data

[0551] The device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for transmission.

[0552] 4. Processing by the server

[0553] The server analyzes the received question data and sentiment data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[0554] 5. Transfer to a generative artificial intelligence engine

[0555] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[0556] 6. Generating the answer

[0557] A generative artificial intelligence engine analyzes question data and understands the question using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle response such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0558] 7. Return your response

[0559] The generative artificial intelligence engine sends the generated response text back to the server. The server then sends the received response data back to the terminal.

[0560] 8. Display on the device

[0561] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0562] 9. User Verification

[0563] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0564] 10. Providing content for parent users

[0565] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[0566] Specific example

[0567] For example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures the user's facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine. The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) confirms it. Additionally, content explaining the benefits and usage of the emotion engine is displayed for the parent user.

[0568] In this way, the system of the present invention not only solves children's questions but also utilizes an emotion engine to provide appropriate answers that correspond to the user's emotions, thereby promoting parents' understanding and use of generative artificial intelligence technology.

[0569] The following describes the processing flow.

[0570] Step 1:

[0571] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[0572] Step 2:

[0573] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[0574] Step 3:

[0575] When the user clicks the submit button, the device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for submission.

[0576] Step 4:

[0577] The server analyzes the received question data and sentiment data and temporarily stores them in a database. The server then converts the question data and sentiment data into a format suitable for a generative artificial intelligence engine.

[0578] Step 5:

[0579] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[0580] Step 6:

[0581] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate answer that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle-toned answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0582] Step 7:

[0583] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[0584] Step 8:

[0585] The server sends the received response text back to the terminal. The server embeds the response data into an HTML template and sends it to the terminal as an HTTP response.

[0586] Step 9:

[0587] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0588] Step 10:

[0589] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0590] Step 11:

[0591] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[0592] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions, and utilizing the emotion engine to provide appropriate answers that respond to the user's emotions, thereby promoting the parent's understanding and use of generative artificial intelligence technology.

[0593] (Example 2)

[0594] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0595] Traditional question-answering systems often generate answers without considering the user's emotions, resulting in low levels of user understanding and satisfaction. Furthermore, the lack of content explaining the system's usefulness and usage for parental users made effective use of the system difficult.

[0596] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the terminal to collect the user's facial expressions and tone of voice in real time and generate emotion data, means for transmitting the collected emotion data to the server, and means for adjusting the generated response text based on the emotion data. This makes it possible to provide an appropriate response that corresponds to the user's emotions and to improve the usefulness of the system and user satisfaction.

[0597] "User" refers to an individual or their representative who uses the system.

[0598] A "terminal" refers to an electronic device used by a user to operate (for example, a smartphone, tablet, or personal computer).

[0599] "Questions and inquiries" refer to text-based input from users regarding information they want to know or problems they want to solve.

[0600] "Text format" refers to a form of writing that uses characters and symbols.

[0601] "Question data" refers to the text information of questions and doubts entered by users.

[0602] A "server" refers to a central computer that is connected to by multiple terminals via a network.

[0603] "Emotional data" refers to emotional information analyzed based on the user's facial expressions and tone of voice.

[0604] A "generative artificial intelligence engine" refers to artificial intelligence technology that analyzes user questions and generates appropriate answers.

[0605] "Answer text" refers to the content of the answers to user questions and inquiries generated by a generative artificial intelligence engine.

[0606] An "API request" refers to a request for data made through an application programming interface.

[0607] A "database" refers to a system for organizing and managing data.

[0608] "Natural language processing technology" refers to the technology used to process natural language used by humans using computers.

[0609] An "HTTP POST request" refers to a request method for sending data to a server using the HTTP protocol.

[0610] This invention relates to a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. In this system, the server transfers the question data to a generative artificial intelligence engine, which analyzes the input content, generates an appropriate answer, and sends it back to the user. Furthermore, by combining it with an emotion engine, the system also includes means to recognize the user's emotions and provide answers more effectively.

[0611] First, the user accesses the website using their device and enters a question or inquiry in text format. For example, they might type the question "Why is the sky blue?" into the text box and click the submit button. At this stage, the device retrieves the user's input.

[0612] Next, as the user enters a question, the camera and microphone on the device collect facial expressions and voice tone in real time. This emotion data is analyzed by an emotion engine to identify the user's emotions (joy, confusion, anger, etc.). This emotion data is sent to the server along with the question data using an HTTP POST request.

[0613] The server analyzes the received question data and sentiment data and temporarily stores this data in a database. The server converts the question data into a format suitable for a generative artificial intelligence engine, and the sentiment data is tagged and formatted appropriately. The server then transfers the converted question data and sentiment data to the generative artificial intelligence engine using an API request.

[0614] The generative artificial intelligence engine analyzes the question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, the engine might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0615] The generated response text is adjusted based on sentiment data and sent back to the server. The server sends the received response text back to the device, which displays the response on the screen. The user can review the displayed response, resolve their questions, and gain new knowledge. Content explaining the usage and benefits of generative artificial intelligence engines is also provided for parent users.

[0616] To give a concrete example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. The device's camera captures the user's confused expression and collects it as emotion data. This data is sent to a server and transferred to a generative artificial intelligence engine. The generative AI engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) has their question answered. In addition, content explaining the benefits and usage of the emotion engine is displayed for parent users. This allows parents to understand the usefulness of the system and promotes its use at home.

[0617] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0618] Step 1: The user enters their question or inquiry in text format via their device. An input form on a web browser is used for input. The user enters a question such as "Why is the sky blue?" into the input form and clicks the submit button.

[0619] Step 2: The device acquires the entered text and simultaneously uses the camera and microphone to collect the user's facial expressions and voice tone. This data is analyzed by an emotion engine to identify the user's emotions. The input data is the question text, and the output data is the analyzed emotion data.

[0620] Step 3: The terminal packages the question text and sentiment data in JSON format and sends it to the server using an HTTP POST request. The input data is the question text and sentiment data, and the output is the data sent to the server.

[0621] Step 4: The server analyzes the received question text and sentiment data and temporarily stores them in the database. The input data is the received text and sentiment data, and the output is storage in the database. The server converts the question text into a format suitable for a generative artificial intelligence engine. The converted data is then output.

[0622] Step 5: The server transfers the converted question text and sentiment data to the generative artificial intelligence engine using a REST API. The input data is the converted text and sentiment data. The output is the transfer to the generative artificial intelligence engine.

[0623] Step 6: The generative artificial intelligence engine analyzes the question text and understands the question using natural language processing techniques. It then generates an appropriate response that corresponds to the user's emotions, taking sentiment data into consideration. For example, it might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input data consists of the question text and sentiment data, and the output is the generated response text.

[0624] Step 7: The generative AI engine sends the generated response text back to the server. This is also done using a REST API. The input is the generated response text, and the output is the text sent back to the server.

[0625] Step 8: The server sends the received answer text back to the terminal, and the terminal displays this answer on its screen. The input data is the answer text, and the output is the display on the device. For example, the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" will be displayed on the screen.

[0626] Step 9: The user confirms the displayed answer, and the question is resolved. The input is the answer displayed on the screen, and the output is the user's understanding.

[0627] Step 10: The device displays content for the parent user explaining how to use the generative artificial intelligence engine and its benefits. There is no input, and the output is explanatory content for the parent user. For example, it might display something like, "By using this system, you can answer your child's questions quickly and accurately."

[0628] (Application Example 2)

[0629] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0630] Traditional question-solving systems have resulted in low user satisfaction because they lack a system that can appropriately respond to users' emotions when they input questions. Furthermore, answers are simply displayed as text information, lacking features to deepen user understanding. Additionally, the lack of educational content for parents hinders the adoption of generative artificial intelligence engines in the home.

[0631] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0632] In this invention, the server includes means for collecting emotional data while the user inputs a question, means for transferring the collected emotional data to a generative artificial intelligence engine and reflecting it in the generated answer, and means for the user to confirm the displayed answer after the generated answer text is displayed on the terminal. This makes it possible to provide an appropriate answer that matches the user's emotions and deepen the user's understanding. Furthermore, by providing content that explains how to use the generative artificial intelligence engine and its advantages for parent users, its use in the home can be promoted.

[0633] definition statement

[0634] A "user" is someone who uses a system to input questions or inquiries and receive answers.

[0635] A "terminal" is an electronic device used by users to input questions or inquiries in text format and display answers.

[0636] "Question data" refers to information about questions and inquiries entered by users in text format.

[0637] A "server" is a central device that receives question data, transfers it to a generative artificial intelligence engine, and sends the generated answer back to the terminal.

[0638] A "generative artificial intelligence engine" is an artificial intelligence system that analyzes question data and generates appropriate answers.

[0639] "Answer text" refers to the information generated by a generative artificial intelligence engine in response to a user's question.

[0640] "Emotional data" refers to information about facial expressions and tone of voice collected while the user is entering questions.

[0641] An "emotion engine" is a device or software that analyzes emotional data to identify a user's emotional state.

[0642] A "parent user" is an adult user related to a child user, who understands and utilizes the features and benefits of a generative artificial intelligence engine.

[0643] "Educational content" refers to information that explains how to use and the benefits of generative artificial intelligence engines for parent users.

[0644] Modes for carrying out the invention

[0645] This invention is a system in which a user inputs questions or inquiries in text format via a terminal, a generative artificial intelligence engine generates appropriate answers based on that information, and further improves the answers using sentiment data. The server processes the question data and sentiment data and transfers them to the generative artificial intelligence engine. The answers corresponding to the user's questions are displayed on the terminal. In addition, content explaining how to use and the convenience of the generative artificial intelligence engine is provided for parent users.

[0646] Hardware and software to be used

[0647] 1. Hardware

[0648] Device: An electronic device such as a smartphone or tablet. It must have a camera and microphone.

[0649] Server: A network-connected computer system that receives, processes, and transfers data.

[0650] 2. Software

[0651] Flask: A Python-based web framework. It handles user requests and communicates with the server.

[0652] OpenAI API: Functions as a generative artificial intelligence engine. It analyzes question data and generates appropriate answers.

[0653] Emotion_recognition library: Analyzes emotional data. It evaluates the user's facial expressions and voice tone in real time.

[0654] System operation

[0655] 1. User enters question

[0656] The user uses their device to input questions or inquiries in text format. For example, they might input the question, "Why is the sky blue?"

[0657] 2. Collection of emotional data

[0658] While the user is entering a question, the emotion engine (Emotion_recognition library) uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time, and collects this as emotion data.

[0659] 3. Submitting Question Data

[0660] The entered question data and collected sentiment data are sent to the server via an HTTP POST request.

[0661] 4. Data processing by the server

[0662] The server analyzes the received question data and sentiment data and temporarily stores them in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API).

[0663] 5. Data transfer to the generative artificial intelligence engine

[0664] The server transfers the converted question data and sentiment data to the generative artificial intelligence engine and sends a request to generate an appropriate answer.

[0665] 6. Generating the answer

[0666] The generative artificial intelligence engine analyzes question data and sentiment data, and uses natural language processing techniques to generate appropriate answers that correspond to the user's emotions. For example, if the question is "Why is the sky blue?" and it is presumed that the user is confused, it will generate an answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0667] 7. Returning and displaying responses

[0668] The generated response text is sent back to the terminal via the server and displayed on the user's terminal.

[0669] Specific example

[0670] For example, a user types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures their facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine (OpenAI). The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user confirms it.

[0671] Example of a prompt

[0672] User question: Why is the sky blue?

[0673] User emotion:

[0674] AI response:

[0675] This system allows users to resolve their questions while simultaneously obtaining answers in a more understandable format using sentiment data. It also provides content explaining the convenience and applications of generative artificial intelligence engines for parental users.

[0676] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0677] Processing steps

[0678] Step 1:

[0679] The user enters questions or inquiries in text format via their device. The entered text is stored on the user's device. A concrete example is the user entering the question, "Why is the sky blue?"

[0680] Step 2:

[0681] The device's camera and microphone capture the user's facial expressions and voice tone in real time. An emotion engine (Emotion_recognition library) is used to analyze the emotional data and recognize the user's emotional state. This data is collected as emotion data. The input is the video and audio data captured by the device, and the output is the analyzed emotion data.

[0682] Step 3:

[0683] The terminal sends the entered question data and collected sentiment data to the server. The data is sent using an HTTP POST request. The input consists of the question data and sentiment data, and the output is a notification that the data transfer to the server is complete.

[0684] Step 4:

[0685] The server temporarily stores the received question data and sentiment data in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API). The input is the received question data and sentiment data, and the output is the converted data format.

[0686] Step 5:

[0687] The server sends the transformed question data and sentiment data to the generative artificial intelligence engine as an API request. The API request includes a prompt. The input is the transformed question data and sentiment data, and the output is the sending of a request to the generative artificial intelligence engine.

[0688] Step 6:

[0689] A generative artificial intelligence engine analyzes question data and sentiment data to generate appropriate answers. For example, it might generate the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight." The input is question data and sentiment data, and the output is the generated answer text.

[0690] Step 7:

[0691] The generated response text is sent back to the server. The server receives the response text and forwards it back to the terminal. The input is the generated response text, and the output is the transmission of the response to the terminal.

[0692] Step 8:

[0693] The device displays the answer text received from the server on the screen. The user checks the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input is the answer text, and the output is the displayed answer text.

[0694] Step 9:

[0695] The device further displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. This promotes the use of the engine at home. The input is educational content, and the output is the displayed educational content.

[0696] These steps provide users with appropriate responses that respond to their emotions, deepen their understanding, and allow parent users to learn how to utilize generative artificial intelligence engines.

[0697] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0698] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0699] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0700] [Third Embodiment]

[0701] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0702] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0703] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0704] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0705] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0706] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0707] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0708] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0709] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0710] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0711] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0712] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0713] This invention describes how to implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, the invention describes a mechanism for providing content to parent users explaining how to use the generative artificial intelligence engine and its benefits.

[0714] Program processing

[0715] 1. User enters question

[0716] Users access the website using their devices and enter their questions or inquiries in text format.

[0717] For example, a user (a child) might input the question, "Why is the sky blue?"

[0718] 2. Submitting Question Data

[0719] The entered question data is sent from the terminal to the server. The server receives this data and stores it temporarily.

[0720] 3. Transfer to a generative artificial intelligence engine

[0721] The server transfers the received question data to the generative artificial intelligence engine.

[0722] The generative artificial intelligence engine analyzes the question data and generates appropriate answers to it.

[0723] 4. Generating the answer

[0724] A generative artificial intelligence engine analyzes the question and generates an answer in natural language. For example, it might generate an answer like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0725] 5. Return your response

[0726] The server sends the generated response text back to the terminal. The terminal displays the received response on its screen.

[0727] The user (child) checks the answer displayed on the screen, and their question is resolved.

[0728] 6. Providing content for parent users

[0729] The device also displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines.

[0730] For example, parents can obtain information such as, "By using generative AI, you can instantly answer your child's questions and improve their learning efficiency."

[0731] Specific example

[0732] As a concrete example, let's explain what happens when a child types "Why is the sky blue?" into a device.

[0733] 1. The user (child) enters "Why is the sky blue?" into the input field on the device and clicks the send button.

[0734] 2. The question data is sent to the server immediately.

[0735] 3. The server receives the question data and passes it to the generative artificial intelligence engine.

[0736] 4. The generative artificial intelligence engine analyzes the question and generates the answer, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0737] 5. The server sends the generated response back to the terminal, and the terminal displays the response on its screen.

[0738] 6. We will also provide content for parent users that explains the convenience of generative artificial intelligence engines and how to use them at home.

[0739] In this way, the system of the present invention is designed to quickly resolve children's questions, and parents can learn about the use and benefits of generative artificial intelligence engines, and apply them to their daily lives.

[0740] The following describes the processing flow.

[0741] Step 1:

[0742] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[0743] Step 2:

[0744] When the user clicks the submit button, the device sends the entered question data to the server. An HTTP POST request is used for submission.

[0745] Step 3:

[0746] The server analyzes the received question data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[0747] Step 4:

[0748] The server transfers the transformed question data to the generative artificial intelligence engine. The question data is sent to the generative artificial intelligence engine using an API request.

[0749] Step 5:

[0750] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. It then generates an appropriate answer to the question.

[0751] Step 6:

[0752] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[0753] Step 7:

[0754] The server sends the received response text back to the user's device. The server embeds the response data into an HTML template and sends it to the device as an HTTP response.

[0755] Step 8:

[0756] The device receives a response from the server and displays the answer on the screen. Specifically, an answer such as, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight," is displayed.

[0757] Step 9:

[0758] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0759] Step 10:

[0760] The device also displays content explaining how to use and the convenience of the generative artificial intelligence engine for parent users. Parent users read the explanations and understand how to use it at home.

[0761] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions and promoting the understanding and application of generative artificial intelligence technology.

[0762] (Example 1)

[0763] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0764] Conventional question-answering systems suffer from an inefficient process, where users input questions in text format and then receive appropriate answers, resulting in slow response times. Furthermore, they lacked user-friendliness and ease of use when utilizing generative artificial intelligence engines. There is a need to address these issues and provide a more efficient and immediate question-answering system.

[0765] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0766] In this invention, the server includes means for sending input question data to the server using an HTTP POST request, means for the server to temporarily store the received question data in a database, and means for the server to send the question data to the API of a generative artificial intelligence engine. This makes it possible to quickly and efficiently generate an appropriate answer after a user inputs and submits a question, and to display that answer to the user. By providing explanatory content for parent users, it is also possible to deepen their understanding of how to use and the convenience of the generative artificial intelligence engine.

[0767] A "user" is an individual or group that uses a system or service.

[0768] A "terminal" refers to an electronic device used by a user to access a system, such as a smartphone, tablet, or personal computer.

[0769] "Questions and doubts" refer to information that users want to know, problems they want to solve, or things they want to confirm.

[0770] "Text format" refers to the representation format of data entered as a string of characters.

[0771] "Question data" refers to data that includes questions and doubts entered by users.

[0772] A "server" is a computer system that provides services over a network, and is a device that receives, processes, and stores question data.

[0773] A "generative artificial intelligence engine" is a program that possesses artificial intelligence technology to analyze input question data and generate appropriate answers.

[0774] "Answer text" refers to the content of the answer to the user's question, generated by a generative artificial intelligence engine.

[0775] "Means of display" refers to a method or function for visually displaying the answer text on the device screen.

[0776] An "HTTP POST request" is a type of HTTP protocol used to send data from a user's device to a server.

[0777] A "database" refers to a system for systematically storing, managing, and retrieving information, and specifically includes relational database management systems (RDBMS).

[0778] An "API" is an interface that allows a specific program or service to communicate with other programs.

[0779] "Content for parent users" refers to content that provides information explaining how to use and the benefits of generative artificial intelligence engines.

[0780] This invention relates to a system that allows users to input questions or doubts via a terminal and quickly generates and provides appropriate answers to those questions. This system has a series of processes that analyze questions using a generative artificial intelligence engine and generate appropriate answers.

[0781] The specific hardware configuration of this system includes a terminal that receives user input, a server that receives, stores, transfers, and returns question data, and a generative artificial intelligence engine that analyzes the question data and generates answers. The software used includes a web browser, a server API for processing HTTP POST requests, a database management system, and the generative artificial intelligence engine.

[0782] Hardware and software

[0783] 1. Terminal

[0784] The user enters the question by opening a web browser on a device (smartphone, tablet, PC, etc.) and accessing a specific website. The device communicates with the server via an internet connection.

[0785] 2. Server

[0786] The server receives the input question data and forwards it to the generative artificial intelligence engine. The server temporarily stores the question data using a database (e.g., MySQL). Then, it sends the question data using the generative artificial intelligence engine's API and receives the generated answer.

[0787] 3. Generative Artificial Intelligence Engine

[0788] A generative artificial intelligence engine (for example, OpenAI's GPT-4) analyzes question data sent from a server and generates the optimal answer to that question. Based on the analysis results, the engine creates an answer in natural language and sends that answer back to the server.

[0789] Specific examples of data processing

[0790] For example, if a user (a child) enters the question "Why is the sky blue?" into the device, the process will proceed as follows:

[0791] 1. The user enters "Why is the sky blue?" into the input field on their device and clicks the send button.

[0792] 2. The terminal sends the entered question data to the server using an HTTP POST request.

[0793] 3. The server receives the question data and temporarily stores it in the database.

[0794] 4. The server sends the question data to the API of the generative artificial intelligence engine.

[0795] Prompt example: "Why is the sky blue?"

[0796] 5. A generative artificial intelligence engine analyzes the question and generates the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[0797] 6. The server sends the generated response text back to the terminal, and the terminal displays the response on its screen.

[0798] 7. The user (child) checks the answer displayed on the device screen.

[0799] 8. The device displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. For example, it might display content such as, "By using generative AI, you can instantly answer your child's questions and improve learning efficiency."

[0800] This system makes it possible to provide users with quick and appropriate answers to their questions, and also helps parents understand the convenience of generative artificial intelligence engines.

[0801] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0802] Step 1:

[0803] The user opens a web browser on their device. The user enters a question or doubt in text format into an input field. For example, they might enter the question, "Why is the sky blue?"

[0804] Input: Questions or inquiries entered by the user in text format.

[0805] Output: Question text displayed in the input field

[0806] Step 2:

[0807] When the user clicks the submit button, the device sends the entered question data to the server using an HTTP POST request.

[0808] Input: Data of the questions or inquiries that were entered.

[0809] Output: HTTP POST request sent to the server

[0810] Step 3:

[0811] The server receives the HTTP POST request and verifies the query data. The server temporarily stores the received query data in a database (e.g., MySQL).

[0812] Input: Question data sent in an HTTP POST request

[0813] Output: Question data temporarily stored in the database

[0814] Step 4:

[0815] The server retrieves the question data from the database and sends it to the API of the generative artificial intelligence engine.

[0816] Input: Question data stored in the database

[0817] Output: Question data sent to the API of a generative artificial intelligence engine.

[0818] Step 5:

[0819] A generative artificial intelligence engine receives the question data and analyzes it. Based on the analysis results, the generative artificial intelligence engine generates an answer in natural language.

[0820] Input: Question data sent to a generative artificial intelligence engine

[0821] Output: Generated answer text

[0822] Specific operation: For example, if the prompt is "Why is the sky blue?", the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight" will be generated.

[0823] Step 6:

[0824] The generative artificial intelligence engine sends the generated response back to the server. The server checks the received response text and sends the response back to the terminal using an HTTP response.

[0825] Input: Response text returned by the generative artificial intelligence engine

[0826] Output: The HTTP response sent to the terminal will include the response text.

[0827] Step 7:

[0828] The terminal receives an HTTP response from the server, parses the received response text, and formats it for display. The terminal displays the response on the screen, and the user confirms it.

[0829] Input: HTTP response sent from the server

[0830] Output: Answer text displayed on the screen

[0831] Specific action: For example, the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight," will be displayed on the device screen.

[0832] Step 8:

[0833] The device further displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. Parent users will understand the educational benefits that can be gained by utilizing generative artificial intelligence engines.

[0834] Input: Details on the advantages and usage of a specific generative artificial intelligence engine.

[0835] Output: Explanatory content displayed for the parent user

[0836] Specific actions: For example, information such as "By using generative AI, children's questions can be answered instantly, improving learning efficiency" will be displayed.

[0837] (Application Example 1)

[0838] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0839] In current virtual stores, it is difficult for users to get immediate and accurate answers to their questions. Furthermore, the lack of mechanisms for personalized suggestions based on customer purchase history makes it difficult to provide optimal product selection and detailed product information. Additionally, systems utilizing generative artificial intelligence engines are difficult for the average user to understand, and their advantages and usage are not clearly communicated, which is another challenge.

[0840] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0841] In this invention, the server includes means for a user to input questions or inquiries in text format via an information terminal; means for transmitting the input question data to an information processing device; means for the information processing device to transfer the question data to a generative artificial intelligence engine; means for the generative artificial intelligence engine to analyze the questions and generate appropriate answers; means for the information processing device to return the generated answer text to the information terminal; means for the information terminal to display the returned answers to the user; means for providing immediate answers to customer inquiries in a virtual space; and means for providing personalized suggestions based on the customer's purchase history. As a result, users can resolve their questions in real time and enjoy a more efficient and satisfying shopping experience by receiving personalized suggestions. Furthermore, by providing parents with information on the advantages and usage of the generative artificial intelligence engine, it is expected that its use within the home will be promoted.

[0842] A "user" refers to an individual who inputs questions or inquiries via an information terminal.

[0843] "Information terminal" is a general term for electronic devices used by users to input questions or inquiries, and includes smartphones, smart glasses, and head-mounted displays.

[0844] "Questions and doubts" refer to things that users want to know or things they are unsure about, expressed in text format.

[0845] An "information processing device" refers to a device that receives question data sent by a user and transfers it to a generative artificial intelligence engine.

[0846] A "generative artificial intelligence engine" refers to artificial intelligence that analyzes user questions and generates appropriate answers.

[0847] "Answer text" refers to the response generated by a generative artificial intelligence engine in response to a question, and is written in natural language.

[0848] "Virtual space" refers to a general term for three-dimensional spaces virtually constructed on a computer system, providing an environment where users can act as if they were actually present.

[0849] "Customer purchase history" refers to a record of products that a customer has purchased in the past, and this information is used to provide personalized recommendations.

[0850] "Personalized recommendations" refer to recommendations for products and services tailored to individual preferences, based on a customer's purchase history.

[0851] This invention is a system that provides immediate answers to user questions and offers personalized suggestions. Specific embodiments are described below.

[0852] First, the user inputs questions or inquiries via an information terminal. These terminals include, for example, smartphones, smart glasses, and head-mounted displays. These terminals operate within a virtual space and provide an interface that allows the user to input questions in text format.

[0853] The input question data is sent to the information processing device. The information processing device receives this question data and stores it temporarily. Next, the information processing device transfers the received question data to a generative artificial intelligence engine. This generative artificial intelligence engine uses GPT-4 or an equivalent AI model. The generative artificial intelligence engine analyzes the question data and generates an appropriate answer to it.

[0854] The generated response text is sent back to the information terminal by the information processing device. The information terminal displays the received response to the user. This allows the user to obtain answers to their questions in real time.

[0855] Furthermore, this includes means of providing instant answers to customer questions in a virtual space, as well as means of making personalized suggestions based on customer purchase history. Customer purchase history data is stored in an information processing device, and an AI engine analyzes it to suggest the most suitable products and services for each individual user.

[0856] The system's operation is illustrated with a concrete example. For instance, if a user enters "Please tell me the features of the new smartphone released at this store," the question data is sent to the information processing device and passed to a generative artificial intelligence engine. The AI ​​engine performs analysis and generates an answer such as "The new model has a longer battery life and improved camera performance," which is then sent back and displayed on the information terminal.

[0857] An example of a prompt would be, "I am a 40-year-old man. Please advise me on what kind of clothes would suit me." In response to this prompt, the generative artificial intelligence engine will generate an answer that includes advice on fashion styles suitable for a 40-year-old man and provide it to the user.

[0858] In this way, the present invention quickly resolves user questions and improves the purchasing experience. By providing parents with information on the advantages and usage of the generative artificial intelligence engine, it can also promote its use within the home.

[0859] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0860] Step 1:

[0861] The user uses an information terminal to input a question or inquiry in text format and presses the send button. The entered question is captured by the information terminal as text data. This input data is then formatted for subsequent processing. Specifically, the user inputs "Please tell me about the features of the new smartphone."

[0862] Step 2:

[0863] The entered question data is sent from the information terminal to the information processing device (server). Here, the information terminal sends the data using an HTTP POST request. The server receives this and temporarily stores the question data. This step verifies that the input data has been successfully delivered to the server.

[0864] Step 3:

[0865] The server forwards the received question data to the generative artificial intelligence engine. Specifically, the server sends the question data again to the AI ​​engine's API endpoint using an HTTP POST request. At this time, the data is converted to JSON format. After sending, the server waits for a response from the generative artificial intelligence engine.

[0866] Step 4:

[0867] The generative artificial intelligence engine analyzes the received question data and generates an appropriate answer. An AI model (e.g., GPT-4) is used in this process. Specifically, it receives the question text as an input prompt, performs natural language processing, and generates the most appropriate answer. For example, it might generate the answer, "The new model has a longer battery life and improved camera performance."

[0868] Step 5:

[0869] The generated response text is sent back from the generative artificial intelligence engine to the server. The server receives it and temporarily stores it again. The returned data is in JSON format, and after receiving this data, the server prepares to convert it to an appropriate format.

[0870] Step 6:

[0871] The server sends the generated response text back to the information terminal. It then uses an HTTP POST request again to send the data to the appropriate endpoint. At this point, the data is converted to a text format suitable for display. After sending, the server verifies the response.

[0872] Step 7:

[0873] The information terminal receives the returned response and displays it to the user. Specifically, a display area is created on the interface the user is using, and the response text is displayed there. For example, in response to the question, "What are the features of the new smartphone?", the answer displayed might be, "The new model has a longer battery life and improved camera performance."

[0874] Step 8:

[0875] The information terminal also displays personalized suggestions to the user. Based on past purchase history data, the server suggests the most suitable products and services for the user. This allows users to see recommended products based on their purchase history. For example, if a user has purchased many camera-related products in the past, new cameras and accessories will be suggested.

[0876] Example of a prompt:

[0877] "I'm a 40-year-old man. Please give me some advice on what kind of clothes would suit me."

[0878] This entire process allows users to enjoy a real-time and personalized shopping experience.

[0879] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0880] This invention describes how to specifically implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, by incorporating an emotion engine, the system includes means to recognize the user's emotions and provide more effective answers. The specific operation is described below.

[0881] Program processing

[0882] 1. User enters question

[0883] A user accesses a website using their device and enters a question or inquiry in text format. For example, they might enter the question, "Why is the sky blue?"

[0884] 2. Collection of emotional data

[0885] As the user enters a question, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[0886] 3. Submitting Question Data

[0887] The device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for transmission.

[0888] 4. Processing by the server

[0889] The server analyzes the received question data and sentiment data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[0890] 5. Transfer to a generative artificial intelligence engine

[0891] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[0892] 6. Generating the answer

[0893] A generative artificial intelligence engine analyzes question data and understands the question using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle response such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0894] 7. Return your response

[0895] The generative artificial intelligence engine sends the generated response text back to the server. The server then sends the received response data back to the terminal.

[0896] 8. Display on the device

[0897] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0898] 9. User Verification

[0899] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0900] 10. Providing content for parent users

[0901] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[0902] Specific example

[0903] For example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures the user's facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine. The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) confirms it. Additionally, content explaining the benefits and usage of the emotion engine is displayed for the parent user.

[0904] In this way, the system of the present invention not only solves children's questions but also utilizes an emotion engine to provide appropriate answers that correspond to the user's emotions, thereby promoting parents' understanding and use of generative artificial intelligence technology.

[0905] The following describes the processing flow.

[0906] Step 1:

[0907] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[0908] Step 2:

[0909] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[0910] Step 3:

[0911] When the user clicks the submit button, the device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for submission.

[0912] Step 4:

[0913] The server analyzes the received question data and sentiment data and temporarily stores them in a database. The server then converts the question data and sentiment data into a format suitable for a generative artificial intelligence engine.

[0914] Step 5:

[0915] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[0916] Step 6:

[0917] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate answer that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle-toned answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0918] Step 7:

[0919] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[0920] Step 8:

[0921] The server sends the received response text back to the terminal. The server embeds the response data into an HTML template and sends it to the terminal as an HTTP response.

[0922] Step 9:

[0923] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0924] Step 10:

[0925] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[0926] Step 11:

[0927] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[0928] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions, and utilizing the emotion engine to provide appropriate answers that respond to the user's emotions, thereby promoting the parent's understanding and use of generative artificial intelligence technology.

[0929] (Example 2)

[0930] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0931] Traditional question-answering systems often generate answers without considering the user's emotions, resulting in low levels of user understanding and satisfaction. Furthermore, the lack of content explaining the system's usefulness and usage for parental users made effective use of the system difficult.

[0932] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the terminal to collect the user's facial expressions and tone of voice in real time and generate emotion data, means for transmitting the collected emotion data to the server, and means for adjusting the generated response text based on the emotion data. This makes it possible to provide an appropriate response that corresponds to the user's emotions and to improve the usefulness of the system and user satisfaction.

[0933] "User" refers to an individual or their representative who uses the system.

[0934] A "terminal" refers to an electronic device used by a user to operate (for example, a smartphone, tablet, or personal computer).

[0935] "Questions and inquiries" refer to text-based input from users regarding information they want to know or problems they want to solve.

[0936] "Text format" refers to a form of writing that uses characters and symbols.

[0937] "Question data" refers to the text information of questions and doubts entered by users.

[0938] A "server" refers to a central computer that is connected to by multiple terminals via a network.

[0939] "Emotional data" refers to emotional information analyzed based on the user's facial expressions and tone of voice.

[0940] A "generative artificial intelligence engine" refers to artificial intelligence technology that analyzes user questions and generates appropriate answers.

[0941] "Answer text" refers to the content of the answers to user questions and inquiries generated by a generative artificial intelligence engine.

[0942] An "API request" refers to a request for data made through an application programming interface.

[0943] A "database" refers to a system for organizing and managing data.

[0944] "Natural language processing technology" refers to the technology used to process natural language used by humans using computers.

[0945] An "HTTP POST request" refers to a request method for sending data to a server using the HTTP protocol.

[0946] This invention relates to a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. In this system, the server transfers the question data to a generative artificial intelligence engine, which analyzes the input content, generates an appropriate answer, and sends it back to the user. Furthermore, by combining it with an emotion engine, the system also includes means to recognize the user's emotions and provide answers more effectively.

[0947] First, the user accesses the website using their device and enters a question or inquiry in text format. For example, they might type the question "Why is the sky blue?" into the text box and click the submit button. At this stage, the device retrieves the user's input.

[0948] Next, as the user enters a question, the camera and microphone on the device collect facial expressions and voice tone in real time. This emotion data is analyzed by an emotion engine to identify the user's emotions (joy, confusion, anger, etc.). This emotion data is sent to the server along with the question data using an HTTP POST request.

[0949] The server analyzes the received question data and sentiment data and temporarily stores this data in a database. The server converts the question data into a format suitable for a generative artificial intelligence engine, and the sentiment data is tagged and formatted appropriately. The server then transfers the converted question data and sentiment data to the generative artificial intelligence engine using an API request.

[0950] The generative artificial intelligence engine analyzes the question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, the engine might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[0951] The generated response text is adjusted based on sentiment data and sent back to the server. The server sends the received response text back to the device, which displays the response on the screen. The user can review the displayed response, resolve their questions, and gain new knowledge. Content explaining the usage and benefits of generative artificial intelligence engines is also provided for parent users.

[0952] To give a concrete example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. The device's camera captures the user's confused expression and collects it as emotion data. This data is sent to a server and transferred to a generative artificial intelligence engine. The generative AI engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) has their question answered. In addition, content explaining the benefits and usage of the emotion engine is displayed for parent users. This allows parents to understand the usefulness of the system and promotes its use at home.

[0953] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0954] Step 1: The user enters their question or inquiry in text format via their device. An input form on a web browser is used for input. The user enters a question such as "Why is the sky blue?" into the input form and clicks the submit button.

[0955] Step 2: The device acquires the entered text and simultaneously uses the camera and microphone to collect the user's facial expressions and voice tone. This data is analyzed by an emotion engine to identify the user's emotions. The input data is the question text, and the output data is the analyzed emotion data.

[0956] Step 3: The terminal packages the question text and sentiment data in JSON format and sends it to the server using an HTTP POST request. The input data is the question text and sentiment data, and the output is the data sent to the server.

[0957] Step 4: The server analyzes the received question text and sentiment data and temporarily stores them in the database. The input data is the received text and sentiment data, and the output is storage in the database. The server converts the question text into a format suitable for a generative artificial intelligence engine. The converted data is then output.

[0958] Step 5: The server transfers the converted question text and sentiment data to the generative artificial intelligence engine using a REST API. The input data is the converted text and sentiment data. The output is the transfer to the generative artificial intelligence engine.

[0959] Step 6: The generative artificial intelligence engine analyzes the question text and understands the question using natural language processing techniques. It then generates an appropriate response that corresponds to the user's emotions, taking sentiment data into consideration. For example, it might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input data consists of the question text and sentiment data, and the output is the generated response text.

[0960] Step 7: The generative AI engine sends the generated response text back to the server. This is also done using a REST API. The input is the generated response text, and the output is the text sent back to the server.

[0961] Step 8: The server sends the received answer text back to the terminal, and the terminal displays this answer on its screen. The input data is the answer text, and the output is the display on the device. For example, the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" will be displayed on the screen.

[0962] Step 9: The user confirms the displayed answer, and the question is resolved. The input is the answer displayed on the screen, and the output is the user's understanding.

[0963] Step 10: The device displays content for the parent user explaining how to use the generative artificial intelligence engine and its benefits. There is no input, and the output is explanatory content for the parent user. For example, it might display something like, "By using this system, you can answer your child's questions quickly and accurately."

[0964] (Application Example 2)

[0965] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0966] Traditional question-solving systems have resulted in low user satisfaction because they lack a system that can appropriately respond to users' emotions when they input questions. Furthermore, answers are simply displayed as text information, lacking features to deepen user understanding. Additionally, the lack of educational content for parents hinders the adoption of generative artificial intelligence engines in the home.

[0967] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0968] In this invention, the server includes means for collecting emotional data while the user inputs a question, means for transferring the collected emotional data to a generative artificial intelligence engine and reflecting it in the generated answer, and means for the user to confirm the displayed answer after the generated answer text is displayed on the terminal. This makes it possible to provide an appropriate answer that matches the user's emotions and deepen the user's understanding. Furthermore, by providing content that explains how to use the generative artificial intelligence engine and its advantages for parent users, its use in the home can be promoted.

[0969] definition statement

[0970] A "user" is someone who uses a system to input questions or inquiries and receive answers.

[0971] A "terminal" is an electronic device used by users to input questions or inquiries in text format and display answers.

[0972] "Question data" refers to information about questions and inquiries entered by users in text format.

[0973] A "server" is a central device that receives question data, transfers it to a generative artificial intelligence engine, and sends the generated answer back to the terminal.

[0974] A "generative artificial intelligence engine" is an artificial intelligence system that analyzes question data and generates appropriate answers.

[0975] "Answer text" refers to the information generated by a generative artificial intelligence engine in response to a user's question.

[0976] "Emotional data" refers to information about facial expressions and tone of voice collected while the user is entering questions.

[0977] An "emotion engine" is a device or software that analyzes emotional data to identify a user's emotional state.

[0978] A "parent user" is an adult user related to a child user, who understands and utilizes the features and benefits of a generative artificial intelligence engine.

[0979] "Educational content" refers to information that explains how to use and the benefits of generative artificial intelligence engines for parent users.

[0980] Modes for carrying out the invention

[0981] This invention is a system in which a user inputs questions or inquiries in text format via a terminal, a generative artificial intelligence engine generates appropriate answers based on that information, and further improves the answers using sentiment data. The server processes the question data and sentiment data and transfers them to the generative artificial intelligence engine. The answers corresponding to the user's questions are displayed on the terminal. In addition, content explaining how to use and the convenience of the generative artificial intelligence engine is provided for parent users.

[0982] Hardware and software to be used

[0983] 1. Hardware

[0984] Device: An electronic device such as a smartphone or tablet. It must have a camera and microphone.

[0985] Server: A network-connected computer system that receives, processes, and transfers data.

[0986] 2. Software

[0987] Flask: A Python-based web framework. It handles user requests and communicates with the server.

[0988] OpenAI API: Functions as a generative artificial intelligence engine. It analyzes question data and generates appropriate answers.

[0989] Emotion_recognition library: Analyzes emotional data. It evaluates the user's facial expressions and voice tone in real time.

[0990] System operation

[0991] 1. User enters question

[0992] The user uses their device to input questions or inquiries in text format. For example, they might input the question, "Why is the sky blue?"

[0993] 2. Collection of emotional data

[0994] While the user is entering a question, the emotion engine (Emotion_recognition library) uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time, and collects this as emotion data.

[0995] 3. Submitting Question Data

[0996] The entered question data and collected sentiment data are sent to the server via an HTTP POST request.

[0997] 4. Data processing by the server

[0998] The server analyzes the received question data and sentiment data and temporarily stores them in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API).

[0999] 5. Data transfer to the generative artificial intelligence engine

[1000] The server transfers the converted question data and sentiment data to the generative artificial intelligence engine and sends a request to generate an appropriate answer.

[1001] 6. Generating the answer

[1002] The generative artificial intelligence engine analyzes question data and sentiment data, and uses natural language processing techniques to generate appropriate answers that correspond to the user's emotions. For example, if the question is "Why is the sky blue?" and it is presumed that the user is confused, it will generate an answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1003] 7. Returning and displaying responses

[1004] The generated response text is sent back to the terminal via the server and displayed on the user's terminal.

[1005] Specific example

[1006] For example, a user types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures their facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine (OpenAI). The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user confirms it.

[1007] Example of a prompt

[1008] User question: Why is the sky blue?

[1009] User emotion:

[1010] AI response:

[1011] This system allows users to resolve their questions while simultaneously obtaining answers in a more understandable format using sentiment data. It also provides content explaining the convenience and applications of generative artificial intelligence engines for parental users.

[1012] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1013] Processing steps

[1014] Step 1:

[1015] The user enters questions or inquiries in text format via their device. The entered text is stored on the user's device. A concrete example is the user entering the question, "Why is the sky blue?"

[1016] Step 2:

[1017] The device's camera and microphone capture the user's facial expressions and voice tone in real time. An emotion engine (Emotion_recognition library) is used to analyze the emotional data and recognize the user's emotional state. This data is collected as emotion data. The input is the video and audio data captured by the device, and the output is the analyzed emotion data.

[1018] Step 3:

[1019] The terminal sends the entered question data and collected sentiment data to the server. The data is sent using an HTTP POST request. The input consists of the question data and sentiment data, and the output is a notification that the data transfer to the server is complete.

[1020] Step 4:

[1021] The server temporarily stores the received question data and sentiment data in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API). The input is the received question data and sentiment data, and the output is the converted data format.

[1022] Step 5:

[1023] The server sends the transformed question data and sentiment data to the generative artificial intelligence engine as an API request. The API request includes a prompt. The input is the transformed question data and sentiment data, and the output is the sending of a request to the generative artificial intelligence engine.

[1024] Step 6:

[1025] A generative artificial intelligence engine analyzes question data and sentiment data to generate appropriate answers. For example, it might generate the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight." The input is question data and sentiment data, and the output is the generated answer text.

[1026] Step 7:

[1027] The generated response text is sent back to the server. The server receives the response text and forwards it back to the terminal. The input is the generated response text, and the output is the transmission of the response to the terminal.

[1028] Step 8:

[1029] The device displays the answer text received from the server on the screen. The user checks the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input is the answer text, and the output is the displayed answer text.

[1030] Step 9:

[1031] The device further displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. This promotes the use of the engine at home. The input is educational content, and the output is the displayed educational content.

[1032] These steps provide users with appropriate responses that respond to their emotions, deepen their understanding, and allow parent users to learn how to utilize generative artificial intelligence engines.

[1033] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1034] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1035] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1036] [Fourth Embodiment]

[1037] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1038] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1039] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1040] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1041] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1042] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1043] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1044] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1045] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1046] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1047] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1048] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1049] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1050] This invention describes how to implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, the invention describes a mechanism for providing content to parent users explaining how to use the generative artificial intelligence engine and its benefits.

[1051] Program processing

[1052] 1. User enters question

[1053] Users access the website using their devices and enter their questions or inquiries in text format.

[1054] For example, a user (a child) might input the question, "Why is the sky blue?"

[1055] 2. Submitting Question Data

[1056] The entered question data is sent from the terminal to the server. The server receives this data and stores it temporarily.

[1057] 3. Transfer to a generative artificial intelligence engine

[1058] The server transfers the received question data to the generative artificial intelligence engine.

[1059] The generative artificial intelligence engine analyzes the question data and generates appropriate answers to it.

[1060] 4. Generating the answer

[1061] A generative artificial intelligence engine analyzes the question and generates an answer in natural language. For example, it might generate an answer like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[1062] 5. Return your response

[1063] The server sends the generated response text back to the terminal. The terminal displays the received response on its screen.

[1064] The user (child) checks the answer displayed on the screen, and their question is resolved.

[1065] 6. Providing content for parent users

[1066] The device also displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines.

[1067] For example, parents can obtain information such as, "By using generative AI, you can instantly answer your child's questions and improve their learning efficiency."

[1068] Specific example

[1069] As a concrete example, let's explain what happens when a child types "Why is the sky blue?" into a device.

[1070] 1. The user (child) enters "Why is the sky blue?" into the input field on the device and clicks the send button.

[1071] 2. The question data is sent to the server immediately.

[1072] 3. The server receives the question data and passes it to the generative artificial intelligence engine.

[1073] 4. The generative artificial intelligence engine analyzes the question and generates the answer, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[1074] 5. The server sends the generated response back to the terminal, and the terminal displays the response on its screen.

[1075] 6. We will also provide content for parent users that explains the convenience of generative artificial intelligence engines and how to use them at home.

[1076] In this way, the system of the present invention is designed to quickly resolve children's questions, and parents can learn about the use and benefits of generative artificial intelligence engines, and apply them to their daily lives.

[1077] The following describes the processing flow.

[1078] Step 1:

[1079] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[1080] Step 2:

[1081] When the user clicks the submit button, the device sends the entered question data to the server. An HTTP POST request is used for submission.

[1082] Step 3:

[1083] The server analyzes the received question data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[1084] Step 4:

[1085] The server transfers the transformed question data to the generative artificial intelligence engine. The question data is sent to the generative artificial intelligence engine using an API request.

[1086] Step 5:

[1087] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. It then generates an appropriate answer to the question.

[1088] Step 6:

[1089] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[1090] Step 7:

[1091] The server sends the received response text back to the user's device. The server embeds the response data into an HTML template and sends it to the device as an HTTP response.

[1092] Step 8:

[1093] The device receives a response from the server and displays the answer on the screen. Specifically, an answer such as, "The sky appears blue because tiny molecules in the Earth's atmosphere scatter sunlight," is displayed.

[1094] Step 9:

[1095] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[1096] Step 10:

[1097] The device also displays content explaining how to use and the convenience of the generative artificial intelligence engine for parent users. Parent users read the explanations and understand how to use it at home.

[1098] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions and promoting the understanding and application of generative artificial intelligence technology.

[1099] (Example 1)

[1100] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1101] Conventional question-answering systems suffer from an inefficient process, where users input questions in text format and then receive appropriate answers, resulting in slow response times. Furthermore, they lacked user-friendliness and ease of use when utilizing generative artificial intelligence engines. There is a need to address these issues and provide a more efficient and immediate question-answering system.

[1102] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1103] In this invention, the server includes means for sending input question data to the server using an HTTP POST request, means for the server to temporarily store the received question data in a database, and means for the server to send the question data to the API of a generative artificial intelligence engine. This makes it possible to quickly and efficiently generate an appropriate answer after a user inputs and submits a question, and to display that answer to the user. By providing explanatory content for parent users, it is also possible to deepen their understanding of how to use and the convenience of the generative artificial intelligence engine.

[1104] A "user" is an individual or group that uses a system or service.

[1105] A "terminal" refers to an electronic device used by a user to access a system, such as a smartphone, tablet, or personal computer.

[1106] "Questions and doubts" refer to information that users want to know, problems they want to solve, or things they want to confirm.

[1107] "Text format" refers to the representation format of data entered as a string of characters.

[1108] "Question data" refers to data that includes questions and doubts entered by users.

[1109] A "server" is a computer system that provides services over a network, and is a device that receives, processes, and stores question data.

[1110] A "generative artificial intelligence engine" is a program that possesses artificial intelligence technology to analyze input question data and generate appropriate answers.

[1111] "Answer text" refers to the content of the answer to the user's question, generated by a generative artificial intelligence engine.

[1112] "Means of display" refers to a method or function for visually displaying the answer text on the device screen.

[1113] An "HTTP POST request" is a type of HTTP protocol used to send data from a user's device to a server.

[1114] A "database" refers to a system for systematically storing, managing, and retrieving information, and specifically includes relational database management systems (RDBMS).

[1115] An "API" is an interface that allows a specific program or service to communicate with other programs.

[1116] "Content for parent users" refers to content that provides information explaining how to use and the benefits of generative artificial intelligence engines.

[1117] This invention relates to a system that allows users to input questions or doubts via a terminal and quickly generates and provides appropriate answers to those questions. This system has a series of processes that analyze questions using a generative artificial intelligence engine and generate appropriate answers.

[1118] The specific hardware configuration of this system includes a terminal that receives user input, a server that receives, stores, transfers, and returns question data, and a generative artificial intelligence engine that analyzes the question data and generates answers. The software used includes a web browser, a server API for processing HTTP POST requests, a database management system, and the generative artificial intelligence engine.

[1119] Hardware and software

[1120] 1. Terminal

[1121] The user enters the question by opening a web browser on a device (smartphone, tablet, PC, etc.) and accessing a specific website. The device communicates with the server via an internet connection.

[1122] 2. Server

[1123] The server receives the input question data and forwards it to the generative artificial intelligence engine. The server temporarily stores the question data using a database (e.g., MySQL). Then, it sends the question data using the generative artificial intelligence engine's API and receives the generated answer.

[1124] 3. Generative Artificial Intelligence Engine

[1125] A generative artificial intelligence engine (for example, OpenAI's GPT-4) analyzes question data sent from a server and generates the optimal answer to that question. Based on the analysis results, the engine creates an answer in natural language and sends that answer back to the server.

[1126] Specific examples of data processing

[1127] For example, if a user (a child) enters the question "Why is the sky blue?" into the device, the process will proceed as follows:

[1128] 1. The user enters "Why is the sky blue?" into the input field on their device and clicks the send button.

[1129] 2. The terminal sends the entered question data to the server using an HTTP POST request.

[1130] 3. The server receives the question data and temporarily stores it in the database.

[1131] 4. The server sends the question data to the API of the generative artificial intelligence engine.

[1132] Prompt example: "Why is the sky blue?"

[1133] 5. A generative artificial intelligence engine analyzes the question and generates the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight."

[1134] 6. The server sends the generated response text back to the terminal, and the terminal displays the response on its screen.

[1135] 7. The user (child) checks the answer displayed on the device screen.

[1136] 8. The device displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. For example, it might display content such as, "By using generative AI, you can instantly answer your child's questions and improve learning efficiency."

[1137] This system makes it possible to provide users with quick and appropriate answers to their questions, and also helps parents understand the convenience of generative artificial intelligence engines.

[1138] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1139] Step 1:

[1140] The user opens a web browser on their device. The user enters a question or doubt in text format into an input field. For example, they might enter the question, "Why is the sky blue?"

[1141] Input: Questions or inquiries entered by the user in text format.

[1142] Output: Question text displayed in the input field

[1143] Step 2:

[1144] When the user clicks the submit button, the device sends the entered question data to the server using an HTTP POST request.

[1145] Input: Data of the questions or inquiries that were entered.

[1146] Output: HTTP POST request sent to the server

[1147] Step 3:

[1148] The server receives the HTTP POST request and verifies the query data. The server temporarily stores the received query data in a database (e.g., MySQL).

[1149] Input: Question data sent in an HTTP POST request

[1150] Output: Question data temporarily stored in the database

[1151] Step 4:

[1152] The server retrieves the question data from the database and sends it to the API of the generative artificial intelligence engine.

[1153] Input: Question data stored in the database

[1154] Output: Question data sent to the API of a generative artificial intelligence engine.

[1155] Step 5:

[1156] A generative artificial intelligence engine receives the question data and analyzes it. Based on the analysis results, the generative artificial intelligence engine generates an answer in natural language.

[1157] Input: Question data sent to a generative artificial intelligence engine

[1158] Output: Generated answer text

[1159] Specific operation: For example, if the prompt is "Why is the sky blue?", the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight" will be generated.

[1160] Step 6:

[1161] The generative artificial intelligence engine sends the generated response back to the server. The server checks the received response text and sends the response back to the terminal using an HTTP response.

[1162] Input: Response text returned by the generative artificial intelligence engine

[1163] Output: The HTTP response sent to the terminal will include the response text.

[1164] Step 7:

[1165] The terminal receives an HTTP response from the server, parses the received response text, and formats it for display. The terminal displays the response on the screen, and the user confirms it.

[1166] Input: HTTP response sent from the server

[1167] Output: Answer text displayed on the screen

[1168] Specific action: For example, the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight," will be displayed on the device screen.

[1169] Step 8:

[1170] The device further displays content for parent users explaining how to use and the benefits of generative artificial intelligence engines. Parent users will understand the educational benefits that can be gained by utilizing generative artificial intelligence engines.

[1171] Input: Details on the advantages and usage of a specific generative artificial intelligence engine.

[1172] Output: Explanatory content displayed for the parent user

[1173] Specific actions: For example, information such as "By using generative AI, children's questions can be answered instantly, improving learning efficiency" will be displayed.

[1174] (Application Example 1)

[1175] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1176] In current virtual stores, it is difficult for users to get immediate and accurate answers to their questions. Furthermore, the lack of mechanisms for personalized suggestions based on customer purchase history makes it difficult to provide optimal product selection and detailed product information. Additionally, systems utilizing generative artificial intelligence engines are difficult for the average user to understand, and their advantages and usage are not clearly communicated, which is another challenge.

[1177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1178] In this invention, the server includes means for a user to input questions or inquiries in text format via an information terminal; means for transmitting the input question data to an information processing device; means for the information processing device to transfer the question data to a generative artificial intelligence engine; means for the generative artificial intelligence engine to analyze the questions and generate appropriate answers; means for the information processing device to return the generated answer text to the information terminal; means for the information terminal to display the returned answers to the user; means for providing immediate answers to customer inquiries in a virtual space; and means for providing personalized suggestions based on the customer's purchase history. As a result, users can resolve their questions in real time and enjoy a more efficient and satisfying shopping experience by receiving personalized suggestions. Furthermore, by providing parents with information on the advantages and usage of the generative artificial intelligence engine, it is expected that its use within the home will be promoted.

[1179] A "user" refers to an individual who inputs questions or inquiries via an information terminal.

[1180] "Information terminal" is a general term for electronic devices used by users to input questions or inquiries, and includes smartphones, smart glasses, and head-mounted displays.

[1181] "Questions and doubts" refer to things that users want to know or things they are unsure about, expressed in text format.

[1182] An "information processing device" refers to a device that receives question data sent by a user and transfers it to a generative artificial intelligence engine.

[1183] A "generative artificial intelligence engine" refers to artificial intelligence that analyzes user questions and generates appropriate answers.

[1184] "Answer text" refers to the response generated by a generative artificial intelligence engine in response to a question, and is written in natural language.

[1185] "Virtual space" refers to a general term for three-dimensional spaces virtually constructed on a computer system, providing an environment where users can act as if they were actually present.

[1186] "Customer purchase history" refers to a record of products that a customer has purchased in the past, and this information is used to provide personalized recommendations.

[1187] "Personalized recommendations" refer to recommendations for products and services tailored to individual preferences, based on a customer's purchase history.

[1188] This invention is a system that provides immediate answers to user questions and offers personalized suggestions. Specific embodiments are described below.

[1189] First, the user inputs questions or inquiries via an information terminal. These terminals include, for example, smartphones, smart glasses, and head-mounted displays. These terminals operate within a virtual space and provide an interface that allows the user to input questions in text format.

[1190] The input question data is sent to the information processing device. The information processing device receives this question data and stores it temporarily. Next, the information processing device transfers the received question data to a generative artificial intelligence engine. This generative artificial intelligence engine uses GPT-4 or an equivalent AI model. The generative artificial intelligence engine analyzes the question data and generates an appropriate answer to it.

[1191] The generated response text is sent back to the information terminal by the information processing device. The information terminal displays the received response to the user. This allows the user to obtain answers to their questions in real time.

[1192] Furthermore, this includes means of providing instant answers to customer questions in a virtual space, as well as means of making personalized suggestions based on customer purchase history. Customer purchase history data is stored in an information processing device, and an AI engine analyzes it to suggest the most suitable products and services for each individual user.

[1193] The system's operation is illustrated with a concrete example. For instance, if a user enters "Please tell me the features of the new smartphone released at this store," the question data is sent to the information processing device and passed to a generative artificial intelligence engine. The AI ​​engine performs analysis and generates an answer such as "The new model has a longer battery life and improved camera performance," which is then sent back and displayed on the information terminal.

[1194] An example of a prompt would be, "I am a 40-year-old man. Please advise me on what kind of clothes would suit me." In response to this prompt, the generative artificial intelligence engine will generate an answer that includes advice on fashion styles suitable for a 40-year-old man and provide it to the user.

[1195] In this way, the present invention quickly resolves user questions and improves the purchasing experience. By providing parents with information on the advantages and usage of the generative artificial intelligence engine, it can also promote its use within the home.

[1196] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1197] Step 1:

[1198] The user uses an information terminal to input a question or inquiry in text format and presses the send button. The entered question is captured by the information terminal as text data. This input data is then formatted for subsequent processing. Specifically, the user inputs "Please tell me about the features of the new smartphone."

[1199] Step 2:

[1200] The entered question data is sent from the information terminal to the information processing device (server). Here, the information terminal sends the data using an HTTP POST request. The server receives this and temporarily stores the question data. This step verifies that the input data has been successfully delivered to the server.

[1201] Step 3:

[1202] The server forwards the received question data to the generative artificial intelligence engine. Specifically, the server sends the question data again to the AI ​​engine's API endpoint using an HTTP POST request. At this time, the data is converted to JSON format. After sending, the server waits for a response from the generative artificial intelligence engine.

[1203] Step 4:

[1204] The generative artificial intelligence engine analyzes the received question data and generates an appropriate answer. An AI model (e.g., GPT-4) is used in this process. Specifically, it receives the question text as an input prompt, performs natural language processing, and generates the most appropriate answer. For example, it might generate the answer, "The new model has a longer battery life and improved camera performance."

[1205] Step 5:

[1206] The generated response text is sent back from the generative artificial intelligence engine to the server. The server receives it and temporarily stores it again. The returned data is in JSON format, and after receiving this data, the server prepares to convert it to an appropriate format.

[1207] Step 6:

[1208] The server sends the generated response text back to the information terminal. It then uses an HTTP POST request again to send the data to the appropriate endpoint. At this point, the data is converted to a text format suitable for display. After sending, the server verifies the response.

[1209] Step 7:

[1210] The information terminal receives the returned response and displays it to the user. Specifically, a display area is created on the interface the user is using, and the response text is displayed there. For example, in response to the question, "What are the features of the new smartphone?", the answer displayed might be, "The new model has a longer battery life and improved camera performance."

[1211] Step 8:

[1212] The information terminal also displays personalized suggestions to the user. Based on past purchase history data, the server suggests the most suitable products and services for the user. This allows users to see recommended products based on their purchase history. For example, if a user has purchased many camera-related products in the past, new cameras and accessories will be suggested.

[1213] Example of a prompt:

[1214] "I'm a 40-year-old man. Please give me some advice on what kind of clothes would suit me."

[1215] This entire process allows users to enjoy a real-time and personalized shopping experience.

[1216] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1217] This invention describes how to specifically implement a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. This system involves the server transferring the question data to a generative artificial intelligence engine, which analyzes the questions, generates appropriate answers, and returns them to the user. Furthermore, by incorporating an emotion engine, the system includes means to recognize the user's emotions and provide more effective answers. The specific operation is described below.

[1218] Program processing

[1219] 1. User enters question

[1220] A user accesses a website using their device and enters a question or inquiry in text format. For example, they might enter the question, "Why is the sky blue?"

[1221] 2. Collection of emotional data

[1222] As the user enters a question, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[1223] 3. Submitting Question Data

[1224] The device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for transmission.

[1225] 4. Processing by the server

[1226] The server analyzes the received question data and sentiment data and temporarily stores it in a database. The server then converts the question data into a format suitable for a generative artificial intelligence engine.

[1227] 5. Transfer to a generative artificial intelligence engine

[1228] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[1229] 6. Generating the answer

[1230] A generative artificial intelligence engine analyzes question data and understands the question using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle response such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1231] 7. Return your response

[1232] The generative artificial intelligence engine sends the generated response text back to the server. The server then sends the received response data back to the terminal.

[1233] 8. Display on the device

[1234] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1235] 9. User Verification

[1236] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[1237] 10. Providing content for parent users

[1238] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[1239] Specific example

[1240] For example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures the user's facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine. The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) confirms it. Additionally, content explaining the benefits and usage of the emotion engine is displayed for the parent user.

[1241] In this way, the system of the present invention not only solves children's questions but also utilizes an emotion engine to provide appropriate answers that correspond to the user's emotions, thereby promoting parents' understanding and use of generative artificial intelligence technology.

[1242] The following describes the processing flow.

[1243] Step 1:

[1244] The user (child) uses a device to access a website and enters questions or doubts in text format. For example, they might type "Why is the sky blue?" into the text input field.

[1245] Step 2:

[1246] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice tone in real time. It captures the user's facial expressions and voice tone while they are typing and collects this data as emotion data.

[1247] Step 3:

[1248] When the user clicks the submit button, the device sends the entered question data and collected sentiment data to the server. An HTTP POST request is used for submission.

[1249] Step 4:

[1250] The server analyzes the received question data and sentiment data and temporarily stores them in a database. The server then converts the question data and sentiment data into a format suitable for a generative artificial intelligence engine.

[1251] Step 5:

[1252] The server transfers the transformed question data and sentiment data to the generative artificial intelligence engine. This data is sent using API requests.

[1253] Step 6:

[1254] The generative artificial intelligence engine analyzes the received question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate answer that corresponds to the user's emotions. For example, if the question is "Why is the sky blue?" and the user is expressing confusion, it will generate a gentle-toned answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1255] Step 7:

[1256] The generative artificial intelligence engine sends the generated response text back to the server. The response data is sent to the server as an API response.

[1257] Step 8:

[1258] The server sends the received response text back to the terminal. The server embeds the response data into an HTML template and sends it to the terminal as an HTTP response.

[1259] Step 9:

[1260] The device receives a response from the server and displays the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1261] Step 10:

[1262] The user (child) checks the displayed answer. The question is resolved, and learning progresses.

[1263] Step 11:

[1264] The device also displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. Parent users read the explanations and understand how to use it at home.

[1265] In this way, processing progresses step by step, supporting the resolution of user (parent / child) questions, and utilizing the emotion engine to provide appropriate answers that respond to the user's emotions, thereby promoting the parent's understanding and use of generative artificial intelligence technology.

[1266] (Example 2)

[1267] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1268] Traditional question-answering systems often generate answers without considering the user's emotions, resulting in low levels of user understanding and satisfaction. Furthermore, the lack of content explaining the system's usefulness and usage for parental users made effective use of the system difficult.

[1269] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the terminal to collect the user's facial expressions and tone of voice in real time and generate emotion data, means for transmitting the collected emotion data to the server, and means for adjusting the generated response text based on the emotion data. This makes it possible to provide an appropriate response that corresponds to the user's emotions and to improve the usefulness of the system and user satisfaction.

[1270] "User" refers to an individual or their representative who uses the system.

[1271] A "terminal" refers to an electronic device used by a user to operate (for example, a smartphone, tablet, or personal computer).

[1272] "Questions and inquiries" refer to text-based input from users regarding information they want to know or problems they want to solve.

[1273] "Text format" refers to a form of writing that uses characters and symbols.

[1274] "Question data" refers to the text information of questions and doubts entered by users.

[1275] A "server" refers to a central computer that is connected to by multiple terminals via a network.

[1276] "Emotional data" refers to emotional information analyzed based on the user's facial expressions and tone of voice.

[1277] A "generative artificial intelligence engine" refers to artificial intelligence technology that analyzes user questions and generates appropriate answers.

[1278] "Answer text" refers to the content of the answers to user questions and inquiries generated by a generative artificial intelligence engine.

[1279] An "API request" refers to a request for data made through an application programming interface.

[1280] A "database" refers to a system for organizing and managing data.

[1281] "Natural language processing technology" refers to the technology used to process natural language used by humans using computers.

[1282] An "HTTP POST request" refers to a request method for sending data to a server using the HTTP protocol.

[1283] This invention relates to a system in which a user inputs questions or doubts in text format via a terminal and sends the input question data to a server. In this system, the server transfers the question data to a generative artificial intelligence engine, which analyzes the input content, generates an appropriate answer, and sends it back to the user. Furthermore, by combining it with an emotion engine, the system also includes means to recognize the user's emotions and provide answers more effectively.

[1284] First, the user accesses the website using their device and enters a question or inquiry in text format. For example, they might type the question "Why is the sky blue?" into the text box and click the submit button. At this stage, the device retrieves the user's input.

[1285] Next, as the user enters a question, the camera and microphone on the device collect facial expressions and voice tone in real time. This emotion data is analyzed by an emotion engine to identify the user's emotions (joy, confusion, anger, etc.). This emotion data is sent to the server along with the question data using an HTTP POST request.

[1286] The server analyzes the received question data and sentiment data and temporarily stores this data in a database. The server converts the question data into a format suitable for a generative artificial intelligence engine, and the sentiment data is tagged and formatted appropriately. The server then transfers the converted question data and sentiment data to the generative artificial intelligence engine using an API request.

[1287] The generative artificial intelligence engine analyzes the question data and understands the question content using natural language processing techniques. Furthermore, it considers sentiment data and generates an appropriate response that corresponds to the user's emotions. For example, the engine might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1288] The generated response text is adjusted based on sentiment data and sent back to the server. The server sends the received response text back to the device, which displays the response on the screen. The user can review the displayed response, resolve their questions, and gain new knowledge. Content explaining the usage and benefits of generative artificial intelligence engines is also provided for parent users.

[1289] To give a concrete example, a user (child) types "Why is the sky blue?" into the device and clicks the send button. The device's camera captures the user's confused expression and collects it as emotion data. This data is sent to a server and transferred to a generative artificial intelligence engine. The generative AI engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user (child) has their question answered. In addition, content explaining the benefits and usage of the emotion engine is displayed for parent users. This allows parents to understand the usefulness of the system and promotes its use at home.

[1290] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1291] Step 1: The user enters their question or inquiry in text format via their device. An input form on a web browser is used for input. The user enters a question such as "Why is the sky blue?" into the input form and clicks the submit button.

[1292] Step 2: The device acquires the entered text and simultaneously uses the camera and microphone to collect the user's facial expressions and voice tone. This data is analyzed by an emotion engine to identify the user's emotions. The input data is the question text, and the output data is the analyzed emotion data.

[1293] Step 3: The terminal packages the question text and sentiment data in JSON format and sends it to the server using an HTTP POST request. The input data is the question text and sentiment data, and the output is the data sent to the server.

[1294] Step 4: The server analyzes the received question text and sentiment data and temporarily stores them in the database. The input data is the received text and sentiment data, and the output is storage in the database. The server converts the question text into a format suitable for a generative artificial intelligence engine. The converted data is then output.

[1295] Step 5: The server transfers the converted question text and sentiment data to the generative artificial intelligence engine using a REST API. The input data is the converted text and sentiment data. The output is the transfer to the generative artificial intelligence engine.

[1296] Step 6: The generative artificial intelligence engine analyzes the question text and understands the question using natural language processing techniques. It then generates an appropriate response that corresponds to the user's emotions, taking sentiment data into consideration. For example, it might generate a response like, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input data consists of the question text and sentiment data, and the output is the generated response text.

[1297] Step 7: The generative AI engine sends the generated response text back to the server. This is also done using a REST API. The input is the generated response text, and the output is the text sent back to the server.

[1298] Step 8: The server sends the received answer text back to the terminal, and the terminal displays this answer on its screen. The input data is the answer text, and the output is the display on the device. For example, the answer "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" will be displayed on the screen.

[1299] Step 9: The user confirms the displayed answer, and the question is resolved. The input is the answer displayed on the screen, and the output is the user's understanding.

[1300] Step 10: The device displays content for the parent user explaining how to use the generative artificial intelligence engine and its benefits. There is no input, and the output is explanatory content for the parent user. For example, it might display something like, "By using this system, you can answer your child's questions quickly and accurately."

[1301] (Application Example 2)

[1302] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1303] Traditional question-solving systems have resulted in low user satisfaction because they lack a system that can appropriately respond to users' emotions when they input questions. Furthermore, answers are simply displayed as text information, lacking features to deepen user understanding. Additionally, the lack of educational content for parents hinders the adoption of generative artificial intelligence engines in the home.

[1304] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1305] In this invention, the server includes means for collecting emotional data while the user inputs a question, means for transferring the collected emotional data to a generative artificial intelligence engine and reflecting it in the generated answer, and means for the user to confirm the displayed answer after the generated answer text is displayed on the terminal. This makes it possible to provide an appropriate answer that matches the user's emotions and deepen the user's understanding. Furthermore, by providing content that explains how to use the generative artificial intelligence engine and its advantages for parent users, its use in the home can be promoted.

[1306] definition statement

[1307] A "user" is someone who uses a system to input questions or inquiries and receive answers.

[1308] A "terminal" is an electronic device used by users to input questions or inquiries in text format and display answers.

[1309] "Question data" refers to information about questions and inquiries entered by users in text format.

[1310] A "server" is a central device that receives question data, transfers it to a generative artificial intelligence engine, and sends the generated answer back to the terminal.

[1311] A "generative artificial intelligence engine" is an artificial intelligence system that analyzes question data and generates appropriate answers.

[1312] "Answer text" refers to the information generated by a generative artificial intelligence engine in response to a user's question.

[1313] "Emotional data" refers to information about facial expressions and tone of voice collected while the user is entering questions.

[1314] An "emotion engine" is a device or software that analyzes emotional data to identify a user's emotional state.

[1315] A "parent user" is an adult user related to a child user, who understands and utilizes the features and benefits of a generative artificial intelligence engine.

[1316] "Educational content" refers to information that explains how to use and the benefits of generative artificial intelligence engines for parent users.

[1317] Modes for carrying out the invention

[1318] This invention is a system in which a user inputs questions or inquiries in text format via a terminal, a generative artificial intelligence engine generates appropriate answers based on that information, and further improves the answers using sentiment data. The server processes the question data and sentiment data and transfers them to the generative artificial intelligence engine. The answers corresponding to the user's questions are displayed on the terminal. In addition, content explaining how to use and the convenience of the generative artificial intelligence engine is provided for parent users.

[1319] Hardware and software to be used

[1320] 1. Hardware

[1321] Device: An electronic device such as a smartphone or tablet. It must have a camera and microphone.

[1322] Server: A network-connected computer system that receives, processes, and transfers data.

[1323] 2. Software

[1324] Flask: A Python-based web framework. It handles user requests and communicates with the server.

[1325] OpenAI API: Functions as a generative artificial intelligence engine. It analyzes question data and generates appropriate answers.

[1326] Emotion_recognition library: Analyzes emotional data. It evaluates the user's facial expressions and voice tone in real time.

[1327] System operation

[1328] 1. User enters question

[1329] The user uses their device to input questions or inquiries in text format. For example, they might input the question, "Why is the sky blue?"

[1330] 2. Collection of emotional data

[1331] While the user is entering a question, the emotion engine (Emotion_recognition library) uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time, and collects this as emotion data.

[1332] 3. Submitting Question Data

[1333] The entered question data and collected sentiment data are sent to the server via an HTTP POST request.

[1334] 4. Data processing by the server

[1335] The server analyzes the received question data and sentiment data and temporarily stores them in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API).

[1336] 5. Data transfer to the generative artificial intelligence engine

[1337] The server transfers the converted question data and sentiment data to the generative artificial intelligence engine and sends a request to generate an appropriate answer.

[1338] 6. Generating the answer

[1339] The generative artificial intelligence engine analyzes question data and sentiment data, and uses natural language processing techniques to generate appropriate answers that correspond to the user's emotions. For example, if the question is "Why is the sky blue?" and it is presumed that the user is confused, it will generate an answer such as, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?"

[1340] 7. Returning and displaying responses

[1341] The generated response text is sent back to the terminal via the server and displayed on the user's terminal.

[1342] Specific example

[1343] For example, a user types "Why is the sky blue?" into the device and clicks the send button. As the user types, the device's camera captures their facial expression, and any confused expressions are collected as emotion data. The question data and emotion data are sent to a server, which then forwards them to a generative artificial intelligence engine (OpenAI). The engine generates an answer such as "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" and sends it back to the device via the server. The device displays the answer on the screen, and the user confirms it.

[1344] Example of a prompt

[1345] User question: Why is the sky blue?

[1346] User emotion:

[1347] AI response:

[1348] This system allows users to resolve their questions while simultaneously obtaining answers in a more understandable format using sentiment data. It also provides content explaining the convenience and applications of generative artificial intelligence engines for parental users.

[1349] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1350] Processing steps

[1351] Step 1:

[1352] The user enters questions or inquiries in text format via their device. The entered text is stored on the user's device. A concrete example is the user entering the question, "Why is the sky blue?"

[1353] Step 2:

[1354] The device's camera and microphone capture the user's facial expressions and voice tone in real time. An emotion engine (Emotion_recognition library) is used to analyze the emotional data and recognize the user's emotional state. This data is collected as emotion data. The input is the video and audio data captured by the device, and the output is the analyzed emotion data.

[1355] Step 3:

[1356] The terminal sends the entered question data and collected sentiment data to the server. The data is sent using an HTTP POST request. The input consists of the question data and sentiment data, and the output is a notification that the data transfer to the server is complete.

[1357] Step 4:

[1358] The server temporarily stores the received question data and sentiment data in a database. Furthermore, it converts the question data into a format suitable for a generative artificial intelligence engine (OpenAI API). The input is the received question data and sentiment data, and the output is the converted data format.

[1359] Step 5:

[1360] The server sends the transformed question data and sentiment data to the generative artificial intelligence engine as an API request. The API request includes a prompt. The input is the transformed question data and sentiment data, and the output is the sending of a request to the generative artificial intelligence engine.

[1361] Step 6:

[1362] A generative artificial intelligence engine analyzes question data and sentiment data to generate appropriate answers. For example, it might generate the answer, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight." The input is question data and sentiment data, and the output is the generated answer text.

[1363] Step 7:

[1364] The generated response text is sent back to the server. The server receives the response text and forwards it back to the terminal. The input is the generated response text, and the output is the transmission of the response to the terminal.

[1365] Step 8:

[1366] The device displays the answer text received from the server on the screen. The user checks the answer on the screen. For example, the answer displayed might be, "The sky is blue because tiny molecules in the Earth's atmosphere scatter sunlight. Was that easy to understand?" The input is the answer text, and the output is the displayed answer text.

[1367] Step 9:

[1368] The device further displays content for parent users explaining how to use and utilize the generative artificial intelligence engine. This promotes the use of the engine at home. The input is educational content, and the output is the displayed educational content.

[1369] These steps provide users with appropriate responses that respond to their emotions, deepen their understanding, and allow parent users to learn how to utilize generative artificial intelligence engines.

[1370] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1371] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1372] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1373] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1374] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1375] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1376] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1377] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1378] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1379] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1380] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1381] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1382] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1383] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1384] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1385] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1386] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1387] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1388] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1389] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1390] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1391] The following is further disclosed regarding the embodiments described above.

[1392] (Claim 1)

[1393] A means for users to input questions or inquiries in text format via their device,

[1394] A means for sending the entered question data to the server,

[1395] A means for the server to transfer question data to a generative artificial intelligence engine,

[1396] A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer,

[1397] A means by which the server sends the generated response text back to the terminal,

[1398] A means of displaying the returned response to the user,

[1399] A system that includes this.

[1400] (Claim 2)

[1401] The system according to claim 1, further comprising means for the user to confirm the displayed answer after the generated answer text has been displayed on the terminal.

[1402] (Claim 3)

[1403] The system according to claim 1, further comprising means for providing content to parent users explaining how to use and the benefits of a generative artificial intelligence engine.

[1404] "Example 1"

[1405] (Claim 1)

[1406] A means for users to input questions or inquiries in text format via their device,

[1407] A means for sending the entered question data to the server,

[1408] A means for the server to transfer question data to a generative artificial intelligence engine,

[1409] A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer,

[1410] A means by which the server sends the generated response text back to the terminal,

[1411] A means of displaying the returned response to the user,

[1412] A system that includes this.

[1413] (Claim 2)

[1414] The system according to claim 1, further comprising means for the user to confirm the displayed answer after the generated answer text has been displayed on the terminal.

[1415] (Claim 3)

[1416] The system according to claim 1, further comprising means for sending question data entered by a terminal to a server using an HTTP POST request.

[1417] (Claim 4)

[1418] The system according to claim 1, further comprising means for temporarily storing the received question data in a database.

[1419] (Claim 5)

[1420] The system according to claim 1, further comprising means for a server to send question data to an API of a generative artificial intelligence engine.

[1421] (Claim 6)

[1422] The system according to claim 1, further comprising means for providing content to parent users explaining how to use and the benefits of a generative artificial intelligence engine.

[1423] "Application Example 1"

[1424] (Claim 1)

[1425] A means for users to input questions or inquiries in text format via an information terminal,

[1426] Means for transmitting input question data to an information processing device,

[1427] A means by which an information processing device transfers question data to a generative artificial intelligence engine,

[1428] A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer,

[1429] A means by which the information processing device sends the generated response text back to the information terminal,

[1430] A means of displaying the returned response to the user via an information terminal,

[1431] A means of providing immediate answers to customer questions in a virtual space,

[1432] A means of providing personalized suggestions based on the customer's purchase history,

[1433] A system that includes this.

[1434] (Claim 2)

[1435] The system according to claim 1, further comprising means for the user to confirm the displayed answer after the generated answer text has been displayed on an information terminal.

[1436] (Claim 3)

[1437] The system according to claim 1, further comprising means for providing content to parent users explaining how to use and the benefits of a generative artificial intelligence engine.

[1438] "Example 2 of combining an emotion engine"

[1439] (Claim 1)

[1440] A means for users to input questions or inquiries in text format via their device,

[1441] A means for sending the entered question data to the server,

[1442] A means by which a device collects the user's facial expressions and voice tone in real time and generates emotional data,

[1443] A means of sending the collected emotional data to a server,

[1444] A means for the server to transfer question data and sentiment data to a generative artificial intelligence engine,

[1445] A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer,

[1446] A means by which the generated response text is adjusted based on sentiment data,

[1447] A means by which the server sends the generated response text back to the terminal,

[1448] A means of displaying the returned response to the user,

[1449] A system that includes this.

[1450] (Claim 2)

[1451] The system according to claim 1, further comprising means for the user to confirm the displayed answer after the generated answer text has been displayed on the terminal.

[1452] (Claim 3)

[1453] The system according to claim 1, further comprising means for providing content to parent users explaining how to use and the benefits of a generative artificial intelligence engine.

[1454] "Application example 2 when combining with an emotional engine"

[1455] (Claim 1)

[1456] A means for users to input questions or inquiries in text format via their device,

[1457] A means for sending the entered question data to the server,

[1458] A means for the server to transfer question data to a generative artificial intelligence engine,

[1459] A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer,

[1460] A means by which the server sends the generated response text back to the terminal,

[1461] A means of displaying the returned response to the user,

[1462] A means of collecting sentiment data while the user is entering a question,

[1463] A means of transferring collected emotional data to a generative artificial intelligence engine and reflecting it in the generated response,

[1464] A system that includes this.

[1465] (Claim 2)

[1466] The system according to claim 1, further comprising means for the user to confirm the displayed answer after the generated answer text has been displayed on the terminal.

[1467] (Claim 3)

[1468] The system according to claim 1, further comprising means for analyzing emotional data collected by a terminal to identify the user's emotional state.

[1469] (Claim 4)

[1470] The system according to claim 1, further comprising means for providing content to parent users explaining how to use and the benefits of a generative artificial intelligence engine. [Explanation of Symbols]

[1471] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to input questions or inquiries in text format via their device, A means for sending the entered question data to the server, A means for the server to transfer question data to a generative artificial intelligence engine, A means by which a generative artificial intelligence engine analyzes a question and generates an appropriate answer, A means by which the server sends the generated response text back to the terminal, A means of displaying the returned response to the user, A system that includes this.

2. The system according to claim 1, further comprising means for the user to confirm the displayed answer after the generated answer text has been displayed on the terminal.

3. The system according to claim 1, further comprising means for providing content to parent users explaining how to use and the advantages of a generative artificial intelligence engine.

Images