System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a query input by a user; means for optimizing the received query; and means for transmitting the optimized query to a generative artificial intelligence.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Creating appropriate prompts is essential for the effective use of generative artificial intelligence (generative AI), but there is a problem in that it is difficult for all users to perform this task accurately. As a result, the accuracy of the information obtained from generative AI decreases, and user satisfaction decreases. In addition, since users must take the time to optimize the prompts, using generative AI becomes cumbersome. This is a factor that hinders the widespread use of generative AI. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, a system is provided that receives a query when the user inputs it. Next, a means is provided for optimizing the received query. In this optimization process, formatting and additional information are added to enable the generation AI to generate a highly accurate answer. By providing a system with a means for sending this optimized query to the generation AI, the user can obtain highly accurate information without any effort. Furthermore, in certain embodiments, a browser extension can be used to temporarily prevent the user from inputting a query, allowing the optimization process to be carried out seamlessly. In this way, the present invention makes it possible to use the generation AI more effectively and easily.

[0006] A "user" is a person who uses the system to input queries into the generative artificial intelligence and receive the results.

[0007] A "query" is text that a user enters into the generated AI to ask the user the information they want to obtain.

[0008] "Generative artificial intelligence" (generative AI) refers to computer-based systems that can generate text data in a human-like manner.

[0009] A "prompt" is a question or instruction text that is input to the generative artificial intelligence.

[0010] "Optimization" refers to the process of adding the necessary formatting and additional information to the generated AI to return more accurate information in response to the query received.

[0011] A "browser extension" is a small program that can be added to a browser and used to extend the browser's functionality.

[0012] "Receiving" refers to the process by which the system takes in a query entered by a user.

[0013] "Submit" refers to the act and process of passing an optimized query to a generating artificial intelligence.

[0014] The "system" refers to a comprehensive computer-based solution that includes all of the above means and operates in conjunction with each other. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[0037] First, when a user uses the generative AI, a browser extension is used as a means of receiving the entered query. When the user enters a query and clicks the "Submit" button, the query is temporarily suppressed and sent to the server.

[0038] The server then analyzes and optimizes the received query, adding formatting and additional information to the query to encourage a detailed and informative answer. This optimization process involves adding statements such as "Please provide a detailed and informative answer to the following question:" to help the AI ​​generate the right answer.

[0039] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[0040] Generative AI generates detailed and accurate information based on optimized prompts, allowing users to obtain high-quality information without any effort.

[0041] Specific examples

[0042] 1. User enters question to the generative AI:

[0043] For example: "What is the latest weather forecast?"

[0044] On your device: The browser extension detects your input and temporarily prevents it from being sent.

[0045] 2. Send the query to the server:

[0046] Terminal: Send the user's query in JSON format to "http: / / localhost:5000 / optimize".

[0047] 3. The server receives the query:

[0048] Server: Receives the JSON data and extracts the prompt string.

[0049] 4. Optimize the prompt:

[0050] Server: Transforms the input prompt into a detailed and informative answer. For example, "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0051] 5. Send optimized prompts back to the device:

[0052] Server: Returns the optimized prompt in JSON format.

[0053] 6. Your device will receive an optimized prompt:

[0054] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[0055] 7. Generative AI returns highly accurate results:

[0056] Generative AI: Generate detailed weather forecasts based on optimized prompts.

[0057] 8. User sees the results:

[0058] User: Check the detailed weather forecast returned by the generated AI.

[0059] This embodiment allows users to automatically generate optimal prompts and obtain highly accurate information from the generation AI. This system frees users from the trouble of creating prompts, making the use of the generation AI easier and more effective.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[0063] Step 2:

[0064] The device (browser extension) detects the user's query input and clicks the send button, and temporarily blocks transmission at this stage.

[0065] Step 3:

[0066] The terminal converts the query entered by the user into JSON format and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[0067] Step 4:

[0068] The server receives the request sent from the device. The received data is in JSON format and the "prompt" key contains the user's query.

[0069] Step 5:

[0070] The server analyzes the received query to optimize it, specifically by adding phrases such as "Please provide a detailed and informative answer to the following question:" to the user's query, transforming the prompt into a more detailed and informative one.

[0071] Step 6:

[0072] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[0073] Step 7:

[0074] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[0075] Step 8:

[0076] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[0077] Step 9:

[0078] Generative AI generates detailed and highly accurate information based on optimized prompts received from the device, such as detailed answers about the latest weather forecast.

[0079] Step 10:

[0080] The user receives and confirms the detailed answers returned by the generating AI, allowing the user to obtain highly accurate and detailed information without any hassle.

[0081] This series of processes enables users to effectively utilize generative AI and realize a system that can automatically obtain highly accurate information.

[0082] Example 1

[0083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] When inputting a question into a generative AI, users have to take the time to create an appropriate prompt. In addition, because the accuracy of the answer obtained from the generative AI depends on the prompt entered by the user, there is a problem that the quality of the information obtained varies depending on the ability to create an optimal prompt. As a result, it can take time and effort for users to obtain a satisfactory answer.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0086] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to an AI that generates it, means for detecting the user's input and temporarily suppressing transmission, means for converting the received query into JSON format and sending it to the server, and means for receiving an optimized prompt from the server and reflecting the prompt. This relieves the user from the trouble of creating prompts and enables them to quickly obtain highly accurate and detailed answers from the AI ​​that generates them.

[0087] "User" refers to a person who uses this system to input queries into the generating AI and obtain information.

[0088] A "query" refers to a question or request that a user sends to a generative AI.

[0089] "Means for receiving" refers to the function of detecting an input query and temporarily storing it as data.

[0090] "Means to optimize" refers to the ability to analyze incoming queries and convert them into a format that makes it easier for the generative AI to generate accurate answers.

[0091] "Means for sending" refers to the function of sending optimized queries to the generation AI and receiving the results.

[0092] "Means to temporarily suppress transmission" refers to a function that temporarily stops transmission of a query when the user enters it, rather than sending the query immediately.

[0093] "Means for converting to JSON format" refers to a function for converting a received query into JSON format data.

[0094] "Browser extension" refers to additional software for a web browser that detects user query input and temporarily blocks that input.

[0095] An "optimized prompt" is a statement that transforms a query into a form that is more likely to elicit a detailed and effective answer.

[0096] "Generative AI" refers to an AI system that generates detailed and accurate information based on input prompts.

[0097] "Means of reflection" refers to the function that reflects the optimization prompt returned from the server in the user's input field and sends it to the generating AI.

[0098] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, allowing the user to obtain more accurate information. This system is implemented through the following steps.

[0099] First, when a user uses the generative AI, they use a browser extension to input a query. This browser extension temporarily suppresses the query when the user inputs the query and clicks the "Submit" button. The suppressed query is temporarily stored in the device's storage.

[0100] Next, the device's browser extension converts the stored query into JSON format and sends it to the server. Specifically, it sends the query to the URL "http: / / localhost:5000 / optimize". At this time, the data sent will be in a format such as {"query": "Please tell me the latest weather forecast"}.

[0101] When the server receives this request, it extracts the query string from the JSON data. For example, the extracted query might be "What is the latest weather forecast?". The server then analyzes the query and optimizes it for the most appropriate prompt. This optimization process involves adding statements to the query that request a detailed and informative answer. Specifically, the query might be converted to a format such as "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0102] The optimized prompt is then sent back to the device as JSON data from the server, such as {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}.

[0103] The device's browser extension receives the returned JSON data and extracts an optimized prompt, which is then automatically reflected in the user's query input field and sent back to the AI ​​generator, which then generates a detailed and accurate answer.

[0104] The answer from the generation AI is displayed in the user's browser, allowing the user to check the results. This system frees users from the hassle of creating prompts, allowing them to quickly and easily obtain highly accurate information from the generation AI. As an example of a specific prompt, when the user enters "Tell me about recent technology trends," the optimized prompt will be "Please provide a detailed and informative answer to the following question: Tell me about recent technology trends."

[0105] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0106] Step 1:

[0107] The user enters a query.

[0108] Input: The user types a question or request into the generated AI. Example: "What is the latest weather forecast?"

[0109] How it works: A user enters a query into the input field in their browser and clicks the "Submit" button.

[0110] Output: The query entered is detected by the browser extension.

[0111] Step 2:

[0112] The browser extension temporarily suppresses the query.

[0113] Input: The query for which the user clicked the submit button.

[0114] What it does: The extension detects the query and temporarily stores it in storage, but does not send the query yet.

[0115] Output: The suppressed queries are saved to storage.

[0116] Step 3:

[0117] The query is converted to JSON format and sent to the server.

[0118] Input: The query saved in the browser extension.

[0119] What it does: Converts a query into JSON format, e.g. {"query": "What is the latest weather forecast?"}. Sends this data to "http: / / localhost:5000 / optimize".

[0120] Output: JSON formatted data is sent to the server.

[0121] Step 4:

[0122] The server receives the query.

[0123] Input: JSON data sent from the terminal.

[0124] How it works: The server extracts the query from the received JSON data. Example: "query": "What is the latest weather forecast?"

[0125] Output: The extracted queries are made available on the server.

[0126] Step 5:

[0127] Optimize prompts.

[0128] Input: The query extracted on the server.

[0129] What it does: Parses the query and appends a statement that requests a detailed and informative answer. Example: "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0130] Output: An optimized prompt is generated.

[0131] Step 6:

[0132] Sends an optimized prompt back to the terminal.

[0133] Input: Optimized prompts.

[0134] What it does: Converts the optimized prompt into JSON format and sends it back to the device. Example: {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}

[0135] Output: An optimized prompt in JSON format is sent to the terminal.

[0136] Step 7:

[0137] Your device will receive an optimized prompt.

[0138] Input: The optimization prompt JSON data returned by the server.

[0139] How it works: The device browser extension receives the JSON data, extracts the optimization prompt, and reflects it in the original query input field.

[0140] Output: An optimized prompt is displayed in the query input field.

[0141] Step 8:

[0142] Generative AI produces highly accurate results.

[0143] Input: Optimized prompts.

[0144] How it works: Generative AI generates detailed and accurate answers based on optimized prompts.

[0145] Output: Detailed information is generated from the generative AI.

[0146] Step 9:

[0147] The user checks the results.

[0148] Input: The answer returned by the generation AI.

[0149] How it works: The user sees the generated answer in their browser.

[0150] Output: The user is satisfied with the highly accurate answer.

[0151] (Application example 1)

[0152] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0153] Conventional information generation systems using generative AI simply send queries entered by users, making it difficult to obtain optimal answers. Furthermore, automated driving vehicles are required to obtain highly accurate information on the optimal route to a destination, but this also presents similar problems. Therefore, there is a need for a system that allows users to easily obtain high-quality route information.

[0154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0155] In this invention, the server includes means for receiving a query input by a user, means for optimizing the received query, means for transmitting the optimized query to the generating artificial intelligence, and means for generating optimal route information for an autonomously driven vehicle. This enables a user to optimize a query and transmit it to the generating artificial intelligence, thereby easily obtaining high-quality route information for an autonomously driven vehicle.

[0156] "User" means any person or entity that utilizes the System to enter a query and obtain optimized information.

[0157] A "query" refers to a question or request that a user enters when requesting information from a generative artificial intelligence.

[0158] "Means for receiving" refers to a function or device that allows the system to obtain a query entered by a user.

[0159] "Optimizing means" refers to a process or device that analyzes a user-entered query and adds formatting and additional information to elicit a detailed and informative answer.

[0160] "Generative artificial intelligence" refers to advanced computer programs and systems that generate answers and information in natural language based on user queries.

[0161] "Means for sending" refers to a communication means or device for sending the optimized query to the generating artificial intelligence.

[0162] An "autonomous vehicle" refers to a vehicle equipped with technology that allows it to drive autonomously without human operation.

[0163] "Optimal route information" refers to information that shows the most efficient and safe route between the user's specified departure and destination, taking into account traffic conditions and other factors.

[0164] A "system" refers to a set of related components and processes combined to achieve a specific function or purpose.

[0165] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[0166] First, a user inputs information about the destination and route of the autonomous vehicle using a device such as a smartphone, smart glasses, or an in-vehicle display. This input information is captured as a query and temporarily suppressed by the browser extension.

[0167] The suppressed query is then sent to the server in JSON format. The server receives this JSON data, parses the query, and optimizes the received query to elicit a detailed and informative answer. For example, it adds a statement such as "Please provide a detailed and informative answer to the following question:". This optimization makes it easier for the generative AI to generate accurate answers.

[0168] The optimized prompt is then sent back to the device in JSON format. The device then reflects the received optimized prompt in the original query input field and finally sends it to the generation AI. The browser extension helps with this process. The generation AI generates detailed and accurate route and destination information based on the optimized prompt. This allows users to obtain efficient and safe route information.

[0169] As a concrete example, if a user types "What is the best route from Shinjuku to Haneda Airport?", the query is temporarily blocked by the browser extension and sent to the server. The server analyzes and optimizes the query, generating a prompt that reads, "Please provide a detailed and informative answer to the following question: What is the best route from Shinjuku to Haneda Airport?" This is then analyzed by the generative AI, and highly accurate route guidance is provided to the user.

[0170] The hardware used includes smartphones, smart glasses, in-vehicle displays, and servers, while the software used includes browser extensions and generative AI APIs (e.g., GCP and Azure generative AI services).

[0171] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0172] Step 1:

[0173] A user inputs information about a destination or intermediate destination using a device such as a smartphone, smart glasses, or an in-car display. This information is captured as a query.

[0174] Input: A query entered by a user (e.g., "What is the best route from Shinjuku to Haneda Airport?")

[0175] Output: The query is retrieved.

[0176] Step 2:

[0177] A browser extension in your device will temporarily prevent this query from being sent. This query will be temporarily saved.

[0178] Input: Retrieved query

[0179] Output: The query is suppressed and temporarily saved.

[0180] Step 3:

[0181] The browser extension sends the saved query in JSON format to the server, which receives the data.

[0182] Input: Saved query

[0183] Output: The query is sent to the server.

[0184] Step 4:

[0185] The server analyzes the query it receives, understands the content of the query, and performs optimizations to the data, such as adding formatting and additional information to provide a detailed and informative answer.

[0186] Input: The query received by the server

[0187] Output: Optimized queries with formatting and additional information to elicit detailed and informative answers

[0188] Step 5:

[0189] The server returns the optimized query in JSON format to the device, which receives the data.

[0190] Input: Optimized query

[0191] Output: The optimized query is sent back to the device.

[0192] Step 6:

[0193] The device reflects the optimized query in the original query input field, and the browser extension sends the query again to the generation AI.

[0194] Input: Optimized query

[0195] Output: The query sent to the generation AI

[0196] Step 7:

[0197] Based on the optimized query, the generative AI generates detailed and accurate route and destination information, which is then sent back to the device and provided to the user.

[0198] Input: The optimized query sent to the generative AI

[0199] Output: Highly accurate route information generated by the generative AI

[0200] Step 8:

[0201] The user can view the detailed route information generated and use the self-driving vehicle based on this information.

[0202] Input: Route information returned from the generation AI

[0203] Output: High-quality route information available to users

[0204] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0205] The present invention combines a system that uses a generative AI to process queries entered by users and sends optimized prompts to the generative AI with an emotion engine that recognizes the user's emotions, thereby optimizing the generative AI's responses according to the user's emotional state and improving the accuracy of the responses.

[0206] First, when a user uses generative AI, a browser extension is used as a means of receiving the query they enter. When the user enters a query and clicks the "Send" button, the browser extension temporarily suppresses the query and analyzes the user's emotional state along with the query.

[0207] The server then receives the query and emotion data sent from the device. The server analyzes the received query and uses an emotion engine to recognize the user's emotion. It then optimizes the query according to the user's emotional state. For example, if the user is dissatisfied, the server adds formatting and additional information to prompt the AI ​​to provide a more polite and detailed response.

[0208] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[0209] Based on optimized prompts, the AI ​​generates detailed and highly accurate information that takes into account the user's emotional state, allowing users to obtain high-quality information that is tailored to their emotions without any effort.

[0210] Specific examples

[0211] 1. User enters question to the generative AI:

[0212] A user types, "What's the latest weather forecast?"

[0213] Device: The browser extension collects emotional data from the user's input, facial expressions, voice, etc., and temporarily prevents it from being sent.

[0214] 2. Send the query and sentiment data to the server:

[0215] Terminal: Send the user's query and sentiment data in JSON format to "http: / / localhost:5000 / optimize".

[0216] 3. The server receives the query and emotion data:

[0217] Server: Receives JSON data and extracts prompt string and emotion data.

[0218] 4. Recognize user emotions:

[0219] Server: Analyzes the user's emotions using an emotion engine and recognizes their emotional state, for example, "dissatisfied" or "confused."

[0220] 5. Optimize the prompt:

[0221] Server: In addition to the entered prompt, add formatting and additional information that corresponds to the user's emotional state. Example: "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0222] 6. Send optimized prompts back to the device:

[0223] Server: Returns the optimized prompt in JSON format.

[0224] 7. Your device will receive an optimized prompt:

[0225] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[0226] 8. Generative AI returns highly accurate results:

[0227] Generative AI: Generate detailed, emotion-aware weather forecasts based on optimized prompts.

[0228] 9. User sees the results:

[0229] User: Check the detailed weather forecast returned by the generative AI. For example, "Today's weather is sunny, with a temperature of 23 degrees and calm winds."

[0230] This implementation allows users to interact with the AI ​​through prompts tailored to their emotional state, resulting in more accurate information. This system increases user satisfaction and significantly improves the AI ​​user experience.

[0231] The processing flow will be explained below.

[0232] Step 1:

[0233] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[0234] Step 2:

[0235] The device (browser extension) detects when the user enters a query and clicks the send button. At this time, it temporarily stops sending the query and begins collecting the user's facial expression and voice data.

[0236] Step 3:

[0237] The device analyzes the user's emotion data collected along with the query, converts the query and emotion data into JSON format, and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[0238] Step 4:

[0239] The server receives the request sent from the device. The received data is in JSON format, with the user's query in the "prompt" key and the user's emotion data in the "emotion" key.

[0240] Step 5:

[0241] The server analyzes the received query and uses an emotion engine to recognize the user's emotion. For example, data indicating that the user is feeling dissatisfied or confused may be detected.

[0242] Step 6:

[0243] The server then optimizes the query based on the user's perceived emotion, adding additional information and formatting based on the user's emotional state, such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0244] Step 7:

[0245] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[0246] Step 8:

[0247] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[0248] Step 9:

[0249] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[0250] Step 10:

[0251] Generative AI generates detailed and highly accurate information based on optimization prompts received from the device, such as a weather forecast with thoughtful and emotionally sensitive explanations.

[0252] Step 11:

[0253] The user receives and confirms detailed, emotionally relevant answers returned by the generative AI, including friendly messages such as, "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm. Have a nice day."

[0254] Through this series of processes, users can effectively utilize generative AI and automatically obtain highly accurate information in line with their own emotional state.

[0255] Example 2

[0256] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] When using generative AI models, conventional systems process queries without considering the user's emotional state, resulting in problems such as reduced accuracy of answers and reduced satisfaction. Furthermore, simply sending the query entered by the user to the generative AI as is often insufficient to obtain the answer the user expects. Therefore, there is a need for a system that can generate prompts optimized according to the user's emotional state and provide more accurate information.

[0258] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a query entered by a user, means for collecting the received query and the user's emotion data, means for transmitting the collected query and emotion data to the server, means for analyzing the query and emotion data received by the server and recognizing the user's emotion, means for optimizing the query based on the recognized emotion, means for transmitting the optimized query to the generating AI, and means for providing the user with information returned from the generating AI. This makes it possible to transmit an optimized prompt that reflects the user's emotional state to the generating AI, thereby enabling more accurate information acquisition.

[0259] "User" means a terminal user who uses the system to input a query and obtain information.

[0260] A "query" refers to a question or request that a user inputs to a generative AI model.

[0261] "Emotional data" refers to information about a user's emotional state collected through facial expressions, voice, text analysis, etc. when entering information.

[0262] A "browser extension" is a software program that can be added to a web browser to extend its functionality, and acts to intercept user queries and collect emotional data.

[0263] "Server" means a device or system that receives and analyzes data submitted by a user, and creates and transmits optimized prompts to a generative AI model.

[0264] An "emotion engine" is software or algorithm that analyzes collected emotional data and recognizes the user's emotions.

[0265] A "prompt" is a message containing a query and additional information optimized to be sent to a generative AI model.

[0266] A "generative AI model" is an artificial intelligence system that generates appropriate answers or information for a user based on the prompts it receives.

[0267] The "JSON format" is a text format used to structure data, and is designed to be easy to read for both humans and machines.

[0268] "Information provision means" is a general term for methods and devices for displaying and providing information returned from a generative AI model to a user.

[0269] The present invention relates to a system that optimizes a query entered by a user using a generative AI model and obtains highly accurate information from the generative AI model according to the user's emotional state. The system includes the steps of entering a user query, collecting emotional data, and optimizing prompts based on the emotion.

[0270] System Overview

[0271] The system includes the following components:

[0272] 1. A browser extension for users to enter queries

[0273] 2. A device to collect queries and emotion data and send them to the server.

[0274] 3. A server that analyzes the received data and generates optimized prompts using an emotion engine to recognize emotions.

[0275] 4. A means to send optimized prompts to the generative AI model and provide the resulting information to the user.

[0276] Hardware and software used

[0277] Terminal: A personal computer or smartphone that serves as the user interface

[0278] Browser extension: Software that receives user queries and collects sentiment data (e.g., Google Chrome extension)

[0279] Server: A central processing unit for analyzing data and optimizing prompts (e.g., Apache server, Nginx server)

[0280] Emotion engine: Software for analyzing emotion data (e.g., Microsoft Azure's Sentiment Analysis API, OpenCV)

[0281] Generative AI model: An artificial intelligence system that generates information based on optimized prompts (e.g., the GPT series)

[0282] Program processing

[0283] The system begins by having the user enter a query through the browser extension. When the user enters a query and presses the submit button, the browser extension temporarily suppresses the query and simultaneously collects emotional data from the user's facial expressions and voice. The collected data is converted into JSON format and sent to the server.

[0284] The server parses the received JSON data and extracts the query and emotion data. It then uses the emotion engine to analyze and recognize the user's emotional state (e.g., "frustrated," "confused," "excited," etc.) and generates an optimized prompt based on this. For example, if the user is frustrated, the prompt will be optimized to the format "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0285] The optimized prompt is sent to a generative AI model, which generates highly accurate information based on the prompt. The generated information is sent back to the browser extension via the server and provided to the user. For example, a user can type "What is the latest weather forecast?" and receive a detailed weather forecast such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm."

[0286] This embodiment makes it possible to send prompts to the generative AI model that are optimized to suit the user's emotional state, allowing the user to efficiently obtain highly accurate information, improving user satisfaction and significantly improving the usability of the generative AI model.

[0287] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0288] Step 1:

[0289] A user enters a query. Specifically, the user uses the browser extension to type "What is the latest weather forecast?" and presses the submit button. The input is a query (a question or request). The output is a temporary suppression of the query by the browser extension.

[0290] Step 2:

[0291] The device temporarily suppresses the user's queries while simultaneously collecting emotion data from the user's facial expressions and voice. For example, OpenCV is used to analyze the user's facial expressions, and emotion data is collected using an emotion recognition API. The input is the user's query and the user's facial expressions and voice. The output is JSON-formatted data containing the query and emotion data.

[0292] Step 3:

[0293] The query and emotion data collected by the device is converted into JSON format and sent to the server. The input is local data including the query and emotion data. The output is an HTTP POST request to "http: / / localhost:5000 / optimize".

[0294] Step 4:

[0295] Parse the JSON data received by the server and extract the query string and emotion data. For example, get the data as data = request.get_json() and save it as query = data['query'] and emotion = data['emotion'], respectively. The input is the JSON data received by the server. The output is the query string and emotion data.

[0296] Step 5:

[0297] The server uses an emotion engine to analyze and recognize the user's emotions. Specifically, it uses emotion recognition software to determine emotional states such as "frustrated" or "confused." The input is the query and emotion data. The output is the recognized emotional state.

[0298] Step 6:

[0299] The server generates an optimized prompt based on the recognized emotional state. For example, if the recognized emotion is "unhappy," it generates a prompt such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?" The input is the recognized emotional state and the query string. The output is the optimized prompt.

[0300] Step 7:

[0301] The server converts the optimized prompt into JSON format and sends it back to the device. The input is the optimized prompt. The output is the JSON response.

[0302] Step 8:

[0303] The optimized prompt received by the device is reflected in the original query input field and sent to the generative AI model. Specifically, the optimized prompt is set in the query input field and reflected in the form document.getElementById("query-input").value = response.optimized_prompt. The input is the optimized prompt. The output is the transmission of the optimized prompt to the generative AI model.

[0304] Step 9:

[0305] The generative AI model generates information based on the optimized prompt. The generative AI model generates detailed information such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm." The input is the optimized prompt. The output is the generated, highly accurate information.

[0306] Step 10:

[0307] The user checks the detailed weather forecast returned by the generative AI model in a browser. Specifically, the results from the generative AI model are displayed in the browser's display area. The input is detailed information from the generative AI model. The output is information provided to the user.

[0308] (Application example 2)

[0309] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] Conventional generative AI systems are unable to properly consider the emotional nuances of user queries, resulting in answers that are unsatisfactory for users. Furthermore, in services like food delivery, responses that do not properly consider emotions in response to user reviews and feedback have been difficult to generate, making it difficult to improve service quality. To address these issues, a system that can analyze a user's emotional state and generate optimized prompts accordingly is needed.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0312] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to the generating AI, means for analyzing the emotional state of the user, means for optimizing the query based on the emotional state, and means for generating a prompt according to the emotional state. As a result, an optimized prompt is generated taking the user's emotions into consideration, so that the generated AI response meets the user's expectations, thereby improving satisfaction.

[0313] "User" means an individual who utilizes the System to enter queries and obtain answers.

[0314] A "query" is a question or request that a user enters into a system.

[0315] "Receiving" means acquiring a query entered by a user within the system.

[0316] "Optimization" is the process of transforming a user's query into a better prompt.

[0317] "Generative AI" is artificial intelligence that generates answers based on input prompts.

[0318] "Sending" refers to sending optimized queries and prompts to other systems or AI.

[0319] "Emotional state" refers to a user's current psychological state or feelings.

[0320] "Analysis" refers to assessing the user's emotional state using an emotion engine or similar.

[0321] A "prompt" is an instruction that tells the generating AI how to respond.

[0322] A "browser extension" is a software feature added to a web browser that temporarily suppresses user queries.

[0323] "Format" refers to the structure or form of a query or prompt.

[0324] "Additional information" is supplemental information provided along with a query or prompt.

[0325] "Food ordering" refers to the act of a user ordering food using a food delivery system.

[0326] A "review" is an evaluation or opinion entered by a user about a service or product.

[0327] A "system" is a set of devices and software that comprehensively executes these means.

[0328] This invention is a system that analyzes user-entered queries, generates optimized prompts based on the user's emotional state, and sends them to a generative AI, which is particularly effective for user reviews and comments in food delivery services.

[0329] When a user orders food through a food delivery app, they can enter queries and reviews about the order, which are then temporarily suppressed through a browser extension that can be installed on popular browsers such as Google Chrome and Firefox.

[0330] The device sends the user's input query and emotion data to the server. The emotion data is acquired using tools such as facial expression recognition and voice analysis. For example, an emotion engine called "sentiment_analysis" is used for emotion analysis.

[0331] The server receives this data and analyzes the query and emotion data. It then optimizes the query according to the user's emotion. This is done using the "prompt_optimization" engine, which recognizes emotional states such as "confused" or "frustrated" and generates corresponding prompts. For example, for a user with the emotion "confused," the prompt format is "Please provide a detailed and calming answer to the following question:"

[0332] The optimized prompt is sent back to the device in JSON format. The device then uses this to reflect the optimized prompt in the user's query input field and sends it to the generation AI. Based on this optimized prompt, the generation AI generates detailed and highly accurate information that takes the user's emotional state into account.

[0333] For example, if a user inputs "This dish was cold when it arrived. Why?" and the emotional state of "confused" is detected, the optimized prompt will be "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?" Based on this prompt, the generative AI will generate a detailed and reassuring answer and provide it to the user.

[0334] This system allows users to receive answers that reflect their feelings, improving satisfaction with food delivery services, and also allows service providers to better process user feedback and improve the quality of their services.

[0335] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0336] Step 1:

[0337] When a user orders food through a food delivery app, they enter queries and reviews about the order, which are captured using a temporary suppression feature in the browser extension.

[0338] Input: User query (e.g., "This dish was cold when it arrived. Why?")

[0339] Output: Temporarily suppressed queries and sentiment data ready to send

[0340] Step 2:

[0341] The device sends the query and emotion data entered by the user to the server. The emotion data is obtained using facial expression recognition and voice analysis tools.

[0342] Input: Temporarily suppressed query and emotion data (e.g., user facial expression data, voice data)

[0343] Output: Sends JSON format data to the server

[0344] Step 3:

[0345] The server receives the user's query and emotion data. When receiving the data, it extracts the prompt string and emotion data from the JSON data.

[0346] Input: JSON format data sent from the terminal

[0347] Output: Extracted prompt string and emotion data

[0348] Step 4:

[0349] The server uses an emotion engine to analyze the user's emotional state. For example, the emotion analysis engine "sentiment_analysis" is used to recognize emotional states such as "confusion" and "dissatisfaction."

[0350] Input: Extracted emotion data (e.g., user facial expressions and voice data)

[0351] Output: Perceived emotional state (e.g., "confused")

[0352] Step 5:

[0353] To optimize queries based on emotional state, the server uses the prompt optimization engine "prompt_optimization" to generate optimized prompts according to emotions.

[0354] Input: Extracted prompt string and recognized emotional state (e.g., "This dish was cold when it arrived. Why?" and "Confused")

[0355] Output: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[0356] Step 6:

[0357] The server sends the optimized prompt back to the device in JSON format, which presents the optimized prompt to the user.

[0358] Input: Optimized prompts

[0359] Output: Send to the terminal as JSON format data

[0360] Step 7:

[0361] The device reflects the optimized prompt received in the user's query input field and sends the optimized prompt to the generation AI.

[0362] Input: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[0363] Output: Send to the generation AI

[0364] Step 8:

[0365] Based on optimized prompts, the generative AI generates detailed and highly accurate information that takes into account the user's emotional state.

[0366] Input: Optimized prompts

[0367] Output: Generated, highly accurate answer (e.g., "The food arrived cold. One possible reason is that the temperature was not properly controlled during the delivery process. Please rest assured that we will take extra care to ensure temperature control the next time you order.")

[0368] Step 9:

[0369] The user checks the detailed answer returned by the generated AI and is satisfied with its content.

[0370] Input: Generated, highly accurate answers

[0371] Output: High user satisfaction (e.g., "I was relieved to receive a detailed explanation of why my food arrived cold and what steps they would take.")

[0372] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0373] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0374] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0375] [Second embodiment]

[0376] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0377] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0378] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0379] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0380] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0381] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0382] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0383] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0384] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0385] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0386] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0387] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0388] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[0389] First, when a user uses the generative AI, a browser extension is used as a means of receiving the entered query. When the user enters a query and clicks the "Submit" button, the query is temporarily suppressed and sent to the server.

[0390] The server then analyzes and optimizes the received query, adding formatting and additional information to the query to encourage a detailed and informative answer. This optimization process involves adding statements such as "Please provide a detailed and informative answer to the following question:" to help the AI ​​generate the right answer.

[0391] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[0392] Generative AI generates detailed and accurate information based on optimized prompts, allowing users to obtain high-quality information without any effort.

[0393] Specific examples

[0394] 1. User enters question to the generative AI:

[0395] For example: "What is the latest weather forecast?"

[0396] On your device: The browser extension detects your input and temporarily prevents it from being sent.

[0397] 2. Send the query to the server:

[0398] Terminal: Send the user's query in JSON format to "http: / / localhost:5000 / optimize".

[0399] 3. The server receives the query:

[0400] Server: Receives the JSON data and extracts the prompt string.

[0401] 4. Optimize the prompt:

[0402] Server: Transforms the input prompt into a detailed and informative answer. For example, "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0403] 5. Send optimized prompts back to the device:

[0404] Server: Returns the optimized prompt in JSON format.

[0405] 6. Your device will receive an optimized prompt:

[0406] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[0407] 7. Generative AI returns highly accurate results:

[0408] Generative AI: Generate detailed weather forecasts based on optimized prompts.

[0409] 8. User sees the results:

[0410] User: Check the detailed weather forecast returned by the generated AI.

[0411] This embodiment allows users to automatically generate optimal prompts and obtain highly accurate information from the generation AI. This system frees users from the trouble of creating prompts, making the use of the generation AI easier and more effective.

[0412] The processing flow will be explained below.

[0413] Step 1:

[0414] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[0415] Step 2:

[0416] The device (browser extension) detects the user's query input and clicks the send button, and temporarily blocks transmission at this stage.

[0417] Step 3:

[0418] The terminal converts the query entered by the user into JSON format and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[0419] Step 4:

[0420] The server receives the request sent from the device. The received data is in JSON format and the "prompt" key contains the user's query.

[0421] Step 5:

[0422] The server analyzes the received query to optimize it, specifically by adding phrases such as "Please provide a detailed and informative answer to the following question:" to the user's query, transforming the prompt into a more detailed and informative one.

[0423] Step 6:

[0424] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[0425] Step 7:

[0426] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[0427] Step 8:

[0428] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[0429] Step 9:

[0430] Generative AI generates detailed and highly accurate information based on optimized prompts received from the device, such as detailed answers about the latest weather forecast.

[0431] Step 10:

[0432] The user receives and confirms the detailed answers returned by the generating AI, allowing the user to obtain highly accurate and detailed information without any hassle.

[0433] This series of processes enables users to effectively utilize generative AI and realize a system that can automatically obtain highly accurate information.

[0434] Example 1

[0435] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0436] When inputting a question into a generative AI, users have to take the time to create an appropriate prompt. In addition, because the accuracy of the answer obtained from the generative AI depends on the prompt entered by the user, there is a problem that the quality of the information obtained varies depending on the ability to create an optimal prompt. As a result, it can take time and effort for users to obtain a satisfactory answer.

[0437] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0438] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to an AI that generates it, means for detecting the user's input and temporarily suppressing transmission, means for converting the received query into JSON format and sending it to the server, and means for receiving an optimized prompt from the server and reflecting the prompt. This relieves the user from the trouble of creating prompts and enables them to quickly obtain highly accurate and detailed answers from the AI ​​that generates them.

[0439] "User" refers to a person who uses this system to input queries into the generating AI and obtain information.

[0440] A "query" refers to a question or request that a user sends to a generative AI.

[0441] "Means for receiving" refers to the function of detecting an input query and temporarily storing it as data.

[0442] "Means to optimize" refers to the ability to analyze incoming queries and convert them into a format that makes it easier for the generative AI to generate accurate answers.

[0443] "Means for sending" refers to the function of sending optimized queries to the generation AI and receiving the results.

[0444] "Means to temporarily suppress transmission" refers to a function that temporarily stops transmission of a query when the user enters it, rather than sending the query immediately.

[0445] "Means for converting to JSON format" refers to a function for converting a received query into JSON format data.

[0446] "Browser extension" refers to additional software for a web browser that detects user query input and temporarily blocks that input.

[0447] An "optimized prompt" is a statement that transforms a query into a form that is more likely to elicit a detailed and effective answer.

[0448] "Generative AI" refers to an AI system that generates detailed and accurate information based on input prompts.

[0449] "Means of reflection" refers to the function that reflects the optimization prompt returned from the server in the user's input field and sends it to the generating AI.

[0450] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, allowing the user to obtain more accurate information. This system is implemented through the following steps.

[0451] First, when a user uses the generative AI, they use a browser extension to input a query. This browser extension temporarily suppresses the query when the user inputs the query and clicks the "Submit" button. The suppressed query is temporarily stored in the device's storage.

[0452] Next, the device's browser extension converts the stored query into JSON format and sends it to the server. Specifically, it sends the query to the URL "http: / / localhost:5000 / optimize". At this time, the data sent will be in a format such as {"query": "Please tell me the latest weather forecast"}.

[0453] When the server receives this request, it extracts the query string from the JSON data. For example, the extracted query might be "What is the latest weather forecast?". The server then analyzes the query and optimizes it for the most appropriate prompt. This optimization process involves adding statements to the query that request a detailed and informative answer. Specifically, the query might be converted to a format such as "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0454] The optimized prompt is then sent back to the device as JSON data from the server, such as {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}.

[0455] The device's browser extension receives the returned JSON data and extracts an optimized prompt, which is then automatically reflected in the user's query input field and sent back to the AI ​​generator, which then generates a detailed and accurate answer.

[0456] The answer from the generation AI is displayed in the user's browser, allowing the user to check the results. This system frees users from the hassle of creating prompts, allowing them to quickly and easily obtain highly accurate information from the generation AI. As an example of a specific prompt, when the user enters "Tell me about recent technology trends," the optimized prompt will be "Please provide a detailed and informative answer to the following question: Tell me about recent technology trends."

[0457] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0458] Step 1:

[0459] The user enters a query.

[0460] Input: The user types a question or request into the generated AI. Example: "What is the latest weather forecast?"

[0461] How it works: A user enters a query into the input field in their browser and clicks the "Submit" button.

[0462] Output: The query entered is detected by the browser extension.

[0463] Step 2:

[0464] The browser extension temporarily suppresses the query.

[0465] Input: The query for which the user clicked the submit button.

[0466] What it does: The extension detects the query and temporarily stores it in storage, but does not send the query yet.

[0467] Output: The suppressed queries are saved to storage.

[0468] Step 3:

[0469] The query is converted to JSON format and sent to the server.

[0470] Input: The query saved in the browser extension.

[0471] What it does: Converts a query into JSON format, e.g. {"query": "What is the latest weather forecast?"}. Sends this data to "http: / / localhost:5000 / optimize".

[0472] Output: JSON formatted data is sent to the server.

[0473] Step 4:

[0474] The server receives the query.

[0475] Input: JSON data sent from the terminal.

[0476] How it works: The server extracts the query from the received JSON data. Example: "query": "What is the latest weather forecast?"

[0477] Output: The extracted queries are made available on the server.

[0478] Step 5:

[0479] Optimize prompts.

[0480] Input: The query extracted on the server.

[0481] What it does: Parses the query and appends a statement that requests a detailed and informative answer. Example: "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0482] Output: An optimized prompt is generated.

[0483] Step 6:

[0484] Sends an optimized prompt back to the terminal.

[0485] Input: Optimized prompts.

[0486] What it does: Converts the optimized prompt into JSON format and sends it back to the device. Example: {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}

[0487] Output: An optimized prompt in JSON format is sent to the terminal.

[0488] Step 7:

[0489] Your device will receive an optimized prompt.

[0490] Input: The optimization prompt JSON data returned by the server.

[0491] How it works: The device browser extension receives the JSON data, extracts the optimization prompt, and reflects it in the original query input field.

[0492] Output: An optimized prompt is displayed in the query input field.

[0493] Step 8:

[0494] Generative AI produces highly accurate results.

[0495] Input: Optimized prompts.

[0496] How it works: Generative AI generates detailed and accurate answers based on optimized prompts.

[0497] Output: Detailed information is generated from the generative AI.

[0498] Step 9:

[0499] The user checks the results.

[0500] Input: The answer returned by the generation AI.

[0501] How it works: The user sees the generated answer in their browser.

[0502] Output: The user is satisfied with the highly accurate answer.

[0503] (Application example 1)

[0504] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0505] Conventional information generation systems using generative AI simply send queries entered by users, making it difficult to obtain optimal answers. Furthermore, automated driving vehicles are required to obtain highly accurate information on the optimal route to a destination, but this also presents similar problems. Therefore, there is a need for a system that allows users to easily obtain high-quality route information.

[0506] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0507] In this invention, the server includes means for receiving a query input by a user, means for optimizing the received query, means for transmitting the optimized query to the generating artificial intelligence, and means for generating optimal route information for an autonomously driven vehicle. This enables a user to optimize a query and transmit it to the generating artificial intelligence, thereby easily obtaining high-quality route information for an autonomously driven vehicle.

[0508] "User" means any person or entity that utilizes the System to enter a query and obtain optimized information.

[0509] A "query" refers to a question or request that a user enters when requesting information from a generative artificial intelligence.

[0510] "Means for receiving" refers to a function or device that allows the system to obtain a query entered by a user.

[0511] "Optimizing means" refers to a process or device that analyzes a user-entered query and adds formatting and additional information to elicit a detailed and informative answer.

[0512] "Generative artificial intelligence" refers to advanced computer programs and systems that generate answers and information in natural language based on user queries.

[0513] "Means for sending" refers to a communication means or device for sending the optimized query to the generating artificial intelligence.

[0514] An "autonomous vehicle" refers to a vehicle equipped with technology that allows it to drive autonomously without human operation.

[0515] "Optimal route information" refers to information that shows the most efficient and safe route between the user's specified departure and destination, taking into account traffic conditions and other factors.

[0516] A "system" refers to a set of related components and processes combined to achieve a specific function or purpose.

[0517] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[0518] First, a user inputs information about the destination and route of the autonomous vehicle using a device such as a smartphone, smart glasses, or an in-vehicle display. This input information is captured as a query and temporarily suppressed by the browser extension.

[0519] The suppressed query is then sent to the server in JSON format. The server receives this JSON data, parses the query, and optimizes the received query to elicit a detailed and informative answer. For example, it adds a statement such as "Please provide a detailed and informative answer to the following question:". This optimization makes it easier for the generative AI to generate accurate answers.

[0520] The optimized prompt is then sent back to the device in JSON format. The device then reflects the received optimized prompt in the original query input field and finally sends it to the generation AI. The browser extension helps with this process. The generation AI generates detailed and accurate route and destination information based on the optimized prompt. This allows users to obtain efficient and safe route information.

[0521] As a concrete example, if a user types "What is the best route from Shinjuku to Haneda Airport?", the query is temporarily blocked by the browser extension and sent to the server. The server analyzes and optimizes the query, generating a prompt that reads, "Please provide a detailed and informative answer to the following question: What is the best route from Shinjuku to Haneda Airport?" This is then analyzed by the generative AI, and highly accurate route guidance is provided to the user.

[0522] The hardware used includes smartphones, smart glasses, in-vehicle displays, and servers, while the software used includes browser extensions and generative AI APIs (e.g., GCP and Azure generative AI services).

[0523] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0524] Step 1:

[0525] A user inputs information about a destination or intermediate destination using a device such as a smartphone, smart glasses, or an in-car display. This information is captured as a query.

[0526] Input: A query entered by a user (e.g., "What is the best route from Shinjuku to Haneda Airport?")

[0527] Output: The query is retrieved.

[0528] Step 2:

[0529] A browser extension in your device will temporarily prevent this query from being sent. This query will be temporarily saved.

[0530] Input: Retrieved query

[0531] Output: The query is suppressed and temporarily saved.

[0532] Step 3:

[0533] The browser extension sends the saved query in JSON format to the server, which receives the data.

[0534] Input: Saved query

[0535] Output: The query is sent to the server.

[0536] Step 4:

[0537] The server analyzes the query it receives, understands the content of the query, and performs optimizations to the data, such as adding formatting and additional information to provide a detailed and informative answer.

[0538] Input: The query received by the server

[0539] Output: Optimized queries with formatting and additional information to elicit detailed and informative answers

[0540] Step 5:

[0541] The server returns the optimized query in JSON format to the device, which receives the data.

[0542] Input: Optimized query

[0543] Output: The optimized query is sent back to the device.

[0544] Step 6:

[0545] The device reflects the optimized query in the original query input field, and the browser extension sends the query again to the generation AI.

[0546] Input: Optimized query

[0547] Output: The query sent to the generation AI

[0548] Step 7:

[0549] Based on the optimized query, the generative AI generates detailed and accurate route and destination information, which is then sent back to the device and provided to the user.

[0550] Input: The optimized query sent to the generative AI

[0551] Output: Highly accurate route information generated by the generative AI

[0552] Step 8:

[0553] The user can view the detailed route information generated and use the self-driving vehicle based on this information.

[0554] Input: Route information returned from the generation AI

[0555] Output: High-quality route information available to users

[0556] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0557] The present invention combines a system that uses a generative AI to process queries entered by users and sends optimized prompts to the generative AI with an emotion engine that recognizes the user's emotions, thereby optimizing the generative AI's responses according to the user's emotional state and improving the accuracy of the responses.

[0558] First, when a user uses generative AI, a browser extension is used as a means of receiving the query they enter. When the user enters a query and clicks the "Send" button, the browser extension temporarily suppresses the query and analyzes the user's emotional state along with the query.

[0559] The server then receives the query and emotion data sent from the device. The server analyzes the received query and uses an emotion engine to recognize the user's emotion. It then optimizes the query according to the user's emotional state. For example, if the user is dissatisfied, the server adds formatting and additional information to prompt the AI ​​to provide a more polite and detailed response.

[0560] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[0561] Based on optimized prompts, the AI ​​generates detailed and highly accurate information that takes into account the user's emotional state, allowing users to obtain high-quality information that is tailored to their emotions without any effort.

[0562] Specific examples

[0563] 1. User enters question to the generative AI:

[0564] A user types, "What's the latest weather forecast?"

[0565] Device: The browser extension collects emotional data from the user's input, facial expressions, voice, etc., and temporarily prevents it from being sent.

[0566] 2. Send the query and sentiment data to the server:

[0567] Terminal: Send the user's query and sentiment data in JSON format to "http: / / localhost:5000 / optimize".

[0568] 3. The server receives the query and emotion data:

[0569] Server: Receives JSON data and extracts prompt string and emotion data.

[0570] 4. Recognize user emotions:

[0571] Server: Analyzes the user's emotions using an emotion engine and recognizes their emotional state, for example, "dissatisfied" or "confused."

[0572] 5. Optimize the prompt:

[0573] Server: In addition to the entered prompt, add formatting and additional information that corresponds to the user's emotional state. Example: "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0574] 6. Send optimized prompts back to the device:

[0575] Server: Returns the optimized prompt in JSON format.

[0576] 7. Your device will receive an optimized prompt:

[0577] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[0578] 8. Generative AI returns highly accurate results:

[0579] Generative AI: Generate detailed, emotion-aware weather forecasts based on optimized prompts.

[0580] 9. User sees the results:

[0581] User: Check the detailed weather forecast returned by the generative AI. For example, "Today's weather is sunny, with a temperature of 23 degrees and calm winds."

[0582] This implementation allows users to interact with the AI ​​through prompts tailored to their emotional state, resulting in more accurate information. This system increases user satisfaction and significantly improves the AI ​​user experience.

[0583] The processing flow will be explained below.

[0584] Step 1:

[0585] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[0586] Step 2:

[0587] The device (browser extension) detects when the user enters a query and clicks the send button. At this time, it temporarily stops sending the query and begins collecting the user's facial expression and voice data.

[0588] Step 3:

[0589] The device analyzes the user's emotion data collected along with the query, converts the query and emotion data into JSON format, and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[0590] Step 4:

[0591] The server receives the request sent from the device. The received data is in JSON format, with the user's query in the "prompt" key and the user's emotion data in the "emotion" key.

[0592] Step 5:

[0593] The server analyzes the received query and uses an emotion engine to recognize the user's emotion. For example, data indicating that the user is feeling dissatisfied or confused may be detected.

[0594] Step 6:

[0595] The server then optimizes the query based on the user's perceived emotion, adding additional information and formatting based on the user's emotional state, such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0596] Step 7:

[0597] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[0598] Step 8:

[0599] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[0600] Step 9:

[0601] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[0602] Step 10:

[0603] Generative AI generates detailed and highly accurate information based on optimization prompts received from the device, such as a weather forecast with thoughtful and emotionally sensitive explanations.

[0604] Step 11:

[0605] The user receives and confirms detailed, emotionally relevant answers returned by the generative AI, including friendly messages such as, "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm. Have a nice day."

[0606] Through this series of processes, users can effectively utilize generative AI and automatically obtain highly accurate information in line with their own emotional state.

[0607] Example 2

[0608] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0609] When using generative AI models, conventional systems process queries without considering the user's emotional state, resulting in problems such as reduced accuracy of answers and reduced satisfaction. Furthermore, simply sending the query entered by the user to the generative AI as is often insufficient to obtain the answer the user expects. Therefore, there is a need for a system that can generate prompts optimized according to the user's emotional state and provide more accurate information.

[0610] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a query entered by a user, means for collecting the received query and the user's emotion data, means for transmitting the collected query and emotion data to the server, means for analyzing the query and emotion data received by the server and recognizing the user's emotion, means for optimizing the query based on the recognized emotion, means for transmitting the optimized query to the generating AI, and means for providing the user with information returned from the generating AI. This makes it possible to transmit an optimized prompt that reflects the user's emotional state to the generating AI, thereby enabling more accurate information acquisition.

[0611] "User" means a terminal user who uses the system to input a query and obtain information.

[0612] A "query" refers to a question or request that a user inputs to a generative AI model.

[0613] "Emotional data" refers to information about a user's emotional state collected through facial expressions, voice, text analysis, etc. when entering information.

[0614] A "browser extension" is a software program that can be added to a web browser to extend its functionality, and acts to intercept user queries and collect emotional data.

[0615] "Server" means a device or system that receives and analyzes data submitted by a user, and creates and transmits optimized prompts to a generative AI model.

[0616] An "emotion engine" is software or algorithm that analyzes collected emotional data and recognizes the user's emotions.

[0617] A "prompt" is a message containing a query and additional information optimized to be sent to a generative AI model.

[0618] A "generative AI model" is an artificial intelligence system that generates appropriate answers or information for a user based on the prompts it receives.

[0619] The "JSON format" is a text format used to structure data, and is designed to be easy to read for both humans and machines.

[0620] "Information provision means" is a general term for methods and devices for displaying and providing information returned from a generative AI model to a user.

[0621] The present invention relates to a system that optimizes a query entered by a user using a generative AI model and obtains highly accurate information from the generative AI model according to the user's emotional state. The system includes the steps of entering a user query, collecting emotional data, and optimizing prompts based on the emotion.

[0622] System Overview

[0623] The system includes the following components:

[0624] 1. A browser extension for users to enter queries

[0625] 2. A device to collect queries and emotion data and send them to the server.

[0626] 3. A server that analyzes the received data and generates optimized prompts using an emotion engine to recognize emotions.

[0627] 4. A means to send optimized prompts to the generative AI model and provide the resulting information to the user.

[0628] Hardware and software used

[0629] Terminal: A personal computer or smartphone that serves as the user interface

[0630] Browser extension: Software that receives user queries and collects sentiment data (e.g., Google Chrome extension)

[0631] Server: A central processing unit for analyzing data and optimizing prompts (e.g., Apache server, Nginx server)

[0632] Emotion engine: Software for analyzing emotion data (e.g., Microsoft Azure's Sentiment Analysis API, OpenCV)

[0633] Generative AI model: An artificial intelligence system that generates information based on optimized prompts (e.g., the GPT series)

[0634] Program processing

[0635] The system begins by having the user enter a query through the browser extension. When the user enters a query and presses the submit button, the browser extension temporarily suppresses the query and simultaneously collects emotional data from the user's facial expressions and voice. The collected data is converted into JSON format and sent to the server.

[0636] The server parses the received JSON data and extracts the query and emotion data. It then uses the emotion engine to analyze and recognize the user's emotional state (e.g., "frustrated," "confused," "excited," etc.) and generates an optimized prompt based on this. For example, if the user is frustrated, the prompt will be optimized to the format "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0637] The optimized prompt is sent to a generative AI model, which generates highly accurate information based on the prompt. The generated information is sent back to the browser extension via the server and provided to the user. For example, a user can type "What is the latest weather forecast?" and receive a detailed weather forecast such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm."

[0638] This embodiment makes it possible to send prompts to the generative AI model that are optimized to suit the user's emotional state, allowing the user to efficiently obtain highly accurate information, improving user satisfaction and significantly improving the usability of the generative AI model.

[0639] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0640] Step 1:

[0641] A user enters a query. Specifically, the user uses the browser extension to type "What is the latest weather forecast?" and presses the submit button. The input is a query (a question or request). The output is a temporary suppression of the query by the browser extension.

[0642] Step 2:

[0643] The device temporarily suppresses the user's queries while simultaneously collecting emotion data from the user's facial expressions and voice. For example, OpenCV is used to analyze the user's facial expressions, and emotion data is collected using an emotion recognition API. The input is the user's query and the user's facial expressions and voice. The output is JSON-formatted data containing the query and emotion data.

[0644] Step 3:

[0645] The query and emotion data collected by the device is converted into JSON format and sent to the server. The input is local data including the query and emotion data. The output is an HTTP POST request to "http: / / localhost:5000 / optimize".

[0646] Step 4:

[0647] Parse the JSON data received by the server and extract the query string and emotion data. For example, get the data as data = request.get_json() and save it as query = data['query'] and emotion = data['emotion'], respectively. The input is the JSON data received by the server. The output is the query string and emotion data.

[0648] Step 5:

[0649] The server uses an emotion engine to analyze and recognize the user's emotions. Specifically, it uses emotion recognition software to determine emotional states such as "frustrated" or "confused." The input is the query and emotion data. The output is the recognized emotional state.

[0650] Step 6:

[0651] The server generates an optimized prompt based on the recognized emotional state. For example, if the recognized emotion is "unhappy," it generates a prompt such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?" The input is the recognized emotional state and the query string. The output is the optimized prompt.

[0652] Step 7:

[0653] The server converts the optimized prompt into JSON format and sends it back to the device. The input is the optimized prompt. The output is the JSON response.

[0654] Step 8:

[0655] The optimized prompt received by the device is reflected in the original query input field and sent to the generative AI model. Specifically, the optimized prompt is set in the query input field and reflected in the form document.getElementById("query-input").value = response.optimized_prompt. The input is the optimized prompt. The output is the transmission of the optimized prompt to the generative AI model.

[0656] Step 9:

[0657] The generative AI model generates information based on the optimized prompt. The generative AI model generates detailed information such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm." The input is the optimized prompt. The output is the generated, highly accurate information.

[0658] Step 10:

[0659] The user checks the detailed weather forecast returned by the generative AI model in a browser. Specifically, the results from the generative AI model are displayed in the browser's display area. The input is detailed information from the generative AI model. The output is information provided to the user.

[0660] (Application example 2)

[0661] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0662] Conventional generative AI systems are unable to properly consider the emotional nuances of user queries, resulting in answers that are unsatisfactory for users. Furthermore, in services like food delivery, responses that do not properly consider emotions in response to user reviews and feedback have been difficult to generate, making it difficult to improve service quality. To address these issues, a system that can analyze a user's emotional state and generate optimized prompts accordingly is needed.

[0663] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0664] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to the generating AI, means for analyzing the emotional state of the user, means for optimizing the query based on the emotional state, and means for generating a prompt according to the emotional state. As a result, an optimized prompt is generated taking the user's emotions into consideration, so that the generated AI response meets the user's expectations, thereby improving satisfaction.

[0665] "User" means an individual who utilizes the System to enter queries and obtain answers.

[0666] A "query" is a question or request that a user enters into a system.

[0667] "Receiving" means acquiring a query entered by a user within the system.

[0668] "Optimization" is the process of transforming a user's query into a better prompt.

[0669] "Generative AI" is artificial intelligence that generates answers based on input prompts.

[0670] "Sending" refers to sending optimized queries and prompts to other systems or AI.

[0671] "Emotional state" refers to a user's current psychological state or feelings.

[0672] "Analysis" refers to assessing the user's emotional state using an emotion engine or similar.

[0673] A "prompt" is an instruction that tells the generating AI how to respond.

[0674] A "browser extension" is a software feature added to a web browser that temporarily suppresses user queries.

[0675] "Format" refers to the structure or form of a query or prompt.

[0676] "Additional information" is supplemental information provided along with a query or prompt.

[0677] "Food ordering" refers to the act of a user ordering food using a food delivery system.

[0678] A "review" is an evaluation or opinion entered by a user about a service or product.

[0679] A "system" is a set of devices and software that comprehensively executes these means.

[0680] This invention is a system that analyzes user-entered queries, generates optimized prompts based on the user's emotional state, and sends them to a generative AI, which is particularly effective for user reviews and comments in food delivery services.

[0681] When a user orders food through a food delivery app, they can enter queries and reviews about the order, which are then temporarily suppressed through a browser extension that can be installed on popular browsers such as Google Chrome and Firefox.

[0682] The device sends the user's input query and emotion data to the server. The emotion data is acquired using tools such as facial expression recognition and voice analysis. For example, an emotion engine called "sentiment_analysis" is used for emotion analysis.

[0683] The server receives this data and analyzes the query and emotion data. It then optimizes the query according to the user's emotion. This is done using the "prompt_optimization" engine, which recognizes emotional states such as "confused" or "frustrated" and generates corresponding prompts. For example, for a user with the emotion "confused," the prompt format is "Please provide a detailed and calming answer to the following question:"

[0684] The optimized prompt is sent back to the device in JSON format. The device then uses this to reflect the optimized prompt in the user's query input field and sends it to the generation AI. Based on this optimized prompt, the generation AI generates detailed and highly accurate information that takes the user's emotional state into account.

[0685] For example, if a user inputs "This dish was cold when it arrived. Why?" and the emotional state of "confused" is detected, the optimized prompt will be "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?" Based on this prompt, the generative AI will generate a detailed and reassuring answer and provide it to the user.

[0686] This system allows users to receive answers that reflect their feelings, improving satisfaction with food delivery services, and also allows service providers to better process user feedback and improve the quality of their services.

[0687] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0688] Step 1:

[0689] When a user orders food through a food delivery app, they enter queries and reviews about the order, which are captured using a temporary suppression feature in the browser extension.

[0690] Input: User query (e.g., "This dish was cold when it arrived. Why?")

[0691] Output: Temporarily suppressed queries and sentiment data ready to send

[0692] Step 2:

[0693] The device sends the query and emotion data entered by the user to the server. The emotion data is obtained using facial expression recognition and voice analysis tools.

[0694] Input: Temporarily suppressed query and emotion data (e.g., user facial expression data, voice data)

[0695] Output: Sends JSON format data to the server

[0696] Step 3:

[0697] The server receives the user's query and emotion data. When receiving the data, it extracts the prompt string and emotion data from the JSON data.

[0698] Input: JSON format data sent from the terminal

[0699] Output: Extracted prompt string and emotion data

[0700] Step 4:

[0701] The server uses an emotion engine to analyze the user's emotional state. For example, the emotion analysis engine "sentiment_analysis" is used to recognize emotional states such as "confusion" and "dissatisfaction."

[0702] Input: Extracted emotion data (e.g., user facial expressions and voice data)

[0703] Output: Perceived emotional state (e.g., "confused")

[0704] Step 5:

[0705] To optimize queries based on emotional state, the server uses the prompt optimization engine "prompt_optimization" to generate optimized prompts according to emotions.

[0706] Input: Extracted prompt string and recognized emotional state (e.g., "This dish was cold when it arrived. Why?" and "Confused")

[0707] Output: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[0708] Step 6:

[0709] The server sends the optimized prompt back to the device in JSON format, which presents the optimized prompt to the user.

[0710] Input: Optimized prompts

[0711] Output: Send to the terminal as JSON format data

[0712] Step 7:

[0713] The device reflects the optimized prompt received in the user's query input field and sends the optimized prompt to the generation AI.

[0714] Input: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[0715] Output: Send to the generation AI

[0716] Step 8:

[0717] Based on optimized prompts, the generative AI generates detailed and highly accurate information that takes into account the user's emotional state.

[0718] Input: Optimized prompts

[0719] Output: Generated, highly accurate answer (e.g., "The food arrived cold. One possible reason is that the temperature was not properly controlled during the delivery process. Please rest assured that we will take extra care to ensure temperature control the next time you order.")

[0720] Step 9:

[0721] The user checks the detailed answer returned by the generated AI and is satisfied with its content.

[0722] Input: Generated, highly accurate answers

[0723] Output: High user satisfaction (e.g., "I was relieved to receive a detailed explanation of why my food arrived cold and what steps they would take.")

[0724] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0725] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0726] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0727] [Third embodiment]

[0728] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0729] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0730] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0731] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0732] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0733] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0734] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0735] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0736] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0737] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0738] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0739] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0740] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[0741] First, when a user uses the generative AI, a browser extension is used as a means of receiving the entered query. When the user enters a query and clicks the "Submit" button, the query is temporarily suppressed and sent to the server.

[0742] The server then analyzes and optimizes the received query, adding formatting and additional information to the query to encourage a detailed and informative answer. This optimization process involves adding statements such as "Please provide a detailed and informative answer to the following question:" to help the AI ​​generate the right answer.

[0743] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[0744] Generative AI generates detailed and accurate information based on optimized prompts, allowing users to obtain high-quality information without any effort.

[0745] Specific examples

[0746] 1. User enters question to the generative AI:

[0747] For example: "What is the latest weather forecast?"

[0748] On your device: The browser extension detects your input and temporarily prevents it from being sent.

[0749] 2. Send the query to the server:

[0750] Terminal: Send the user's query in JSON format to "http: / / localhost:5000 / optimize".

[0751] 3. The server receives the query:

[0752] Server: Receives the JSON data and extracts the prompt string.

[0753] 4. Optimize the prompt:

[0754] Server: Transforms the input prompt into a detailed and informative answer. For example, "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0755] 5. Send optimized prompts back to the device:

[0756] Server: Returns the optimized prompt in JSON format.

[0757] 6. Your device will receive an optimized prompt:

[0758] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[0759] 7. Generative AI returns highly accurate results:

[0760] Generative AI: Generate detailed weather forecasts based on optimized prompts.

[0761] 8. User sees the results:

[0762] User: Check the detailed weather forecast returned by the generated AI.

[0763] This embodiment allows users to automatically generate optimal prompts and obtain highly accurate information from the generation AI. This system frees users from the trouble of creating prompts, making the use of the generation AI easier and more effective.

[0764] The processing flow will be explained below.

[0765] Step 1:

[0766] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[0767] Step 2:

[0768] The device (browser extension) detects the user's query input and clicks the send button, and temporarily blocks transmission at this stage.

[0769] Step 3:

[0770] The terminal converts the query entered by the user into JSON format and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[0771] Step 4:

[0772] The server receives the request sent from the device. The received data is in JSON format and the "prompt" key contains the user's query.

[0773] Step 5:

[0774] The server analyzes the received query to optimize it, specifically by adding phrases such as "Please provide a detailed and informative answer to the following question:" to the user's query, transforming the prompt into a more detailed and informative one.

[0775] Step 6:

[0776] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[0777] Step 7:

[0778] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[0779] Step 8:

[0780] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[0781] Step 9:

[0782] Generative AI generates detailed and highly accurate information based on optimized prompts received from the device, such as detailed answers about the latest weather forecast.

[0783] Step 10:

[0784] The user receives and confirms the detailed answers returned by the generating AI, allowing the user to obtain highly accurate and detailed information without any hassle.

[0785] This series of processes enables users to effectively utilize generative AI and realize a system that can automatically obtain highly accurate information.

[0786] Example 1

[0787] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0788] When inputting a question into a generative AI, users have to take the time to create an appropriate prompt. In addition, because the accuracy of the answer obtained from the generative AI depends on the prompt entered by the user, there is a problem that the quality of the information obtained varies depending on the ability to create an optimal prompt. As a result, it can take time and effort for users to obtain a satisfactory answer.

[0789] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0790] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to an AI that generates it, means for detecting the user's input and temporarily suppressing transmission, means for converting the received query into JSON format and sending it to the server, and means for receiving an optimized prompt from the server and reflecting the prompt. This relieves the user from the trouble of creating prompts and enables them to quickly obtain highly accurate and detailed answers from the AI ​​that generates them.

[0791] "User" refers to a person who uses this system to input queries into the generating AI and obtain information.

[0792] A "query" refers to a question or request that a user sends to a generative AI.

[0793] "Means for receiving" refers to the function of detecting an input query and temporarily storing it as data.

[0794] "Means to optimize" refers to the ability to analyze incoming queries and convert them into a format that makes it easier for the generative AI to generate accurate answers.

[0795] "Means for sending" refers to the function of sending optimized queries to the generation AI and receiving the results.

[0796] "Means to temporarily suppress transmission" refers to a function that temporarily stops transmission of a query when the user enters it, rather than sending the query immediately.

[0797] "Means for converting to JSON format" refers to a function for converting a received query into JSON format data.

[0798] "Browser extension" refers to additional software for a web browser that detects user query input and temporarily blocks that input.

[0799] An "optimized prompt" is a statement that transforms a query into a form that is more likely to elicit a detailed and effective answer.

[0800] "Generative AI" refers to an AI system that generates detailed and accurate information based on input prompts.

[0801] "Means of reflection" refers to the function that reflects the optimization prompt returned from the server in the user's input field and sends it to the generating AI.

[0802] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, allowing the user to obtain more accurate information. This system is implemented through the following steps.

[0803] First, when a user uses the generative AI, they use a browser extension to input a query. This browser extension temporarily suppresses the query when the user inputs the query and clicks the "Submit" button. The suppressed query is temporarily stored in the device's storage.

[0804] Next, the device's browser extension converts the stored query into JSON format and sends it to the server. Specifically, it sends the query to the URL "http: / / localhost:5000 / optimize". At this time, the data sent will be in a format such as {"query": "Please tell me the latest weather forecast"}.

[0805] When the server receives this request, it extracts the query string from the JSON data. For example, the extracted query might be "What is the latest weather forecast?". The server then analyzes the query and optimizes it for the most appropriate prompt. This optimization process involves adding statements to the query that request a detailed and informative answer. Specifically, the query might be converted to a format such as "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0806] The optimized prompt is then sent back to the device as JSON data from the server, such as {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}.

[0807] The device's browser extension receives the returned JSON data and extracts an optimized prompt, which is then automatically reflected in the user's query input field and sent back to the AI ​​generator, which then generates a detailed and accurate answer.

[0808] The answer from the generation AI is displayed in the user's browser, allowing the user to check the results. This system frees users from the hassle of creating prompts, allowing them to quickly and easily obtain highly accurate information from the generation AI. As an example of a specific prompt, when the user enters "Tell me about recent technology trends," the optimized prompt will be "Please provide a detailed and informative answer to the following question: Tell me about recent technology trends."

[0809] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0810] Step 1:

[0811] The user enters a query.

[0812] Input: The user types a question or request into the generated AI. Example: "What is the latest weather forecast?"

[0813] How it works: A user enters a query into the input field in their browser and clicks the "Submit" button.

[0814] Output: The query entered is detected by the browser extension.

[0815] Step 2:

[0816] The browser extension temporarily suppresses the query.

[0817] Input: The query for which the user clicked the submit button.

[0818] What it does: The extension detects the query and temporarily stores it in storage, but does not send the query yet.

[0819] Output: The suppressed queries are saved to storage.

[0820] Step 3:

[0821] The query is converted to JSON format and sent to the server.

[0822] Input: The query saved in the browser extension.

[0823] What it does: Converts a query into JSON format, e.g. {"query": "What is the latest weather forecast?"}. Sends this data to "http: / / localhost:5000 / optimize".

[0824] Output: JSON formatted data is sent to the server.

[0825] Step 4:

[0826] The server receives the query.

[0827] Input: JSON data sent from the terminal.

[0828] How it works: The server extracts the query from the received JSON data. Example: "query": "What is the latest weather forecast?"

[0829] Output: The extracted queries are made available on the server.

[0830] Step 5:

[0831] Optimize prompts.

[0832] Input: The query extracted on the server.

[0833] What it does: Parses the query and appends a statement that requests a detailed and informative answer. Example: "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[0834] Output: An optimized prompt is generated.

[0835] Step 6:

[0836] Sends an optimized prompt back to the terminal.

[0837] Input: Optimized prompts.

[0838] What it does: Converts the optimized prompt into JSON format and sends it back to the device. Example: {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}

[0839] Output: An optimized prompt in JSON format is sent to the terminal.

[0840] Step 7:

[0841] Your device will receive an optimized prompt.

[0842] Input: The optimization prompt JSON data returned by the server.

[0843] How it works: The device browser extension receives the JSON data, extracts the optimization prompt, and reflects it in the original query input field.

[0844] Output: An optimized prompt is displayed in the query input field.

[0845] Step 8:

[0846] Generative AI produces highly accurate results.

[0847] Input: Optimized prompts.

[0848] How it works: Generative AI generates detailed and accurate answers based on optimized prompts.

[0849] Output: Detailed information is generated from the generative AI.

[0850] Step 9:

[0851] The user checks the results.

[0852] Input: The answer returned by the generation AI.

[0853] How it works: The user sees the generated answer in their browser.

[0854] Output: The user is satisfied with the highly accurate answer.

[0855] (Application example 1)

[0856] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0857] Conventional information generation systems using generative AI simply send queries entered by users, making it difficult to obtain optimal answers. Furthermore, automated driving vehicles are required to obtain highly accurate information on the optimal route to a destination, but this also presents similar problems. Therefore, there is a need for a system that allows users to easily obtain high-quality route information.

[0858] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0859] In this invention, the server includes means for receiving a query input by a user, means for optimizing the received query, means for transmitting the optimized query to the generating artificial intelligence, and means for generating optimal route information for an autonomously driven vehicle. This enables a user to optimize a query and transmit it to the generating artificial intelligence, thereby easily obtaining high-quality route information for an autonomously driven vehicle.

[0860] "User" means any person or entity that utilizes the System to enter a query and obtain optimized information.

[0861] A "query" refers to a question or request that a user enters when requesting information from a generative artificial intelligence.

[0862] "Means for receiving" refers to a function or device that allows the system to obtain a query entered by a user.

[0863] "Optimizing means" refers to a process or device that analyzes a user-entered query and adds formatting and additional information to elicit a detailed and informative answer.

[0864] "Generative artificial intelligence" refers to advanced computer programs and systems that generate answers and information in natural language based on user queries.

[0865] "Means for sending" refers to a communication means or device for sending the optimized query to the generating artificial intelligence.

[0866] An "autonomous vehicle" refers to a vehicle equipped with technology that allows it to drive autonomously without human operation.

[0867] "Optimal route information" refers to information that shows the most efficient and safe route between the user's specified departure and destination, taking into account traffic conditions and other factors.

[0868] A "system" refers to a set of related components and processes combined to achieve a specific function or purpose.

[0869] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[0870] First, a user inputs information about the destination and route of the autonomous vehicle using a device such as a smartphone, smart glasses, or an in-vehicle display. This input information is captured as a query and temporarily suppressed by the browser extension.

[0871] The suppressed query is then sent to the server in JSON format. The server receives this JSON data, parses the query, and optimizes the received query to elicit a detailed and informative answer. For example, it adds a statement such as "Please provide a detailed and informative answer to the following question:". This optimization makes it easier for the generative AI to generate accurate answers.

[0872] The optimized prompt is then sent back to the device in JSON format. The device then reflects the received optimized prompt in the original query input field and finally sends it to the generation AI. The browser extension helps with this process. The generation AI generates detailed and accurate route and destination information based on the optimized prompt. This allows users to obtain efficient and safe route information.

[0873] As a concrete example, if a user types "What is the best route from Shinjuku to Haneda Airport?", the query is temporarily blocked by the browser extension and sent to the server. The server analyzes and optimizes the query, generating a prompt that reads, "Please provide a detailed and informative answer to the following question: What is the best route from Shinjuku to Haneda Airport?" This is then analyzed by the generative AI, and highly accurate route guidance is provided to the user.

[0874] The hardware used includes smartphones, smart glasses, in-vehicle displays, and servers, while the software used includes browser extensions and generative AI APIs (e.g., GCP and Azure generative AI services).

[0875] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0876] Step 1:

[0877] A user inputs information about a destination or intermediate destination using a device such as a smartphone, smart glasses, or an in-car display. This information is captured as a query.

[0878] Input: A query entered by a user (e.g., "What is the best route from Shinjuku to Haneda Airport?")

[0879] Output: The query is retrieved.

[0880] Step 2:

[0881] A browser extension in your device will temporarily prevent this query from being sent. This query will be temporarily saved.

[0882] Input: Retrieved query

[0883] Output: The query is suppressed and temporarily saved.

[0884] Step 3:

[0885] The browser extension sends the saved query in JSON format to the server, which receives the data.

[0886] Input: Saved query

[0887] Output: The query is sent to the server.

[0888] Step 4:

[0889] The server analyzes the query it receives, understands the content of the query, and performs optimizations to the data, such as adding formatting and additional information to provide a detailed and informative answer.

[0890] Input: The query received by the server

[0891] Output: Optimized queries with formatting and additional information to elicit detailed and informative answers

[0892] Step 5:

[0893] The server returns the optimized query in JSON format to the device, which receives the data.

[0894] Input: Optimized query

[0895] Output: The optimized query is sent back to the device.

[0896] Step 6:

[0897] The device reflects the optimized query in the original query input field, and the browser extension sends the query again to the generation AI.

[0898] Input: Optimized query

[0899] Output: The query sent to the generation AI

[0900] Step 7:

[0901] Based on the optimized query, the generative AI generates detailed and accurate route and destination information, which is then sent back to the device and provided to the user.

[0902] Input: The optimized query sent to the generative AI

[0903] Output: Highly accurate route information generated by the generative AI

[0904] Step 8:

[0905] The user can view the detailed route information generated and use the self-driving vehicle based on this information.

[0906] Input: Route information returned from the generation AI

[0907] Output: High-quality route information available to users

[0908] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0909] The present invention combines a system that uses a generative AI to process queries entered by users and sends optimized prompts to the generative AI with an emotion engine that recognizes the user's emotions, thereby optimizing the generative AI's responses according to the user's emotional state and improving the accuracy of the responses.

[0910] First, when a user uses generative AI, a browser extension is used as a means of receiving the query they enter. When the user enters a query and clicks the "Send" button, the browser extension temporarily suppresses the query and analyzes the user's emotional state along with the query.

[0911] The server then receives the query and emotion data sent from the device. The server analyzes the received query and uses an emotion engine to recognize the user's emotion. It then optimizes the query according to the user's emotional state. For example, if the user is dissatisfied, the server adds formatting and additional information to prompt the AI ​​to provide a more polite and detailed response.

[0912] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[0913] Based on optimized prompts, the AI ​​generates detailed and highly accurate information that takes into account the user's emotional state, allowing users to obtain high-quality information that is tailored to their emotions without any effort.

[0914] Specific examples

[0915] 1. User enters question to the generative AI:

[0916] A user types, "What's the latest weather forecast?"

[0917] Device: The browser extension collects emotional data from the user's input, facial expressions, voice, etc., and temporarily prevents it from being sent.

[0918] 2. Send the query and sentiment data to the server:

[0919] Terminal: Send the user's query and sentiment data in JSON format to "http: / / localhost:5000 / optimize".

[0920] 3. The server receives the query and emotion data:

[0921] Server: Receives JSON data and extracts prompt string and emotion data.

[0922] 4. Recognize user emotions:

[0923] Server: Analyzes the user's emotions using an emotion engine and recognizes their emotional state, for example, "dissatisfied" or "confused."

[0924] 5. Optimize the prompt:

[0925] Server: In addition to the entered prompt, add formatting and additional information that corresponds to the user's emotional state. Example: "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0926] 6. Send optimized prompts back to the device:

[0927] Server: Returns the optimized prompt in JSON format.

[0928] 7. Your device will receive an optimized prompt:

[0929] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[0930] 8. Generative AI returns highly accurate results:

[0931] Generative AI: Generate detailed, emotion-aware weather forecasts based on optimized prompts.

[0932] 9. User sees the results:

[0933] User: Check the detailed weather forecast returned by the generative AI. For example, "Today's weather is sunny, with a temperature of 23 degrees and calm winds."

[0934] This implementation allows users to interact with the AI ​​through prompts tailored to their emotional state, resulting in more accurate information. This system increases user satisfaction and significantly improves the AI ​​user experience.

[0935] The processing flow will be explained below.

[0936] Step 1:

[0937] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[0938] Step 2:

[0939] The device (browser extension) detects when the user enters a query and clicks the send button. At this time, it temporarily stops sending the query and begins collecting the user's facial expression and voice data.

[0940] Step 3:

[0941] The device analyzes the user's emotion data collected along with the query, converts the query and emotion data into JSON format, and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[0942] Step 4:

[0943] The server receives the request sent from the device. The received data is in JSON format, with the user's query in the "prompt" key and the user's emotion data in the "emotion" key.

[0944] Step 5:

[0945] The server analyzes the received query and uses an emotion engine to recognize the user's emotion. For example, data indicating that the user is feeling dissatisfied or confused may be detected.

[0946] Step 6:

[0947] The server then optimizes the query based on the user's perceived emotion, adding additional information and formatting based on the user's emotional state, such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0948] Step 7:

[0949] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[0950] Step 8:

[0951] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[0952] Step 9:

[0953] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[0954] Step 10:

[0955] Generative AI generates detailed and highly accurate information based on optimization prompts received from the device, such as a weather forecast with thoughtful and emotionally sensitive explanations.

[0956] Step 11:

[0957] The user receives and confirms detailed, emotionally relevant answers returned by the generative AI, including friendly messages such as, "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm. Have a nice day."

[0958] Through this series of processes, users can effectively utilize generative AI and automatically obtain highly accurate information in line with their own emotional state.

[0959] Example 2

[0960] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0961] When using generative AI models, conventional systems process queries without considering the user's emotional state, resulting in problems such as reduced accuracy of answers and reduced satisfaction. Furthermore, simply sending the query entered by the user to the generative AI as is often insufficient to obtain the answer the user expects. Therefore, there is a need for a system that can generate prompts optimized according to the user's emotional state and provide more accurate information.

[0962] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a query entered by a user, means for collecting the received query and the user's emotion data, means for transmitting the collected query and emotion data to the server, means for analyzing the query and emotion data received by the server and recognizing the user's emotion, means for optimizing the query based on the recognized emotion, means for transmitting the optimized query to the generating AI, and means for providing the user with information returned from the generating AI. This makes it possible to transmit an optimized prompt that reflects the user's emotional state to the generating AI, thereby enabling more accurate information acquisition.

[0963] "User" means a terminal user who uses the system to input a query and obtain information.

[0964] A "query" refers to a question or request that a user inputs to a generative AI model.

[0965] "Emotional data" refers to information about a user's emotional state collected through facial expressions, voice, text analysis, etc. when entering information.

[0966] A "browser extension" is a software program that can be added to a web browser to extend its functionality, and acts to intercept user queries and collect emotional data.

[0967] "Server" means a device or system that receives and analyzes data submitted by a user, and creates and transmits optimized prompts to a generative AI model.

[0968] An "emotion engine" is software or algorithm that analyzes collected emotional data and recognizes the user's emotions.

[0969] A "prompt" is a message containing a query and additional information optimized to be sent to a generative AI model.

[0970] A "generative AI model" is an artificial intelligence system that generates appropriate answers or information for a user based on the prompts it receives.

[0971] The "JSON format" is a text format used to structure data, and is designed to be easy to read for both humans and machines.

[0972] "Information provision means" is a general term for methods and devices for displaying and providing information returned from a generative AI model to a user.

[0973] The present invention relates to a system that optimizes a query entered by a user using a generative AI model and obtains highly accurate information from the generative AI model according to the user's emotional state. The system includes the steps of entering a user query, collecting emotional data, and optimizing prompts based on the emotion.

[0974] System Overview

[0975] The system includes the following components:

[0976] 1. A browser extension for users to enter queries

[0977] 2. A device to collect queries and emotion data and send them to the server.

[0978] 3. A server that analyzes the received data and generates optimized prompts using an emotion engine to recognize emotions.

[0979] 4. A means to send optimized prompts to the generative AI model and provide the resulting information to the user.

[0980] Hardware and software used

[0981] Terminal: A personal computer or smartphone that serves as the user interface

[0982] Browser extension: Software that receives user queries and collects sentiment data (e.g., Google Chrome extension)

[0983] Server: A central processing unit for analyzing data and optimizing prompts (e.g., Apache server, Nginx server)

[0984] Emotion engine: Software for analyzing emotion data (e.g., Microsoft Azure's Sentiment Analysis API, OpenCV)

[0985] Generative AI model: An artificial intelligence system that generates information based on optimized prompts (e.g., the GPT series)

[0986] Program processing

[0987] The system begins by having the user enter a query through the browser extension. When the user enters a query and presses the submit button, the browser extension temporarily suppresses the query and simultaneously collects emotional data from the user's facial expressions and voice. The collected data is converted into JSON format and sent to the server.

[0988] The server parses the received JSON data and extracts the query and emotion data. It then uses the emotion engine to analyze and recognize the user's emotional state (e.g., "frustrated," "confused," "excited," etc.) and generates an optimized prompt based on this. For example, if the user is frustrated, the prompt will be optimized to the format "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[0989] The optimized prompt is sent to a generative AI model, which generates highly accurate information based on the prompt. The generated information is sent back to the browser extension via the server and provided to the user. For example, a user can type "What is the latest weather forecast?" and receive a detailed weather forecast such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm."

[0990] This embodiment makes it possible to send prompts to the generative AI model that are optimized to suit the user's emotional state, allowing the user to efficiently obtain highly accurate information, improving user satisfaction and significantly improving the usability of the generative AI model.

[0991] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0992] Step 1:

[0993] A user enters a query. Specifically, the user uses the browser extension to type "What is the latest weather forecast?" and presses the submit button. The input is a query (a question or request). The output is a temporary suppression of the query by the browser extension.

[0994] Step 2:

[0995] The device temporarily suppresses the user's queries while simultaneously collecting emotion data from the user's facial expressions and voice. For example, OpenCV is used to analyze the user's facial expressions, and emotion data is collected using an emotion recognition API. The input is the user's query and the user's facial expressions and voice. The output is JSON-formatted data containing the query and emotion data.

[0996] Step 3:

[0997] The query and emotion data collected by the device is converted into JSON format and sent to the server. The input is local data including the query and emotion data. The output is an HTTP POST request to "http: / / localhost:5000 / optimize".

[0998] Step 4:

[0999] Parse the JSON data received by the server and extract the query string and emotion data. For example, get the data as data = request.get_json() and save it as query = data['query'] and emotion = data['emotion'], respectively. The input is the JSON data received by the server. The output is the query string and emotion data.

[1000] Step 5:

[1001] The server uses an emotion engine to analyze and recognize the user's emotions. Specifically, it uses emotion recognition software to determine emotional states such as "frustrated" or "confused." The input is the query and emotion data. The output is the recognized emotional state.

[1002] Step 6:

[1003] The server generates an optimized prompt based on the recognized emotional state. For example, if the recognized emotion is "unhappy," it generates a prompt such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?" The input is the recognized emotional state and the query string. The output is the optimized prompt.

[1004] Step 7:

[1005] The server converts the optimized prompt into JSON format and sends it back to the device. The input is the optimized prompt. The output is the JSON response.

[1006] Step 8:

[1007] The optimized prompt received by the device is reflected in the original query input field and sent to the generative AI model. Specifically, the optimized prompt is set in the query input field and reflected in the form document.getElementById("query-input").value = response.optimized_prompt. The input is the optimized prompt. The output is the transmission of the optimized prompt to the generative AI model.

[1008] Step 9:

[1009] The generative AI model generates information based on the optimized prompt. The generative AI model generates detailed information such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm." The input is the optimized prompt. The output is the generated, highly accurate information.

[1010] Step 10:

[1011] The user checks the detailed weather forecast returned by the generative AI model in a browser. Specifically, the results from the generative AI model are displayed in the browser's display area. The input is detailed information from the generative AI model. The output is information provided to the user.

[1012] (Application example 2)

[1013] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1014] Conventional generative AI systems are unable to properly consider the emotional nuances of user queries, resulting in answers that are unsatisfactory for users. Furthermore, in services like food delivery, responses that do not properly consider emotions in response to user reviews and feedback have been difficult to generate, making it difficult to improve service quality. To address these issues, a system that can analyze a user's emotional state and generate optimized prompts accordingly is needed.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1016] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to the generating AI, means for analyzing the emotional state of the user, means for optimizing the query based on the emotional state, and means for generating a prompt according to the emotional state. As a result, an optimized prompt is generated taking the user's emotions into consideration, so that the generated AI response meets the user's expectations, thereby improving satisfaction.

[1017] "User" means an individual who utilizes the System to enter queries and obtain answers.

[1018] A "query" is a question or request that a user enters into a system.

[1019] "Receiving" means acquiring a query entered by a user within the system.

[1020] "Optimization" is the process of transforming a user's query into a better prompt.

[1021] "Generative AI" is artificial intelligence that generates answers based on input prompts.

[1022] "Sending" refers to sending optimized queries and prompts to other systems or AI.

[1023] "Emotional state" refers to a user's current psychological state or feelings.

[1024] "Analysis" refers to assessing the user's emotional state using an emotion engine or similar.

[1025] A "prompt" is an instruction that tells the generating AI how to respond.

[1026] A "browser extension" is a software feature added to a web browser that temporarily suppresses user queries.

[1027] "Format" refers to the structure or form of a query or prompt.

[1028] "Additional information" is supplemental information provided along with a query or prompt.

[1029] "Food ordering" refers to the act of a user ordering food using a food delivery system.

[1030] A "review" is an evaluation or opinion entered by a user about a service or product.

[1031] A "system" is a set of devices and software that comprehensively executes these means.

[1032] This invention is a system that analyzes user-entered queries, generates optimized prompts based on the user's emotional state, and sends them to a generative AI, which is particularly effective for user reviews and comments in food delivery services.

[1033] When a user orders food through a food delivery app, they can enter queries and reviews about the order, which are then temporarily suppressed through a browser extension that can be installed on popular browsers such as Google Chrome and Firefox.

[1034] The device sends the user's input query and emotion data to the server. The emotion data is acquired using tools such as facial expression recognition and voice analysis. For example, an emotion engine called "sentiment_analysis" is used for emotion analysis.

[1035] The server receives this data and analyzes the query and emotion data. It then optimizes the query according to the user's emotion. This is done using the "prompt_optimization" engine, which recognizes emotional states such as "confused" or "frustrated" and generates corresponding prompts. For example, for a user with the emotion "confused," the prompt format is "Please provide a detailed and calming answer to the following question:"

[1036] The optimized prompt is sent back to the device in JSON format. The device then uses this to reflect the optimized prompt in the user's query input field and sends it to the generation AI. Based on this optimized prompt, the generation AI generates detailed and highly accurate information that takes the user's emotional state into account.

[1037] For example, if a user inputs "This dish was cold when it arrived. Why?" and the emotional state of "confused" is detected, the optimized prompt will be "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?" Based on this prompt, the generative AI will generate a detailed and reassuring answer and provide it to the user.

[1038] This system allows users to receive answers that reflect their feelings, improving satisfaction with food delivery services, and also allows service providers to better process user feedback and improve the quality of their services.

[1039] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1040] Step 1:

[1041] When a user orders food through a food delivery app, they enter queries and reviews about the order, which are captured using a temporary suppression feature in the browser extension.

[1042] Input: User query (e.g., "This dish was cold when it arrived. Why?")

[1043] Output: Temporarily suppressed queries and sentiment data ready to send

[1044] Step 2:

[1045] The device sends the query and emotion data entered by the user to the server. The emotion data is obtained using facial expression recognition and voice analysis tools.

[1046] Input: Temporarily suppressed query and emotion data (e.g., user facial expression data, voice data)

[1047] Output: Sends JSON format data to the server

[1048] Step 3:

[1049] The server receives the user's query and emotion data. When receiving the data, it extracts the prompt string and emotion data from the JSON data.

[1050] Input: JSON format data sent from the terminal

[1051] Output: Extracted prompt string and emotion data

[1052] Step 4:

[1053] The server uses an emotion engine to analyze the user's emotional state. For example, the emotion analysis engine "sentiment_analysis" is used to recognize emotional states such as "confusion" and "dissatisfaction."

[1054] Input: Extracted emotion data (e.g., user facial expressions and voice data)

[1055] Output: Perceived emotional state (e.g., "confused")

[1056] Step 5:

[1057] To optimize queries based on emotional state, the server uses the prompt optimization engine "prompt_optimization" to generate optimized prompts according to emotions.

[1058] Input: Extracted prompt string and recognized emotional state (e.g., "This dish was cold when it arrived. Why?" and "Confused")

[1059] Output: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[1060] Step 6:

[1061] The server sends the optimized prompt back to the device in JSON format, which presents the optimized prompt to the user.

[1062] Input: Optimized prompts

[1063] Output: Send to the terminal as JSON format data

[1064] Step 7:

[1065] The device reflects the optimized prompt received in the user's query input field and sends the optimized prompt to the generation AI.

[1066] Input: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[1067] Output: Send to the generation AI

[1068] Step 8:

[1069] Based on optimized prompts, the generative AI generates detailed and highly accurate information that takes into account the user's emotional state.

[1070] Input: Optimized prompts

[1071] Output: Generated, highly accurate answer (e.g., "The food arrived cold. One possible reason is that the temperature was not properly controlled during the delivery process. Please rest assured that we will take extra care to ensure temperature control the next time you order.")

[1072] Step 9:

[1073] The user checks the detailed answer returned by the generated AI and is satisfied with its content.

[1074] Input: Generated, highly accurate answers

[1075] Output: High user satisfaction (e.g., "I was relieved to receive a detailed explanation of why my food arrived cold and what steps they would take.")

[1076] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1077] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1078] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1079] [Fourth embodiment]

[1080] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1081] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1082] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1083] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1084] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1085] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1086] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1087] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1088] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1089] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1090] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1091] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1092] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1093] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[1094] First, when a user uses the generative AI, a browser extension is used as a means of receiving the entered query. When the user enters a query and clicks the "Submit" button, the query is temporarily suppressed and sent to the server.

[1095] The server then analyzes and optimizes the received query, adding formatting and additional information to the query to encourage a detailed and informative answer. This optimization process involves adding statements such as "Please provide a detailed and informative answer to the following question:" to help the AI ​​generate the right answer.

[1096] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[1097] Generative AI generates detailed and accurate information based on optimized prompts, allowing users to obtain high-quality information without any effort.

[1098] Specific examples

[1099] 1. User enters question to the generative AI:

[1100] For example: "What is the latest weather forecast?"

[1101] On your device: The browser extension detects your input and temporarily prevents it from being sent.

[1102] 2. Send the query to the server:

[1103] Terminal: Send the user's query in JSON format to "http: / / localhost:5000 / optimize".

[1104] 3. The server receives the query:

[1105] Server: Receives the JSON data and extracts the prompt string.

[1106] 4. Optimize the prompt:

[1107] Server: Transforms the input prompt into a detailed and informative answer. For example, "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[1108] 5. Send optimized prompts back to the device:

[1109] Server: Returns the optimized prompt in JSON format.

[1110] 6. Your device will receive an optimized prompt:

[1111] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[1112] 7. Generative AI returns highly accurate results:

[1113] Generative AI: Generate detailed weather forecasts based on optimized prompts.

[1114] 8. User sees the results:

[1115] User: Check the detailed weather forecast returned by the generated AI.

[1116] This embodiment allows users to automatically generate optimal prompts and obtain highly accurate information from the generation AI. This system frees users from the trouble of creating prompts, making the use of the generation AI easier and more effective.

[1117] The processing flow will be explained below.

[1118] Step 1:

[1119] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[1120] Step 2:

[1121] The device (browser extension) detects the user's query input and clicks the send button, and temporarily blocks transmission at this stage.

[1122] Step 3:

[1123] The terminal converts the query entered by the user into JSON format and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[1124] Step 4:

[1125] The server receives the request sent from the device. The received data is in JSON format and the "prompt" key contains the user's query.

[1126] Step 5:

[1127] The server analyzes the received query to optimize it, specifically by adding phrases such as "Please provide a detailed and informative answer to the following question:" to the user's query, transforming the prompt into a more detailed and informative one.

[1128] Step 6:

[1129] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[1130] Step 7:

[1131] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[1132] Step 8:

[1133] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[1134] Step 9:

[1135] Generative AI generates detailed and highly accurate information based on optimized prompts received from the device, such as detailed answers about the latest weather forecast.

[1136] Step 10:

[1137] The user receives and confirms the detailed answers returned by the generating AI, allowing the user to obtain highly accurate and detailed information without any hassle.

[1138] This series of processes enables users to effectively utilize generative AI and realize a system that can automatically obtain highly accurate information.

[1139] Example 1

[1140] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1141] When inputting a question into a generative AI, users have to take the time to create an appropriate prompt. In addition, because the accuracy of the answer obtained from the generative AI depends on the prompt entered by the user, there is a problem that the quality of the information obtained varies depending on the ability to create an optimal prompt. As a result, it can take time and effort for users to obtain a satisfactory answer.

[1142] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1143] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to an AI that generates it, means for detecting the user's input and temporarily suppressing transmission, means for converting the received query into JSON format and sending it to the server, and means for receiving an optimized prompt from the server and reflecting the prompt. This relieves the user from the trouble of creating prompts and enables them to quickly obtain highly accurate and detailed answers from the AI ​​that generates them.

[1144] "User" refers to a person who uses this system to input queries into the generating AI and obtain information.

[1145] A "query" refers to a question or request that a user sends to a generative AI.

[1146] "Means for receiving" refers to the function of detecting an input query and temporarily storing it as data.

[1147] "Means to optimize" refers to the ability to analyze incoming queries and convert them into a format that makes it easier for the generative AI to generate accurate answers.

[1148] "Means for sending" refers to the function of sending optimized queries to the generation AI and receiving the results.

[1149] "Means to temporarily suppress transmission" refers to a function that temporarily stops transmission of a query when the user enters it, rather than sending the query immediately.

[1150] "Means for converting to JSON format" refers to a function for converting a received query into JSON format data.

[1151] "Browser extension" refers to additional software for a web browser that detects user query input and temporarily blocks that input.

[1152] An "optimized prompt" is a statement that transforms a query into a form that is more likely to elicit a detailed and effective answer.

[1153] "Generative AI" refers to an AI system that generates detailed and accurate information based on input prompts.

[1154] "Means of reflection" refers to the function that reflects the optimization prompt returned from the server in the user's input field and sends it to the generating AI.

[1155] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, allowing the user to obtain more accurate information. This system is implemented through the following steps.

[1156] First, when a user uses the generative AI, they use a browser extension to input a query. This browser extension temporarily suppresses the query when the user inputs the query and clicks the "Submit" button. The suppressed query is temporarily stored in the device's storage.

[1157] Next, the device's browser extension converts the stored query into JSON format and sends it to the server. Specifically, it sends the query to the URL "http: / / localhost:5000 / optimize". At this time, the data sent will be in a format such as {"query": "Please tell me the latest weather forecast"}.

[1158] When the server receives this request, it extracts the query string from the JSON data. For example, the extracted query might be "What is the latest weather forecast?". The server then analyzes the query and optimizes it for the most appropriate prompt. This optimization process involves adding statements to the query that request a detailed and informative answer. Specifically, the query might be converted to a format such as "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[1159] The optimized prompt is then sent back to the device as JSON data from the server, such as {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}.

[1160] The device's browser extension receives the returned JSON data and extracts an optimized prompt, which is then automatically reflected in the user's query input field and sent back to the AI ​​generator, which then generates a detailed and accurate answer.

[1161] The answer from the generation AI is displayed in the user's browser, allowing the user to check the results. This system frees users from the hassle of creating prompts, allowing them to quickly and easily obtain highly accurate information from the generation AI. As an example of a specific prompt, when the user enters "Tell me about recent technology trends," the optimized prompt will be "Please provide a detailed and informative answer to the following question: Tell me about recent technology trends."

[1162] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1163] Step 1:

[1164] The user enters a query.

[1165] Input: The user types a question or request into the generated AI. Example: "What is the latest weather forecast?"

[1166] How it works: A user enters a query into the input field in their browser and clicks the "Submit" button.

[1167] Output: The query entered is detected by the browser extension.

[1168] Step 2:

[1169] The browser extension temporarily suppresses the query.

[1170] Input: The query for which the user clicked the submit button.

[1171] What it does: The extension detects the query and temporarily stores it in storage, but does not send the query yet.

[1172] Output: The suppressed queries are saved to storage.

[1173] Step 3:

[1174] The query is converted to JSON format and sent to the server.

[1175] Input: The query saved in the browser extension.

[1176] What it does: Converts a query into JSON format, e.g. {"query": "What is the latest weather forecast?"}. Sends this data to "http: / / localhost:5000 / optimize".

[1177] Output: JSON formatted data is sent to the server.

[1178] Step 4:

[1179] The server receives the query.

[1180] Input: JSON data sent from the terminal.

[1181] How it works: The server extracts the query from the received JSON data. Example: "query": "What is the latest weather forecast?"

[1182] Output: The extracted queries are made available on the server.

[1183] Step 5:

[1184] Optimize prompts.

[1185] Input: The query extracted on the server.

[1186] What it does: Parses the query and appends a statement that requests a detailed and informative answer. Example: "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"

[1187] Output: An optimized prompt is generated.

[1188] Step 6:

[1189] Sends an optimized prompt back to the terminal.

[1190] Input: Optimized prompts.

[1191] What it does: Converts the optimized prompt into JSON format and sends it back to the device. Example: {"optimized_query": "Please provide a detailed and informative answer to the following question: What is the latest weather forecast?"}

[1192] Output: An optimized prompt in JSON format is sent to the terminal.

[1193] Step 7:

[1194] Your device will receive an optimized prompt.

[1195] Input: The optimization prompt JSON data returned by the server.

[1196] How it works: The device browser extension receives the JSON data, extracts the optimization prompt, and reflects it in the original query input field.

[1197] Output: An optimized prompt is displayed in the query input field.

[1198] Step 8:

[1199] Generative AI produces highly accurate results.

[1200] Input: Optimized prompts.

[1201] How it works: Generative AI generates detailed and accurate answers based on optimized prompts.

[1202] Output: Detailed information is generated from the generative AI.

[1203] Step 9:

[1204] The user checks the results.

[1205] Input: The answer returned by the generation AI.

[1206] How it works: The user sees the generated answer in their browser.

[1207] Output: The user is satisfied with the highly accurate answer.

[1208] (Application example 1)

[1209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1210] Conventional information generation systems using generative AI simply send queries entered by users, making it difficult to obtain optimal answers. Furthermore, automated driving vehicles are required to obtain highly accurate information on the optimal route to a destination, but this also presents similar problems. Therefore, there is a need for a system that allows users to easily obtain high-quality route information.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1212] In this invention, the server includes means for receiving a query input by a user, means for optimizing the received query, means for transmitting the optimized query to the generating artificial intelligence, and means for generating optimal route information for an autonomously driven vehicle. This enables a user to optimize a query and transmit it to the generating artificial intelligence, thereby easily obtaining high-quality route information for an autonomously driven vehicle.

[1213] "User" means any person or entity that utilizes the System to enter a query and obtain optimized information.

[1214] A "query" refers to a question or request that a user enters when requesting information from a generative artificial intelligence.

[1215] "Means for receiving" refers to a function or device that allows the system to obtain a query entered by a user.

[1216] "Optimizing means" refers to a process or device that analyzes a user-entered query and adds formatting and additional information to elicit a detailed and informative answer.

[1217] "Generative artificial intelligence" refers to advanced computer programs and systems that generate answers and information in natural language based on user queries.

[1218] "Means for sending" refers to a communication means or device for sending the optimized query to the generating artificial intelligence.

[1219] An "autonomous vehicle" refers to a vehicle equipped with technology that allows it to drive autonomously without human operation.

[1220] "Optimal route information" refers to information that shows the most efficient and safe route between the user's specified departure and destination, taking into account traffic conditions and other factors.

[1221] A "system" refers to a set of related components and processes combined to achieve a specific function or purpose.

[1222] The present invention relates to a system that automatically generates optimal prompts when a user inputs a query to a generation AI and sends them to the generation AI, thereby enabling the user to obtain more accurate information.

[1223] First, a user inputs information about the destination and route of the autonomous vehicle using a device such as a smartphone, smart glasses, or an in-vehicle display. This input information is captured as a query and temporarily suppressed by the browser extension.

[1224] The suppressed query is then sent to the server in JSON format. The server receives this JSON data, parses the query, and optimizes the received query to elicit a detailed and informative answer. For example, it adds a statement such as "Please provide a detailed and informative answer to the following question:". This optimization makes it easier for the generative AI to generate accurate answers.

[1225] The optimized prompt is then sent back to the device in JSON format. The device then reflects the received optimized prompt in the original query input field and finally sends it to the generation AI. The browser extension helps with this process. The generation AI generates detailed and accurate route and destination information based on the optimized prompt. This allows users to obtain efficient and safe route information.

[1226] As a concrete example, if a user types "What is the best route from Shinjuku to Haneda Airport?", the query is temporarily blocked by the browser extension and sent to the server. The server analyzes and optimizes the query, generating a prompt that reads, "Please provide a detailed and informative answer to the following question: What is the best route from Shinjuku to Haneda Airport?" This is then analyzed by the generative AI, and highly accurate route guidance is provided to the user.

[1227] The hardware used includes smartphones, smart glasses, in-vehicle displays, and servers, while the software used includes browser extensions and generative AI APIs (e.g., GCP and Azure generative AI services).

[1228] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1229] Step 1:

[1230] A user inputs information about a destination or intermediate destination using a device such as a smartphone, smart glasses, or an in-car display. This information is captured as a query.

[1231] Input: A query entered by a user (e.g., "What is the best route from Shinjuku to Haneda Airport?")

[1232] Output: The query is retrieved.

[1233] Step 2:

[1234] A browser extension in your device will temporarily prevent this query from being sent. This query will be temporarily saved.

[1235] Input: Retrieved query

[1236] Output: The query is suppressed and temporarily saved.

[1237] Step 3:

[1238] The browser extension sends the saved query in JSON format to the server, which receives the data.

[1239] Input: Saved query

[1240] Output: The query is sent to the server.

[1241] Step 4:

[1242] The server analyzes the query it receives, understands the content of the query, and performs optimizations to the data, such as adding formatting and additional information to provide a detailed and informative answer.

[1243] Input: The query received by the server

[1244] Output: Optimized queries with formatting and additional information to elicit detailed and informative answers

[1245] Step 5:

[1246] The server returns the optimized query in JSON format to the device, which receives the data.

[1247] Input: Optimized query

[1248] Output: The optimized query is sent back to the device.

[1249] Step 6:

[1250] The device reflects the optimized query in the original query input field, and the browser extension sends the query again to the generation AI.

[1251] Input: Optimized query

[1252] Output: The query sent to the generation AI

[1253] Step 7:

[1254] Based on the optimized query, the generative AI generates detailed and accurate route and destination information, which is then sent back to the device and provided to the user.

[1255] Input: The optimized query sent to the generative AI

[1256] Output: Highly accurate route information generated by the generative AI

[1257] Step 8:

[1258] The user can view the detailed route information generated and use the self-driving vehicle based on this information.

[1259] Input: Route information returned from the generation AI

[1260] Output: High-quality route information available to users

[1261] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1262] The present invention combines a system that uses a generative AI to process queries entered by users and sends optimized prompts to the generative AI with an emotion engine that recognizes the user's emotions, thereby optimizing the generative AI's responses according to the user's emotional state and improving the accuracy of the responses.

[1263] First, when a user uses generative AI, a browser extension is used as a means of receiving the query they enter. When the user enters a query and clicks the "Send" button, the browser extension temporarily suppresses the query and analyzes the user's emotional state along with the query.

[1264] The server then receives the query and emotion data sent from the device. The server analyzes the received query and uses an emotion engine to recognize the user's emotion. It then optimizes the query according to the user's emotional state. For example, if the user is dissatisfied, the server adds formatting and additional information to prompt the AI ​​to provide a more polite and detailed response.

[1265] The optimized prompt is sent back to the browser extension in JSON format, which then replaces the user's query field with the optimized prompt and sends it back to the generation AI.

[1266] Based on optimized prompts, the AI ​​generates detailed and highly accurate information that takes into account the user's emotional state, allowing users to obtain high-quality information that is tailored to their emotions without any effort.

[1267] Specific examples

[1268] 1. User enters question to the generative AI:

[1269] A user types, "What's the latest weather forecast?"

[1270] Device: The browser extension collects emotional data from the user's input, facial expressions, voice, etc., and temporarily prevents it from being sent.

[1271] 2. Send the query and sentiment data to the server:

[1272] Terminal: Send the user's query and sentiment data in JSON format to "http: / / localhost:5000 / optimize".

[1273] 3. The server receives the query and emotion data:

[1274] Server: Receives JSON data and extracts prompt string and emotion data.

[1275] 4. Recognize user emotions:

[1276] Server: Analyzes the user's emotions using an emotion engine and recognizes their emotional state, for example, "dissatisfied" or "confused."

[1277] 5. Optimize the prompt:

[1278] Server: In addition to the entered prompt, add formatting and additional information that corresponds to the user's emotional state. Example: "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[1279] 6. Send optimized prompts back to the device:

[1280] Server: Returns the optimized prompt in JSON format.

[1281] 7. Your device will receive an optimized prompt:

[1282] Terminal: Reflect the optimized prompt in the original input field and send it to the generating AI.

[1283] 8. Generative AI returns highly accurate results:

[1284] Generative AI: Generate detailed, emotion-aware weather forecasts based on optimized prompts.

[1285] 9. User sees the results:

[1286] User: Check the detailed weather forecast returned by the generative AI. For example, "Today's weather is sunny, with a temperature of 23 degrees and calm winds."

[1287] This implementation allows users to interact with the AI ​​through prompts tailored to their emotional state, resulting in more accurate information. This system increases user satisfaction and significantly improves the AI ​​user experience.

[1288] The processing flow will be explained below.

[1289] Step 1:

[1290] The user inputs a question to the AI ​​generator, such as a query like "What is the latest weather forecast?"

[1291] Step 2:

[1292] The device (browser extension) detects when the user enters a query and clicks the send button. At this time, it temporarily stops sending the query and begins collecting the user's facial expression and voice data.

[1293] Step 3:

[1294] The device analyzes the user's emotion data collected along with the query, converts the query and emotion data into JSON format, and sends it to the provided server endpoint (e.g., "http: / / localhost:5000 / optimize").

[1295] Step 4:

[1296] The server receives the request sent from the device. The received data is in JSON format, with the user's query in the "prompt" key and the user's emotion data in the "emotion" key.

[1297] Step 5:

[1298] The server analyzes the received query and uses an emotion engine to recognize the user's emotion. For example, data indicating that the user is feeling dissatisfied or confused may be detected.

[1299] Step 6:

[1300] The server then optimizes the query based on the user's perceived emotion, adding additional information and formatting based on the user's emotional state, such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[1301] Step 7:

[1302] The server generates an optimized prompt as a response in JSON format and sends it back to the device.

[1303] Step 8:

[1304] The terminal receives the returned optimized prompt and reflects it in the user's original query input field, automatically re-entering the optimized prompt in the user's input field.

[1305] Step 9:

[1306] The terminal then sends the optimized prompt to the generating AI again, following the normal sending procedure.

[1307] Step 10:

[1308] Generative AI generates detailed and highly accurate information based on optimization prompts received from the device, such as a weather forecast with thoughtful and emotionally sensitive explanations.

[1309] Step 11:

[1310] The user receives and confirms detailed, emotionally relevant answers returned by the generative AI, including friendly messages such as, "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm. Have a nice day."

[1311] Through this series of processes, users can effectively utilize generative AI and automatically obtain highly accurate information in line with their own emotional state.

[1312] Example 2

[1313] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1314] When using generative AI models, conventional systems process queries without considering the user's emotional state, resulting in problems such as reduced accuracy of answers and reduced satisfaction. Furthermore, simply sending the query entered by the user to the generative AI as is often insufficient to obtain the answer the user expects. Therefore, there is a need for a system that can generate prompts optimized according to the user's emotional state and provide more accurate information.

[1315] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving a query entered by a user, means for collecting the received query and the user's emotion data, means for transmitting the collected query and emotion data to the server, means for analyzing the query and emotion data received by the server and recognizing the user's emotion, means for optimizing the query based on the recognized emotion, means for transmitting the optimized query to the generating AI, and means for providing the user with information returned from the generating AI. This makes it possible to transmit an optimized prompt that reflects the user's emotional state to the generating AI, thereby enabling more accurate information acquisition.

[1316] "User" means a terminal user who uses the system to input a query and obtain information.

[1317] A "query" refers to a question or request that a user inputs to a generative AI model.

[1318] "Emotional data" refers to information about a user's emotional state collected through facial expressions, voice, text analysis, etc. when entering information.

[1319] A "browser extension" is a software program that can be added to a web browser to extend its functionality, and acts to intercept user queries and collect emotional data.

[1320] "Server" means a device or system that receives and analyzes data submitted by a user, and creates and transmits optimized prompts to a generative AI model.

[1321] An "emotion engine" is software or algorithm that analyzes collected emotional data and recognizes the user's emotions.

[1322] A "prompt" is a message containing a query and additional information optimized to be sent to a generative AI model.

[1323] A "generative AI model" is an artificial intelligence system that generates appropriate answers or information for a user based on the prompts it receives.

[1324] The "JSON format" is a text format used to structure data, and is designed to be easy to read for both humans and machines.

[1325] "Information provision means" is a general term for methods and devices for displaying and providing information returned from a generative AI model to a user.

[1326] The present invention relates to a system that optimizes a query entered by a user using a generative AI model and obtains highly accurate information from the generative AI model according to the user's emotional state. The system includes the steps of entering a user query, collecting emotional data, and optimizing prompts based on the emotion.

[1327] System Overview

[1328] The system includes the following components:

[1329] 1. A browser extension for users to enter queries

[1330] 2. A device to collect queries and emotion data and send them to the server.

[1331] 3. A server that analyzes the received data and generates optimized prompts using an emotion engine to recognize emotions.

[1332] 4. A means to send optimized prompts to the generative AI model and provide the resulting information to the user.

[1333] Hardware and software used

[1334] Terminal: A personal computer or smartphone that serves as the user interface

[1335] Browser extension: Software that receives user queries and collects sentiment data (e.g., Google Chrome extension)

[1336] Server: A central processing unit for analyzing data and optimizing prompts (e.g., Apache server, Nginx server)

[1337] Emotion engine: Software for analyzing emotion data (e.g., Microsoft Azure's Sentiment Analysis API, OpenCV)

[1338] Generative AI model: An artificial intelligence system that generates information based on optimized prompts (e.g., the GPT series)

[1339] Program processing

[1340] The system begins by having the user enter a query through the browser extension. When the user enters a query and presses the submit button, the browser extension temporarily suppresses the query and simultaneously collects emotional data from the user's facial expressions and voice. The collected data is converted into JSON format and sent to the server.

[1341] The server parses the received JSON data and extracts the query and emotion data. It then uses the emotion engine to analyze and recognize the user's emotional state (e.g., "frustrated," "confused," "excited," etc.) and generates an optimized prompt based on this. For example, if the user is frustrated, the prompt will be optimized to the format "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?"

[1342] The optimized prompt is sent to a generative AI model, which generates highly accurate information based on the prompt. The generated information is sent back to the browser extension via the server and provided to the user. For example, a user can type "What is the latest weather forecast?" and receive a detailed weather forecast such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm."

[1343] This embodiment makes it possible to send prompts to the generative AI model that are optimized to suit the user's emotional state, allowing the user to efficiently obtain highly accurate information, improving user satisfaction and significantly improving the usability of the generative AI model.

[1344] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1345] Step 1:

[1346] A user enters a query. Specifically, the user uses the browser extension to type "What is the latest weather forecast?" and presses the submit button. The input is a query (a question or request). The output is a temporary suppression of the query by the browser extension.

[1347] Step 2:

[1348] The device temporarily suppresses the user's queries while simultaneously collecting emotion data from the user's facial expressions and voice. For example, OpenCV is used to analyze the user's facial expressions, and emotion data is collected using an emotion recognition API. The input is the user's query and the user's facial expressions and voice. The output is JSON-formatted data containing the query and emotion data.

[1349] Step 3:

[1350] The query and emotion data collected by the device is converted into JSON format and sent to the server. The input is local data including the query and emotion data. The output is an HTTP POST request to "http: / / localhost:5000 / optimize".

[1351] Step 4:

[1352] Parse the JSON data received by the server and extract the query string and emotion data. For example, get the data as data = request.get_json() and save it as query = data['query'] and emotion = data['emotion'], respectively. The input is the JSON data received by the server. The output is the query string and emotion data.

[1353] Step 5:

[1354] The server uses an emotion engine to analyze and recognize the user's emotions. Specifically, it uses emotion recognition software to determine emotional states such as "frustrated" or "confused." The input is the query and emotion data. The output is the recognized emotional state.

[1355] Step 6:

[1356] The server generates an optimized prompt based on the recognized emotional state. For example, if the recognized emotion is "unhappy," it generates a prompt such as "Please provide a detailed and reassuring answer to the following question: What is the latest weather forecast?" The input is the recognized emotional state and the query string. The output is the optimized prompt.

[1357] Step 7:

[1358] The server converts the optimized prompt into JSON format and sends it back to the device. The input is the optimized prompt. The output is the JSON response.

[1359] Step 8:

[1360] The optimized prompt received by the device is reflected in the original query input field and sent to the generative AI model. Specifically, the optimized prompt is set in the query input field and reflected in the form document.getElementById("query-input").value = response.optimized_prompt. The input is the optimized prompt. The output is the transmission of the optimized prompt to the generative AI model.

[1361] Step 9:

[1362] The generative AI model generates information based on the optimized prompt. The generative AI model generates detailed information such as "Today's weather is sunny, the temperature is 23 degrees, and the wind is calm." The input is the optimized prompt. The output is the generated, highly accurate information.

[1363] Step 10:

[1364] The user checks the detailed weather forecast returned by the generative AI model in a browser. Specifically, the results from the generative AI model are displayed in the browser's display area. The input is detailed information from the generative AI model. The output is information provided to the user.

[1365] (Application example 2)

[1366] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1367] Conventional generative AI systems are unable to properly consider the emotional nuances of user queries, resulting in answers that are unsatisfactory for users. Furthermore, in services like food delivery, responses that do not properly consider emotions in response to user reviews and feedback have been difficult to generate, making it difficult to improve service quality. To address these issues, a system that can analyze a user's emotional state and generate optimized prompts accordingly is needed.

[1368] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1369] In this invention, the server includes means for receiving a query entered by a user, means for optimizing the received query, means for sending the optimized query to the generating AI, means for analyzing the emotional state of the user, means for optimizing the query based on the emotional state, and means for generating a prompt according to the emotional state. As a result, an optimized prompt is generated taking the user's emotions into consideration, so that the generated AI response meets the user's expectations, thereby improving satisfaction.

[1370] "User" means an individual who utilizes the System to enter queries and obtain answers.

[1371] A "query" is a question or request that a user enters into a system.

[1372] "Receiving" means acquiring a query entered by a user within the system.

[1373] "Optimization" is the process of transforming a user's query into a better prompt.

[1374] "Generative AI" is artificial intelligence that generates answers based on input prompts.

[1375] "Sending" refers to sending optimized queries and prompts to other systems or AI.

[1376] "Emotional state" refers to a user's current psychological state or feelings.

[1377] "Analysis" refers to assessing the user's emotional state using an emotion engine or similar.

[1378] A "prompt" is an instruction that tells the generating AI how to respond.

[1379] A "browser extension" is a software feature added to a web browser that temporarily suppresses user queries.

[1380] "Format" refers to the structure or form of a query or prompt.

[1381] "Additional information" is supplemental information provided along with a query or prompt.

[1382] "Food ordering" refers to the act of a user ordering food using a food delivery system.

[1383] A "review" is an evaluation or opinion entered by a user about a service or product.

[1384] A "system" is a set of devices and software that comprehensively executes these means.

[1385] This invention is a system that analyzes user-entered queries, generates optimized prompts based on the user's emotional state, and sends them to a generative AI, which is particularly effective for user reviews and comments in food delivery services.

[1386] When a user orders food through a food delivery app, they can enter queries and reviews about the order, which are then temporarily suppressed through a browser extension that can be installed on popular browsers such as Google Chrome and Firefox.

[1387] The device sends the user's input query and emotion data to the server. The emotion data is acquired using tools such as facial expression recognition and voice analysis. For example, an emotion engine called "sentiment_analysis" is used for emotion analysis.

[1388] The server receives this data and analyzes the query and emotion data. It then optimizes the query according to the user's emotion. This is done using the "prompt_optimization" engine, which recognizes emotional states such as "confused" or "frustrated" and generates corresponding prompts. For example, for a user with the emotion "confused," the prompt format is "Please provide a detailed and calming answer to the following question:"

[1389] The optimized prompt is sent back to the device in JSON format. The device then uses this to reflect the optimized prompt in the user's query input field and sends it to the generation AI. Based on this optimized prompt, the generation AI generates detailed and highly accurate information that takes the user's emotional state into account.

[1390] For example, if a user inputs "This dish was cold when it arrived. Why?" and the emotional state of "confused" is detected, the optimized prompt will be "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?" Based on this prompt, the generative AI will generate a detailed and reassuring answer and provide it to the user.

[1391] This system allows users to receive answers that reflect their feelings, improving satisfaction with food delivery services, and also allows service providers to better process user feedback and improve the quality of their services.

[1392] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1393] Step 1:

[1394] When a user orders food through a food delivery app, they enter queries and reviews about the order, which are captured using a temporary suppression feature in the browser extension.

[1395] Input: User query (e.g., "This dish was cold when it arrived. Why?")

[1396] Output: Temporarily suppressed queries and sentiment data ready to send

[1397] Step 2:

[1398] The device sends the query and emotion data entered by the user to the server. The emotion data is obtained using facial expression recognition and voice analysis tools.

[1399] Input: Temporarily suppressed query and emotion data (e.g., user facial expression data, voice data)

[1400] Output: Sends JSON format data to the server

[1401] Step 3:

[1402] The server receives the user's query and emotion data. When receiving the data, it extracts the prompt string and emotion data from the JSON data.

[1403] Input: JSON format data sent from the terminal

[1404] Output: Extracted prompt string and emotion data

[1405] Step 4:

[1406] The server uses an emotion engine to analyze the user's emotional state. For example, the emotion analysis engine "sentiment_analysis" is used to recognize emotional states such as "confusion" and "dissatisfaction."

[1407] Input: Extracted emotion data (e.g., user facial expressions and voice data)

[1408] Output: Perceived emotional state (e.g., "confused")

[1409] Step 5:

[1410] To optimize queries based on emotional state, the server uses the prompt optimization engine "prompt_optimization" to generate optimized prompts according to emotions.

[1411] Input: Extracted prompt string and recognized emotional state (e.g., "This dish was cold when it arrived. Why?" and "Confused")

[1412] Output: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[1413] Step 6:

[1414] The server sends the optimized prompt back to the device in JSON format, which presents the optimized prompt to the user.

[1415] Input: Optimized prompts

[1416] Output: Send to the terminal as JSON format data

[1417] Step 7:

[1418] The device reflects the optimized prompt received in the user's query input field and sends the optimized prompt to the generation AI.

[1419] Input: Optimized prompts (e.g., "Please provide a detailed and calming answer to the following question: This dish was cold when it arrived. Why?")

[1420] Output: Send to the generation AI

[1421] Step 8:

[1422] Based on optimized prompts, the generative AI generates detailed and highly accurate information that takes into account the user's emotional state.

[1423] Input: Optimized prompts

[1424] Output: Generated, highly accurate answer (e.g., "The food arrived cold. One possible reason is that the temperature was not properly controlled during the delivery process. Please rest assured that we will take extra care to ensure temperature control the next time you order.")

[1425] Step 9:

[1426] The user checks the detailed answer returned by the generated AI and is satisfied with its content.

[1427] Input: Generated, highly accurate answers

[1428] Output: High user satisfaction (e.g., "I was relieved to receive a detailed explanation of why my food arrived cold and what steps they would take.")

[1429] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1430] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1431] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1432] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1433] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1434] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1435] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1436] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1437] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1438] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1439] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1440] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1441] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1442] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1443] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1444] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1445] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1446] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1447] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1448] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1449] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1450] The following is further disclosed regarding the above embodiment.

[1451] (Claim 1)

[1452] means for receiving a query entered by a user;

[1453] a means for optimizing the received query;

[1454] a means for sending the optimized query to a generative artificial intelligence;

[1455] A system including:

[1456] (Claim 2)

[1457] 10. The system of claim 1, further comprising: means for temporarily inhibiting a user from entering a query using a browser extension.

[1458] (Claim 3)

[1459] 10. The system of claim 1, further comprising: means for providing formatting and additional information to the optimized query.

[1460] "Example 1"

[1461] (Claim 1)

[1462] means for receiving a query entered by a user;

[1463] a means for optimizing the received query;

[1464] a means for sending the optimized query to an artificial intelligence for generation;

[1465] A means for detecting user input and temporarily suppressing transmission;

[1466] A means of converting received queries into JSON format and sending them to the server;

[1467] means for receiving the optimized prompt from the server and reflecting the prompt;

[1468] A system including:

[1469] (Claim 2)

[1470] 10. The system of claim 1, further comprising: means for temporarily inhibiting a user from entering a query using a browser extension.

[1471] (Claim 3)

[1472] 10. The system of claim 1, further comprising: means for providing formatting and additional information to the optimized query.

[1473] "Application Example 1"

[1474] (Claim 1)

[1475] means for receiving a query entered by a user;

[1476] a means for optimizing the received query;

[1477] a means for sending the optimized query to a generative artificial intelligence;

[1478] A means for generating optimal route information for an autonomous vehicle;

[1479] A system including:

[1480] (Claim 2)

[1481] 10. The system of claim 1, further comprising: means for temporarily inhibiting a user from entering a query using a browser extension.

[1482] (Claim 3)

[1483] 10. The system of claim 1, further comprising: means for providing formatting and additional information to the optimized query.

[1484] "Example 2: Combining Emotion Engines"

[1485] (Claim 1)

[1486] means for receiving a query entered by a user;

[1487] a means for collecting received query and user sentiment data;

[1488] means for transmitting the collected query and sentiment data to a server;

[1489] a means for analyzing the received query and emotion data by the server and recognizing the emotion of the user;

[1490] a means for optimizing queries based on the recognized sentiment;

[1491] a means for sending the optimized query to a generative artificial intelligence;

[1492] A means for providing the information returned from the generating artificial intelligence to the user;

[1493] A system including:

[1494] (Claim 2)

[1495] The system of claim 1, further comprising: means for temporarily suppressing a user's query input using a browser extension to collect emotion data.

[1496] (Claim 3)

[1497] 10. The system of claim 1, further comprising: means for providing formatting and additional information to the optimized query.

[1498] "Application example 2 when combining emotion engines"

[1499] (Claim 1)

[1500] means for receiving a query entered by a user;

[1501] a means for optimizing the received query;

[1502] a means for sending the optimized query to a generative artificial intelligence;

[1503] a means for analyzing the emotional state of a user;

[1504] a means for optimizing queries based on emotional state;

[1505] means for generating prompts according to an emotional state;

[1506] A system including:

[1507] (Claim 2)

[1508] 10. The system of claim 1, further comprising: means for temporarily inhibiting a user from entering a query using a browser extension.

[1509] (Claim 3)

[1510] 10. The system of claim 1, further comprising: means for providing formatting and additional information to the optimized query.

[1511] (Claim 4)

[1512] The system of claim 1, further comprising: means for receiving queries and reviews entered when ordering food.

[1513] (Claim 5)

[1514] The system of claim 4 further comprising: means for analyzing an emotional state based on queries and reviews entered when ordering food, and generating optimized prompts.

[1515] (Claim 6)

[1516] 6. The system of claim 4 or 5, further comprising means for generating prompts according to the emotional state and transmitting the prompts to the generating artificial intelligence. [Explanation of symbols]

[1517] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a query entered by a user; a means for optimizing the received query; a means for sending the optimized query to a generative artificial intelligence; A system including:

2. The system of claim 1 , further comprising: means for temporarily inhibiting a user from entering a query using a browser extension.

3. The system of claim 1 further comprising means for providing formatting and additional information to the optimized query.

Images