System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving an unclear instruction input in natural language; means for parsing the instruction and decomposing the instruction into a plurality of concrete elements; means for generating an intermediate representation based on the concrete elements; means for querying an external artificial intelligence model using the intermediate representation; and means for receiving a response from the artificial intelligence model and generating a result to be presented to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern artificial intelligence (AI) systems require a high level of expertise to understand unclear instructions from users and provide appropriate results based on those instructions. This makes it difficult for ordinary users to effectively utilize AI. In particular, to achieve good results, instructions must be written in English and detailed prompts must be designed, which requires a certain level of proficiency. This situation is a barrier to the widespread adoption and improved convenience of AI. [Means for solving the problem]

[0005] The present invention provides a system with the ability to analyze unclear instructions input in natural language, break them down into specific elements, and generate an intermediate representation that is easy for AI to understand. Specifically, the system first includes a means for receiving instructions input in natural language from a user. Next, the system includes a means for analyzing the instructions and breaking them down into multiple specific elements such as date and time, location, number of people, type of event, and user preferences. The system further includes a means for generating an intermediate representation that is easy for AI to understand based on these specific elements. The system then includes a means for querying an external AI model using the generated intermediate representation. Finally, the system provides a means for receiving responses from the AI ​​model and generating results to present to the user. This allows even ordinary users to easily communicate appropriate instructions to AI and maximize its performance.

[0006] A "natural language" is a language used by humans on a daily basis, which arises and develops naturally, as opposed to a specific programming language or symbol system.

[0007] "Unclear instructions" are instructions that are difficult for an AI to interpret accurately because they lack specific details or are ambiguous.

[0008] "Analysis" refers to the process of breaking down input data or information and analyzing each element to understand its meaning and structure.

[0009] "Specific elements" refer to clear and independent elements that constitute instructions obtained through analysis, such as date and time, location, number of people, type of event, and user preferences.

[0010] An "intermediate representation" is a collection of concrete elements that have been converted into a form that is easy for an AI to understand, usually written in a specific format or language.

[0011] An "artificial intelligence model" is a software model designed to perform a specific task, taking data as input and performing computations to produce a specific output.

[0012] "Query" refers to the process by which a system sends a request to an external AI model for some information or processing result.

[0013] A "response" refers to the information or processing results returned to the system from an external AI model, and refers to an answer to a query.

[0014] "Results" refer to information generated based on the AI's response and ultimately provided to the user, and are in line with the user's desired objectives. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for an AI to understand, and provides specific results. This system is composed of a server, a terminal, and a user.

[0037] 1. Overview of program processing

[0038] The user enters instructions

[0039] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday."

[0040] The device sends instructions to the server

[0041] The terminal receives the user's instructions and sends them to the server, which communicates with the terminal via a network.

[0042] The server parses the instructions and generates an intermediate representation

[0043] The server analyzes the user's instructions received from the device. Natural language processing technology is used for the analysis, and specific elements such as date and time, location, number of people, type of event, and user preferences are extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for the AI ​​to understand is generated.

[0044] The server sends a query to the AI ​​model based on the intermediate representation

[0045] The server uses the generated intermediate representation to query an external AI model, which is accessed through an API pre-configured by the system.

[0046] The AI ​​model returns the results

[0047] The AI ​​model generates results in response to queries and sends them back to the server.

[0048] The server processes the results for presentation to the user

[0049] The server receives the results returned by the AI ​​model and processes them in a format that is easy for users to understand, for example, by reorganizing the generated cooking recipe in Japanese.

[0050] The terminal displays the results to the user

[0051] The server sends the processed results to the terminal, which displays them to the user, who then checks the displayed information and achieves their goal.

[0052] 2. Specific examples of program processing

[0053] For example, consider a specific example where a user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[0054] 1. The user inputs instructions into the terminal in natural language.

[0055] 2. The device sends instructions to the server.

[0056] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[0057] 4. The server generates an intermediate representation based on the extracted elements, such as:

[0058] Generate recipes for a party on next Saturday. Consider the following details:

[0059] Event: Party

[0060] Date: Next Saturday

[0061] Location: User's home

[0062] Number of Guests: 10

[0063] Preferences: Easy to cook, delicious

[0064] 5. The server sends the intermediate representation to the AI ​​model.

[0065] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0066] 7. The server receives the results and reorganizes them in Japanese.

[0067] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[0068] The above is a specific embodiment of the present invention.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday."

[0072] Step 2:

[0073] The device receives the user's instructions and stores them internally as text data. It then checks to see if the input has been properly saved.

[0074] Step 3:

[0075] The device sends the saved instructions to the server, using a method such as an HTTP request to send the instructions to the server as structured data.

[0076] Step 4:

[0077] The server receives instructions from the terminal, decodes the received data, and extracts the instruction content in text format.

[0078] Step 5:

[0079] The server analyzes the received instructions using a natural language processing engine, extracting specific elements from the instructions, such as the date, time, location, number of people, type of event, and user preferences.

[0080] Step 6:

[0081] The server generates an intermediate representation based on the extracted specific elements. This intermediate representation is generated in a format that is easy for the AI ​​model to understand. For example, it might be something like "Generate recipes for a party on next Saturday. Consider the following details: ...".

[0082] Step 7:

[0083] The server sends the generated intermediate representation to the external AI model as a query, sending a request to the AI ​​model's API endpoint.

[0084] Step 8:

[0085] The AI ​​model generates appropriate results based on queries from the server, which are then returned as a response to the request.

[0086] Step 9:

[0087] The server receives the response from the AI ​​model, decodes the received results, translates them into Japanese if necessary, and prepares the data to be presented to the user.

[0088] Step 10:

[0089] The server then sends the results of the analysis to the terminal, also in structured data format.

[0090] Step 11:

[0091] The terminal displays the results received from the server, and visually presents the results to the user through a user interface.

[0092] Step 12:

[0093] The user checks the content displayed on the device and performs a task based on the information provided, for example, preparing a dish based on a displayed recipe.

[0094] Example 1

[0095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0096] Conventional systems have difficulty accurately analyzing vague instructions entered by users in natural language and providing appropriate results. Furthermore, they have problems with outputting results in different languages ​​and not taking into account a variety of specific factors. Systems that adjust to a variety of requirements often result in a poor user experience.

[0097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0098] In this invention, the server includes means for receiving an unclear instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, means for processing the result for presentation to the user, and means for displaying the processed result to the user. This makes it possible to analyze unclear natural language input, convert it into a format that is easy for AI to understand, and provide appropriate results.

[0099] "Natural language" refers to the language used by humans in everyday life, including spoken and written forms.

[0100] "Unclear instructions" are abstract or vague words that do not specify specific content or requests.

[0101] "Analysis" is the process of breaking down input information, extracting its components, and understanding them.

[0102] "Specific elements" are detailed information extracted from the analyzed instructions, and include elements such as the date and time, location, number of people, type of event, and user preferences.

[0103] An "intermediate representation" is data that has been converted into a format that is easy for an artificial intelligence model to understand, based on specific elements extracted through analysis.

[0104] An "artificial intelligence model" is a computational model that uses techniques such as machine learning and deep learning to learn patterns and relationships from input data and generate output.

[0105] An "inquiry" is the act of sending a request to an external system or service to obtain specific information.

[0106] A "response" is information returned from an external system or service in response to a query.

[0107] "Generating results" refers to the process of creating tangible output for presentation to the user based on the responses received.

[0108] "Processing" refers to the steps taken to convert the generated results into a format that is easy for the user to understand.

[0109] "Display" refers to the act of providing the processed results to the user visually or audibly.

[0110] This invention is a system that analyzes vague instructions entered by users in natural language, converts them into a format that is easy for AI to understand, and provides specific results. This system consists of a server, a terminal, and a user.

[0111] System configuration

[0112] The user enters instructions

[0113] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday." This can be done using keyboard input or voice input.

[0114] The device sends instructions to the server

[0115] The device receives the user's instructions and sends them to the server. The device and server communicate via a network using HTTP requests and data formats such as JSON.

[0116] The server parses the instructions and generates an intermediate representation

[0117] The server uses natural language processing (NLP) technology to analyze the user instructions received from the device. Python libraries such as NLTK and SpaCy are often used for analysis. Specific elements such as the date and time, location, number of people, type of event, and user preferences are also extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for AI to understand is generated.

[0118] As a concrete example, the following intermediate representation is generated from the instruction "Please tell me a simple and delicious recipe for a party next Saturday":

[0119] Generate recipes for a party on next Saturday. Consider the following details:

[0120] Event: Party

[0121] Date: Next Saturday

[0122] Location: User's home

[0123] Number of Guests: 10

[0124] Preferences: Easy to cook, delicious

[0125] The server sends a query to the AI ​​model based on the intermediate representation

[0126] The server uses the generated intermediate representation to query the external AI model. This query is performed via a pre-configured API endpoint, typically a POST request that includes authentication information such as an API key or token.

[0127] The AI ​​model returns the results

[0128] The external generative AI model processes the data based on the received intermediate representation and generates results. These results are sent back to the server in JSON format. For example, recipe suggestions might be returned as a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0129] The server processes the results for presentation to the user

[0130] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts the JSON data into Japanese sentences that are easy for humans to understand. For example, it processes the results into a format such as "We suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0131] The terminal displays the results to the user

[0132] The server sends the processed results to the terminal, which receives them and displays them to the user. The results can be displayed as text on the screen or read aloud using speech synthesis technology. The user can check the results and obtain the desired information.

[0133] Specific examples

[0134] Here is a specific example where the user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[0135] 1. The user inputs instructions into the terminal in natural language.

[0136] 2. The device sends instructions to the server.

[0137] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[0138] 4. The server generates an intermediate representation based on the extracted elements.

[0139] 5. The server sends the intermediate representation to the AI ​​model.

[0140] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0141] 7. The server receives the results and reorganizes them in Japanese.

[0142] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[0143] As a result of the above, it is possible to implement a system that analyzes unclear instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results.

[0144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0145] Step 1: User Enters Instructions

[0146] The user inputs instructions into the terminal in natural language. Specifically, the user inputs instructions such as "Please tell me a recipe for an easy and delicious dish for a party next Saturday" using a keyboard or voice input. The input data at this time is text or voice data. The input instructions are output and saved as text data on the terminal.

[0147] Step 2: The device sends instructions to the server

[0148] The device receives user instructions and sends them to the server. Specifically, it uses an HTTP request to send text instructions to the server's API endpoint. The input is the text data entered by the user, and the output is the data received by the server.

[0149] Step 3: The server parses the instructions and generates an intermediate representation

[0150] The server analyzes the user's instructions received from the device. For analysis, it uses natural language processing libraries such as Python's NLTK or SpaCy. The input is the user's text instructions, which the server breaks down into specific elements such as date and time, location, number of people, type of event, and user preferences. The output is the generation of an intermediate representation containing these elements.

[0151] Step 4: The server queries the AI ​​model based on the intermediate representation.

[0152] The server uses the generated intermediate representation to query the external AI model. Specifically, it sends the intermediate representation to the external API in JSON format. The input is the generated intermediate representation, and the output is the response from the AI ​​model. The server performs operations to send the request securely, including the API key and token.

[0153] Step 5: The AI ​​model returns the results

[0154] The external generative AI model generates results based on the received intermediate representation and sends them back to the server. The input is the JSON-formatted intermediate representation sent from the server, and the output is the resulting data, such as a cooking recipe. For example, a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad" is returned.

[0155] Step 6: The server processes the results for presentation to the user

[0156] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts JSON data into Japanese sentences that are easy for humans to understand. The input is the response from the AI ​​model, and the output is text reformatted into an easy-to-understand format. For example, it might convert it to "Suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0157] Step 7: The terminal displays the results to the user

[0158] The server sends the processed results to the terminal, which receives them and displays them visually or audibly to the user. The input is the processed results sent from the server, and the output is the information presented to the user. Specifically, the terminal displays the received data on a screen or reads it out through a speaker.

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] Conventional food delivery services have the drawback of requiring users to manually select menu items and complete the ordering process when deciding on specific orders, which is time-consuming. Additionally, there is a lack of a system that can select the optimal menu based on vague instructions entered by users in natural language, leaving room for improvement in the user experience.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes means for receiving an imprecise instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, and means for converting the result into an order content related to a food delivery service, and means for processing the generated order content into a form that can be confirmed by the user and presenting it again. This enables an optimal food delivery order to be automatically generated based on the imprecise instruction input by the user in natural language, significantly improving the user experience.

[0164] A "natural language" is a language that humans use on a daily basis as a means of communication.

[0165] "Unclear instructions" are instructions that lack specific details and are therefore open to interpretation.

[0166] "Analysis" means analyzing input information and understanding its meaning and structure.

[0167] "Specific elements" are detailed information extracted from vague instructions, and include the date and time, type of food, and user preferences.

[0168] An "intermediate representation" is a representation generated based on analyzed concrete elements, in a form that is easy for an external artificial intelligence model to understand.

[0169] An "artificial intelligence model" is a program or algorithm that uses artificial intelligence technology to solve a specific problem.

[0170] A "query" is the act of sending a question or request to a system or model to obtain specific information or results.

[0171] A "response" is the information or result returned by an artificial intelligence model or system in response to a query.

[0172] A "food delivery service" is a service that delivers food ordered by a user to a specified location.

[0173] "Order details" is a list of dishes and ingredients that the user specifies to the food delivery service.

[0174] "Processing" means arranging the received information or results in a form that is easy for the user to understand.

[0175] "Re-presentation" refers to the act of displaying the processed information or results to the user again.

[0176] The present invention is a system that analyzes vague instructions entered by a user in natural language and generates specific food delivery order details. Specific embodiments for carrying out the present invention are described below.

[0177] Hardware Configuration

[0178] Smartphone: A device that allows users to input instructions in natural language and display the results.

[0179] Server: A back-end system that analyzes instructions and works with artificial intelligence models to generate optimal results.

[0180] Software Configuration

[0181] Natural language processing libraries: used to parse inputted, imprecise instructions, such as spaCy and NLTK.

[0182] HTTP communication library: Used to communicate between the server and the artificial intelligence model. A specific example is requests (a Python library).

[0183] Artificial intelligence models: Models for optimizing food delivery service orders, such as OpenAI GPT-3 and Google Cloud AI.

[0184] System Operation

[0185] 1. Entering and receiving instructions

[0186] A user opens the app on their smartphone and enters a command in natural language, such as "I'd like to order the recommended pizza for dinner tonight."

[0187] 2. Parsing the instructions

[0188] The smartphone sends this instruction to the server, which then uses a natural language processing library to analyze the received instruction and break it down into specific elements (date and time, type of food, user preferences, etc.).

[0189] 3. Generating Intermediate Representations

[0190] The server generates an intermediate representation based on the parsing result, such as the following prompt:

[0191] Generate order for tonight's dinner.

[0192] Consider the following details:

[0193] Food: Pizza

[0194] Time: Tonight

[0195] Preferences: Recommendation

[0196] 4. Querying the AI ​​model

[0197] The server uses this intermediate representation to query an external AI model, which then generates the optimal dish and order.

[0198] 5. Generating and displaying results

[0199] The server receives the response from the AI ​​model and generates a result to present to the user. For example, it may recommend "Margherita Pizza" and "Quattro Formaggi Pizza." This result is sent to the smartphone and displayed to the user.

[0200] Specific examples

[0201] When a user inputs a command such as "I'd like to order a recommended pizza for dinner tonight," the server analyzes the command and generates the following prompt for the AI ​​model:

[0202] Generate order for tonight's dinner.

[0203] Consider the following details:

[0204] Food: Pizza

[0205] Time: Tonight

[0206] Preferences: Recommendation

[0207] Based on this, the AI ​​model suggests the best pizzas, and the server reprocesses the suggestions and presents them to the user as "Margherita Pizza" and "Quattro Formaggi Pizza."

[0208] In this way, the system of the present invention allows users to easily place food delivery orders based on non-specific instructions entered in natural language.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] A user accesses an application on their smartphone and inputs an order in natural language, for example, "I'd like to order the recommended pizza for dinner tonight." The input data is received by the smartphone.

[0212] Step 2:

[0213] The device sends the user's natural language input to the server, and the smartphone sends the data to the server via the network, including the input text.

[0214] Step 3:

[0215] The server parses the received natural language input. It uses a natural language processing library (e.g., spaCy or NLTK) to break down the instructions into specific elements (date and time, type of cuisine, user preferences, etc.) and outputs the input text as parsed data.

[0216] Step 4:

[0217] The server generates an intermediate representation based on the parsing results, which has the following format:

[0218] Generate order for tonight's dinner.

[0219] Consider the following details:

[0220] Food: Pizza

[0221] Time: Tonight

[0222] Preferences: Recommendation

[0223] The generated intermediate representation is output by the server as a prompt sentence.

[0224] Step 5:

[0225] The server queries the generated intermediate representation to the external AI model. Using an HTTP communication library (e.g., requests), it sends the intermediate representation to the AI ​​model and waits for a response from the model. The sent data is the intermediate representation, and the received data is the response from the AI ​​model.

[0226] Step 6:

[0227] The AI ​​model generates the optimal dish or order based on the query, such as "Margherita pizza" or "Quattro Formaggi pizza." This recommendation data is returned to the server from the AI ​​model.

[0228] Step 7:

[0229] The server processes the returned order information. In particular, it organizes the results to make them easier for the user to understand, and converts them into a format that can be displayed in Japanese. The final output data is an organized recommendation menu.

[0230] Step 8:

[0231] The smartphone presents the organized results to the user. The results are displayed on the device, allowing the user to check the recommended menu items provided and easily use food delivery. The displayed data is a list of recommended dishes.

[0232] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0233] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results by combining an emotion engine with a system that provides appropriate results that also take the user's emotions into consideration. This system is composed of a server, a terminal, a user, and an emotion engine.

[0234] 1. Overview of program processing

[0235] The user enters instructions

[0236] The user inputs instructions into the terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[0237] Emotion engine analyzes emotions

[0238] The device first receives the input instructions and then sends the text and voice data contained in the input to the emotion engine, which then analyzes the emotional information from the user's input and extracts an emotional status such as "joy," "sadness," or "excitement."

[0239] The device sends instructions and emotional information to the server.

[0240] The terminal transmits the user's instruction text and emotion information from the emotion engine together to the server. The instruction content and emotion information are sent to the server as structured data.

[0241] The server parses the instructions and generates an intermediate representation

[0242] The server combines the instructions and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted from the instructions, and the emotional information is then integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[0243] The server sends a query to the AI ​​model based on the intermediate representation

[0244] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and also includes emotional information. For example,

[0245] Generate recipes for a party on next Saturday. Consider the following details:

[0246] Event: Party

[0247] Date: Next Saturday

[0248] Location: User's home

[0249] Number of Guests: 10

[0250] Preferences: Easy to cook, delicious

[0251] User Emotion: Excited

[0252] It looks like this.

[0253] The server sends the intermediate representation to an external AI model.

[0254] The server uses this intermediate representation to query the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[0255] The AI ​​model returns the results

[0256] The AI ​​model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the AI ​​model will prioritize results that return cooking recipes to get the user in a party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" are likely to be returned.

[0257] The server receives and processes the results

[0258] The server receives the results returned by the AI ​​model. It decodes the results, translates them into Japanese if necessary, and prepares them as data to be presented to the user. In particular, the results are processed in a format that takes into account emotional information and provides content that matches the user's emotions.

[0259] The terminal displays the results to the user

[0260] The server sends the results of the analysis to the device, which then displays them to the user. Appropriate information reflecting the user's emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on a displayed recipe.

[0261] The above is a specific embodiment of the present invention.

[0262] The processing flow will be explained below.

[0263] Step 1:

[0264] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." Input can be in the form of text or voice.

[0265] Step 2:

[0266] The device receives the input instructions and saves them as text data. After confirming that the input has been saved, it sends the input data to the emotion engine.

[0267] Step 3:

[0268] The emotion engine installed in the device analyzes emotional information from the user's input data (text or voice). Specifically, it extracts emotional status such as "happiness," "sadness," and "excitement" from the user's vocabulary, tone of voice, and content.

[0269] Step 4:

[0270] The device sends the user's instruction text, including the parsed emotion information, to the server, usually in a structured format (e.g., JSON format).

[0271] Step 5:

[0272] The server receives the instruction and emotion information sent from the terminal. The received data is first decoded to extract the instruction text and emotion information.

[0273] Step 6:

[0274] The server analyzes the received instructions using a natural language processing engine. This analysis extracts specific elements from the instructions, such as the date and time, location, number of people, type of event, and the user's preferences. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" can be obtained.

[0275] Step 7:

[0276] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and includes emotional information. For example, the following intermediate representation is generated:

[0277] Generate recipes for a party on next Saturday. Consider the following details:

[0278] Event: Party

[0279] Date: Next Saturday

[0280] Location: User's home

[0281] Number of Guests: 10

[0282] Preferences: Easy to cook, delicious

[0283] User Emotion: Excited

[0284] Step 8:

[0285] The server sends this intermediate representation as a query to the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[0286] Step 9:

[0287] The AI ​​model generates appropriate results based on the intermediate representation from the server. For example, it takes into account the user's emotions (excitement) and prioritizes generating cooking recipes that will further enhance the party atmosphere. The generated results are sent back to the server as a response to the request.

[0288] Step 10:

[0289] The server receives the response from the AI ​​model. The received results are decoded, translated into Japanese as needed, and organized into data to be presented to the user. As a result of taking emotional information into consideration, the data is organized in a format that matches the user's emotions.

[0290] Step 11:

[0291] The server sends the prepared results to the terminal, also in a structured format.

[0292] Step 12:

[0293] The device displays the results received from the server to the user. Appropriate information reflecting emotional information is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on the displayed recipe.

[0294] Example 2

[0295] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0296] Conventional AI systems have had difficulty taking into account the user's emotional information when analyzing ambiguous instructions entered in natural language by the user. As a result, it has been difficult to provide an appropriate response that matches the user's emotions, resulting in a poor user experience. Furthermore, if emotional information is not taken into account, the results provided often do not meet the user's expectations.

[0297] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving an unclear instruction input in natural language, a means for extracting emotional information, and a means for analyzing the instruction and emotional information and breaking it down into multiple specific elements. This makes it possible to provide an appropriate result that takes the user's emotions into consideration. The server also includes a means for generating an intermediate representation based on the specific elements and emotional information, and a means for querying an external artificial intelligence model using the intermediate representation. This enables queries to be made in a format that is easy for the artificial intelligence model to understand, and a highly accurate response is returned. The server also includes a means for receiving a response from the artificial intelligence model, generating a result based on the user's emotions, and a means for presenting the result to the user. This provides information that matches the user's emotions, improving the user experience.

[0298] "Natural language" refers to the forms of language used by humans on a daily basis, including speech and writing.

[0299] "Unclear instructions" refer to user input that is difficult to interpret because specific information is not clearly stated.

[0300] "Emotion information" refers to data that indicates an emotional state, such as "happiness," "sadness," or "excitement," extracted from instructions input by the user.

[0301] "Specific elements" refer to specific information such as the date and time, location, number of people, type of event, and user preferences extracted from the analyzed instructions.

[0302] An "intermediate representation" refers to data generated based on instructions and emotional information in a format that is easy for an external artificial intelligence model to understand.

[0303] An "artificial intelligence model" refers to a system that has algorithms for data analysis and information generation (e.g., machine learning models and deep learning models).

[0304] "Reception" refers to the process by which a terminal or a server receives information that has been transmitted remotely.

[0305] "Analysis" refers to the process of interpreting received information and converting it into meaningful data.

[0306] "Query" refers to the act of sending a request for information to an external system.

[0307] A "response" refers to the information or results returned by an external system in response to a query.

[0308] "Presenting" refers to the act of showing the received results or information to the user through a user interface.

[0309] This invention relates to a system that analyzes vague instructions entered by a user in natural language, takes emotional information into consideration, and then provides appropriate results using an external artificial intelligence model. This system is composed of a server, a terminal, a user, and an emotion engine.

[0310] A user inputs instructions into the device in natural language. For example, "Please tell me a recipe for an easy and delicious dish for a party next Saturday." This input can be in text or voice format. The device first receives the input instruction and sends the text or voice data contained in the input to the emotion engine. The emotion engine analyzes the emotion information from the user's input and extracts an emotional status such as "joy" or "excitement."

[0311] The device sends the user's instruction text and emotional information from the emotion engine together to the server. The instruction content and emotional information are sent to the server as structured data. The server combines the instruction and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date and time, location, number of people, type of event, and user preferences are extracted from the instruction, and the emotional information is further integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[0312] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for a generative AI model (such as OpenAI's GPT-4) to understand, and also includes emotional information. For example, it might look like this: "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[0313] The server uses this intermediate representation to query an external generative AI model. The intermediate representation is sent as the request body to the API endpoint of the generative AI model. The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the generated results will prioritize cooking recipes to get the user in the party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" could be returned.

[0314] The server receives the results returned from the generative AI model. The results are decoded, translated into Japanese as needed, and organized into data for presentation to the administrator. The results are processed into a format that provides content that matches the user's emotions, taking into particular account emotional information. The organized results are sent to the device, which displays them to the user. Appropriate information that reflects emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal; for example, preparing a dish based on the displayed recipe.

[0315] Specific examples

[0316] Example prompt sentence:

[0317] "Can you give me some easy and delicious recipes for a party next Saturday? I'm super excited."

[0318] The user's input is analyzed as an instruction such as "Tell me a simple and delicious recipe for a party next Saturday" and the emotional information of "excited." Based on this information, the server sends the appropriate intermediate representation to the generative AI model, and the returned result is recipes suitable for an excited user, such as "spaghetti arrabbiata" or "garlic shrimp."

[0319] In this way, users can receive information optimized for their input instructions and emotions and put it into action.

[0320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0321] Step 1: User Enters Instructions

[0322] Specific operation: A user inputs a command into a terminal in natural language. For example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[0323] Input: User's natural language instructions (text or voice)

[0324] Output: The terminal receives the user's instructions.

[0325] Step 2: The emotion engine analyzes the emotion information

[0326] Specific operation: The device receives input instructions and sends text and voice data to the emotion engine. The emotion engine analyzes this and extracts the user's emotional status. For example, it extracts the emotional information of "excitement" from the user's instructions.

[0327] Input: User instructions, text or voice data

[0328] Output: Emotional information (e.g., "excitement")

[0329] Step 3: The device sends instructions and emotion information to the server.

[0330] Specific operation: The device sends the user's instruction text and the emotion information analyzed by the emotion engine to the server. This data is sent as structured data in JSON format or similar.

[0331] Input: User instructions and emotional information

[0332] Output: Structured data sent to the server

[0333] Step 4: The server parses the instructions and generates an intermediate representation

[0334] Specific operation: The server uses a natural language processing engine to analyze the received instructions and emotional information. Through the analysis, specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted and integrated with the emotional information. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[0335] Input: User instructions and emotional information

[0336] Output: Specific elements and emotional information

[0337] Step 5: The server queries the AI ​​model based on the intermediate representation.

[0338] Specific behavior: The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is in a format that is easy for the generative AI model to understand. For example, it might be in the format "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[0339] Input: concrete elements and emotional information

[0340] Output: The generated intermediate representation

[0341] Step 6: The server sends the intermediate representation to an external AI model

[0342] Specific operation: The server uses the intermediate representation to query the external generative AI model, and sends the intermediate representation to the API endpoint using the POST method.

[0343] Input: Intermediate Representation

[0344] Output: Sending an API request

[0345] Step 7: The AI ​​model returns the results

[0346] Specific operation: The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, for an "excited" user, recipes that will get them in a party mood will be prioritized. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" will be generated.

[0347] Input: Intermediate Representation

[0348] Output: The results generated

[0349] Step 8: The server receives and processes the results

[0350] Specific operation: The server receives the results from the generative AI model and decodes the response body. If necessary, it translates the results into Japanese and prepares them as data for presentation to the administrator. In particular, it processes the results based on emotional information into a format that matches the user's emotions.

[0351] Input: Generated results

[0352] Output: Cleaned data

[0353] Step 9: The terminal displays the results to the user

[0354] Specific actions: The device receives the preparation results sent from the server and visually displays them through the user interface. The user checks the displayed information and takes specific action, such as preparing a dish based on the presented recipe.

[0355] Input: Organized data

[0356] Output: The results displayed in the user interface

[0357] (Application example 2)

[0358] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0359] Conventional systems have the problem that they only analyze vague instructions entered by users in natural language and are unable to provide results that take the user's emotions into account. In particular, food delivery services are required to recommend the most appropriate dishes and restaurants based on the user's emotions, but current systems are inadequate in this regard. This makes it difficult to improve the user experience.

[0360] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0361] In this invention, the server includes means for receiving vague instructions input in natural language, means for analyzing the user's instructions and breaking them down into multiple specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external artificial intelligence model and receiving a response, means for analyzing the user's emotions using an emotion engine and integrating the emotion information into the intermediate representation, and means for providing an optimal result that matches the user's emotions based on the emotion information. This allows for careful consideration of the user's emotions, and in the case of a food delivery service, makes it possible to recommend the most suitable dishes and restaurants to the user.

[0362] "Natural language" refers to language that humans use on a daily basis, which does not have a specific form or structure but contains meaning.

[0363] An "unclear instruction" refers to an instruction that is input by a user in natural language, but whose specific content or request is not clear unless analyzed.

[0364] "Analysis" is the process of analyzing input information and converting it into an understandable structure.

[0365] "Concrete elements" are the specific items or information extracted from the parsed imprecise instructions.

[0366] An "intermediate representation" is data generated based on specific elements in a format that is easy for an artificial intelligence model to understand.

[0367] An "artificial intelligence model" is a system that uses machine learning, deep learning, etc. to make predictions and classifications from input data.

[0368] An "emotion engine" is a system that analyzes emotions from user input and extracts that emotional information.

[0369] "Emotion information" is data relating to the user's emotional state obtained as a result of analysis by the emotion engine.

[0370] A "query" is an operation in which a system sends a request for data processing to an external artificial intelligence model.

[0371] "Response" refers to the processing result returned from an external artificial intelligence model.

[0372] The "means for generating a result" is a method or device that generates a result to be presented to the user based on the received response.

[0373] A "means for providing results" is a method or device for displaying the generated results to the user and enhancing the user experience.

[0374] The present invention relates to a system that analyzes unclear instructions entered by a user in natural language and provides optimal results based on the user's emotional information. A specific method for realizing this invention is described below.

[0375] System Configuration

[0376] This system consists of a user terminal, a server, an emotion engine, and an external artificial intelligence model.

[0377] Software / hardware used

[0378] Terminal: A device that allows users to input information, such as a smartphone or tablet

[0379] Server: A server that performs data analysis, sentiment analysis, intermediate representation generation, queries to external AI models, and results organization.

[0380] Emotion Engine: Software for Analyzing User Emotions

[0381] External Artificial Intelligence Models: AI models that use machine learning or deep learning to perform specific tasks, such as recipe recommendations or restaurant recommendations.

[0382] Processing flow

[0383] 1. The user enters instructions

[0384] Users input requests to food delivery apps in natural language, for example, "I'm tired, so please recommend some food that will help me relax."

[0385] 2. The device sends instructions to the server

[0386] The device receives the input and sends it to a server for sentiment analysis.

[0387] 3. The server analyzes emotions

[0388] The server uses an emotion engine to analyze emotions from the user's input, for example, extracting emotion information indicating "fatigue."

[0389] 4. The server generates an intermediate representation

[0390] The server generates an intermediate representation based on the user's instructions and emotional information. This intermediate representation is in a format that is easy for the AI ​​model to understand. For example, it may have the following format:

[0391] "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[0392] 5. The server queries the AI ​​model

[0393] The server uses the intermediate representation to query an external artificial intelligence model, which retrieves a list of the best dishes and restaurants.

[0394] 6. The server prepares the response

[0395] The server receives the response from the AI ​​model and prepares the results in the most optimal way for the user.

[0396] 7. The device will display the results

[0397] The device receives the results and displays them to the user, recommending dishes such as "Japanese sushi set" or "Thai green curry."

[0398] Examples of prompt statements

[0399] A prompt sentence is generated in the following format and sent to the AI ​​model:

[0400] "Generate recommendations for dishes that can uplift the mood. Consider the user is feeling 'happy'."

[0401] "Provide food suggestions that can help a tired user feel relaxed."

[0402] This allows for the provision of specific and effective results that correspond to the user's emotions.

[0403] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0404] Step 1:

[0405] The user inputs instructions.

[0406] A user inputs a request in natural language into a device such as a smartphone or tablet. For example, they input an instruction such as "I'm tired, so please recommend some dishes that will help me relax" by voice or text. This input is then received by the device.

[0407] Step 2:

[0408] The terminal sends instructions to the server.

[0409] The terminal transmits the received user input data to the server. Specifically, the user's text or voice data is transferred to the server. The input at this time is in natural language.

[0410] Step 3:

[0411] The server analyzes the emotions.

[0412] The server uses an emotion engine to analyze the received user input data. Through this analysis, the server extracts the user's emotion information (e.g., "fatigue"). The acquired emotion information is saved in the format "Emotion: Tired."

[0413] Step 4:

[0414] The server integrates the instructions and emotion information to generate an intermediate representation.

[0415] The server generates an intermediate representation based on the analyzed user instructions and emotional information. This intermediate representation is formatted to suit the AI ​​model. For example, it may contain content such as "The user is looking for relaxing recipes. The user is tired." Specifically, the following prompt sentence is generated: "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[0416] Step 5:

[0417] The server queries the AI ​​model for the intermediate representation.

[0418] The server sends the generated intermediate representation to an external artificial intelligence model. Specifically, it sends the intermediate representation as a request to the AI ​​model's API endpoint. In this case, the input is the intermediate representation, and the output is the recommendation result from the AI ​​model.

[0419] Step 6:

[0420] The server receives and organizes the response from the AI ​​model.

[0421] The server receives the response from the AI ​​model and formats it accordingly for the user. For example, it formats the recipes and restaurant lists obtained from the AI ​​model and converts them into a format that can be displayed in the user's language. This results in specific suggestions such as "Japanese sushi set" or "Thai green curry."

[0422] Step 7:

[0423] The terminal displays the results to the user.

[0424] The device displays the results received from the server to the user. Specifically, the results are presented in a user-friendly format using the smartphone's user interface. The user can then review the displayed results and select, for example, recommended dishes or restaurants.

[0425] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0426] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0427] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0428] [Second embodiment]

[0429] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0430] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0431] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0432] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0433] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0434] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0435] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0436] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0437] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0438] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0439] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0440] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0441] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for an AI to understand, and provides specific results. This system is composed of a server, a terminal, and a user.

[0442] 1. Overview of program processing

[0443] The user enters instructions

[0444] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday."

[0445] The device sends instructions to the server

[0446] The terminal receives the user's instructions and sends them to the server, which communicates with the terminal via a network.

[0447] The server parses the instructions and generates an intermediate representation

[0448] The server analyzes the user's instructions received from the device. Natural language processing technology is used for the analysis, and specific elements such as date and time, location, number of people, type of event, and user preferences are extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for the AI ​​to understand is generated.

[0449] The server sends a query to the AI ​​model based on the intermediate representation

[0450] The server uses the generated intermediate representation to query an external AI model, which is accessed through an API pre-configured by the system.

[0451] The AI ​​model returns the results

[0452] The AI ​​model generates results in response to queries and sends them back to the server.

[0453] The server processes the results for presentation to the user

[0454] The server receives the results returned by the AI ​​model and processes them in a format that is easy for users to understand, for example, by reorganizing the generated cooking recipe in Japanese.

[0455] The terminal displays the results to the user

[0456] The server sends the processed results to the terminal, which displays them to the user, who then checks the displayed information and achieves their goal.

[0457] 2. Specific examples of program processing

[0458] For example, consider a specific example where a user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[0459] 1. The user inputs instructions into the terminal in natural language.

[0460] 2. The device sends instructions to the server.

[0461] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[0462] 4. The server generates an intermediate representation based on the extracted elements, such as:

[0463] Generate recipes for a party on next Saturday. Consider the following details:

[0464] Event: Party

[0465] Date: Next Saturday

[0466] Location: User's home

[0467] Number of Guests: 10

[0468] Preferences: Easy to cook, delicious

[0469] 5. The server sends the intermediate representation to the AI ​​model.

[0470] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0471] 7. The server receives the results and reorganizes them in Japanese.

[0472] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[0473] The above is a specific embodiment of the present invention.

[0474] The processing flow will be explained below.

[0475] Step 1:

[0476] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday."

[0477] Step 2:

[0478] The device receives the user's instructions and stores them internally as text data. It then checks to see if the input has been properly saved.

[0479] Step 3:

[0480] The device sends the saved instructions to the server, using a method such as an HTTP request to send the instructions to the server as structured data.

[0481] Step 4:

[0482] The server receives instructions from the terminal, decodes the received data, and extracts the instruction content in text format.

[0483] Step 5:

[0484] The server analyzes the received instructions using a natural language processing engine, extracting specific elements from the instructions, such as the date, time, location, number of people, type of event, and user preferences.

[0485] Step 6:

[0486] The server generates an intermediate representation based on the extracted specific elements. This intermediate representation is generated in a format that is easy for the AI ​​model to understand. For example, it might be something like "Generate recipes for a party on next Saturday. Consider the following details: ...".

[0487] Step 7:

[0488] The server sends the generated intermediate representation to the external AI model as a query, sending a request to the AI ​​model's API endpoint.

[0489] Step 8:

[0490] The AI ​​model generates appropriate results based on queries from the server, which are then returned as a response to the request.

[0491] Step 9:

[0492] The server receives the response from the AI ​​model, decodes the received results, translates them into Japanese if necessary, and prepares the data to be presented to the user.

[0493] Step 10:

[0494] The server then sends the results of the analysis to the terminal, also in structured data format.

[0495] Step 11:

[0496] The terminal displays the results received from the server, and visually presents the results to the user through a user interface.

[0497] Step 12:

[0498] The user checks the content displayed on the device and performs a task based on the information provided, for example, preparing a dish based on a displayed recipe.

[0499] Example 1

[0500] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0501] Conventional systems have difficulty accurately analyzing vague instructions entered by users in natural language and providing appropriate results. Furthermore, they have problems with outputting results in different languages ​​and not taking into account a variety of specific factors. Systems that adjust to a variety of requirements often result in a poor user experience.

[0502] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0503] In this invention, the server includes means for receiving an unclear instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, means for processing the result for presentation to the user, and means for displaying the processed result to the user. This makes it possible to analyze unclear natural language input, convert it into a format that is easy for AI to understand, and provide appropriate results.

[0504] "Natural language" refers to the language used by humans in everyday life, including spoken and written forms.

[0505] "Unclear instructions" are abstract or vague words that do not specify specific content or requests.

[0506] "Analysis" is the process of breaking down input information, extracting its components, and understanding them.

[0507] "Specific elements" are detailed information extracted from the analyzed instructions, and include elements such as the date and time, location, number of people, type of event, and user preferences.

[0508] An "intermediate representation" is data that has been converted into a format that is easy for an artificial intelligence model to understand, based on specific elements extracted through analysis.

[0509] An "artificial intelligence model" is a computational model that uses techniques such as machine learning and deep learning to learn patterns and relationships from input data and generate output.

[0510] An "inquiry" is the act of sending a request to an external system or service to obtain specific information.

[0511] A "response" is information returned from an external system or service in response to a query.

[0512] "Generating results" refers to the process of creating tangible output for presentation to the user based on the responses received.

[0513] "Processing" refers to the steps taken to convert the generated results into a format that is easy for the user to understand.

[0514] "Display" refers to the act of providing the processed results to the user visually or audibly.

[0515] This invention is a system that analyzes vague instructions entered by users in natural language, converts them into a format that is easy for AI to understand, and provides specific results. This system consists of a server, a terminal, and a user.

[0516] System configuration

[0517] The user enters instructions

[0518] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday." This can be done using keyboard input or voice input.

[0519] The device sends instructions to the server

[0520] The device receives the user's instructions and sends them to the server. The device and server communicate via a network using HTTP requests and data formats such as JSON.

[0521] The server parses the instructions and generates an intermediate representation

[0522] The server uses natural language processing (NLP) technology to analyze the user instructions received from the device. Python libraries such as NLTK and SpaCy are often used for analysis. Specific elements such as the date and time, location, number of people, type of event, and user preferences are also extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for AI to understand is generated.

[0523] As a concrete example, the following intermediate representation is generated from the instruction "Please tell me a simple and delicious recipe for a party next Saturday":

[0524] Generate recipes for a party on next Saturday. Consider the following details:

[0525] Event: Party

[0526] Date: Next Saturday

[0527] Location: User's home

[0528] Number of Guests: 10

[0529] Preferences: Easy to cook, delicious

[0530] The server sends a query to the AI ​​model based on the intermediate representation

[0531] The server uses the generated intermediate representation to query the external AI model. This query is performed via a pre-configured API endpoint, typically a POST request that includes authentication information such as an API key or token.

[0532] The AI ​​model returns the results

[0533] The external generative AI model processes the data based on the received intermediate representation and generates results. These results are sent back to the server in JSON format. For example, recipe suggestions might be returned as a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0534] The server processes the results for presentation to the user

[0535] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts the JSON data into Japanese sentences that are easy for humans to understand. For example, it processes the results into a format such as "We suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0536] The terminal displays the results to the user

[0537] The server sends the processed results to the terminal, which receives them and displays them to the user. The results can be displayed as text on the screen or read aloud using speech synthesis technology. The user can check the results and obtain the desired information.

[0538] Specific examples

[0539] Here is a specific example where the user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[0540] 1. The user inputs instructions into the terminal in natural language.

[0541] 2. The device sends instructions to the server.

[0542] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[0543] 4. The server generates an intermediate representation based on the extracted elements.

[0544] 5. The server sends the intermediate representation to the AI ​​model.

[0545] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0546] 7. The server receives the results and reorganizes them in Japanese.

[0547] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[0548] As a result of the above, it is possible to implement a system that analyzes unclear instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results.

[0549] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0550] Step 1: User Enters Instructions

[0551] The user inputs instructions into the terminal in natural language. Specifically, the user inputs instructions such as "Please tell me a recipe for an easy and delicious dish for a party next Saturday" using a keyboard or voice input. The input data at this time is text or voice data. The input instructions are output and saved as text data on the terminal.

[0552] Step 2: The device sends instructions to the server

[0553] The device receives user instructions and sends them to the server. Specifically, it uses an HTTP request to send text instructions to the server's API endpoint. The input is the text data entered by the user, and the output is the data received by the server.

[0554] Step 3: The server parses the instructions and generates an intermediate representation

[0555] The server analyzes the user's instructions received from the device. For analysis, it uses natural language processing libraries such as Python's NLTK or SpaCy. The input is the user's text instructions, which the server breaks down into specific elements such as date and time, location, number of people, type of event, and user preferences. The output is the generation of an intermediate representation containing these elements.

[0556] Step 4: The server queries the AI ​​model based on the intermediate representation.

[0557] The server uses the generated intermediate representation to query the external AI model. Specifically, it sends the intermediate representation to the external API in JSON format. The input is the generated intermediate representation, and the output is the response from the AI ​​model. The server performs operations to send the request securely, including the API key and token.

[0558] Step 5: The AI ​​model returns the results

[0559] The external generative AI model generates results based on the received intermediate representation and sends them back to the server. The input is the JSON-formatted intermediate representation sent from the server, and the output is the resulting data, such as a cooking recipe. For example, a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad" is returned.

[0560] Step 6: The server processes the results for presentation to the user

[0561] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts JSON data into Japanese sentences that are easy for humans to understand. The input is the response from the AI ​​model, and the output is text reformatted into an easy-to-understand format. For example, it might convert it to "Suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0562] Step 7: The terminal displays the results to the user

[0563] The server sends the processed results to the terminal, which receives them and displays them visually or audibly to the user. The input is the processed results sent from the server, and the output is the information presented to the user. Specifically, the terminal displays the received data on a screen or reads it out through a speaker.

[0564] (Application example 1)

[0565] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0566] Conventional food delivery services have the drawback of requiring users to manually select menu items and complete the ordering process when deciding on specific orders, which is time-consuming. Additionally, there is a lack of a system that can select the optimal menu based on vague instructions entered by users in natural language, leaving room for improvement in the user experience.

[0567] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0568] In this invention, the server includes means for receiving an imprecise instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, and means for converting the result into an order content related to a food delivery service, and means for processing the generated order content into a form that can be confirmed by the user and presenting it again. This enables an optimal food delivery order to be automatically generated based on the imprecise instruction input by the user in natural language, significantly improving the user experience.

[0569] A "natural language" is a language that humans use on a daily basis as a means of communication.

[0570] "Unclear instructions" are instructions that lack specific details and are therefore open to interpretation.

[0571] "Analysis" means analyzing input information and understanding its meaning and structure.

[0572] "Specific elements" are detailed information extracted from vague instructions, and include the date and time, type of food, and user preferences.

[0573] An "intermediate representation" is a representation generated based on analyzed concrete elements, in a form that is easy for an external artificial intelligence model to understand.

[0574] An "artificial intelligence model" is a program or algorithm that uses artificial intelligence technology to solve a specific problem.

[0575] A "query" is the act of sending a question or request to a system or model to obtain specific information or results.

[0576] A "response" is the information or result returned by an artificial intelligence model or system in response to a query.

[0577] A "food delivery service" is a service that delivers food ordered by a user to a specified location.

[0578] "Order details" is a list of dishes and ingredients that the user specifies to the food delivery service.

[0579] "Processing" means arranging the received information or results in a form that is easy for the user to understand.

[0580] "Re-presentation" refers to the act of displaying the processed information or results to the user again.

[0581] The present invention is a system that analyzes vague instructions entered by a user in natural language and generates specific food delivery order details. Specific embodiments for carrying out the present invention are described below.

[0582] Hardware Configuration

[0583] Smartphone: A device that allows users to input instructions in natural language and display the results.

[0584] Server: A back-end system that analyzes instructions and works with artificial intelligence models to generate optimal results.

[0585] Software Configuration

[0586] Natural language processing libraries: used to parse inputted, imprecise instructions, such as spaCy and NLTK.

[0587] HTTP communication library: Used to communicate between the server and the artificial intelligence model. A specific example is requests (a Python library).

[0588] Artificial intelligence models: Models for optimizing food delivery service orders, such as OpenAI GPT-3 and Google Cloud AI.

[0589] System Operation

[0590] 1. Entering and receiving instructions

[0591] A user opens the app on their smartphone and enters a command in natural language, such as "I'd like to order the recommended pizza for dinner tonight."

[0592] 2. Parsing the instructions

[0593] The smartphone sends this instruction to the server, which then uses a natural language processing library to analyze the received instruction and break it down into specific elements (date and time, type of food, user preferences, etc.).

[0594] 3. Generating Intermediate Representations

[0595] The server generates an intermediate representation based on the parsing result, such as the following prompt:

[0596] Generate order for tonight's dinner.

[0597] Consider the following details:

[0598] Food: Pizza

[0599] Time: Tonight

[0600] Preferences: Recommendation

[0601] 4. Querying the AI ​​model

[0602] The server uses this intermediate representation to query an external AI model, which then generates the optimal dish and order.

[0603] 5. Generating and displaying results

[0604] The server receives the response from the AI ​​model and generates a result to present to the user. For example, it may recommend "Margherita Pizza" and "Quattro Formaggi Pizza." This result is sent to the smartphone and displayed to the user.

[0605] Specific examples

[0606] When a user inputs a command such as "I'd like to order a recommended pizza for dinner tonight," the server analyzes the command and generates the following prompt for the AI ​​model:

[0607] Generate order for tonight's dinner.

[0608] Consider the following details:

[0609] Food: Pizza

[0610] Time: Tonight

[0611] Preferences: Recommendation

[0612] Based on this, the AI ​​model suggests the best pizzas, and the server reprocesses the suggestions and presents them to the user as "Margherita Pizza" and "Quattro Formaggi Pizza."

[0613] In this way, the system of the present invention allows users to easily place food delivery orders based on non-specific instructions entered in natural language.

[0614] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0615] Step 1:

[0616] A user accesses an application on their smartphone and inputs an order in natural language, for example, "I'd like to order the recommended pizza for dinner tonight." The input data is received by the smartphone.

[0617] Step 2:

[0618] The device sends the user's natural language input to the server, and the smartphone sends the data to the server via the network, including the input text.

[0619] Step 3:

[0620] The server parses the received natural language input. It uses a natural language processing library (e.g., spaCy or NLTK) to break down the instructions into specific elements (date and time, type of cuisine, user preferences, etc.) and outputs the input text as parsed data.

[0621] Step 4:

[0622] The server generates an intermediate representation based on the parsing results, which has the following format:

[0623] Generate order for tonight's dinner.

[0624] Consider the following details:

[0625] Food: Pizza

[0626] Time: Tonight

[0627] Preferences: Recommendation

[0628] The generated intermediate representation is output by the server as a prompt sentence.

[0629] Step 5:

[0630] The server queries the generated intermediate representation to the external AI model. Using an HTTP communication library (e.g., requests), it sends the intermediate representation to the AI ​​model and waits for a response from the model. The sent data is the intermediate representation, and the received data is the response from the AI ​​model.

[0631] Step 6:

[0632] The AI ​​model generates the optimal dish or order based on the query, such as "Margherita pizza" or "Quattro Formaggi pizza." This recommendation data is returned to the server from the AI ​​model.

[0633] Step 7:

[0634] The server processes the returned order information. In particular, it organizes the results to make them easier for the user to understand, and converts them into a format that can be displayed in Japanese. The final output data is an organized recommendation menu.

[0635] Step 8:

[0636] The smartphone presents the organized results to the user. The results are displayed on the device, allowing the user to check the recommended menu items provided and easily use food delivery. The displayed data is a list of recommended dishes.

[0637] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0638] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results by combining an emotion engine with a system that provides appropriate results that also take the user's emotions into consideration. This system is composed of a server, a terminal, a user, and an emotion engine.

[0639] 1. Overview of program processing

[0640] The user enters instructions

[0641] The user inputs instructions into the terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[0642] Emotion engine analyzes emotions

[0643] The device first receives the input instructions and then sends the text and voice data contained in the input to the emotion engine, which then analyzes the emotional information from the user's input and extracts an emotional status such as "joy," "sadness," or "excitement."

[0644] The device sends instructions and emotional information to the server.

[0645] The terminal transmits the user's instruction text and emotion information from the emotion engine together to the server. The instruction content and emotion information are sent to the server as structured data.

[0646] The server parses the instructions and generates an intermediate representation

[0647] The server combines the instructions and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted from the instructions, and the emotional information is then integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[0648] The server sends a query to the AI ​​model based on the intermediate representation

[0649] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and also includes emotional information. For example,

[0650] Generate recipes for a party on next Saturday. Consider the following details:

[0651] Event: Party

[0652] Date: Next Saturday

[0653] Location: User's home

[0654] Number of Guests: 10

[0655] Preferences: Easy to cook, delicious

[0656] User Emotion: Excited

[0657] It looks like this.

[0658] The server sends the intermediate representation to an external AI model.

[0659] The server uses this intermediate representation to query the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[0660] The AI ​​model returns the results

[0661] The AI ​​model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the AI ​​model will prioritize results that return cooking recipes to get the user in a party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" are likely to be returned.

[0662] The server receives and processes the results

[0663] The server receives the results returned by the AI ​​model. It decodes the results, translates them into Japanese if necessary, and prepares them as data to be presented to the user. In particular, the results are processed in a format that takes into account emotional information and provides content that matches the user's emotions.

[0664] The terminal displays the results to the user

[0665] The server sends the results of the analysis to the device, which then displays them to the user. Appropriate information reflecting the user's emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on a displayed recipe.

[0666] The above is a specific embodiment of the present invention.

[0667] The processing flow will be explained below.

[0668] Step 1:

[0669] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." Input can be in the form of text or voice.

[0670] Step 2:

[0671] The device receives the input instructions and saves them as text data. After confirming that the input has been saved, it sends the input data to the emotion engine.

[0672] Step 3:

[0673] The emotion engine installed in the device analyzes emotional information from the user's input data (text or voice). Specifically, it extracts emotional status such as "happiness," "sadness," and "excitement" from the user's vocabulary, tone of voice, and content.

[0674] Step 4:

[0675] The device sends the user's instruction text, including the parsed emotion information, to the server, usually in a structured format (e.g., JSON format).

[0676] Step 5:

[0677] The server receives the instruction and emotion information sent from the terminal. The received data is first decoded to extract the instruction text and emotion information.

[0678] Step 6:

[0679] The server analyzes the received instructions using a natural language processing engine. This analysis extracts specific elements from the instructions, such as the date and time, location, number of people, type of event, and the user's preferences. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" can be obtained.

[0680] Step 7:

[0681] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and includes emotional information. For example, the following intermediate representation is generated:

[0682] Generate recipes for a party on next Saturday. Consider the following details:

[0683] Event: Party

[0684] Date: Next Saturday

[0685] Location: User's home

[0686] Number of Guests: 10

[0687] Preferences: Easy to cook, delicious

[0688] User Emotion: Excited

[0689] Step 8:

[0690] The server sends this intermediate representation as a query to the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[0691] Step 9:

[0692] The AI ​​model generates appropriate results based on the intermediate representation from the server. For example, it takes into account the user's emotions (excitement) and prioritizes generating cooking recipes that will further enhance the party atmosphere. The generated results are sent back to the server as a response to the request.

[0693] Step 10:

[0694] The server receives the response from the AI ​​model. The received results are decoded, translated into Japanese as needed, and organized into data to be presented to the user. As a result of taking emotional information into consideration, the data is organized in a format that matches the user's emotions.

[0695] Step 11:

[0696] The server sends the prepared results to the terminal, also in a structured format.

[0697] Step 12:

[0698] The device displays the results received from the server to the user. Appropriate information reflecting emotional information is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on the displayed recipe.

[0699] Example 2

[0700] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] Conventional AI systems have had difficulty taking into account the user's emotional information when analyzing ambiguous instructions entered in natural language by the user. As a result, it has been difficult to provide an appropriate response that matches the user's emotions, resulting in a poor user experience. Furthermore, if emotional information is not taken into account, the results provided often do not meet the user's expectations.

[0702] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving an unclear instruction input in natural language, a means for extracting emotional information, and a means for analyzing the instruction and emotional information and breaking it down into multiple specific elements. This makes it possible to provide an appropriate result that takes the user's emotions into consideration. The server also includes a means for generating an intermediate representation based on the specific elements and emotional information, and a means for querying an external artificial intelligence model using the intermediate representation. This enables queries to be made in a format that is easy for the artificial intelligence model to understand, and a highly accurate response is returned. The server also includes a means for receiving a response from the artificial intelligence model, generating a result based on the user's emotions, and a means for presenting the result to the user. This provides information that matches the user's emotions, improving the user experience.

[0703] "Natural language" refers to the forms of language used by humans on a daily basis, including speech and writing.

[0704] "Unclear instructions" refer to user input that is difficult to interpret because specific information is not clearly stated.

[0705] "Emotion information" refers to data that indicates an emotional state, such as "happiness," "sadness," or "excitement," extracted from instructions input by the user.

[0706] "Specific elements" refer to specific information such as the date and time, location, number of people, type of event, and user preferences extracted from the analyzed instructions.

[0707] An "intermediate representation" refers to data generated based on instructions and emotional information in a format that is easy for an external artificial intelligence model to understand.

[0708] An "artificial intelligence model" refers to a system that has algorithms for data analysis and information generation (e.g., machine learning models and deep learning models).

[0709] "Reception" refers to the process by which a terminal or a server receives information that has been transmitted remotely.

[0710] "Analysis" refers to the process of interpreting received information and converting it into meaningful data.

[0711] "Query" refers to the act of sending a request for information to an external system.

[0712] A "response" refers to the information or results returned by an external system in response to a query.

[0713] "Presenting" refers to the act of showing the received results or information to the user through a user interface.

[0714] This invention relates to a system that analyzes vague instructions entered by a user in natural language, takes emotional information into consideration, and then provides appropriate results using an external artificial intelligence model. This system is composed of a server, a terminal, a user, and an emotion engine.

[0715] A user inputs instructions into the device in natural language. For example, "Please tell me a recipe for an easy and delicious dish for a party next Saturday." This input can be in text or voice format. The device first receives the input instruction and sends the text or voice data contained in the input to the emotion engine. The emotion engine analyzes the emotion information from the user's input and extracts an emotional status such as "joy" or "excitement."

[0716] The device sends the user's instruction text and emotional information from the emotion engine together to the server. The instruction content and emotional information are sent to the server as structured data. The server combines the instruction and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date and time, location, number of people, type of event, and user preferences are extracted from the instruction, and the emotional information is further integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[0717] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for a generative AI model (such as OpenAI's GPT-4) to understand, and also includes emotional information. For example, it might look like this: "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[0718] The server uses this intermediate representation to query an external generative AI model. The intermediate representation is sent as the request body to the API endpoint of the generative AI model. The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the generated results will prioritize cooking recipes to get the user in the party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" could be returned.

[0719] The server receives the results returned from the generative AI model. The results are decoded, translated into Japanese as needed, and organized into data for presentation to the administrator. The results are processed into a format that provides content that matches the user's emotions, taking into particular account emotional information. The organized results are sent to the device, which displays them to the user. Appropriate information that reflects emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal; for example, preparing a dish based on the displayed recipe.

[0720] Specific examples

[0721] Example prompt sentence:

[0722] "Can you give me some easy and delicious recipes for a party next Saturday? I'm super excited."

[0723] The user's input is analyzed as an instruction such as "Tell me a simple and delicious recipe for a party next Saturday" and the emotional information of "excited." Based on this information, the server sends the appropriate intermediate representation to the generative AI model, and the returned result is recipes suitable for an excited user, such as "spaghetti arrabbiata" or "garlic shrimp."

[0724] In this way, users can receive information optimized for their input instructions and emotions and put it into action.

[0725] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0726] Step 1: User Enters Instructions

[0727] Specific operation: A user inputs a command into a terminal in natural language. For example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[0728] Input: User's natural language instructions (text or voice)

[0729] Output: The terminal receives the user's instructions.

[0730] Step 2: The emotion engine analyzes the emotion information

[0731] Specific operation: The device receives input instructions and sends text and voice data to the emotion engine. The emotion engine analyzes this and extracts the user's emotional status. For example, it extracts the emotional information of "excitement" from the user's instructions.

[0732] Input: User instructions, text or voice data

[0733] Output: Emotional information (e.g., "excitement")

[0734] Step 3: The device sends instructions and emotion information to the server.

[0735] Specific operation: The device sends the user's instruction text and the emotion information analyzed by the emotion engine to the server. This data is sent as structured data in JSON format or similar.

[0736] Input: User instructions and emotional information

[0737] Output: Structured data sent to the server

[0738] Step 4: The server parses the instructions and generates an intermediate representation

[0739] Specific operation: The server uses a natural language processing engine to analyze the received instructions and emotional information. Through the analysis, specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted and integrated with the emotional information. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[0740] Input: User instructions and emotional information

[0741] Output: Specific elements and emotional information

[0742] Step 5: The server queries the AI ​​model based on the intermediate representation.

[0743] Specific behavior: The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is in a format that is easy for the generative AI model to understand. For example, it might be in the format "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[0744] Input: concrete elements and emotional information

[0745] Output: The generated intermediate representation

[0746] Step 6: The server sends the intermediate representation to an external AI model

[0747] Specific operation: The server uses the intermediate representation to query the external generative AI model, and sends the intermediate representation to the API endpoint using the POST method.

[0748] Input: Intermediate Representation

[0749] Output: Sending an API request

[0750] Step 7: The AI ​​model returns the results

[0751] Specific operation: The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, for an "excited" user, recipes that will get them in a party mood will be prioritized. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" will be generated.

[0752] Input: Intermediate Representation

[0753] Output: The results generated

[0754] Step 8: The server receives and processes the results

[0755] Specific operation: The server receives the results from the generative AI model and decodes the response body. If necessary, it translates the results into Japanese and prepares them as data for presentation to the administrator. In particular, it processes the results based on emotional information into a format that matches the user's emotions.

[0756] Input: Generated results

[0757] Output: Cleaned data

[0758] Step 9: The terminal displays the results to the user

[0759] Specific actions: The device receives the preparation results sent from the server and visually displays them through the user interface. The user checks the displayed information and takes specific action, such as preparing a dish based on the presented recipe.

[0760] Input: Organized data

[0761] Output: The results displayed in the user interface

[0762] (Application example 2)

[0763] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0764] Conventional systems have the problem that they only analyze vague instructions entered by users in natural language and are unable to provide results that take the user's emotions into account. In particular, food delivery services are required to recommend the most appropriate dishes and restaurants based on the user's emotions, but current systems are inadequate in this regard. This makes it difficult to improve the user experience.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0766] In this invention, the server includes means for receiving vague instructions input in natural language, means for analyzing the user's instructions and breaking them down into multiple specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external artificial intelligence model and receiving a response, means for analyzing the user's emotions using an emotion engine and integrating the emotion information into the intermediate representation, and means for providing an optimal result that matches the user's emotions based on the emotion information. This allows for careful consideration of the user's emotions, and in the case of a food delivery service, makes it possible to recommend the most suitable dishes and restaurants to the user.

[0767] "Natural language" refers to language that humans use on a daily basis, which does not have a specific form or structure but contains meaning.

[0768] An "unclear instruction" refers to an instruction that is input by a user in natural language, but whose specific content or request is not clear unless analyzed.

[0769] "Analysis" is the process of analyzing input information and converting it into an understandable structure.

[0770] "Concrete elements" are the specific items or information extracted from the parsed imprecise instructions.

[0771] An "intermediate representation" is data generated based on specific elements in a format that is easy for an artificial intelligence model to understand.

[0772] An "artificial intelligence model" is a system that uses machine learning, deep learning, etc. to make predictions and classifications from input data.

[0773] An "emotion engine" is a system that analyzes emotions from user input and extracts that emotional information.

[0774] "Emotion information" is data relating to the user's emotional state obtained as a result of analysis by the emotion engine.

[0775] A "query" is an operation in which a system sends a request for data processing to an external artificial intelligence model.

[0776] "Response" refers to the processing result returned from an external artificial intelligence model.

[0777] The "means for generating a result" is a method or device that generates a result to be presented to the user based on the received response.

[0778] A "means for providing results" is a method or device for displaying the generated results to the user and enhancing the user experience.

[0779] The present invention relates to a system that analyzes unclear instructions entered by a user in natural language and provides optimal results based on the user's emotional information. A specific method for realizing this invention is described below.

[0780] System Configuration

[0781] This system consists of a user terminal, a server, an emotion engine, and an external artificial intelligence model.

[0782] Software / hardware used

[0783] Terminal: A device that allows users to input information, such as a smartphone or tablet

[0784] Server: A server that performs data analysis, sentiment analysis, intermediate representation generation, queries to external AI models, and results organization.

[0785] Emotion Engine: Software for Analyzing User Emotions

[0786] External Artificial Intelligence Models: AI models that use machine learning or deep learning to perform specific tasks, such as recipe recommendations or restaurant recommendations.

[0787] Processing flow

[0788] 1. The user enters instructions

[0789] Users input requests to food delivery apps in natural language, for example, "I'm tired, so please recommend some food that will help me relax."

[0790] 2. The device sends instructions to the server

[0791] The device receives the input and sends it to a server for sentiment analysis.

[0792] 3. The server analyzes emotions

[0793] The server uses an emotion engine to analyze emotions from the user's input, for example, extracting emotion information indicating "fatigue."

[0794] 4. The server generates an intermediate representation

[0795] The server generates an intermediate representation based on the user's instructions and emotional information. This intermediate representation is in a format that is easy for the AI ​​model to understand. For example, it may have the following format:

[0796] "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[0797] 5. The server queries the AI ​​model

[0798] The server uses the intermediate representation to query an external artificial intelligence model, which retrieves a list of the best dishes and restaurants.

[0799] 6. The server prepares the response

[0800] The server receives the response from the AI ​​model and prepares the results in the most optimal way for the user.

[0801] 7. The device will display the results

[0802] The device receives the results and displays them to the user, recommending dishes such as "Japanese sushi set" or "Thai green curry."

[0803] Examples of prompt statements

[0804] A prompt sentence is generated in the following format and sent to the AI ​​model:

[0805] "Generate recommendations for dishes that can uplift the mood. Consider the user is feeling 'happy'."

[0806] "Provide food suggestions that can help a tired user feel relaxed."

[0807] This allows for the provision of specific and effective results that correspond to the user's emotions.

[0808] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0809] Step 1:

[0810] The user inputs instructions.

[0811] A user inputs a request in natural language into a device such as a smartphone or tablet. For example, they input an instruction such as "I'm tired, so please recommend some dishes that will help me relax" by voice or text. This input is then received by the device.

[0812] Step 2:

[0813] The terminal sends instructions to the server.

[0814] The terminal transmits the received user input data to the server. Specifically, the user's text or voice data is transferred to the server. The input at this time is in natural language.

[0815] Step 3:

[0816] The server analyzes the emotions.

[0817] The server uses an emotion engine to analyze the received user input data. Through this analysis, the server extracts the user's emotion information (e.g., "fatigue"). The acquired emotion information is saved in the format "Emotion: Tired."

[0818] Step 4:

[0819] The server integrates the instructions and emotion information to generate an intermediate representation.

[0820] The server generates an intermediate representation based on the analyzed user instructions and emotional information. This intermediate representation is formatted to suit the AI ​​model. For example, it may contain content such as "The user is looking for relaxing recipes. The user is tired." Specifically, the following prompt sentence is generated: "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[0821] Step 5:

[0822] The server queries the AI ​​model for the intermediate representation.

[0823] The server sends the generated intermediate representation to an external artificial intelligence model. Specifically, it sends the intermediate representation as a request to the AI ​​model's API endpoint. In this case, the input is the intermediate representation, and the output is the recommendation result from the AI ​​model.

[0824] Step 6:

[0825] The server receives and organizes the response from the AI ​​model.

[0826] The server receives the response from the AI ​​model and formats it accordingly for the user. For example, it formats the recipes and restaurant lists obtained from the AI ​​model and converts them into a format that can be displayed in the user's language. This results in specific suggestions such as "Japanese sushi set" or "Thai green curry."

[0827] Step 7:

[0828] The terminal displays the results to the user.

[0829] The device displays the results received from the server to the user. Specifically, the results are presented in a user-friendly format using the smartphone's user interface. The user can then review the displayed results and select, for example, recommended dishes or restaurants.

[0830] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0831] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0832] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0833] [Third embodiment]

[0834] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0835] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0836] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0837] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0838] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0839] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0840] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0841] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0842] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0843] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0844] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0845] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0846] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for an AI to understand, and provides specific results. This system is composed of a server, a terminal, and a user.

[0847] 1. Overview of program processing

[0848] The user enters instructions

[0849] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday."

[0850] The device sends instructions to the server

[0851] The terminal receives the user's instructions and sends them to the server, which communicates with the terminal via a network.

[0852] The server parses the instructions and generates an intermediate representation

[0853] The server analyzes the user's instructions received from the device. Natural language processing technology is used for the analysis, and specific elements such as date and time, location, number of people, type of event, and user preferences are extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for the AI ​​to understand is generated.

[0854] The server sends a query to the AI ​​model based on the intermediate representation

[0855] The server uses the generated intermediate representation to query an external AI model, which is accessed through an API pre-configured by the system.

[0856] The AI ​​model returns the results

[0857] The AI ​​model generates results in response to queries and sends them back to the server.

[0858] The server processes the results for presentation to the user

[0859] The server receives the results returned by the AI ​​model and processes them in a format that is easy for users to understand, for example, by reorganizing the generated cooking recipe in Japanese.

[0860] The terminal displays the results to the user

[0861] The server sends the processed results to the terminal, which displays them to the user, who then checks the displayed information and achieves their goal.

[0862] 2. Specific examples of program processing

[0863] For example, consider a specific example where a user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[0864] 1. The user inputs instructions into the terminal in natural language.

[0865] 2. The device sends instructions to the server.

[0866] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[0867] 4. The server generates an intermediate representation based on the extracted elements, such as:

[0868] Generate recipes for a party on next Saturday. Consider the following details:

[0869] Event: Party

[0870] Date: Next Saturday

[0871] Location: User's home

[0872] Number of Guests: 10

[0873] Preferences: Easy to cook, delicious

[0874] 5. The server sends the intermediate representation to the AI ​​model.

[0875] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0876] 7. The server receives the results and reorganizes them in Japanese.

[0877] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[0878] The above is a specific embodiment of the present invention.

[0879] The processing flow will be explained below.

[0880] Step 1:

[0881] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday."

[0882] Step 2:

[0883] The device receives the user's instructions and stores them internally as text data. It then checks to see if the input has been properly saved.

[0884] Step 3:

[0885] The device sends the saved instructions to the server, using a method such as an HTTP request to send the instructions to the server as structured data.

[0886] Step 4:

[0887] The server receives instructions from the terminal, decodes the received data, and extracts the instruction content in text format.

[0888] Step 5:

[0889] The server analyzes the received instructions using a natural language processing engine, extracting specific elements from the instructions, such as the date, time, location, number of people, type of event, and user preferences.

[0890] Step 6:

[0891] The server generates an intermediate representation based on the extracted specific elements. This intermediate representation is generated in a format that is easy for the AI ​​model to understand. For example, it might be something like "Generate recipes for a party on next Saturday. Consider the following details: ...".

[0892] Step 7:

[0893] The server sends the generated intermediate representation to the external AI model as a query, sending a request to the AI ​​model's API endpoint.

[0894] Step 8:

[0895] The AI ​​model generates appropriate results based on queries from the server, which are then returned as a response to the request.

[0896] Step 9:

[0897] The server receives the response from the AI ​​model, decodes the received results, translates them into Japanese if necessary, and prepares the data to be presented to the user.

[0898] Step 10:

[0899] The server then sends the results of the analysis to the terminal, also in structured data format.

[0900] Step 11:

[0901] The terminal displays the results received from the server, and visually presents the results to the user through a user interface.

[0902] Step 12:

[0903] The user checks the content displayed on the device and performs a task based on the information provided, for example, preparing a dish based on a displayed recipe.

[0904] Example 1

[0905] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0906] Conventional systems have difficulty accurately analyzing vague instructions entered by users in natural language and providing appropriate results. Furthermore, they have problems with outputting results in different languages ​​and not taking into account a variety of specific factors. Systems that adjust to a variety of requirements often result in a poor user experience.

[0907] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0908] In this invention, the server includes means for receiving an unclear instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, means for processing the result for presentation to the user, and means for displaying the processed result to the user. This makes it possible to analyze unclear natural language input, convert it into a format that is easy for AI to understand, and provide appropriate results.

[0909] "Natural language" refers to the language used by humans in everyday life, including spoken and written forms.

[0910] "Unclear instructions" are abstract or vague words that do not specify specific content or requests.

[0911] "Analysis" is the process of breaking down input information, extracting its components, and understanding them.

[0912] "Specific elements" are detailed information extracted from the analyzed instructions, and include elements such as the date and time, location, number of people, type of event, and user preferences.

[0913] An "intermediate representation" is data that has been converted into a format that is easy for an artificial intelligence model to understand, based on specific elements extracted through analysis.

[0914] An "artificial intelligence model" is a computational model that uses techniques such as machine learning and deep learning to learn patterns and relationships from input data and generate output.

[0915] An "inquiry" is the act of sending a request to an external system or service to obtain specific information.

[0916] A "response" is information returned from an external system or service in response to a query.

[0917] "Generating results" refers to the process of creating tangible output for presentation to the user based on the responses received.

[0918] "Processing" refers to the steps taken to convert the generated results into a format that is easy for the user to understand.

[0919] "Display" refers to the act of providing the processed results to the user visually or audibly.

[0920] This invention is a system that analyzes vague instructions entered by users in natural language, converts them into a format that is easy for AI to understand, and provides specific results. This system consists of a server, a terminal, and a user.

[0921] System configuration

[0922] The user enters instructions

[0923] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday." This can be done using keyboard input or voice input.

[0924] The device sends instructions to the server

[0925] The device receives the user's instructions and sends them to the server. The device and server communicate via a network using HTTP requests and data formats such as JSON.

[0926] The server parses the instructions and generates an intermediate representation

[0927] The server uses natural language processing (NLP) technology to analyze the user instructions received from the device. Python libraries such as NLTK and SpaCy are often used for analysis. Specific elements such as the date and time, location, number of people, type of event, and user preferences are also extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for AI to understand is generated.

[0928] As a concrete example, the following intermediate representation is generated from the instruction "Please tell me a simple and delicious recipe for a party next Saturday":

[0929] Generate recipes for a party on next Saturday. Consider the following details:

[0930] Event: Party

[0931] Date: Next Saturday

[0932] Location: User's home

[0933] Number of Guests: 10

[0934] Preferences: Easy to cook, delicious

[0935] The server sends a query to the AI ​​model based on the intermediate representation

[0936] The server uses the generated intermediate representation to query the external AI model. This query is performed via a pre-configured API endpoint, typically a POST request that includes authentication information such as an API key or token.

[0937] The AI ​​model returns the results

[0938] The external generative AI model processes the data based on the received intermediate representation and generates results. These results are sent back to the server in JSON format. For example, recipe suggestions might be returned as a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0939] The server processes the results for presentation to the user

[0940] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts the JSON data into Japanese sentences that are easy for humans to understand. For example, it processes the results into a format such as "We suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0941] The terminal displays the results to the user

[0942] The server sends the processed results to the terminal, which receives them and displays them to the user. The results can be displayed as text on the screen or read aloud using speech synthesis technology. The user can check the results and obtain the desired information.

[0943] Specific examples

[0944] Here is a specific example where the user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[0945] 1. The user inputs instructions into the terminal in natural language.

[0946] 2. The device sends instructions to the server.

[0947] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[0948] 4. The server generates an intermediate representation based on the extracted elements.

[0949] 5. The server sends the intermediate representation to the AI ​​model.

[0950] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0951] 7. The server receives the results and reorganizes them in Japanese.

[0952] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[0953] As a result of the above, it is possible to implement a system that analyzes unclear instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results.

[0954] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0955] Step 1: User Enters Instructions

[0956] The user inputs instructions into the terminal in natural language. Specifically, the user inputs instructions such as "Please tell me a recipe for an easy and delicious dish for a party next Saturday" using a keyboard or voice input. The input data at this time is text or voice data. The input instructions are output and saved as text data on the terminal.

[0957] Step 2: The device sends instructions to the server

[0958] The device receives user instructions and sends them to the server. Specifically, it uses an HTTP request to send text instructions to the server's API endpoint. The input is the text data entered by the user, and the output is the data received by the server.

[0959] Step 3: The server parses the instructions and generates an intermediate representation

[0960] The server analyzes the user's instructions received from the device. For analysis, it uses natural language processing libraries such as Python's NLTK or SpaCy. The input is the user's text instructions, which the server breaks down into specific elements such as date and time, location, number of people, type of event, and user preferences. The output is the generation of an intermediate representation containing these elements.

[0961] Step 4: The server queries the AI ​​model based on the intermediate representation.

[0962] The server uses the generated intermediate representation to query the external AI model. Specifically, it sends the intermediate representation to the external API in JSON format. The input is the generated intermediate representation, and the output is the response from the AI ​​model. The server performs operations to send the request securely, including the API key and token.

[0963] Step 5: The AI ​​model returns the results

[0964] The external generative AI model generates results based on the received intermediate representation and sends them back to the server. The input is the JSON-formatted intermediate representation sent from the server, and the output is the resulting data, such as a cooking recipe. For example, a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad" is returned.

[0965] Step 6: The server processes the results for presentation to the user

[0966] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts JSON data into Japanese sentences that are easy for humans to understand. The input is the response from the AI ​​model, and the output is text reformatted into an easy-to-understand format. For example, it might convert it to "Suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[0967] Step 7: The terminal displays the results to the user

[0968] The server sends the processed results to the terminal, which receives them and displays them visually or audibly to the user. The input is the processed results sent from the server, and the output is the information presented to the user. Specifically, the terminal displays the received data on a screen or reads it out through a speaker.

[0969] (Application example 1)

[0970] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0971] Conventional food delivery services have the drawback of requiring users to manually select menu items and complete the ordering process when deciding on specific orders, which is time-consuming. Additionally, there is a lack of a system that can select the optimal menu based on vague instructions entered by users in natural language, leaving room for improvement in the user experience.

[0972] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0973] In this invention, the server includes means for receiving an imprecise instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, and means for converting the result into an order content related to a food delivery service, and means for processing the generated order content into a form that can be confirmed by the user and presenting it again. This enables an optimal food delivery order to be automatically generated based on the imprecise instruction input by the user in natural language, significantly improving the user experience.

[0974] A "natural language" is a language that humans use on a daily basis as a means of communication.

[0975] "Unclear instructions" are instructions that lack specific details and are therefore open to interpretation.

[0976] "Analysis" means analyzing input information and understanding its meaning and structure.

[0977] "Specific elements" are detailed information extracted from vague instructions, and include the date and time, type of food, and user preferences.

[0978] An "intermediate representation" is a representation generated based on analyzed concrete elements, in a form that is easy for an external artificial intelligence model to understand.

[0979] An "artificial intelligence model" is a program or algorithm that uses artificial intelligence technology to solve a specific problem.

[0980] A "query" is the act of sending a question or request to a system or model to obtain specific information or results.

[0981] A "response" is the information or result returned by an artificial intelligence model or system in response to a query.

[0982] A "food delivery service" is a service that delivers food ordered by a user to a specified location.

[0983] "Order details" is a list of dishes and ingredients that the user specifies to the food delivery service.

[0984] "Processing" means arranging the received information or results in a form that is easy for the user to understand.

[0985] "Re-presentation" refers to the act of displaying the processed information or results to the user again.

[0986] The present invention is a system that analyzes vague instructions entered by a user in natural language and generates specific food delivery order details. Specific embodiments for carrying out the present invention are described below.

[0987] Hardware Configuration

[0988] Smartphone: A device that allows users to input instructions in natural language and display the results.

[0989] Server: A back-end system that analyzes instructions and works with artificial intelligence models to generate optimal results.

[0990] Software Configuration

[0991] Natural language processing libraries: used to parse inputted, imprecise instructions, such as spaCy and NLTK.

[0992] HTTP communication library: Used to communicate between the server and the artificial intelligence model. A specific example is requests (a Python library).

[0993] Artificial intelligence models: Models for optimizing food delivery service orders, such as OpenAI GPT-3 and Google Cloud AI.

[0994] System Operation

[0995] 1. Entering and receiving instructions

[0996] A user opens the app on their smartphone and enters a command in natural language, such as "I'd like to order the recommended pizza for dinner tonight."

[0997] 2. Parsing the instructions

[0998] The smartphone sends this instruction to the server, which then uses a natural language processing library to analyze the received instruction and break it down into specific elements (date and time, type of food, user preferences, etc.).

[0999] 3. Generating Intermediate Representations

[1000] The server generates an intermediate representation based on the parsing result, such as the following prompt:

[1001] Generate order for tonight's dinner.

[1002] Consider the following details:

[1003] Food: Pizza

[1004] Time: Tonight

[1005] Preferences: Recommendation

[1006] 4. Querying the AI ​​model

[1007] The server uses this intermediate representation to query an external AI model, which then generates the optimal dish and order.

[1008] 5. Generating and displaying results

[1009] The server receives the response from the AI ​​model and generates a result to present to the user. For example, it may recommend "Margherita Pizza" and "Quattro Formaggi Pizza." This result is sent to the smartphone and displayed to the user.

[1010] Specific examples

[1011] When a user inputs a command such as "I'd like to order a recommended pizza for dinner tonight," the server analyzes the command and generates the following prompt for the AI ​​model:

[1012] Generate order for tonight's dinner.

[1013] Consider the following details:

[1014] Food: Pizza

[1015] Time: Tonight

[1016] Preferences: Recommendation

[1017] Based on this, the AI ​​model suggests the best pizzas, and the server reprocesses the suggestions and presents them to the user as "Margherita Pizza" and "Quattro Formaggi Pizza."

[1018] In this way, the system of the present invention allows users to easily place food delivery orders based on non-specific instructions entered in natural language.

[1019] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1020] Step 1:

[1021] A user accesses an application on their smartphone and inputs an order in natural language, for example, "I'd like to order the recommended pizza for dinner tonight." The input data is received by the smartphone.

[1022] Step 2:

[1023] The device sends the user's natural language input to the server, and the smartphone sends the data to the server via the network, including the input text.

[1024] Step 3:

[1025] The server parses the received natural language input. It uses a natural language processing library (e.g., spaCy or NLTK) to break down the instructions into specific elements (date and time, type of cuisine, user preferences, etc.) and outputs the input text as parsed data.

[1026] Step 4:

[1027] The server generates an intermediate representation based on the parsing results, which has the following format:

[1028] Generate order for tonight's dinner.

[1029] Consider the following details:

[1030] Food: Pizza

[1031] Time: Tonight

[1032] Preferences: Recommendation

[1033] The generated intermediate representation is output by the server as a prompt sentence.

[1034] Step 5:

[1035] The server queries the generated intermediate representation to the external AI model. Using an HTTP communication library (e.g., requests), it sends the intermediate representation to the AI ​​model and waits for a response from the model. The sent data is the intermediate representation, and the received data is the response from the AI ​​model.

[1036] Step 6:

[1037] The AI ​​model generates the optimal dish or order based on the query, such as "Margherita pizza" or "Quattro Formaggi pizza." This recommendation data is returned to the server from the AI ​​model.

[1038] Step 7:

[1039] The server processes the returned order information. In particular, it organizes the results to make them easier for the user to understand, and converts them into a format that can be displayed in Japanese. The final output data is an organized recommendation menu.

[1040] Step 8:

[1041] The smartphone presents the organized results to the user. The results are displayed on the device, allowing the user to check the recommended menu items provided and easily use food delivery. The displayed data is a list of recommended dishes.

[1042] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1043] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results by combining an emotion engine with a system that provides appropriate results that also take the user's emotions into consideration. This system is composed of a server, a terminal, a user, and an emotion engine.

[1044] 1. Overview of program processing

[1045] The user enters instructions

[1046] The user inputs instructions into the terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[1047] Emotion engine analyzes emotions

[1048] The device first receives the input instructions and then sends the text and voice data contained in the input to the emotion engine, which then analyzes the emotional information from the user's input and extracts an emotional status such as "joy," "sadness," or "excitement."

[1049] The device sends instructions and emotional information to the server.

[1050] The terminal transmits the user's instruction text and emotion information from the emotion engine together to the server. The instruction content and emotion information are sent to the server as structured data.

[1051] The server parses the instructions and generates an intermediate representation

[1052] The server combines the instructions and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted from the instructions, and the emotional information is then integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[1053] The server sends a query to the AI ​​model based on the intermediate representation

[1054] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and also includes emotional information. For example,

[1055] Generate recipes for a party on next Saturday. Consider the following details:

[1056] Event: Party

[1057] Date: Next Saturday

[1058] Location: User's home

[1059] Number of Guests: 10

[1060] Preferences: Easy to cook, delicious

[1061] User Emotion: Excited

[1062] It looks like this.

[1063] The server sends the intermediate representation to an external AI model.

[1064] The server uses this intermediate representation to query the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[1065] The AI ​​model returns the results

[1066] The AI ​​model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the AI ​​model will prioritize results that return cooking recipes to get the user in a party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" are likely to be returned.

[1067] The server receives and processes the results

[1068] The server receives the results returned by the AI ​​model. It decodes the results, translates them into Japanese if necessary, and prepares them as data to be presented to the user. In particular, the results are processed in a format that takes into account emotional information and provides content that matches the user's emotions.

[1069] The terminal displays the results to the user

[1070] The server sends the results of the analysis to the device, which then displays them to the user. Appropriate information reflecting the user's emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on a displayed recipe.

[1071] The above is a specific embodiment of the present invention.

[1072] The processing flow will be explained below.

[1073] Step 1:

[1074] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." Input can be in the form of text or voice.

[1075] Step 2:

[1076] The device receives the input instructions and saves them as text data. After confirming that the input has been saved, it sends the input data to the emotion engine.

[1077] Step 3:

[1078] The emotion engine installed in the device analyzes emotional information from the user's input data (text or voice). Specifically, it extracts emotional status such as "happiness," "sadness," and "excitement" from the user's vocabulary, tone of voice, and content.

[1079] Step 4:

[1080] The device sends the user's instruction text, including the parsed emotion information, to the server, usually in a structured format (e.g., JSON format).

[1081] Step 5:

[1082] The server receives the instruction and emotion information sent from the terminal. The received data is first decoded to extract the instruction text and emotion information.

[1083] Step 6:

[1084] The server analyzes the received instructions using a natural language processing engine. This analysis extracts specific elements from the instructions, such as the date and time, location, number of people, type of event, and the user's preferences. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" can be obtained.

[1085] Step 7:

[1086] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and includes emotional information. For example, the following intermediate representation is generated:

[1087] Generate recipes for a party on next Saturday. Consider the following details:

[1088] Event: Party

[1089] Date: Next Saturday

[1090] Location: User's home

[1091] Number of Guests: 10

[1092] Preferences: Easy to cook, delicious

[1093] User Emotion: Excited

[1094] Step 8:

[1095] The server sends this intermediate representation as a query to the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[1096] Step 9:

[1097] The AI ​​model generates appropriate results based on the intermediate representation from the server. For example, it takes into account the user's emotions (excitement) and prioritizes generating cooking recipes that will further enhance the party atmosphere. The generated results are sent back to the server as a response to the request.

[1098] Step 10:

[1099] The server receives the response from the AI ​​model. The received results are decoded, translated into Japanese as needed, and organized into data to be presented to the user. As a result of taking emotional information into consideration, the data is organized in a format that matches the user's emotions.

[1100] Step 11:

[1101] The server sends the prepared results to the terminal, also in a structured format.

[1102] Step 12:

[1103] The device displays the results received from the server to the user. Appropriate information reflecting emotional information is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on the displayed recipe.

[1104] Example 2

[1105] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1106] Conventional AI systems have had difficulty taking into account the user's emotional information when analyzing ambiguous instructions entered in natural language by the user. As a result, it has been difficult to provide an appropriate response that matches the user's emotions, resulting in a poor user experience. Furthermore, if emotional information is not taken into account, the results provided often do not meet the user's expectations.

[1107] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving an unclear instruction input in natural language, a means for extracting emotional information, and a means for analyzing the instruction and emotional information and breaking it down into multiple specific elements. This makes it possible to provide an appropriate result that takes the user's emotions into consideration. The server also includes a means for generating an intermediate representation based on the specific elements and emotional information, and a means for querying an external artificial intelligence model using the intermediate representation. This enables queries to be made in a format that is easy for the artificial intelligence model to understand, and a highly accurate response is returned. The server also includes a means for receiving a response from the artificial intelligence model, generating a result based on the user's emotions, and a means for presenting the result to the user. This provides information that matches the user's emotions, improving the user experience.

[1108] "Natural language" refers to the forms of language used by humans on a daily basis, including speech and writing.

[1109] "Unclear instructions" refer to user input that is difficult to interpret because specific information is not clearly stated.

[1110] "Emotion information" refers to data that indicates an emotional state, such as "happiness," "sadness," or "excitement," extracted from instructions input by the user.

[1111] "Specific elements" refer to specific information such as the date and time, location, number of people, type of event, and user preferences extracted from the analyzed instructions.

[1112] An "intermediate representation" refers to data generated based on instructions and emotional information in a format that is easy for an external artificial intelligence model to understand.

[1113] An "artificial intelligence model" refers to a system that has algorithms for data analysis and information generation (e.g., machine learning models and deep learning models).

[1114] "Reception" refers to the process by which a terminal or a server receives information that has been transmitted remotely.

[1115] "Analysis" refers to the process of interpreting received information and converting it into meaningful data.

[1116] "Query" refers to the act of sending a request for information to an external system.

[1117] A "response" refers to the information or results returned by an external system in response to a query.

[1118] "Presenting" refers to the act of showing the received results or information to the user through a user interface.

[1119] This invention relates to a system that analyzes vague instructions entered by a user in natural language, takes emotional information into consideration, and then provides appropriate results using an external artificial intelligence model. This system is composed of a server, a terminal, a user, and an emotion engine.

[1120] A user inputs instructions into the device in natural language. For example, "Please tell me a recipe for an easy and delicious dish for a party next Saturday." This input can be in text or voice format. The device first receives the input instruction and sends the text or voice data contained in the input to the emotion engine. The emotion engine analyzes the emotion information from the user's input and extracts an emotional status such as "joy" or "excitement."

[1121] The device sends the user's instruction text and emotional information from the emotion engine together to the server. The instruction content and emotional information are sent to the server as structured data. The server combines the instruction and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date and time, location, number of people, type of event, and user preferences are extracted from the instruction, and the emotional information is further integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[1122] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for a generative AI model (such as OpenAI's GPT-4) to understand, and also includes emotional information. For example, it might look like this: "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[1123] The server uses this intermediate representation to query an external generative AI model. The intermediate representation is sent as the request body to the API endpoint of the generative AI model. The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the generated results will prioritize cooking recipes to get the user in the party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" could be returned.

[1124] The server receives the results returned from the generative AI model. The results are decoded, translated into Japanese as needed, and organized into data for presentation to the administrator. The results are processed into a format that provides content that matches the user's emotions, taking into particular account emotional information. The organized results are sent to the device, which displays them to the user. Appropriate information that reflects emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal; for example, preparing a dish based on the displayed recipe.

[1125] Specific examples

[1126] Example prompt sentence:

[1127] "Can you give me some easy and delicious recipes for a party next Saturday? I'm super excited."

[1128] The user's input is analyzed as an instruction such as "Tell me a simple and delicious recipe for a party next Saturday" and the emotional information of "excited." Based on this information, the server sends the appropriate intermediate representation to the generative AI model, and the returned result is recipes suitable for an excited user, such as "spaghetti arrabbiata" or "garlic shrimp."

[1129] In this way, users can receive information optimized for their input instructions and emotions and put it into action.

[1130] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1131] Step 1: User Enters Instructions

[1132] Specific operation: A user inputs a command into a terminal in natural language. For example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[1133] Input: User's natural language instructions (text or voice)

[1134] Output: The terminal receives the user's instructions.

[1135] Step 2: The emotion engine analyzes the emotion information

[1136] Specific operation: The device receives input instructions and sends text and voice data to the emotion engine. The emotion engine analyzes this and extracts the user's emotional status. For example, it extracts the emotional information of "excitement" from the user's instructions.

[1137] Input: User instructions, text or voice data

[1138] Output: Emotional information (e.g., "excitement")

[1139] Step 3: The device sends instructions and emotion information to the server.

[1140] Specific operation: The device sends the user's instruction text and the emotion information analyzed by the emotion engine to the server. This data is sent as structured data in JSON format or similar.

[1141] Input: User instructions and emotional information

[1142] Output: Structured data sent to the server

[1143] Step 4: The server parses the instructions and generates an intermediate representation

[1144] Specific operation: The server uses a natural language processing engine to analyze the received instructions and emotional information. Through the analysis, specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted and integrated with the emotional information. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[1145] Input: User instructions and emotional information

[1146] Output: Specific elements and emotional information

[1147] Step 5: The server queries the AI ​​model based on the intermediate representation.

[1148] Specific behavior: The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is in a format that is easy for the generative AI model to understand. For example, it might be in the format "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[1149] Input: concrete elements and emotional information

[1150] Output: The generated intermediate representation

[1151] Step 6: The server sends the intermediate representation to an external AI model

[1152] Specific operation: The server uses the intermediate representation to query the external generative AI model, and sends the intermediate representation to the API endpoint using the POST method.

[1153] Input: Intermediate Representation

[1154] Output: Sending an API request

[1155] Step 7: The AI ​​model returns the results

[1156] Specific operation: The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, for an "excited" user, recipes that will get them in a party mood will be prioritized. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" will be generated.

[1157] Input: Intermediate Representation

[1158] Output: The results generated

[1159] Step 8: The server receives and processes the results

[1160] Specific operation: The server receives the results from the generative AI model and decodes the response body. If necessary, it translates the results into Japanese and prepares them as data for presentation to the administrator. In particular, it processes the results based on emotional information into a format that matches the user's emotions.

[1161] Input: Generated results

[1162] Output: Cleaned data

[1163] Step 9: The terminal displays the results to the user

[1164] Specific actions: The device receives the preparation results sent from the server and visually displays them through the user interface. The user checks the displayed information and takes specific action, such as preparing a dish based on the presented recipe.

[1165] Input: Organized data

[1166] Output: The results displayed in the user interface

[1167] (Application example 2)

[1168] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1169] Conventional systems have the problem that they only analyze vague instructions entered by users in natural language and are unable to provide results that take the user's emotions into account. In particular, food delivery services are required to recommend the most appropriate dishes and restaurants based on the user's emotions, but current systems are inadequate in this regard. This makes it difficult to improve the user experience.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1171] In this invention, the server includes means for receiving vague instructions input in natural language, means for analyzing the user's instructions and breaking them down into multiple specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external artificial intelligence model and receiving a response, means for analyzing the user's emotions using an emotion engine and integrating the emotion information into the intermediate representation, and means for providing an optimal result that matches the user's emotions based on the emotion information. This allows for careful consideration of the user's emotions, and in the case of a food delivery service, makes it possible to recommend the most suitable dishes and restaurants to the user.

[1172] "Natural language" refers to language that humans use on a daily basis, which does not have a specific form or structure but contains meaning.

[1173] An "unclear instruction" refers to an instruction that is input by a user in natural language, but whose specific content or request is not clear unless analyzed.

[1174] "Analysis" is the process of analyzing input information and converting it into an understandable structure.

[1175] "Concrete elements" are the specific items or information extracted from the parsed imprecise instructions.

[1176] An "intermediate representation" is data generated based on specific elements in a format that is easy for an artificial intelligence model to understand.

[1177] An "artificial intelligence model" is a system that uses machine learning, deep learning, etc. to make predictions and classifications from input data.

[1178] An "emotion engine" is a system that analyzes emotions from user input and extracts that emotional information.

[1179] "Emotion information" is data relating to the user's emotional state obtained as a result of analysis by the emotion engine.

[1180] A "query" is an operation in which a system sends a request for data processing to an external artificial intelligence model.

[1181] "Response" refers to the processing result returned from an external artificial intelligence model.

[1182] The "means for generating a result" is a method or device that generates a result to be presented to the user based on the received response.

[1183] A "means for providing results" is a method or device for displaying the generated results to the user and enhancing the user experience.

[1184] The present invention relates to a system that analyzes unclear instructions entered by a user in natural language and provides optimal results based on the user's emotional information. A specific method for realizing this invention is described below.

[1185] System Configuration

[1186] This system consists of a user terminal, a server, an emotion engine, and an external artificial intelligence model.

[1187] Software / hardware used

[1188] Terminal: A device that allows users to input information, such as a smartphone or tablet

[1189] Server: A server that performs data analysis, sentiment analysis, intermediate representation generation, queries to external AI models, and results organization.

[1190] Emotion Engine: Software for Analyzing User Emotions

[1191] External Artificial Intelligence Models: AI models that use machine learning or deep learning to perform specific tasks, such as recipe recommendations or restaurant recommendations.

[1192] Processing flow

[1193] 1. The user enters instructions

[1194] Users input requests to food delivery apps in natural language, for example, "I'm tired, so please recommend some food that will help me relax."

[1195] 2. The device sends instructions to the server

[1196] The device receives the input and sends it to a server for sentiment analysis.

[1197] 3. The server analyzes emotions

[1198] The server uses an emotion engine to analyze emotions from the user's input, for example, extracting emotion information indicating "fatigue."

[1199] 4. The server generates an intermediate representation

[1200] The server generates an intermediate representation based on the user's instructions and emotional information. This intermediate representation is in a format that is easy for the AI ​​model to understand. For example, it may have the following format:

[1201] "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[1202] 5. The server queries the AI ​​model

[1203] The server uses the intermediate representation to query an external artificial intelligence model, which retrieves a list of the best dishes and restaurants.

[1204] 6. The server prepares the response

[1205] The server receives the response from the AI ​​model and prepares the results in the most optimal way for the user.

[1206] 7. The device will display the results

[1207] The device receives the results and displays them to the user, recommending dishes such as "Japanese sushi set" or "Thai green curry."

[1208] Examples of prompt statements

[1209] A prompt sentence is generated in the following format and sent to the AI ​​model:

[1210] "Generate recommendations for dishes that can uplift the mood. Consider the user is feeling 'happy'."

[1211] "Provide food suggestions that can help a tired user feel relaxed."

[1212] This allows for the provision of specific and effective results that correspond to the user's emotions.

[1213] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1214] Step 1:

[1215] The user inputs instructions.

[1216] A user inputs a request in natural language into a device such as a smartphone or tablet. For example, they input an instruction such as "I'm tired, so please recommend some dishes that will help me relax" by voice or text. This input is then received by the device.

[1217] Step 2:

[1218] The terminal sends instructions to the server.

[1219] The terminal transmits the received user input data to the server. Specifically, the user's text or voice data is transferred to the server. The input at this time is in natural language.

[1220] Step 3:

[1221] The server analyzes the emotions.

[1222] The server uses an emotion engine to analyze the received user input data. Through this analysis, the server extracts the user's emotion information (e.g., "fatigue"). The acquired emotion information is saved in the format "Emotion: Tired."

[1223] Step 4:

[1224] The server integrates the instructions and emotion information to generate an intermediate representation.

[1225] The server generates an intermediate representation based on the analyzed user instructions and emotional information. This intermediate representation is formatted to suit the AI ​​model. For example, it may contain content such as "The user is looking for relaxing recipes. The user is tired." Specifically, the following prompt sentence is generated: "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[1226] Step 5:

[1227] The server queries the AI ​​model for the intermediate representation.

[1228] The server sends the generated intermediate representation to an external artificial intelligence model. Specifically, it sends the intermediate representation as a request to the AI ​​model's API endpoint. In this case, the input is the intermediate representation, and the output is the recommendation result from the AI ​​model.

[1229] Step 6:

[1230] The server receives and organizes the response from the AI ​​model.

[1231] The server receives the response from the AI ​​model and formats it accordingly for the user. For example, it formats the recipes and restaurant lists obtained from the AI ​​model and converts them into a format that can be displayed in the user's language. This results in specific suggestions such as "Japanese sushi set" or "Thai green curry."

[1232] Step 7:

[1233] The terminal displays the results to the user.

[1234] The device displays the results received from the server to the user. Specifically, the results are presented in a user-friendly format using the smartphone's user interface. The user can then review the displayed results and select, for example, recommended dishes or restaurants.

[1235] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1236] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1237] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1238] [Fourth embodiment]

[1239] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1240] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1241] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1242] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1243] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1244] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1245] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1246] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1247] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1248] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1249] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1250] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1251] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1252] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for an AI to understand, and provides specific results. This system is composed of a server, a terminal, and a user.

[1253] 1. Overview of program processing

[1254] The user enters instructions

[1255] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday."

[1256] The device sends instructions to the server

[1257] The terminal receives the user's instructions and sends them to the server, which communicates with the terminal via a network.

[1258] The server parses the instructions and generates an intermediate representation

[1259] The server analyzes the user's instructions received from the device. Natural language processing technology is used for the analysis, and specific elements such as date and time, location, number of people, type of event, and user preferences are extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for the AI ​​to understand is generated.

[1260] The server sends a query to the AI ​​model based on the intermediate representation

[1261] The server uses the generated intermediate representation to query an external AI model, which is accessed through an API pre-configured by the system.

[1262] The AI ​​model returns the results

[1263] The AI ​​model generates results in response to queries and sends them back to the server.

[1264] The server processes the results for presentation to the user

[1265] The server receives the results returned by the AI ​​model and processes them in a format that is easy for users to understand, for example, by reorganizing the generated cooking recipe in Japanese.

[1266] The terminal displays the results to the user

[1267] The server sends the processed results to the terminal, which displays them to the user, who then checks the displayed information and achieves their goal.

[1268] 2. Specific examples of program processing

[1269] For example, consider a specific example where a user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[1270] 1. The user inputs instructions into the terminal in natural language.

[1271] 2. The device sends instructions to the server.

[1272] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[1273] 4. The server generates an intermediate representation based on the extracted elements, such as:

[1274] Generate recipes for a party on next Saturday. Consider the following details:

[1275] Event: Party

[1276] Date: Next Saturday

[1277] Location: User's home

[1278] Number of Guests: 10

[1279] Preferences: Easy to cook, delicious

[1280] 5. The server sends the intermediate representation to the AI ​​model.

[1281] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[1282] 7. The server receives the results and reorganizes them in Japanese.

[1283] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[1284] The above is a specific embodiment of the present invention.

[1285] The processing flow will be explained below.

[1286] Step 1:

[1287] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday."

[1288] Step 2:

[1289] The device receives the user's instructions and stores them internally as text data. It then checks to see if the input has been properly saved.

[1290] Step 3:

[1291] The device sends the saved instructions to the server, using a method such as an HTTP request to send the instructions to the server as structured data.

[1292] Step 4:

[1293] The server receives instructions from the terminal, decodes the received data, and extracts the instruction content in text format.

[1294] Step 5:

[1295] The server analyzes the received instructions using a natural language processing engine, extracting specific elements from the instructions, such as the date, time, location, number of people, type of event, and user preferences.

[1296] Step 6:

[1297] The server generates an intermediate representation based on the extracted specific elements. This intermediate representation is generated in a format that is easy for the AI ​​model to understand. For example, it might be something like "Generate recipes for a party on next Saturday. Consider the following details: ...".

[1298] Step 7:

[1299] The server sends the generated intermediate representation to the external AI model as a query, sending a request to the AI ​​model's API endpoint.

[1300] Step 8:

[1301] The AI ​​model generates appropriate results based on queries from the server, which are then returned as a response to the request.

[1302] Step 9:

[1303] The server receives the response from the AI ​​model, decodes the received results, translates them into Japanese if necessary, and prepares the data to be presented to the user.

[1304] Step 10:

[1305] The server then sends the results of the analysis to the terminal, also in structured data format.

[1306] Step 11:

[1307] The terminal displays the results received from the server, and visually presents the results to the user through a user interface.

[1308] Step 12:

[1309] The user checks the content displayed on the device and performs a task based on the information provided, for example, preparing a dish based on a displayed recipe.

[1310] Example 1

[1311] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1312] Conventional systems have difficulty accurately analyzing vague instructions entered by users in natural language and providing appropriate results. Furthermore, they have problems with outputting results in different languages ​​and not taking into account a variety of specific factors. Systems that adjust to a variety of requirements often result in a poor user experience.

[1313] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1314] In this invention, the server includes means for receiving an unclear instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, means for processing the result for presentation to the user, and means for displaying the processed result to the user. This makes it possible to analyze unclear natural language input, convert it into a format that is easy for AI to understand, and provide appropriate results.

[1315] "Natural language" refers to the language used by humans in everyday life, including spoken and written forms.

[1316] "Unclear instructions" are abstract or vague words that do not specify specific content or requests.

[1317] "Analysis" is the process of breaking down input information, extracting its components, and understanding them.

[1318] "Specific elements" are detailed information extracted from the analyzed instructions, and include elements such as the date and time, location, number of people, type of event, and user preferences.

[1319] An "intermediate representation" is data that has been converted into a format that is easy for an artificial intelligence model to understand, based on specific elements extracted through analysis.

[1320] An "artificial intelligence model" is a computational model that uses techniques such as machine learning and deep learning to learn patterns and relationships from input data and generate output.

[1321] An "inquiry" is the act of sending a request to an external system or service to obtain specific information.

[1322] A "response" is information returned from an external system or service in response to a query.

[1323] "Generating results" refers to the process of creating tangible output for presentation to the user based on the responses received.

[1324] "Processing" refers to the steps taken to convert the generated results into a format that is easy for the user to understand.

[1325] "Display" refers to the act of providing the processed results to the user visually or audibly.

[1326] This invention is a system that analyzes vague instructions entered by users in natural language, converts them into a format that is easy for AI to understand, and provides specific results. This system consists of a server, a terminal, and a user.

[1327] System configuration

[1328] The user enters instructions

[1329] The user inputs instructions into the terminal in natural language, such as "Please tell me a recipe for a simple and delicious dish for a party next Saturday." This can be done using keyboard input or voice input.

[1330] The device sends instructions to the server

[1331] The device receives the user's instructions and sends them to the server. The device and server communicate via a network using HTTP requests and data formats such as JSON.

[1332] The server parses the instructions and generates an intermediate representation

[1333] The server uses natural language processing (NLP) technology to analyze the user instructions received from the device. Python libraries such as NLTK and SpaCy are often used for analysis. Specific elements such as the date and time, location, number of people, type of event, and user preferences are also extracted from the instructions. Based on these extracted elements, an intermediate representation that is easy for AI to understand is generated.

[1334] As a concrete example, the following intermediate representation is generated from the instruction "Please tell me a simple and delicious recipe for a party next Saturday":

[1335] Generate recipes for a party on next Saturday. Consider the following details:

[1336] Event: Party

[1337] Date: Next Saturday

[1338] Location: User's home

[1339] Number of Guests: 10

[1340] Preferences: Easy to cook, delicious

[1341] The server sends a query to the AI ​​model based on the intermediate representation

[1342] The server uses the generated intermediate representation to query the external AI model. This query is performed via a pre-configured API endpoint, typically a POST request that includes authentication information such as an API key or token.

[1343] The AI ​​model returns the results

[1344] The external generative AI model processes the data based on the received intermediate representation and generates results. These results are sent back to the server in JSON format. For example, recipe suggestions might be returned as a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[1345] The server processes the results for presentation to the user

[1346] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts the JSON data into Japanese sentences that are easy for humans to understand. For example, it processes the results into a format such as "We suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[1347] The terminal displays the results to the user

[1348] The server sends the processed results to the terminal, which receives them and displays them to the user. The results can be displayed as text on the screen or read aloud using speech synthesis technology. The user can check the results and obtain the desired information.

[1349] Specific examples

[1350] Here is a specific example where the user inputs "Please tell me some easy and delicious recipes for a party next Saturday."

[1351] 1. The user inputs instructions into the terminal in natural language.

[1352] 2. The device sends instructions to the server.

[1353] 3. The server analyzes the instructions and extracts specific elements such as the date and time (next Saturday), the type of event (party), and the user's preferences (easy and delicious food).

[1354] 4. The server generates an intermediate representation based on the extracted elements.

[1355] 5. The server sends the intermediate representation to the AI ​​model.

[1356] 6. The AI ​​model generates recipes based on the specified criteria and returns results such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[1357] 7. The server receives the results and reorganizes them in Japanese.

[1358] 8. The device displays the results to the user, who can then prepare the dish based on the displayed recipe.

[1359] As a result of the above, it is possible to implement a system that analyzes unclear instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results.

[1360] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1361] Step 1: User Enters Instructions

[1362] The user inputs instructions into the terminal in natural language. Specifically, the user inputs instructions such as "Please tell me a recipe for an easy and delicious dish for a party next Saturday" using a keyboard or voice input. The input data at this time is text or voice data. The input instructions are output and saved as text data on the terminal.

[1363] Step 2: The device sends instructions to the server

[1364] The device receives user instructions and sends them to the server. Specifically, it uses an HTTP request to send text instructions to the server's API endpoint. The input is the text data entered by the user, and the output is the data received by the server.

[1365] Step 3: The server parses the instructions and generates an intermediate representation

[1366] The server analyzes the user's instructions received from the device. For analysis, it uses natural language processing libraries such as Python's NLTK or SpaCy. The input is the user's text instructions, which the server breaks down into specific elements such as date and time, location, number of people, type of event, and user preferences. The output is the generation of an intermediate representation containing these elements.

[1367] Step 4: The server queries the AI ​​model based on the intermediate representation.

[1368] The server uses the generated intermediate representation to query the external AI model. Specifically, it sends the intermediate representation to the external API in JSON format. The input is the generated intermediate representation, and the output is the response from the AI ​​model. The server performs operations to send the request securely, including the API key and token.

[1369] Step 5: The AI ​​model returns the results

[1370] The external generative AI model generates results based on the received intermediate representation and sends them back to the server. The input is the JSON-formatted intermediate representation sent from the server, and the output is the resulting data, such as a cooking recipe. For example, a list such as "1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad" is returned.

[1371] Step 6: The server processes the results for presentation to the user

[1372] The server receives the results returned by the AI ​​model and processes them to present to the user. Specifically, it converts JSON data into Japanese sentences that are easy for humans to understand. The input is the response from the AI ​​model, and the output is text reformatted into an easy-to-understand format. For example, it might convert it to "Suggest the following three dishes as easy and delicious recipes: 1. Spaghetti Arrabbiata, 2. Garlic Shrimp, 3. Caesar Salad."

[1373] Step 7: The terminal displays the results to the user

[1374] The server sends the processed results to the terminal, which receives them and displays them visually or audibly to the user. The input is the processed results sent from the server, and the output is the information presented to the user. Specifically, the terminal displays the received data on a screen or reads it out through a speaker.

[1375] (Application example 1)

[1376] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1377] Conventional food delivery services have the drawback of requiring users to manually select menu items and complete the ordering process when deciding on specific orders, which is time-consuming. Additionally, there is a lack of a system that can select the optimal menu based on vague instructions entered by users in natural language, leaving room for improvement in the user experience.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1379] In this invention, the server includes means for receiving an imprecise instruction input in natural language, means for analyzing the instruction and breaking it down into a plurality of specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external AI model using the intermediate representation, means for receiving a response from the AI ​​model and generating a result to be presented to a user, and means for converting the result into an order content related to a food delivery service, and means for processing the generated order content into a form that can be confirmed by the user and presenting it again. This enables an optimal food delivery order to be automatically generated based on the imprecise instruction input by the user in natural language, significantly improving the user experience.

[1380] A "natural language" is a language that humans use on a daily basis as a means of communication.

[1381] "Unclear instructions" are instructions that lack specific details and are therefore open to interpretation.

[1382] "Analysis" means analyzing input information and understanding its meaning and structure.

[1383] "Specific elements" are detailed information extracted from vague instructions, and include the date and time, type of food, and user preferences.

[1384] An "intermediate representation" is a representation generated based on analyzed concrete elements, in a form that is easy for an external artificial intelligence model to understand.

[1385] An "artificial intelligence model" is a program or algorithm that uses artificial intelligence technology to solve a specific problem.

[1386] A "query" is the act of sending a question or request to a system or model to obtain specific information or results.

[1387] A "response" is the information or result returned by an artificial intelligence model or system in response to a query.

[1388] A "food delivery service" is a service that delivers food ordered by a user to a specified location.

[1389] "Order details" is a list of dishes and ingredients that the user specifies to the food delivery service.

[1390] "Processing" means arranging the received information or results in a form that is easy for the user to understand.

[1391] "Re-presentation" refers to the act of displaying the processed information or results to the user again.

[1392] The present invention is a system that analyzes vague instructions entered by a user in natural language and generates specific food delivery order details. Specific embodiments for carrying out the present invention are described below.

[1393] Hardware Configuration

[1394] Smartphone: A device that allows users to input instructions in natural language and display the results.

[1395] Server: A back-end system that analyzes instructions and works with artificial intelligence models to generate optimal results.

[1396] Software Configuration

[1397] Natural language processing libraries: used to parse inputted, imprecise instructions, such as spaCy and NLTK.

[1398] HTTP communication library: Used to communicate between the server and the artificial intelligence model. A specific example is requests (a Python library).

[1399] Artificial intelligence models: Models for optimizing food delivery service orders, such as OpenAI GPT-3 and Google Cloud AI.

[1400] System Operation

[1401] 1. Entering and receiving instructions

[1402] A user opens the app on their smartphone and enters a command in natural language, such as "I'd like to order the recommended pizza for dinner tonight."

[1403] 2. Parsing the instructions

[1404] The smartphone sends this instruction to the server, which then uses a natural language processing library to analyze the received instruction and break it down into specific elements (date and time, type of food, user preferences, etc.).

[1405] 3. Generating Intermediate Representations

[1406] The server generates an intermediate representation based on the parsing result, such as the following prompt:

[1407] Generate order for tonight's dinner.

[1408] Consider the following details:

[1409] Food: Pizza

[1410] Time: Tonight

[1411] Preferences: Recommendation

[1412] 4. Querying the AI ​​model

[1413] The server uses this intermediate representation to query an external AI model, which then generates the optimal dish and order.

[1414] 5. Generating and displaying results

[1415] The server receives the response from the AI ​​model and generates a result to present to the user. For example, it may recommend "Margherita Pizza" and "Quattro Formaggi Pizza." This result is sent to the smartphone and displayed to the user.

[1416] Specific examples

[1417] When a user inputs a command such as "I'd like to order a recommended pizza for dinner tonight," the server analyzes the command and generates the following prompt for the AI ​​model:

[1418] Generate order for tonight's dinner.

[1419] Consider the following details:

[1420] Food: Pizza

[1421] Time: Tonight

[1422] Preferences: Recommendation

[1423] Based on this, the AI ​​model suggests the best pizzas, and the server reprocesses the suggestions and presents them to the user as "Margherita Pizza" and "Quattro Formaggi Pizza."

[1424] In this way, the system of the present invention allows users to easily place food delivery orders based on non-specific instructions entered in natural language.

[1425] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1426] Step 1:

[1427] A user accesses an application on their smartphone and inputs an order in natural language, for example, "I'd like to order the recommended pizza for dinner tonight." The input data is received by the smartphone.

[1428] Step 2:

[1429] The device sends the user's natural language input to the server, and the smartphone sends the data to the server via the network, including the input text.

[1430] Step 3:

[1431] The server parses the received natural language input. It uses a natural language processing library (e.g., spaCy or NLTK) to break down the instructions into specific elements (date and time, type of cuisine, user preferences, etc.) and outputs the input text as parsed data.

[1432] Step 4:

[1433] The server generates an intermediate representation based on the parsing results, which has the following format:

[1434] Generate order for tonight's dinner.

[1435] Consider the following details:

[1436] Food: Pizza

[1437] Time: Tonight

[1438] Preferences: Recommendation

[1439] The generated intermediate representation is output by the server as a prompt sentence.

[1440] Step 5:

[1441] The server queries the generated intermediate representation to the external AI model. Using an HTTP communication library (e.g., requests), it sends the intermediate representation to the AI ​​model and waits for a response from the model. The sent data is the intermediate representation, and the received data is the response from the AI ​​model.

[1442] Step 6:

[1443] The AI ​​model generates the optimal dish or order based on the query, such as "Margherita pizza" or "Quattro Formaggi pizza." This recommendation data is returned to the server from the AI ​​model.

[1444] Step 7:

[1445] The server processes the returned order information. In particular, it organizes the results to make them easier for the user to understand, and converts them into a format that can be displayed in Japanese. The final output data is an organized recommendation menu.

[1446] Step 8:

[1447] The smartphone presents the organized results to the user. The results are displayed on the device, allowing the user to check the recommended menu items provided and easily use food delivery. The displayed data is a list of recommended dishes.

[1448] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1449] This invention relates to a system that analyzes vague instructions entered by a user in natural language, converts them into a format that is easy for AI to understand, and provides specific results by combining an emotion engine with a system that provides appropriate results that also take the user's emotions into consideration. This system is composed of a server, a terminal, a user, and an emotion engine.

[1450] 1. Overview of program processing

[1451] The user enters instructions

[1452] The user inputs instructions into the terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[1453] Emotion engine analyzes emotions

[1454] The device first receives the input instructions and then sends the text and voice data contained in the input to the emotion engine, which then analyzes the emotional information from the user's input and extracts an emotional status such as "joy," "sadness," or "excitement."

[1455] The device sends instructions and emotional information to the server.

[1456] The terminal transmits the user's instruction text and emotion information from the emotion engine together to the server. The instruction content and emotion information are sent to the server as structured data.

[1457] The server parses the instructions and generates an intermediate representation

[1458] The server combines the instructions and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted from the instructions, and the emotional information is then integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[1459] The server sends a query to the AI ​​model based on the intermediate representation

[1460] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and also includes emotional information. For example,

[1461] Generate recipes for a party on next Saturday. Consider the following details:

[1462] Event: Party

[1463] Date: Next Saturday

[1464] Location: User's home

[1465] Number of Guests: 10

[1466] Preferences: Easy to cook, delicious

[1467] User Emotion: Excited

[1468] It looks like this.

[1469] The server sends the intermediate representation to an external AI model.

[1470] The server uses this intermediate representation to query the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[1471] The AI ​​model returns the results

[1472] The AI ​​model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the AI ​​model will prioritize results that return cooking recipes to get the user in a party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" are likely to be returned.

[1473] The server receives and processes the results

[1474] The server receives the results returned by the AI ​​model. It decodes the results, translates them into Japanese if necessary, and prepares them as data to be presented to the user. In particular, the results are processed in a format that takes into account emotional information and provides content that matches the user's emotions.

[1475] The terminal displays the results to the user

[1476] The server sends the results of the analysis to the device, which then displays them to the user. Appropriate information reflecting the user's emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on a displayed recipe.

[1477] The above is a specific embodiment of the present invention.

[1478] The processing flow will be explained below.

[1479] Step 1:

[1480] A user inputs instructions into a terminal in natural language, for example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." Input can be in the form of text or voice.

[1481] Step 2:

[1482] The device receives the input instructions and saves them as text data. After confirming that the input has been saved, it sends the input data to the emotion engine.

[1483] Step 3:

[1484] The emotion engine installed in the device analyzes emotional information from the user's input data (text or voice). Specifically, it extracts emotional status such as "happiness," "sadness," and "excitement" from the user's vocabulary, tone of voice, and content.

[1485] Step 4:

[1486] The device sends the user's instruction text, including the parsed emotion information, to the server, usually in a structured format (e.g., JSON format).

[1487] Step 5:

[1488] The server receives the instruction and emotion information sent from the terminal. The received data is first decoded to extract the instruction text and emotion information.

[1489] Step 6:

[1490] The server analyzes the received instructions using a natural language processing engine. This analysis extracts specific elements from the instructions, such as the date and time, location, number of people, type of event, and the user's preferences. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" can be obtained.

[1491] Step 7:

[1492] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for the AI ​​model to understand, and includes emotional information. For example, the following intermediate representation is generated:

[1493] Generate recipes for a party on next Saturday. Consider the following details:

[1494] Event: Party

[1495] Date: Next Saturday

[1496] Location: User's home

[1497] Number of Guests: 10

[1498] Preferences: Easy to cook, delicious

[1499] User Emotion: Excited

[1500] Step 8:

[1501] The server sends this intermediate representation as a query to the external AI model, sending the intermediate representation as the request body to the AI ​​model's API endpoint.

[1502] Step 9:

[1503] The AI ​​model generates appropriate results based on the intermediate representation from the server. For example, it takes into account the user's emotions (excitement) and prioritizes generating cooking recipes that will further enhance the party atmosphere. The generated results are sent back to the server as a response to the request.

[1504] Step 10:

[1505] The server receives the response from the AI ​​model. The received results are decoded, translated into Japanese as needed, and organized into data to be presented to the user. As a result of taking emotional information into consideration, the data is organized in a format that matches the user's emotions.

[1506] Step 11:

[1507] The server sends the prepared results to the terminal, also in a structured format.

[1508] Step 12:

[1509] The device displays the results received from the server to the user. Appropriate information reflecting emotional information is visually provided through the user interface. The user confirms the displayed information and achieves their goal, such as preparing a dish based on the displayed recipe.

[1510] Example 2

[1511] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1512] Conventional AI systems have had difficulty taking into account the user's emotional information when analyzing ambiguous instructions entered in natural language by the user. As a result, it has been difficult to provide an appropriate response that matches the user's emotions, resulting in a poor user experience. Furthermore, if emotional information is not taken into account, the results provided often do not meet the user's expectations.

[1513] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving an unclear instruction input in natural language, a means for extracting emotional information, and a means for analyzing the instruction and emotional information and breaking it down into multiple specific elements. This makes it possible to provide an appropriate result that takes the user's emotions into consideration. The server also includes a means for generating an intermediate representation based on the specific elements and emotional information, and a means for querying an external artificial intelligence model using the intermediate representation. This enables queries to be made in a format that is easy for the artificial intelligence model to understand, and a highly accurate response is returned. The server also includes a means for receiving a response from the artificial intelligence model, generating a result based on the user's emotions, and a means for presenting the result to the user. This provides information that matches the user's emotions, improving the user experience.

[1514] "Natural language" refers to the forms of language used by humans on a daily basis, including speech and writing.

[1515] "Unclear instructions" refer to user input that is difficult to interpret because specific information is not clearly stated.

[1516] "Emotion information" refers to data that indicates an emotional state, such as "happiness," "sadness," or "excitement," extracted from instructions input by the user.

[1517] "Specific elements" refer to specific information such as the date and time, location, number of people, type of event, and user preferences extracted from the analyzed instructions.

[1518] An "intermediate representation" refers to data generated based on instructions and emotional information in a format that is easy for an external artificial intelligence model to understand.

[1519] An "artificial intelligence model" refers to a system that has algorithms for data analysis and information generation (e.g., machine learning models and deep learning models).

[1520] "Reception" refers to the process by which a terminal or a server receives information that has been transmitted remotely.

[1521] "Analysis" refers to the process of interpreting received information and converting it into meaningful data.

[1522] "Query" refers to the act of sending a request for information to an external system.

[1523] A "response" refers to the information or results returned by an external system in response to a query.

[1524] "Presenting" refers to the act of showing the received results or information to the user through a user interface.

[1525] This invention relates to a system that analyzes vague instructions entered by a user in natural language, takes emotional information into consideration, and then provides appropriate results using an external artificial intelligence model. This system is composed of a server, a terminal, a user, and an emotion engine.

[1526] A user inputs instructions into the device in natural language. For example, "Please tell me a recipe for an easy and delicious dish for a party next Saturday." This input can be in text or voice format. The device first receives the input instruction and sends the text or voice data contained in the input to the emotion engine. The emotion engine analyzes the emotion information from the user's input and extracts an emotional status such as "joy" or "excitement."

[1527] The device sends the user's instruction text and emotional information from the emotion engine together to the server. The instruction content and emotional information are sent to the server as structured data. The server combines the instruction and emotional information received from the device and analyzes them using a natural language processing engine. Specific elements such as the date and time, location, number of people, type of event, and user preferences are extracted from the instruction, and the emotional information is further integrated. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[1528] The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is generated in a format that is easy for a generative AI model (such as OpenAI's GPT-4) to understand, and also includes emotional information. For example, it might look like this: "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[1529] The server uses this intermediate representation to query an external generative AI model. The intermediate representation is sent as the request body to the API endpoint of the generative AI model. The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, if the user is excited, the generated results will prioritize cooking recipes to get the user in the party mood. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" could be returned.

[1530] The server receives the results returned from the generative AI model. The results are decoded, translated into Japanese as needed, and organized into data for presentation to the administrator. The results are processed into a format that provides content that matches the user's emotions, taking into particular account emotional information. The organized results are sent to the device, which displays them to the user. Appropriate information that reflects emotions is visually provided through the user interface. The user confirms the displayed information and achieves their goal; for example, preparing a dish based on the displayed recipe.

[1531] Specific examples

[1532] Example prompt sentence:

[1533] "Can you give me some easy and delicious recipes for a party next Saturday? I'm super excited."

[1534] The user's input is analyzed as an instruction such as "Tell me a simple and delicious recipe for a party next Saturday" and the emotional information of "excited." Based on this information, the server sends the appropriate intermediate representation to the generative AI model, and the returned result is recipes suitable for an excited user, such as "spaghetti arrabbiata" or "garlic shrimp."

[1535] In this way, users can receive information optimized for their input instructions and emotions and put it into action.

[1536] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1537] Step 1: User Enters Instructions

[1538] Specific operation: A user inputs a command into a terminal in natural language. For example, "Please give me a recipe for a simple and delicious dish for a party next Saturday." This input can be in the form of text or voice.

[1539] Input: User's natural language instructions (text or voice)

[1540] Output: The terminal receives the user's instructions.

[1541] Step 2: The emotion engine analyzes the emotion information

[1542] Specific operation: The device receives input instructions and sends text and voice data to the emotion engine. The emotion engine analyzes this and extracts the user's emotional status. For example, it extracts the emotional information of "excitement" from the user's instructions.

[1543] Input: User instructions, text or voice data

[1544] Output: Emotional information (e.g., "excitement")

[1545] Step 3: The device sends instructions and emotion information to the server.

[1546] Specific operation: The device sends the user's instruction text and the emotion information analyzed by the emotion engine to the server. This data is sent as structured data in JSON format or similar.

[1547] Input: User instructions and emotional information

[1548] Output: Structured data sent to the server

[1549] Step 4: The server parses the instructions and generates an intermediate representation

[1550] Specific operation: The server uses a natural language processing engine to analyze the received instructions and emotional information. Through the analysis, specific elements such as the date, time, location, number of people, type of event, and user preferences are extracted and integrated with the emotional information. For example, elements such as "next Saturday," "party," "easy and delicious cooking," and "the user is excited" are extracted.

[1551] Input: User instructions and emotional information

[1552] Output: Specific elements and emotional information

[1553] Step 5: The server queries the AI ​​model based on the intermediate representation.

[1554] Specific behavior: The server generates an intermediate representation based on the extracted specific elements and emotional information. This intermediate representation is in a format that is easy for the generative AI model to understand. For example, it might be in the format "Generate recipes for a party on next Saturday. Consider the following details: - Event: Party - Date: Next Saturday - Location: User's home - Number of Guests: 10 - Preferences: Easy to cook, delicious - User Emotion: Excited."

[1555] Input: concrete elements and emotional information

[1556] Output: The generated intermediate representation

[1557] Step 6: The server sends the intermediate representation to an external AI model

[1558] Specific operation: The server uses the intermediate representation to query the external generative AI model, and sends the intermediate representation to the API endpoint using the POST method.

[1559] Input: Intermediate Representation

[1560] Output: Sending an API request

[1561] Step 7: The AI ​​model returns the results

[1562] Specific operation: The generative AI model generates appropriate results based on the intermediate representation according to the specified conditions and emotional information. For example, for an "excited" user, recipes that will get them in a party mood will be prioritized. Results such as "spaghetti arrabbiata," "garlic shrimp," and "Caesar salad" will be generated.

[1563] Input: Intermediate Representation

[1564] Output: The results generated

[1565] Step 8: The server receives and processes the results

[1566] Specific operation: The server receives the results from the generative AI model and decodes the response body. If necessary, it translates the results into Japanese and prepares them as data for presentation to the administrator. In particular, it processes the results based on emotional information into a format that matches the user's emotions.

[1567] Input: Generated results

[1568] Output: Cleaned data

[1569] Step 9: The terminal displays the results to the user

[1570] Specific actions: The device receives the preparation results sent from the server and visually displays them through the user interface. The user checks the displayed information and takes specific action, such as preparing a dish based on the presented recipe.

[1571] Input: Organized data

[1572] Output: The results displayed in the user interface

[1573] (Application example 2)

[1574] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1575] Conventional systems have the problem that they only analyze vague instructions entered by users in natural language and are unable to provide results that take the user's emotions into account. In particular, food delivery services are required to recommend the most appropriate dishes and restaurants based on the user's emotions, but current systems are inadequate in this regard. This makes it difficult to improve the user experience.

[1576] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1577] In this invention, the server includes means for receiving vague instructions input in natural language, means for analyzing the user's instructions and breaking them down into multiple specific elements, means for generating an intermediate representation based on the specific elements, means for querying an external artificial intelligence model and receiving a response, means for analyzing the user's emotions using an emotion engine and integrating the emotion information into the intermediate representation, and means for providing an optimal result that matches the user's emotions based on the emotion information. This allows for careful consideration of the user's emotions, and in the case of a food delivery service, makes it possible to recommend the most suitable dishes and restaurants to the user.

[1578] "Natural language" refers to language that humans use on a daily basis, which does not have a specific form or structure but contains meaning.

[1579] An "unclear instruction" refers to an instruction that is input by a user in natural language, but whose specific content or request is not clear unless analyzed.

[1580] "Analysis" is the process of analyzing input information and converting it into an understandable structure.

[1581] "Concrete elements" are the specific items or information extracted from the parsed imprecise instructions.

[1582] An "intermediate representation" is data generated based on specific elements in a format that is easy for an artificial intelligence model to understand.

[1583] An "artificial intelligence model" is a system that uses machine learning, deep learning, etc. to make predictions and classifications from input data.

[1584] An "emotion engine" is a system that analyzes emotions from user input and extracts that emotional information.

[1585] "Emotion information" is data relating to the user's emotional state obtained as a result of analysis by the emotion engine.

[1586] A "query" is an operation in which a system sends a request for data processing to an external artificial intelligence model.

[1587] "Response" refers to the processing result returned from an external artificial intelligence model.

[1588] The "means for generating a result" is a method or device that generates a result to be presented to the user based on the received response.

[1589] A "means for providing results" is a method or device for displaying the generated results to the user and enhancing the user experience.

[1590] The present invention relates to a system that analyzes unclear instructions entered by a user in natural language and provides optimal results based on the user's emotional information. A specific method for realizing this invention is described below.

[1591] System Configuration

[1592] This system consists of a user terminal, a server, an emotion engine, and an external artificial intelligence model.

[1593] Software / hardware used

[1594] Terminal: A device that allows users to input information, such as a smartphone or tablet

[1595] Server: A server that performs data analysis, sentiment analysis, intermediate representation generation, queries to external AI models, and results organization.

[1596] Emotion Engine: Software for Analyzing User Emotions

[1597] External Artificial Intelligence Models: AI models that use machine learning or deep learning to perform specific tasks, such as recipe recommendations or restaurant recommendations.

[1598] Processing flow

[1599] 1. The user enters instructions

[1600] Users input requests to food delivery apps in natural language, for example, "I'm tired, so please recommend some food that will help me relax."

[1601] 2. The device sends instructions to the server

[1602] The device receives the input and sends it to a server for sentiment analysis.

[1603] 3. The server analyzes emotions

[1604] The server uses an emotion engine to analyze emotions from the user's input, for example, extracting emotion information indicating "fatigue."

[1605] 4. The server generates an intermediate representation

[1606] The server generates an intermediate representation based on the user's instructions and emotional information. This intermediate representation is in a format that is easy for the AI ​​model to understand. For example, it may have the following format:

[1607] "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[1608] 5. The server queries the AI ​​model

[1609] The server uses the intermediate representation to query an external artificial intelligence model, which retrieves a list of the best dishes and restaurants.

[1610] 6. The server prepares the response

[1611] The server receives the response from the AI ​​model and prepares the results in the most optimal way for the user.

[1612] 7. The device will display the results

[1613] The device receives the results and displays them to the user, recommending dishes such as "Japanese sushi set" or "Thai green curry."

[1614] Examples of prompt statements

[1615] A prompt sentence is generated in the following format and sent to the AI ​​model:

[1616] "Generate recommendations for dishes that can uplift the mood. Consider the user is feeling 'happy'."

[1617] "Provide food suggestions that can help a tired user feel relaxed."

[1618] This allows for the provision of specific and effective results that correspond to the user's emotions.

[1619] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1620] Step 1:

[1621] The user inputs instructions.

[1622] A user inputs a request in natural language into a device such as a smartphone or tablet. For example, they input an instruction such as "I'm tired, so please recommend some dishes that will help me relax" by voice or text. This input is then received by the device.

[1623] Step 2:

[1624] The terminal sends instructions to the server.

[1625] The terminal transmits the received user input data to the server. Specifically, the user's text or voice data is transferred to the server. The input at this time is in natural language.

[1626] Step 3:

[1627] The server analyzes the emotions.

[1628] The server uses an emotion engine to analyze the received user input data. Through this analysis, the server extracts the user's emotion information (e.g., "fatigue"). The acquired emotion information is saved in the format "Emotion: Tired."

[1629] Step 4:

[1630] The server integrates the instructions and emotion information to generate an intermediate representation.

[1631] The server generates an intermediate representation based on the analyzed user instructions and emotional information. This intermediate representation is formatted to suit the AI ​​model. For example, it may contain content such as "The user is looking for relaxing recipes. The user is tired." Specifically, the following prompt sentence is generated: "Generate recipes that help the user feel relaxed. Consider the user is feeling 'tired'."

[1632] Step 5:

[1633] The server queries the AI ​​model for the intermediate representation.

[1634] The server sends the generated intermediate representation to an external artificial intelligence model. Specifically, it sends the intermediate representation as a request to the AI ​​model's API endpoint. In this case, the input is the intermediate representation, and the output is the recommendation result from the AI ​​model.

[1635] Step 6:

[1636] The server receives and organizes the response from the AI ​​model.

[1637] The server receives the response from the AI ​​model and formats it accordingly for the user. For example, it formats the recipes and restaurant lists obtained from the AI ​​model and converts them into a format that can be displayed in the user's language. This results in specific suggestions such as "Japanese sushi set" or "Thai green curry."

[1638] Step 7:

[1639] The terminal displays the results to the user.

[1640] The device displays the results received from the server to the user. Specifically, the results are presented in a user-friendly format using the smartphone's user interface. The user can then review the displayed results and select, for example, recommended dishes or restaurants.

[1641] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1642] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1643] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1644] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1645] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1646] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1647] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1648] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, motorcycles, and other devices, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1649] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1650] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1651] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1652] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1653] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1654] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1655] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1656] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1657] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1658] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1659] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1660] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1661] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1662] The following is further disclosed regarding the above embodiment.

[1663] (Claim 1)

[1664] means for receiving unspecified instructions input in natural language;

[1665] means for parsing and breaking down the instructions into a plurality of specific elements;

[1666] means for generating an intermediate representation based on the concrete elements;

[1667] means for querying an external artificial intelligence model using said intermediate representation;

[1668] means for receiving responses from the artificial intelligence model and generating results for presentation to a user;

[1669] A system including:

[1670] (Claim 2)

[1671] 2. The system of claim 1, wherein the intermediate representation is generated in a format that is easy for the artificial intelligence model to understand.

[1672] (Claim 3)

[1673] 2. The system of claim 1, wherein the plurality of specific elements include date and time, location, number of people, type of event, and user preferences.

[1674] "Example 1"

[1675] (Claim 1)

[1676] means for receiving unspecified instructions input in natural language;

[1677] means for parsing and breaking down the instructions into a plurality of specific elements;

[1678] means for generating an intermediate representation based on the concrete elements;

[1679] means for querying an external artificial intelligence model using said intermediate representation;

[1680] means for receiving responses from the artificial intelligence model and generating results for presentation to a user;

[1681] means for processing the results for presentation to a user;

[1682] means for displaying the processed result to a user;

[1683] A system including:

[1684] (Claim 2)

[1685] 2. The system of claim 1, wherein the intermediate representation is generated in a format that is easy for the artificial intelligence model to understand.

[1686] (Claim 3)

[1687] 2. The system of claim 1, wherein the plurality of specific elements include date and time, location, number of people, type of event, and user preferences.

[1688] "Application Example 1"

[1689] (Claim 1)

[1690] means for receiving unspecified instructions input in natural language;

[1691] means for parsing and breaking down the instructions into a plurality of specific elements;

[1692] means for generating an intermediate representation based on the concrete elements;

[1693] means for querying an external artificial intelligence model using said intermediate representation;

[1694] means for receiving responses from the artificial intelligence model and generating results for presentation to a user;

[1695] a means for converting the result into order details related to a food delivery service, and processing and re-presenting the generated order details in a form that can be confirmed by the user;

[1696] A system including:

[1697] (Claim 2)

[1698] 2. The system of claim 1, wherein the intermediate representation is generated in a format that is understandable to the artificial intelligence model and is adjusted based on conditions related to the food delivery service.

[1699] (Claim 3)

[1700] The system of claim 1, wherein the plurality of specific elements include date and time, type of food, and user preferences, and automatically generates a food delivery order based on these.

[1701] "Example 2: Combining Emotion Engines"

[1702] (Claim 1)

[1703] means for receiving unspecified instructions input in natural language;

[1704] means for analyzing the instruction and extracting emotion information;

[1705] means for analyzing the instruction and emotion information and breaking it down into a plurality of specific elements;

[1706] means for generating an intermediate representation based on the specific elements and emotion information;

[1707] means for querying an external artificial intelligence model using said intermediate representation;

[1708] means for receiving a response from the artificial intelligence model and generating a result based on the user's sentiment;

[1709] means for presenting the results to a user;

[1710] A system including:

[1711] (Claim 2)

[1712] 2. The system according to claim 1, wherein the intermediate representation is generated in a format that is easy for the artificial intelligence model to understand and includes emotional information.

[1713] (Claim 3)

[1714] 2. The system according to claim 1, wherein the plurality of specific elements include date and time, location, number of people, type of event, user preferences, and user emotions.

[1715] "Application example 2 when combining emotion engines"

[1716] (Claim 1)

[1717] means for receiving unspecified instructions input in natural language;

[1718] means for parsing and breaking down the instructions into a plurality of specific elements;

[1719] means for generating an intermediate representation based on the concrete elements;

[1720] means for querying an external artificial intelligence model using said intermediate representation;

[1721] means for receiving responses from the artificial intelligence model and generating results for presentation to a user;

[1722] means for analyzing a user's emotions using an emotion engine and integrating the emotion information into the intermediate representation;

[1723] means for providing an optimal result that matches the emotion of the user based on the emotion information;

[1724] A system including:

[1725] (Claim 2)

[1726] 2. The system according to claim 1, wherein the intermediate representation is generated in a format that is easy for the artificial intelligence model to understand and includes emotional information.

[1727] (Claim 3)

[1728] 2. The system according to claim 1, wherein the plurality of specific elements include date and time, location, number of people, type of event, user preferences and emotional information. [Explanation of symbols]

[1729] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving unspecified instructions input in natural language; means for parsing and breaking down the instructions into a plurality of specific elements; means for generating an intermediate representation based on the concrete elements; means for querying an external artificial intelligence model using said intermediate representation; means for receiving responses from the artificial intelligence model and generating results for presentation to a user; A system including:

2. 2. The system of claim 1, wherein the intermediate representation is generated in a format that is easy for the artificial intelligence model to understand.

3. The system of claim 1 , wherein the plurality of specific elements include date and time, location, number of people, type of event, and user preferences.

Images