system

Abstract

We provide the system. [Solution] A means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, Means for checking the output and adding the order details to the list based on the output, A means of calculating the total amount and notifying the user, A system that includes this.

Description

Technical Field

[0004] , , , ,

[0005] , , , , ,

[0003] , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional in-store order processing systems have problems such as high labor costs and a high likelihood of human errors because they manually receive, confirm, and process user orders. Also, when customer orders are complex or when accepting multiple orders at once, it is difficult to process smoothly and waiting times become long. To solve such a situation, it is necessary to automate order processing to reduce costs and speed up order processing.

Means for Solving the Problems

[0005] This invention provides a system that includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, and means for calculating the total amount and notifying the user. This allows users to place orders in natural language, and the AI ​​analyzes the input content to identify the order details in real time, add them to a list, and finally calculate the total amount and notify the user. Furthermore, by handling multiple orders from the user, smooth order processing is possible. As a result, labor costs are reduced and order processing efficiency is improved, and customer satisfaction is expected to increase.

[0006] An "artificial intelligence engine" is software or an algorithm that analyzes user input and outputs order details in a specific format.

[0007] "Natural language" refers to the forms of language and writing that humans use on a daily basis.

[0008] "Analysis" is the process of understanding the content based on the input information and extracting information that aligns with the objective.

[0009] A "user" is a person who uses the system to place an order.

[0010] An "order" is a list of items or services that a user wishes to purchase or receive.

[0011] "Format" refers to a specific type or format in which data or information is structured.

[0012] "Output" refers to the act of presenting the analyzed information to the user or passing it to the system.

[0013] A "list" is a data structure that enumerates multiple items.

[0014] "Append" refers to the act of adding new items to an existing list or dataset.

[0015] "Total amount" refers to the amount obtained by summing up the prices of all ordered items.

[0016] "Notification" refers to the act of the system providing information to the user.

[0017] "System" refers to a collection of multiple software and hardware that cooperate to perform order processing.

[0018] [[ID=1,5]] "Order details" refer to the information of items and services specified by the user.

Brief Explanation of Drawings

[0019] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10]Shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0023] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0024] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] This invention relates to an order processing system that uses generative AI to analyze user in-store orders and reduce labor costs. The embodiments for carrying out this invention will be described in detail below.

[0041] System Overview

[0042] System configuration:

[0043] This system primarily consists of three elements: servers, terminals, and users.

[0044] Server: The server sets the API key for the generative AI and manages the list of menu items. It also analyzes user input and outputs the order details in a specific format.

[0045] Terminal: The terminal provides an interface for receiving orders from users in natural language and communicates with the server to process the orders.

[0046] User: The user enters their order via a terminal and receives a response from the system.

[0047] Program Description

[0048] server:

[0049] The server performs the following specific actions:

[0050] 1. API Key Configuration: The server configures API keys for generative AI (such as OpenAI®) to enable the use of specific AI engines.

[0051] 2. Menu Item Management: The server maintains a list of menu items that users can order. This list includes the item's ID, name, and price.

[0052] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[0053] Specific example:

[0054] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[0055] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[0056] Terminal:

[0057] The terminal performs the following specific actions:

[0058] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[0059] 2. Sending the order details: The terminal sends the received order details to the server.

[0060] Specific example:

[0061] When a user types "Please add some french fries," the terminal sends this input to the server.

[0062] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[0063] User:

[0064] The user will perform the following specific operations through this system.

[0065] 1. Order Input: The user enters their order details in natural language.

[0066] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0067] This system allows users to place orders using natural language, which are then analyzed in real time by generative AI, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[0068] The following describes the processing flow.

[0069] Step 1:

[0070] Server: Set up the OpenAI API key and prepare the AI ​​engine to be used (e.g., "text-davinci-003"). Also, define a list of menu items and store the ID, name, and price of each item.

[0071] Step 2:

[0072] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[0073] Step 3:

[0074] User: Enter the items you want to order in natural language. For example, "I'd like one hamburger and a soda."

[0075] Step 4:

[0076] Terminal: Receives user input and sends the input to the server.

[0077] Step 5:

[0078] Server: Receives user input and generates prompts for the AI ​​to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0079] Step 6:

[0080] Server: Sends the generated prompt to the AI ​​engine and waits for the analysis results.

[0081] Step 7:

[0082] AI: Analyzes user input and returns order details as JSON data. For example, it returns in the format {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[0083] Step 8:

[0084] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[0085] Step 9:

[0086] Server: Notifies the user that an order has been added. For example, it might notify them with a message like, "A hamburger has been added," or "A soda has been added."

[0087] Step 10:

[0088] Terminal: Receives notifications from the server and displays them to the user.

[0089] Step 11:

[0090] User: Enter any additional orders as needed. For example, "Please add one more order of french fries."

[0091] Step 12:

[0092] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[0093] Step 13:

[0094] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[0095] Step 14:

[0096] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[0097] Step 15:

[0098] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[0099] Step 16:

[0100] System: At this point, the order process is complete and the program terminates.

[0101] Through the steps described above, this system can automatically process user orders, thereby reducing labor costs and improving efficiency.

[0102] (Example 1)

[0103] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0104] Traditional order processing systems involve numerous manual processes, such as receiving and confirming orders and calculating total amounts, which not only reduces efficiency but also increases the likelihood of human error. Furthermore, the lack of technology to automatically analyze and accurately process user orders results in slow order processing.

[0105] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0106] In this invention, the server includes means for analyzing natural language orders from users using a generative artificial intelligence model and outputting the orders in a specific format; means for verifying the output and adding the order details to a database based on the output; means for receiving user input via a terminal and sending the input to the server; means for the server to send a prompt message to the generative artificial intelligence model and receive a response from the AI; means for verifying the response and, if in the correct format, reflecting it in the order processing system; means for displaying the response and error messages on the terminal; and means for calculating the total amount and notifying the user. This enables efficient and accurate analysis of natural language orders from users and allows for rapid order processing.

[0107] A "generative artificial intelligence model" is a general term for artificial intelligence algorithms and systems that analyze natural language input from users and convert it into a specific format.

[0108] "Natural language" refers to the language that humans use on a daily basis, and is not a specific programming language, but rather language that arises naturally.

[0109] A "prompt" is input text used to provide instructions or information to an artificial intelligence model.

[0110] A "server" refers to a computer or system that provides specific services or functions within a network, and in this invention, it plays the role of analyzing orders using a generative AI model.

[0111] A "terminal" refers to a device or interface used by users to input information, and its role is to receive order details and communicate with the server.

[0112] A "database" is a general term for a system or software used to efficiently store, retrieve, and manage data, and in this invention, it is used to store menu items and order details.

[0113] An "order processing system" is a general term for a system used to manage and process orders from users.

[0114] An "error message" is a message sent to inform the user of the reason why a system or program is not functioning correctly.

[0115] "Response" refers to the result or reply that a generative artificial intelligence model generates based on user input.

[0116] "Total amount" refers to the sum of the prices of all items ordered by the user.

[0117] System Overview

[0118] This invention is a system that uses a generative AI model to analyze a user's order in natural language and processes the order efficiently and accurately without human intervention. This invention mainly consists of three elements: a server, a terminal, and a user.

[0119] Server Configuration and Roles

[0120] The server sets the API key for the generated AI model and manages the menu items. Specifically, it performs the following processes:

[0121] 1. API key settings:

[0122] The server configures the API key for the generated AI model (e.g., OpenAI) and makes the specific AI engine available. The API key is saved in a configuration file and read when the server starts, thereby setting the API key in the application.

[0123] 2. Managing menu items:

[0124] The server stores a list of menu items that users can order in a database (for example, MySQL®). This list includes the item's identifier, name, and price.

[0125] Allow users to add, update, and delete menu items as needed.

[0126] 3. Analysis of user input:

[0127] It receives the user's order details in natural language, generates a prompt, and sends it to the AI ​​model. For example, if the user enters "I'd like a hamburger and a soda," it will generate a prompt like the following.

[0128] Example prompt: "User wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it as a JSON with its ID."

[0129] 4. AI response processing:

[0130] The server receives the response from the generated AI model and verifies that it is in the correct format (JSON format). If it is correct, it is reflected in the order processing system; otherwise, an error message is generated.

[0131] Device configuration and role

[0132] The terminal provides an interface for receiving order input from users and communicates with the server to process the orders.

[0133] 1. Accepting user input:

[0134] A text input field is displayed on the terminal, allowing the user to enter their order details in natural language.

[0135] 2. Submit your order:

[0136] The order details entered by the user in the text input field are sent to the server.

[0137] 3. Display of response:

[0138] The analysis results and error messages received from the server are displayed to the user.

[0139] User roles

[0140] Users enter their orders using natural language via a terminal and confirm the system's response.

[0141] 1. Enter your order:

[0142] The user enters their order details into a text input field on the device. For example, they might type, "I'd like a hamburger and a soda."

[0143] 2. Order confirmation and completion:

[0144] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[0145] This system allows users to place orders using natural language, which are then analyzed in real time by a generative AI model, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[0146] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0147] Step 1:

[0148] API key settings (server)

[0149] The server configures the API key for the generated AI model. Specifically, it saves the API key in a configuration file and reads this file when the server starts up, thereby setting the API key in the application.

[0150] Input: Configuration file (e.g., config.json)

[0151] Specific operation: Read the API key from the configuration file and enable access to the AI ​​engine.

[0152] Output: Generative AI model available for use.

[0153] Step 2:

[0154] Menu item management (server)

[0155] The server stores a list of menu items in a database. This list includes the item's identifier, name, and price. It allows for the addition, updating, and deletion of menu items as needed.

[0156] Input: Menu item data (e.g., item identifier, name, price)

[0157] Specific actions: Save, update, and delete menu items in the database.

[0158] Output: Latest menu list

[0159] Step 3:

[0160] User input reception (terminal)

[0161] The terminal provides an interface for receiving order input from users.

[0162] Input: User's order in natural language (e.g., "I'd like a hamburger and a soda, please")

[0163] Specific action: The user enters the order details into a text input field.

[0164] Output: User input

[0165] Step 4:

[0166] Sending user input (terminal)

[0167] The terminal sends the order details entered by the user to the server.

[0168] Input: User input (e.g., "I'd like a hamburger and a soda, please")

[0169] Specific action: Send the input content to the server in JSON format.

[0170] Output: Sending order details to the server

[0171] Step 5:

[0172] User input analysis prompt generation (server)

[0173] The server generates prompt messages to parse the order details received from the user.

[0174] Input: User input and menu list

[0175] Specific operation: Generate a prompt message in the format "Items the user wants to order: (User input). Menu items are as follows: (Menu list). Identify the order and return it in JSON format including the ID."

[0176] Output: Generated prompt message

[0177] Example of a specific prompt: "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0178] Step 6:

[0179] Sending prompts to the generative artificial intelligence model (server)

[0180] The server sends the generated prompt message to the artificial intelligence model.

[0181] Input: Prompt message

[0182] Specific operation: Send a prompt message to the API endpoint of the AI ​​model that generates prompts.

[0183] Output: Response from the generative AI model

[0184] Step 7:

[0185] AI response processing (server)

[0186] The server receives the response from the generated AI model and verifies that it is in the correct format. If it is abnormal, it generates an error message.

[0187] Input: Response from the generative AI model (in JSON format)

[0188] Specific actions: Check the format of the response and, if correct, reflect it in the order processing system. If incorrect, generate an error message.

[0189] Output: Correct order data or error message

[0190] Step 8:

[0191] Response display (terminal)

[0192] The terminal displays the analysis results and error messages received from the server to the user.

[0193] Input: Response from the server (analysis result or error message)

[0194] Specific action: The response content is displayed on the device's screen.

[0195] Output: Order result or error message displayed to the user

[0196] Step 9:

[0197] Order confirmation and completion (user)

[0198] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[0199] Input: Order result or error message displayed on the device

[0200] Specific actions: The user reviews and modifies the order details, and places additional orders or cancels the order.

[0201] Output: Final confirmed order or additional order

[0202] (Application Example 1)

[0203] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0204] Conventional order processing systems have problems efficiently and accurately analyzing user orders in natural language, checking inventory in real time, and confirming order processing. Furthermore, they are prone to human error and processing delays, leading to high labor costs.

[0205] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0206] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user. It also includes means for comparing the order details with an inventory database to check for availability, and means for confirming the order or reordering based on the confirmed inventory status. As a result, when a user places an order in natural language, the order details are analyzed by a generative AI, and inventory is checked in real time, efficient and accurate order processing becomes possible.

[0207] A "specific artificial intelligence engine" is a software engine with advanced algorithms designed to analyze natural language input from users and output it in a specific format.

[0208] "Ordering in natural language" refers to orders placed by users using everyday language, and is a free format that does not require a specific format or protocol.

[0209] "Outputting in a specific format" means that the order details analyzed by the AI ​​are output in a structured format that can be used in subsequent processing, such as JSON format.

[0210] "Adding order details to a list" refers to adding the analyzed order details to a specific database or list structure and managing them in conjunction with other orders.

[0211] "Calculate the total amount and notify the user" means adding up the prices of each item in the order that has been added to the list and informing the user of the total amount.

[0212] An "inventory database" is a database that manages inventory information for products in stores and warehouses, recording the current inventory status of each product.

[0213] "Inventory check" is the process of verifying whether an ordered item exists in the inventory database and confirming that it is in stock.

[0214] "Order confirmation" refers to the formal processing of an order after verifying that the user's order details match the inventory status and are feasible.

[0215] "Suggesting a reorder" refers to suggesting alternative products or encouraging users to place another order when inventory is insufficient.

[0216] A "generative AI model" is an artificial intelligence model that analyzes input text or audio, understands its meaning, and generates appropriate responses or data.

[0217] A "prompt statement" is an input statement used to instruct a generative AI model on what kind of analysis to perform, and it includes specific questions and instructions.

[0218] This invention relates to a system that uses generative AI to analyze a user's order in natural language and efficiently process the order details. The embodiments for carrying out this invention will be described in detail below.

[0219] System Overview

[0220] This system primarily consists of three elements: servers, terminals, and users. Specifically, the following processes are performed:

[0221] server

[0222] The server is responsible for setting the API key for the generative AI and analyzing the user's order details. Specifically, it uses the following hardware and software:

[0223] Hardware: Cloud servers or on-premises servers

[0224] Software: Python, Flask, OpenAI's GPT-4® API

[0225] The server will perform the following steps:

[0226] 1. Setting the API key for the generative AI: The server first sets the API key for the generative AI to make the AI ​​engine available for use.

[0227] 2. Analysis of user input: The system receives orders from the user in natural language, generates prompt sentences for analysis, and sends them to the generative AI.

[0228] 3. Processing AI responses: Analyze the response from the AI ​​and output the order details in JSON format.

[0229] 4. Inventory check: The server checks the order details against the inventory database to confirm whether the item is in stock.

[0230] 5. Confirmation and Notification: Notify the user of confirmed order details and inventory status.

[0231] For example, if a user types "I'd like a hamburger and a soda," the server will generate a prompt message like this:

[0232] "The items the user wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format, including the ID."

[0233] terminal

[0234] The terminal provides an interface for receiving order input from users in natural language. Specifically, it performs the following processes:

[0235] Hardware: Smartphone (iOS or Android®)

[0236] Software: Mobile applications

[0237] 1. Accepting user input: The terminal provides a screen for accepting order input from the user in natural language.

[0238] 2. Sending the order details: The terminal sends the order details received from the user to the server.

[0239] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[0240] User

[0241] The user will perform the following specific operations through this system.

[0242] 1. Order Input: The user enters their order details in natural language.

[0243] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0244] Specific example

[0245] When a user enters "I'd like a hamburger and a soda" into a smartphone application, the order is sent to a server. The server generates a prompt for a generative AI model and sends it to the AI ​​engine. The AI ​​analyzes the order and returns a response in JSON format. The server then checks the inventory database to confirm availability. The confirmed results are returned to the device and displayed to the user.

[0246] In this way, the present invention enables users to place orders using natural language, analyze the order details in real time, check inventory levels, and ultimately achieve efficient and error-free order processing.

[0247] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0248] Step 1:

[0249] The user enters their order using natural language.

[0250] Specific operation:

[0251] The user enters their order using a smartphone application. Example: "I'd like a hamburger and a soda, please."

[0252] input:

[0253] A user's order in natural language.

[0254] output:

[0255] The order text displayed in the input field on the smartphone app.

[0256] Step 2:

[0257] The terminal sends the order details to the server.

[0258] Specific operation:

[0259] The terminal sends user input to the server in text format. Specifically, it is sent as an HTTP request.

[0260] input:

[0261] A natural language order from the user.

[0262] output:

[0263] Order data received by the server.

[0264] Step 3:

[0265] The server generates prompt text for the generative AI.

[0266] Specific operation:

[0267] The server generates prompts for the generative AI based on the user's order. Example: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0268] input:

[0269] User order data sent to the server.

[0270] output:

[0271] A prompt message to send to a generative AI.

[0272] Step 4:

[0273] The server sends a prompt sentence to the generative AI and receives a response.

[0274] Specific operation:

[0275] The server sends a prompt sentence to the generative AI (e.g., GPT-4) and receives an analysis result from the AI. This analysis result is order data in JSON format.

[0276] Input:

[0277] A prompt sentence for the generative AI.

[0278] Output:

[0279] A response of order data in JSON format.

[0280] Step 5:

[0281] The server analyzes the order data in JSON format and matches it with the inventory database.

[0282] Specific operation:

[0283] The server analyzes the received order data, obtains the ID of each item. Next, it matches with the inventory database to check the availability of the inventory.

[0284] Input:

[0285] Order data in JSON format. <00_{}00903><00_{}00904><00_{}00905>Output:<00_{}00906><00_{}00907><00_{}00908>Inventory check result. <00_{}_{}00909>[[ID=6,3]]<00_{}00910><00_{}00911>Step 6:<00_{}00912><00_{}00913><00_{}00914>The server determines the order or proposes a reorder based on the inventory status.

[0290] Specific operation:

[0291] Based on inventory levels, the server generates confirmed orders and suggestions for reorders for items that are out of stock.

[0292] input:

[0293] Inventory check results and order data in JSON format.

[0294] output:

[0295] Order confirmation information and reorder suggestion information.

[0296] Step 7:

[0297] The server sends order confirmation information and reorder suggestion information to the terminal.

[0298] Specific operation:

[0299] The server returns order confirmation information and a suggestion to reorder to the terminal as an HTTP response.

[0300] input:

[0301] Order confirmation information and reorder suggestion information.

[0302] output:

[0303] Order confirmation information and reorder suggestion information received by the terminal.

[0304] Step 8:

[0305] The terminal displays order confirmation information and reorder suggestion information to the user.

[0306] Specific operation:

[0307] The terminal displays the received information on the user interface, notifying the user of the confirmed order details and the proposed reorder details.

[0308] Input:

[0309] Order confirmation information and reorder proposal information received from the server.

[0310] Output:

[0311] Order confirmation information and reorder proposal information displayed on the user's smartphone screen.

[0312] In this way, the order in the user's natural language is analyzed by the generative AI, and efficient and accurate order processing is performed.

[0313] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion recognition model 59 and perform specific processing using the user's emotion.

[0314] The present invention relates to an order processing system that analyzes the user's in-store order using a generative AI and an emotion engine, and further optimizes the response by recognizing the user's emotion. Hereinafter, embodiments for implementing the present invention will be described in detail.

[0315] Overview of the System

[0316] System Configuration:

[0317] This system is mainly composed of three elements: a server, a terminal, and a user.

[0318] Server: The server sets the API keys of the generative AI and the emotion recognition engine, enabling the use of each engine. It also manages the list of menu items, analyzes the user's input, and plays the role of outputting the order content in a specific format.

[0319] Terminal: The terminal provides an interface for receiving orders from users in natural language, and communicates with the server to process orders and perform sentiment analysis.

[0320] User: The user enters their order via a terminal and receives responses and recommendations from the system.

[0321] Program Description

[0322] server:

[0323] The server performs the following specific actions:

[0324] 1. API Key Configuration: The server configures the API keys for the generative AI and emotion recognition engines, enabling the use of each engine.

[0325] 2. Menu Item Management: The server maintains a list of menu items that users can order, and this list includes the item's ID, name, and price.

[0326] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[0327] Specific example:

[0328] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[0329] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[0330] 5. Emotion Recognition: If the response is in the correct format, the user's input is also sent to the emotion recognition engine to analyze the user's emotions.

[0331] Specific example:

[0332] When a user types "I'd like a hamburger and a soda," the emotion recognition engine identifies the user's emotional state (e.g., joy, anger, sadness, etc.) from their input.

[0333] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the response content and displayed messages.

[0334] Terminal:

[0335] The terminal performs the following specific actions:

[0336] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[0337] 2. Sending the order details: The terminal sends the received order details to the server.

[0338] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[0339] User:

[0340] The user will perform the following specific operations through this system.

[0341] 1. Order Input: The user enters their order details in natural language.

[0342] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0343] This system allows users to place orders using natural language. A generative AI analyzes the input to identify and add order details in real time, ultimately calculating the total amount and notifying the user. Furthermore, by utilizing an emotion recognition engine, the system can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[0344] The following describes the processing flow.

[0345] Step 1:

[0346] Server: Configure API keys for OpenAI and the emotion recognition engine, making each engine available. Also, define a list of menu items available to the user, and maintain the ID, name, and price of each item.

[0347] Step 2:

[0348] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[0349] Step 3:

[0350] User: Enter the items you want to order in natural language. For example, enter "I'd like a hamburger and a soda, please."

[0351] Step 4:

[0352] Terminal: Receives user input and sends the input to the server.

[0353] Step 5:

[0354] Server: Receives user input and generates prompts for the generative AI to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0355] Step 6:

[0356] Server: Sends the generated prompt to the generative AI and waits for the analysis results.

[0357] Step 7:

[0358] AI: Analyzes user input and returns order details as JSON data. For example, it returns {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[0359] Step 8:

[0360] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[0361] Step 9:

[0362] Server: Based on the added order, it sends a prompt to the emotion recognition engine to recognize the user's emotions.

[0363] Step 10:

[0364] Emotion recognition engine: Identifies emotions from user input and returns the result to the server. For example, it identifies the user's emotional state, such as "happy" or "angry."

[0365] Step 11:

[0366] Server: Based on the results of sentiment analysis, it generates messages to provide appropriate responses and suggestions. For example, if the user is "happy," it generates a message such as "Thank you for your order! Have a great day!"

[0367] Step 12:

[0368] Server: Sends the generated response message to the terminal.

[0369] Step 13:

[0370] Terminal: Displays messages received from the server to the user.

[0371] Step 14:

[0372] User: Enter any additional orders as needed. For example, "Please add some french fries."

[0373] Step 15:

[0374] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[0375] Step 16:

[0376] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[0377] Step 17:

[0378] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[0379] Step 18:

[0380] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[0381] Step 19:

[0382] System: At this point, the order process is complete and the program terminates.

[0383] Through the steps described above, this system can automatically process user orders and further improve customer satisfaction by analyzing user emotions and providing optimal responses.

[0384] (Example 2)

[0385] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0386] In modern restaurants and retail businesses, efficiently processing and analyzing customer orders is a challenge. Furthermore, understanding customer emotions and providing appropriate responses based on those emotions is essential for improving customer satisfaction. Existing systems often fail to adequately analyze order content and recognize emotions, potentially leading to a diminished user experience. There is also a need for more efficient order processing and reduced labor costs.

[0387] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0388] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format; means for confirming the output and adding the order details to a list based on the output; means for calculating the total amount and notifying the user; means for generating a prompt for a generative artificial intelligence engine and obtaining the engine's response; means for sending the user's input to an emotion recognition engine and analyzing the user's emotions; and means for adjusting the response content based on the emotion analysis results. This makes it possible to efficiently analyze a user's order in natural language and provide an appropriate response in real time. Furthermore, by providing a response that corresponds to the user's emotional state, it is expected that customer satisfaction will improve.

[0389] A "specific artificial intelligence engine" is software or hardware that has the function of analyzing user input data and outputting it in a specific format.

[0390] "Natural language" refers to the words and sentences that humans use on a daily basis, in a format that can be understood and analyzed by machines.

[0391] A "prompt" is input data used to give instructions or questions to an artificial intelligence engine, and is intended to guide its analysis and response.

[0392] An "emotion recognition engine" is software or hardware that analyzes user input data to identify emotional states (e.g., joy, anger, sadness, etc.).

[0393] A "specific format" is a format that includes the ID, name, and price of the ordered items, allowing the system to clearly identify the order.

[0394] A "list" is a collection of order details, containing detailed information about each ordered item.

[0395] The "total amount" refers to the total price of all items ordered by the user and is displayed to the user.

[0396] A "generative artificial intelligence engine" refers to an artificial intelligence model that can generate the optimal response to a user's prompt.

[0397] "Response content" refers to the reactions or messages that the system generates in response to user input, and is the information presented to the user.

[0398] "Adjustment" refers to modifying or changing the response content based on the results of the user's sentiment analysis, with the aim of increasing user satisfaction.

[0399] This invention relates to a system in which a user inputs an order in natural language, and the order content is analyzed and optimized using a generative artificial intelligence engine and an emotion recognition engine. This system consists of three main elements: a server, a terminal, and a user.

[0400] Server Role

[0401] The server primarily performs the following processes:

[0402] 1. API key settings:

[0403] The server sets the API keys for the generative artificial intelligence engine and the emotion recognition engine, and makes these engines available for use.

[0404] Hardware and software used: Cloud storage and API management services.

[0405] 2. Managing menu items:

[0406] The server manages a list of menu items that users can order. This list includes the item's ID, name, and price.

[0407] Hardware and software used: Database management system.

[0408] Specific example: The menu list includes entries such as "ID: 1, Name: Hamburger, Price: 500 yen", "ID: 2, Name: Soda, Price: 150 yen", etc.

[0409] 3. Analysis of user input:

[0410] The system receives order details entered by the user in natural language and generates prompts for the generative artificial intelligence engine.

[0411] Example: If a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu options are as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0412] 4. Inquiries to generative artificial intelligence engines:

[0413] The server sends a prompt to the generative artificial intelligence engine to analyze the order details.

[0414] Hardware and software used: Cloud-based generative artificial intelligence API.

[0415] 5. Processing AI responses:

[0416] The server receives the response from the generative artificial intelligence engine, verifies that it is in the correct format (e.g., JSON format), and returns an error message if it is invalid.

[0417] Specific example: Verify that the response is in the format "{"items": [{"id": 1, "name": "hamburger"}, {"id": 2, "name": "soda"}]}".

[0418] 6. Emotion recognition:

[0419] The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions.

[0420] Hardware and software used: Cloud-based emotion recognition API.

[0421] Specific example: If a user types "I'd like a hamburger and a soda," the emotion recognition engine detects "joy."

[0422] 7. Adjusting the response:

[0423] Based on the results of the emotion analysis, the response content and displayed messages are adjusted.

[0424] Specific example: If the user expresses "joy," respond with, "Thank you! I'll bring your hamburger and soda right away."

[0425] Terminal role

[0426] The terminal receives order input from the user and provides an interface for communicating with the server.

[0427] 1. Accepting user input:

[0428] The terminal accepts order input from users in natural language.

[0429] Hardware and software to be used: Touchscreen input device and speech recognition software.

[0430] 2. Submit your order:

[0431] The terminal sends the received order details to the server.

[0432] 3. Display of emotion analysis results:

[0433] The terminal displays the analysis results and sentiment analysis results received from the server to the user.

[0434] Hardware and software to be used: Display unit and display software.

[0435] User roles

[0436] Users place orders through this system.

[0437] 1. Enter your order:

[0438] Users enter their order details in natural language.

[0439] Example: Enter "I'd like a hamburger and a soda."

[0440] 2. Order confirmation and completion:

[0441] The user reviews their order, places additional orders as needed, and finally enters "Finish" to complete the ordering process.

[0442] Overall picture of the operation

[0443] This system allows users to place orders using natural language, and a generative artificial intelligence engine analyzes the input to identify the order details in real time. Furthermore, by utilizing an emotion recognition engine, it can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[0444] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0445] Step 1:

[0446] User-generated order entry using natural language

[0447] Input: The user enters the order details in natural language.

[0448] Specific operation: The user uses the input interface of the in-store terminal to enter their order details in natural language, such as "I'd like a hamburger and a soda, please."

[0449] Output: Order data in natural language input.

[0450] Step 2:

[0451] Sending order details via terminal

[0452] Input: Order details entered by the user (in natural language).

[0453] Specific operation: The terminal sends the natural language order data entered by the user to the server.

[0454] Output: Order data sent to the server.

[0455] Step 3:

[0456] Server-driven prompt generation

[0457] Input: Order data in natural language sent from the terminal.

[0458] Specific operation: The server generates a prompt to request analysis from the generative artificial intelligence engine. It generates a prompt that says, "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON including the ID."

[0459] Output: The prompt text to send to the generative artificial intelligence engine.

[0460] Step 4:

[0461] Server queries to the generative artificial intelligence engine

[0462] Input: The generated prompt message.

[0463] Specific operation: The server sends a prompt message to the generative artificial intelligence engine, which then analyzes the order details.

[0464] Output: Analysis results returned by the generative artificial intelligence engine (data identifying the order details).

[0465] Step 5:

[0466] Server-based verification and analysis of AI responses.

[0467] Input: Analysis results returned from a generative artificial intelligence engine.

[0468] Specific operation: The server checks whether the response from the generative artificial intelligence engine is in the correct format (e.g., JSON format). If it is invalid, it generates an error message and returns it to the user.

[0469] Output: Order details data in the correct format, or an error message.

[0470] Step 6:

[0471] Server queries emotion recognition engine

[0472] Input: User's order text.

[0473] Specific operation: The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions. For example, when a user enters "I'd like a hamburger and a soda, please," the server sends this text to the emotion recognition engine.

[0474] Output: Emotion analysis results from the emotion recognition engine (e.g., joy, anger, sadness, etc.).

[0475] Step 7:

[0476] Server-side adjustment of response content

[0477] Input: Sentiment analysis results from the emotion recognition engine, and correctly formatted order data.

[0478] Specific operation: The server adjusts the response based on the results of the sentiment analysis. For example, if the user is feeling "joyful," the response message will be "Thank you! I'll bring your hamburger and soda right away."

[0479] Output: Optimized response content.

[0480] Step 8:

[0481] Server sends response data to terminal

[0482] Input: Optimized response content.

[0483] Specific operation: The server sends the optimized response to the terminal.

[0484] Output: Response data sent to the terminal.

[0485] Step 9:

[0486] Displaying the response from the terminal

[0487] Input: Response data sent from the server.

[0488] Specific operation: The terminal displays the received response on its screen and informs the user of the result.

[0489] Output: The response message displayed on the terminal's screen.

[0490] (Application Example 2)

[0491] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0492] Currently, many stores rely on manual order entry by staff, posing challenges to improving order efficiency and customer experience. Furthermore, providing appropriate responses tailored to customer emotions is difficult, limiting the potential for increased customer satisfaction. Therefore, there is a need for a system that automatically handles natural language order taking, provides optimal product suggestions based on those orders, and offers responses that take customer emotions into consideration.

[0493] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for analyzing a natural language order from a user using a specific generation technology engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user, means for identifying emotions from the user's input using an emotion analysis engine, and means for adjusting the response to the user based on the results of the emotion analysis. This not only automates the analysis and processing of natural language orders, but also enables responses that respond to the user's emotions, thereby improving customer satisfaction and operational efficiency.

[0494] A "generative technology engine" is an artificial intelligence engine that analyzes a user's order in natural language and outputs it in a specific format.

[0495] An "emotion analysis engine" is an artificial intelligence engine that identifies emotions from user input and outputs the analysis results.

[0496] "Specific format" refers to a format that includes the identifier, name, and price of the ordered items.

[0497] "Means for adding order details to a list" refers to a processing method or apparatus for adding analyzed order details to an existing order list.

[0498] "Means for calculating the total amount" refers to a processing method or apparatus for calculating the total amount by summing up the prices of the listed order items.

[0499] "Means for adjusting the content of responses to the user" refers to a processing method or apparatus for appropriately changing the content of responses to the user based on the results of the sentiment analysis engine.

[0500] This invention relates to a system that uses a generation technology engine and an emotion analysis engine to analyze a user's natural language order, recognize the user's emotional state, and provide an optimal response. This system mainly consists of three elements: a server, a terminal, and a user.

[0501] Server configuration and functionality

[0502] The server has the following roles:

[0503] 1. Setting API keys for the generative technology engine and sentiment analysis engine: The server sets the API keys for the generative technology engine and sentiment analysis engine to enable the use of each engine.

[0504] 2. Menu Item Management: The server manages a list of menu items that users can order, and this list includes the identifier, name, and price of each item.

[0505] 3. Parsing user input: The system receives order details entered by the user in natural language and generates prompt sentences for parsing using a generation technology engine.

[0506] Example: If a user enters "Please recommend a coffee and a cake," the server will generate a prompt like this: "User order: Please recommend a coffee and a cake. Menu: [{menu list}]. Identify the order and respond in JSON format."

[0507] 4. AI response processing: The server receives the response from the generation technology engine and verifies that it is in the correct format (JSON format). If it is invalid, it returns an error message to the user.

[0508] 5. Emotion Recognition: The server sends the user's input text to the emotion analysis engine, which then analyzes the user's emotions.

[0509] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the content of the response to the user.

[0510] Device configuration and functions

[0511] The terminal is for use by store staff and performs the following tasks:

[0512] 1. Receiving user input: The terminal provides an interface for receiving order input from the user in natural language.

[0513] 2. Sending the order details: The terminal sends the received order details to the server.

[0514] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[0515] User roles

[0516] The user performs the following actions:

[0517] 1. Order Input: The user enters their order details in natural language.

[0518] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0519] Overall system flow

[0520] To implement this invention, the server, terminal, and user must each fulfill their respective roles and connect and communicate seamlessly. By using a generative technology engine and an emotion analysis engine in combination, a system is built that analyzes the user's natural language input and automatically provides responses that correspond to their emotions. This makes it possible to achieve real-time order processing and high customer satisfaction.

[0521] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0522] Step 1:

[0523] The terminal accepts orders from users via voice or text input using natural language. The input data consists of the user's order details, which serves as the starting point for processing the entire system.

[0524] Step 2:

[0525] The terminal sends the received order details to the server. The data is in natural language text format, and its content is necessary for analysis on the server side.

[0526] Step 3:

[0527] The server requests the AI ​​to analyze the order details. The input data is the user's order text and menu list, which are sent as prompts to the AI ​​model. "User Order: [Order Details] Menu: [{Menu List}]. Identify the order details and respond in JSON format."

[0528] Step 4:

[0529] The generative AI model receives a prompt, parses the order details, and outputs them in a specific format (JSON format). This output includes the identifier, name, and price of each ordered item.

[0530] Step 5:

[0531] The server receives the response from the generated AI model and verifies that the output is in the correct format. If it is invalid, it generates an error message and sends it to the terminal to notify the user; otherwise, it proceeds to the next step.

[0532] Step 6:

[0533] The server sends correctly formatted output to the sentiment analysis engine to identify the user's emotions. The input data is the text of the order details, and the sentiment recognition engine analyzes the text to output the emotional state (e.g., joy, anger, sadness, etc.).

[0534] Step 7:

[0535] The server receives the sentiment analysis results and adjusts its response to the user based on the analysis. For example, if the user is happy, it generates a response that will make them even happier. Specific example: "Certainly. I'm glad! Your order is a cafe latte and cheesecake."

[0536] Step 8:

[0537] The server sends the adjusted response to the terminal. This response reflects the analysis results from the generative AI model and the sentiment analysis engine.

[0538] Step 9:

[0539] The terminal displays or verbally communicates the response received from the server to the user. This allows the user to review the information and take additional actions (such as placing an additional order) as needed.

[0540] Step 10:

[0541] If the user places an additional order, the process returns to step 1 and repeats. Once the final order is confirmed, a termination command is sent from the terminal to the server.

[0542] Step 11:

[0543] The server calculates the total amount based on the list of final order items and sends the result to the terminal. This notifies the user, and the order process is completed.

[0544] This series of steps results in a system that analyzes the user's natural language order and provides the most appropriate response based on their emotions.

[0545] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0546] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0547] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0548] [Second Embodiment]

[0549] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0550] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0551] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0552] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0553] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0555] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0556] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0557] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0558] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0559] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0560] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0561] This invention relates to an order processing system that uses generative AI to analyze user in-store orders and reduce labor costs. The embodiments for carrying out this invention will be described in detail below.

[0562] System Overview

[0563] System configuration:

[0564] This system primarily consists of three elements: servers, terminals, and users.

[0565] Server: The server sets the API key for the generative AI and manages the list of menu items. It also analyzes user input and outputs the order details in a specific format.

[0566] Terminal: The terminal provides an interface for receiving orders from users in natural language and communicates with the server to process the orders.

[0567] User: The user enters their order via a terminal and receives a response from the system.

[0568] Program Description

[0569] server:

[0570] The server performs the following specific actions:

[0571] 1. API Key Configuration: The server configures API keys for generative AI (such as OpenAI) to enable the use of specific AI engines.

[0572] 2. Menu Item Management: The server maintains a list of menu items that users can order. This list includes the item's ID, name, and price.

[0573] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[0574] Specific example:

[0575] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[0576] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[0577] Terminal:

[0578] The terminal performs the following specific actions:

[0579] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[0580] 2. Sending the order details: The terminal sends the received order details to the server.

[0581] Specific example:

[0582] When a user types "Please add some french fries," the terminal sends this input to the server.

[0583] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[0584] User:

[0585] The user will perform the following specific operations through this system.

[0586] 1. Order Input: The user enters their order details in natural language.

[0587] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0588] This system allows users to place orders using natural language, which are then analyzed in real time by generative AI, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[0589] The following describes the processing flow.

[0590] Step 1:

[0591] Server: Set up the OpenAI API key and prepare the AI ​​engine to be used (e.g., "text-davinci-003"). Also, define a list of menu items and store the ID, name, and price of each item.

[0592] Step 2:

[0593] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[0594] Step 3:

[0595] User: Enter the items you want to order in natural language. For example, "I'd like one hamburger and a soda."

[0596] Step 4:

[0597] Terminal: Receives user input and sends the input to the server.

[0598] Step 5:

[0599] Server: Receives user input and generates prompts for the AI ​​to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0600] Step 6:

[0601] Server: Sends the generated prompt to the AI ​​engine and waits for the analysis results.

[0602] Step 7:

[0603] AI: Analyzes user input and returns order details as JSON data. For example, it returns in the format {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[0604] Step 8:

[0605] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[0606] Step 9:

[0607] Server: Notifies the user that an order has been added. For example, it might notify them with a message like, "A hamburger has been added," or "A soda has been added."

[0608] Step 10:

[0609] Terminal: Receives notifications from the server and displays them to the user.

[0610] Step 11:

[0611] User: Enter any additional orders as needed. For example, "Please add one more order of french fries."

[0612] Step 12:

[0613] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[0614] Step 13:

[0615] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[0616] Step 14:

[0617] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[0618] Step 15:

[0619] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[0620] Step 16:

[0621] System: At this point, the order process is complete and the program terminates.

[0622] Through the steps described above, this system can automatically process user orders, thereby reducing labor costs and improving efficiency.

[0623] (Example 1)

[0624] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0625] Traditional order processing systems involve numerous manual processes, such as receiving and confirming orders and calculating total amounts, which not only reduces efficiency but also increases the likelihood of human error. Furthermore, the lack of technology to automatically analyze and accurately process user orders results in slow order processing.

[0626] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0627] In this invention, the server includes means for analyzing natural language orders from users using a generative artificial intelligence model and outputting the orders in a specific format; means for verifying the output and adding the order details to a database based on the output; means for receiving user input via a terminal and sending the input to the server; means for the server to send a prompt message to the generative artificial intelligence model and receive a response from the AI; means for verifying the response and, if in the correct format, reflecting it in the order processing system; means for displaying the response and error messages on the terminal; and means for calculating the total amount and notifying the user. This enables efficient and accurate analysis of natural language orders from users and allows for rapid order processing.

[0628] A "generative artificial intelligence model" is a general term for artificial intelligence algorithms and systems that analyze natural language input from users and convert it into a specific format.

[0629] "Natural language" refers to the language that humans use on a daily basis, and is not a specific programming language, but rather language that arises naturally.

[0630] A "prompt" is input text used to provide instructions or information to an artificial intelligence model.

[0631] A "server" refers to a computer or system that provides specific services or functions within a network, and in this invention, it plays the role of analyzing orders using a generative AI model.

[0632] A "terminal" refers to a device or interface used by users to input information, and its role is to receive order details and communicate with the server.

[0633] A "database" is a general term for a system or software used to efficiently store, retrieve, and manage data, and in this invention, it is used to store menu items and order details.

[0634] An "order processing system" is a general term for a system used to manage and process orders from users.

[0635] An "error message" is a message sent to inform the user of the reason why a system or program is not functioning correctly.

[0636] "Response" refers to the result or reply that a generative artificial intelligence model generates based on user input.

[0637] "Total amount" refers to the sum of the prices of all items ordered by the user.

[0638] System Overview

[0639] This invention is a system that uses a generative AI model to analyze a user's order in natural language and processes the order efficiently and accurately without human intervention. This invention mainly consists of three elements: a server, a terminal, and a user.

[0640] Server Configuration and Roles

[0641] The server sets the API key for the generated AI model and manages the menu items. Specifically, it performs the following processes:

[0642] 1. API key settings:

[0643] The server configures the API key for the generated AI model (e.g., OpenAI) and makes the specific AI engine available. The API key is saved in a configuration file and read when the server starts, thereby setting the API key in the application.

[0644] 2. Managing menu items:

[0645] The server stores a list of menu items that users can order in a database (e.g., MySQL). This list includes the item's identifier, name, and price.

[0646] Allow users to add, update, and delete menu items as needed.

[0647] 3. Analysis of user input:

[0648] It receives the user's order details in natural language, generates a prompt, and sends it to the AI ​​model. For example, if the user enters "I'd like a hamburger and a soda," it will generate a prompt like the following.

[0649] Example prompt: "User wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it as a JSON with its ID."

[0650] 4. AI response processing:

[0651] The server receives the response from the generated AI model and verifies that it is in the correct format (JSON format). If it is correct, it is reflected in the order processing system; otherwise, an error message is generated.

[0652] Device configuration and role

[0653] The terminal provides an interface for receiving order input from users and communicates with the server to process the orders.

[0654] 1. Accepting user input:

[0655] A text input field is displayed on the terminal, allowing the user to enter their order details in natural language.

[0656] 2. Submit your order:

[0657] The order details entered by the user in the text input field are sent to the server.

[0658] 3. Display of response:

[0659] The analysis results and error messages received from the server are displayed to the user.

[0660] User roles

[0661] Users enter their orders using natural language via a terminal and confirm the system's response.

[0662] 1. Enter your order:

[0663] The user enters their order details into a text input field on the device. For example, they might type, "I'd like a hamburger and a soda."

[0664] 2. Order confirmation and completion:

[0665] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[0666] This system allows users to place orders using natural language, which are then analyzed in real time by a generative AI model, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[0667] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0668] Step 1:

[0669] API key settings (server)

[0670] The server configures the API key for the generated AI model. Specifically, it saves the API key in a configuration file and reads this file when the server starts up, thereby setting the API key in the application.

[0671] Input: Configuration file (e.g., config.json)

[0672] Specific operation: Read the API key from the configuration file and enable access to the AI ​​engine.

[0673] Output: Generative AI model available for use.

[0674] Step 2:

[0675] Menu item management (server)

[0676] The server stores a list of menu items in a database. This list includes the item's identifier, name, and price. It allows for the addition, updating, and deletion of menu items as needed.

[0677] Input: Menu item data (e.g., item identifier, name, price)

[0678] Specific actions: Save, update, and delete menu items in the database.

[0679] Output: Latest menu list

[0680] Step 3:

[0681] User input reception (terminal)

[0682] The terminal provides an interface for receiving order input from users.

[0683] Input: User's order in natural language (e.g., "I'd like a hamburger and a soda, please")

[0684] Specific action: The user enters the order details into a text input field.

[0685] Output: User input

[0686] Step 4:

[0687] Sending user input (terminal)

[0688] The terminal sends the order details entered by the user to the server.

[0689] Input: User input (e.g., "I'd like a hamburger and a soda, please")

[0690] Specific action: Send the input content to the server in JSON format.

[0691] Output: Sending order details to the server

[0692] Step 5:

[0693] User input analysis prompt generation (server)

[0694] The server generates prompt messages to parse the order details received from the user.

[0695] Input: User input and menu list

[0696] Specific operation: Generate a prompt message in the format "Items the user wants to order: (User input). Menu items are as follows: (Menu list). Identify the order and return it in JSON format including the ID."

[0697] Output: Generated prompt message

[0698] Example of a specific prompt: "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0699] Step 6:

[0700] Sending prompts to the generative artificial intelligence model (server)

[0701] The server sends the generated prompt message to the artificial intelligence model.

[0702] Input: Prompt message

[0703] Specific operation: Send a prompt message to the API endpoint of the AI ​​model that generates prompts.

[0704] Output: Response from the generative AI model

[0705] Step 7:

[0706] AI response processing (server)

[0707] The server receives the response from the generated AI model and verifies that it is in the correct format. If it is abnormal, it generates an error message.

[0708] Input: Response from the generative AI model (in JSON format)

[0709] Specific actions: Check the format of the response and, if correct, reflect it in the order processing system. If incorrect, generate an error message.

[0710] Output: Correct order data or error message

[0711] Step 8:

[0712] Response display (terminal)

[0713] The terminal displays the analysis results and error messages received from the server to the user.

[0714] Input: Response from the server (analysis result or error message)

[0715] Specific action: The response content is displayed on the device's screen.

[0716] Output: Order result or error message displayed to the user

[0717] Step 9:

[0718] Order confirmation and completion (user)

[0719] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[0720] Input: Order result or error message displayed on the device

[0721] Specific actions: The user reviews and modifies the order details, and places additional orders or cancels the order.

[0722] Output: Final confirmed order or additional order

[0723] (Application Example 1)

[0724] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0725] Conventional order processing systems have problems efficiently and accurately analyzing user orders in natural language, checking inventory in real time, and confirming order processing. Furthermore, they are prone to human error and processing delays, leading to high labor costs.

[0726] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0727] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user. It also includes means for comparing the order details with an inventory database to check for availability, and means for confirming the order or reordering based on the confirmed inventory status. As a result, when a user places an order in natural language, the order details are analyzed by a generative AI, and inventory is checked in real time, efficient and accurate order processing becomes possible.

[0728] A "specific artificial intelligence engine" is a software engine with advanced algorithms designed to analyze natural language input from users and output it in a specific format.

[0729] "Ordering in natural language" refers to orders placed by users using everyday language, and is a free format that does not require a specific format or protocol.

[0730] "Outputting in a specific format" means that the order details analyzed by the AI ​​are output in a structured format that can be used in subsequent processing, such as JSON format.

[0731] "Adding order details to a list" refers to adding the analyzed order details to a specific database or list structure and managing them in conjunction with other orders.

[0732] "Calculate the total amount and notify the user" means adding up the prices of each item in the order that has been added to the list and informing the user of the total amount.

[0733] An "inventory database" is a database that manages inventory information for products in stores and warehouses, recording the current inventory status of each product.

[0734] "Inventory check" is the process of verifying whether an ordered item exists in the inventory database and confirming that it is in stock.

[0735] "Order confirmation" refers to the formal processing of an order after verifying that the user's order details match the inventory status and are feasible.

[0736] "Suggesting a reorder" refers to suggesting alternative products or encouraging users to place another order when inventory is insufficient.

[0737] A "generative AI model" is an artificial intelligence model that analyzes input text or audio, understands its meaning, and generates appropriate responses or data.

[0738] A "prompt statement" is an input statement used to instruct a generative AI model on what kind of analysis to perform, and it includes specific questions and instructions.

[0739] This invention relates to a system that uses generative AI to analyze a user's order in natural language and efficiently process the order details. The embodiments for carrying out this invention will be described in detail below.

[0740] System Overview

[0741] This system primarily consists of three elements: servers, terminals, and users. Specifically, the following processes are performed:

[0742] server

[0743] The server is responsible for setting the API key for the generative AI and analyzing the user's order details. Specifically, it uses the following hardware and software:

[0744] Hardware: Cloud servers or on-premises servers

[0745] Software: Python, Flask, OpenAI's GPT-4 API

[0746] The server will perform the following steps:

[0747] 1. Setting the API key for the generative AI: The server first sets the API key for the generative AI to make the AI ​​engine available for use.

[0748] 2. Analysis of user input: The system receives orders from the user in natural language, generates prompt sentences for analysis, and sends them to the generative AI.

[0749] 3. Processing AI responses: Analyze the response from the AI ​​and output the order details in JSON format.

[0750] 4. Inventory check: The server checks the order details against the inventory database to confirm whether the item is in stock.

[0751] 5. Confirmation and Notification: Notify the user of confirmed order details and inventory status.

[0752] For example, if a user types "I'd like a hamburger and a soda," the server will generate a prompt message like this:

[0753] "The items the user wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format, including the ID."

[0754] terminal

[0755] The terminal provides an interface for receiving order input from users in natural language. Specifically, it performs the following processes:

[0756] Hardware: Smartphone (iOS or Android)

[0757] Software: Mobile applications

[0758] 1. Accepting user input: The terminal provides a screen for accepting order input from the user in natural language.

[0759] 2. Sending the order details: The terminal sends the order details received from the user to the server.

[0760] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[0761] User

[0762] The user will perform the following specific operations through this system.

[0763] 1. Order Input: The user enters their order details in natural language.

[0764] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0765] Specific example

[0766] When a user enters "I'd like a hamburger and a soda" into a smartphone application, the order is sent to a server. The server generates a prompt for a generative AI model and sends it to the AI ​​engine. The AI ​​analyzes the order and returns a response in JSON format. The server then checks the inventory database to confirm availability. The confirmed results are returned to the device and displayed to the user.

[0767] In this way, the present invention enables users to place orders using natural language, analyze the order details in real time, check inventory levels, and ultimately achieve efficient and error-free order processing.

[0768] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0769] Step 1:

[0770] The user enters their order using natural language.

[0771] Specific operation:

[0772] The user enters their order using a smartphone application. Example: "I'd like a hamburger and a soda, please."

[0773] input:

[0774] A user's order in natural language.

[0775] output:

[0776] The order text displayed in the input field on the smartphone app.

[0777] Step 2:

[0778] The terminal sends the order details to the server.

[0779] Specific operation:

[0780] The terminal sends user input to the server in text format. Specifically, it is sent as an HTTP request.

[0781] input:

[0782] A natural language order from the user.

[0783] output:

[0784] Order data received by the server.

[0785] Step 3:

[0786] The server generates prompt text for the generative AI.

[0787] Specific operation:

[0788] The server generates prompts for the generative AI based on the user's order. Example: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0789] input:

[0790] User order data sent to the server.

[0791] output:

[0792] A prompt message to send to a generative AI.

[0793] Step 4:

[0794] The server sends a prompt message to the generative AI and receives a response.

[0795] Specific operation:

[0796] The server sends prompt messages to a generative AI (e.g., GPT-4) and receives analysis results from the AI. These analysis results are order data in JSON format.

[0797] input:

[0798] A prompt message for a generative AI.

[0799] output:

[0800] Response with order data in JSON format.

[0801] Step 5:

[0802] The server parses the order data in JSON format and compares it with the inventory database.

[0803] Specific operation:

[0804] The server analyzes the received order data and obtains the ID of each item. Next, it compares this with the inventory database to check for availability.

[0805] input:

[0806] Order data in JSON format.

[0807] output:

[0808] Inventory check results.

[0809] Step 6:

[0810] The server confirms the order or suggests reordering based on inventory status.

[0811] Specific operation:

[0812] Based on inventory levels, the server generates confirmed orders and suggestions for reorders for items that are out of stock.

[0813] input:

[0814] Inventory check results and order data in JSON format.

[0815] output:

[0816] Order confirmation information and reorder suggestion information.

[0817] Step 7:

[0818] The server sends order confirmation information and reorder suggestion information to the terminal.

[0819] Specific operation:

[0820] The server returns order confirmation information and a suggestion to reorder to the terminal as an HTTP response.

[0821] input:

[0822] Order confirmation information and reorder suggestion information.

[0823] output:

[0824] Order confirmation information and reorder suggestion information received by the terminal.

[0825] Step 8:

[0826] The terminal displays order confirmation information and reorder suggestion information to the user.

[0827] Specific operation:

[0828] The terminal displays the received information on the user interface, informing the user of the order confirmation details and suggestions for reordering.

[0829] input:

[0830] Order confirmation information and reorder suggestion information received from the server.

[0831] output:

[0832] Order confirmation information and reorder suggestion information displayed on the user's smartphone screen.

[0833] In this way, user orders in natural language are analyzed by generative AI, resulting in efficient and accurate order processing.

[0834] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0835] This invention relates to an order processing system that analyzes a user's in-store order using a generative AI and an emotion engine, and further recognizes the user's emotions to optimize the response. The embodiments for carrying out this invention will be described in detail below.

[0836] System Overview

[0837] System configuration:

[0838] This system primarily consists of three elements: servers, terminals, and users.

[0839] Server: The server sets the API keys for the generative AI and the emotion recognition engine, enabling the use of each engine. It also manages the list of menu items, analyzes user input, and outputs order details in a specific format.

[0840] Terminal: The terminal provides an interface for receiving orders from users in natural language, and communicates with the server to process orders and perform sentiment analysis.

[0841] User: The user enters their order via a terminal and receives responses and recommendations from the system.

[0842] Program Description

[0843] server:

[0844] The server performs the following specific actions:

[0845] 1. API Key Configuration: The server configures the API keys for the generative AI and emotion recognition engines, enabling the use of each engine.

[0846] 2. Menu Item Management: The server maintains a list of menu items that users can order, and this list includes the item's ID, name, and price.

[0847] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[0848] Specific example:

[0849] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[0850] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[0851] 5. Emotion Recognition: If the response is in the correct format, the user's input is also sent to the emotion recognition engine to analyze the user's emotions.

[0852] Specific example:

[0853] When a user types "I'd like a hamburger and a soda," the emotion recognition engine identifies the user's emotional state (e.g., joy, anger, sadness, etc.) from their input.

[0854] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the response content and displayed messages.

[0855] Terminal:

[0856] The terminal performs the following specific actions:

[0857] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[0858] 2. Sending the order details: The terminal sends the received order details to the server.

[0859] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[0860] User:

[0861] The user will perform the following specific operations through this system.

[0862] 1. Order Input: The user enters their order details in natural language.

[0863] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[0864] This system allows users to place orders using natural language. A generative AI analyzes the input to identify and add order details in real time, ultimately calculating the total amount and notifying the user. Furthermore, by utilizing an emotion recognition engine, the system can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[0865] The following describes the processing flow.

[0866] Step 1:

[0867] Server: Configure API keys for OpenAI and the emotion recognition engine, making each engine available. Also, define a list of menu items available to the user, and maintain the ID, name, and price of each item.

[0868] Step 2:

[0869] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[0870] Step 3:

[0871] User: Enter the items you want to order in natural language. For example, enter "I'd like a hamburger and a soda, please."

[0872] Step 4:

[0873] Terminal: Receives user input and sends the input to the server.

[0874] Step 5:

[0875] Server: Receives user input and generates prompts for the generative AI to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0876] Step 6:

[0877] Server: Sends the generated prompt to the generative AI and waits for the analysis results.

[0878] Step 7:

[0879] AI: Analyzes user input and returns order details as JSON data. For example, it returns {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[0880] Step 8:

[0881] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[0882] Step 9:

[0883] Server: Based on the added order, it sends a prompt to the emotion recognition engine to recognize the user's emotions.

[0884] Step 10:

[0885] Emotion recognition engine: Identifies emotions from user input and returns the result to the server. For example, it identifies the user's emotional state, such as "happy" or "angry."

[0886] Step 11:

[0887] Server: Based on the results of sentiment analysis, it generates messages to provide appropriate responses and suggestions. For example, if the user is "happy," it generates a message such as "Thank you for your order! Have a great day!"

[0888] Step 12:

[0889] Server: Sends the generated response message to the terminal.

[0890] Step 13:

[0891] Terminal: Displays messages received from the server to the user.

[0892] Step 14:

[0893] User: Enter any additional orders as needed. For example, "Please add some french fries."

[0894] Step 15:

[0895] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[0896] Step 16:

[0897] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[0898] Step 17:

[0899] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[0900] Step 18:

[0901] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[0902] Step 19:

[0903] System: At this point, the order process is complete and the program terminates.

[0904] Through the steps described above, this system can automatically process user orders and further improve customer satisfaction by analyzing user emotions and providing optimal responses.

[0905] (Example 2)

[0906] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0907] In modern restaurants and retail businesses, efficiently processing and analyzing customer orders is a challenge. Furthermore, understanding customer emotions and providing appropriate responses based on those emotions is essential for improving customer satisfaction. Existing systems often fail to adequately analyze order content and recognize emotions, potentially leading to a diminished user experience. There is also a need for more efficient order processing and reduced labor costs.

[0908] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0909] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format; means for confirming the output and adding the order details to a list based on the output; means for calculating the total amount and notifying the user; means for generating a prompt for a generative artificial intelligence engine and obtaining the engine's response; means for sending the user's input to an emotion recognition engine and analyzing the user's emotions; and means for adjusting the response content based on the emotion analysis results. This makes it possible to efficiently analyze a user's order in natural language and provide an appropriate response in real time. Furthermore, by providing a response that corresponds to the user's emotional state, it is expected that customer satisfaction will improve.

[0910] A "specific artificial intelligence engine" is software or hardware that has the function of analyzing user input data and outputting it in a specific format.

[0911] "Natural language" refers to the words and sentences that humans use on a daily basis, in a format that can be understood and analyzed by machines.

[0912] A "prompt" is input data used to give instructions or questions to an artificial intelligence engine, and is intended to guide its analysis and response.

[0913] An "emotion recognition engine" is software or hardware that analyzes user input data to identify emotional states (e.g., joy, anger, sadness, etc.).

[0914] A "specific format" is a format that includes the ID, name, and price of the ordered items, allowing the system to clearly identify the order.

[0915] A "list" is a collection of order details, containing detailed information about each ordered item.

[0916] The "total amount" refers to the total price of all items ordered by the user and is displayed to the user.

[0917] A "generative artificial intelligence engine" refers to an artificial intelligence model that can generate the optimal response to a user's prompt.

[0918] "Response content" refers to the reactions or messages that the system generates in response to user input, and is the information presented to the user.

[0919] "Adjustment" refers to modifying or changing the response content based on the results of the user's sentiment analysis, with the aim of increasing user satisfaction.

[0920] This invention relates to a system in which a user inputs an order in natural language, and the order content is analyzed and optimized using a generative artificial intelligence engine and an emotion recognition engine. This system consists of three main elements: a server, a terminal, and a user.

[0921] Server Role

[0922] The server primarily performs the following processes:

[0923] 1. API key settings:

[0924] The server sets the API keys for the generative artificial intelligence engine and the emotion recognition engine, and makes these engines available for use.

[0925] Hardware and software used: Cloud storage and API management services.

[0926] 2. Managing menu items:

[0927] The server manages a list of menu items that users can order. This list includes the item's ID, name, and price.

[0928] Hardware and software used: Database management system.

[0929] Specific example: The menu list includes entries such as "ID: 1, Name: Hamburger, Price: 500 yen", "ID: 2, Name: Soda, Price: 150 yen", etc.

[0930] 3. Analysis of user input:

[0931] The system receives order details entered by the user in natural language and generates prompts for the generative artificial intelligence engine.

[0932] Example: If a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu options are as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[0933] 4. Inquiries to generative artificial intelligence engines:

[0934] The server sends a prompt to the generative artificial intelligence engine to analyze the order details.

[0935] Hardware and software used: Cloud-based generative artificial intelligence API.

[0936] 5. Processing AI responses:

[0937] The server receives the response from the generative artificial intelligence engine, verifies that it is in the correct format (e.g., JSON format), and returns an error message if it is invalid.

[0938] Specific example: Verify that the response is in the format "{"items": [{"id": 1, "name": "hamburger"}, {"id": 2, "name": "soda"}]}".

[0939] 6. Emotion recognition:

[0940] The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions.

[0941] Hardware and software used: Cloud-based emotion recognition API.

[0942] Specific example: If a user types "I'd like a hamburger and a soda," the emotion recognition engine detects "joy."

[0943] 7. Adjusting the response:

[0944] Based on the results of the emotion analysis, the response content and displayed messages are adjusted.

[0945] Specific example: If the user expresses "joy," respond with, "Thank you! I'll bring your hamburger and soda right away."

[0946] Terminal role

[0947] The terminal receives order input from the user and provides an interface for communicating with the server.

[0948] 1. Accepting user input:

[0949] The terminal accepts order input from users in natural language.

[0950] Hardware and software to be used: Touchscreen input device and speech recognition software.

[0951] 2. Submit your order:

[0952] The terminal sends the received order details to the server.

[0953] 3. Display of emotion analysis results:

[0954] The terminal displays the analysis results and sentiment analysis results received from the server to the user.

[0955] Hardware and software to be used: Display unit and display software.

[0956] User roles

[0957] Users place orders through this system.

[0958] 1. Enter your order:

[0959] Users enter their order details in natural language.

[0960] Example: Enter "I'd like a hamburger and a soda."

[0961] 2. Order confirmation and completion:

[0962] The user reviews their order, places additional orders as needed, and finally enters "Finish" to complete the ordering process.

[0963] Overall picture of the operation

[0964] This system allows users to place orders using natural language, and a generative artificial intelligence engine analyzes the input to identify the order details in real time. Furthermore, by utilizing an emotion recognition engine, it can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[0965] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0966] Step 1:

[0967] User-generated order entry using natural language

[0968] Input: The user enters the order details in natural language.

[0969] Specific operation: The user uses the input interface of the in-store terminal to enter their order details in natural language, such as "I'd like a hamburger and a soda, please."

[0970] Output: Order data in natural language input.

[0971] Step 2:

[0972] Sending order details via terminal

[0973] Input: Order details entered by the user (in natural language).

[0974] Specific operation: The terminal sends the natural language order data entered by the user to the server.

[0975] Output: Order data sent to the server.

[0976] Step 3:

[0977] Server-driven prompt generation

[0978] Input: Order data in natural language sent from the terminal.

[0979] Specific operation: The server generates a prompt to request analysis from the generative artificial intelligence engine. It generates a prompt that says, "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON including the ID."

[0980] Output: The prompt text to send to the generative artificial intelligence engine.

[0981] Step 4:

[0982] Server queries to the generative artificial intelligence engine

[0983] Input: The generated prompt message.

[0984] Specific operation: The server sends a prompt message to the generative artificial intelligence engine, which then analyzes the order details.

[0985] Output: Analysis results returned by the generative artificial intelligence engine (data identifying the order details).

[0986] Step 5:

[0987] Server-based verification and analysis of AI responses.

[0988] Input: Analysis results returned from a generative artificial intelligence engine.

[0989] Specific operation: The server checks whether the response from the generative artificial intelligence engine is in the correct format (e.g., JSON format). If it is invalid, it generates an error message and returns it to the user.

[0990] Output: Order details data in the correct format, or an error message.

[0991] Step 6:

[0992] Server queries emotion recognition engine

[0993] Input: User's order text.

[0994] Specific operation: The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions. For example, when a user enters "I'd like a hamburger and a soda, please," the server sends this text to the emotion recognition engine.

[0995] Output: Emotion analysis results from the emotion recognition engine (e.g., joy, anger, sadness, etc.).

[0996] Step 7:

[0997] Server-side adjustment of response content

[0998] Input: Sentiment analysis results from the emotion recognition engine, and correctly formatted order data.

[0999] Specific operation: The server adjusts the response based on the results of the sentiment analysis. For example, if the user is feeling "joyful," the response message will be "Thank you! I'll bring your hamburger and soda right away."

[1000] Output: Optimized response content.

[1001] Step 8:

[1002] Server sends response data to terminal

[1003] Input: Optimized response content.

[1004] Specific operation: The server sends the optimized response to the terminal.

[1005] Output: Response data sent to the terminal.

[1006] Step 9:

[1007] Displaying the response from the terminal

[1008] Input: Response data sent from the server.

[1009] Specific operation: The terminal displays the received response on its screen and informs the user of the result.

[1010] Output: The response message displayed on the terminal's screen.

[1011] (Application Example 2)

[1012] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1013] Currently, many stores rely on manual order entry by staff, posing challenges to improving order efficiency and customer experience. Furthermore, providing appropriate responses tailored to customer emotions is difficult, limiting the potential for increased customer satisfaction. Therefore, there is a need for a system that automatically handles natural language order taking, provides optimal product suggestions based on those orders, and offers responses that take customer emotions into consideration.

[1014] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for analyzing a natural language order from a user using a specific generation technology engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user, means for identifying emotions from the user's input using an emotion analysis engine, and means for adjusting the response to the user based on the results of the emotion analysis. This not only automates the analysis and processing of natural language orders, but also enables responses that respond to the user's emotions, thereby improving customer satisfaction and operational efficiency.

[1015] A "generative technology engine" is an artificial intelligence engine that analyzes a user's order in natural language and outputs it in a specific format.

[1016] An "emotion analysis engine" is an artificial intelligence engine that identifies emotions from user input and outputs the analysis results.

[1017] "Specific format" refers to a format that includes the identifier, name, and price of the ordered items.

[1018] "Means for adding order details to a list" refers to a processing method or apparatus for adding analyzed order details to an existing order list.

[1019] "Means for calculating the total amount" refers to a processing method or apparatus for calculating the total amount by summing up the prices of the listed order items.

[1020] "Means for adjusting the content of responses to the user" refers to a processing method or apparatus for appropriately changing the content of responses to the user based on the results of the sentiment analysis engine.

[1021] This invention relates to a system that uses a generation technology engine and an emotion analysis engine to analyze a user's natural language order, recognize the user's emotional state, and provide an optimal response. This system mainly consists of three elements: a server, a terminal, and a user.

[1022] Server configuration and functionality

[1023] The server has the following roles:

[1024] 1. Setting API keys for the generative technology engine and sentiment analysis engine: The server sets the API keys for the generative technology engine and sentiment analysis engine to enable the use of each engine.

[1025] 2. Menu Item Management: The server manages a list of menu items that users can order, and this list includes the identifier, name, and price of each item.

[1026] 3. Parsing user input: The system receives order details entered by the user in natural language and generates prompt sentences for parsing using a generation technology engine.

[1027] Example: If a user enters "Please recommend a coffee and a cake," the server will generate a prompt like this: "User order: Please recommend a coffee and a cake. Menu: [{menu list}]. Identify the order and respond in JSON format."

[1028] 4. AI response processing: The server receives the response from the generation technology engine and verifies that it is in the correct format (JSON format). If it is invalid, it returns an error message to the user.

[1029] 5. Emotion Recognition: The server sends the user's input text to the emotion analysis engine, which then analyzes the user's emotions.

[1030] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the content of the response to the user.

[1031] Device configuration and functions

[1032] The terminal is for use by store staff and performs the following tasks:

[1033] 1. Receiving user input: The terminal provides an interface for receiving order input from the user in natural language.

[1034] 2. Sending the order details: The terminal sends the received order details to the server.

[1035] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[1036] User roles

[1037] The user performs the following actions:

[1038] 1. Order Input: The user enters their order details in natural language.

[1039] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1040] Overall system flow

[1041] To implement this invention, the server, terminal, and user must each fulfill their respective roles and connect and communicate seamlessly. By using a generative technology engine and an emotion analysis engine in combination, a system is built that analyzes the user's natural language input and automatically provides responses that correspond to their emotions. This makes it possible to achieve real-time order processing and high customer satisfaction.

[1042] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1043] Step 1:

[1044] The terminal accepts orders from users via voice or text input using natural language. The input data consists of the user's order details, which serves as the starting point for processing the entire system.

[1045] Step 2:

[1046] The terminal sends the received order details to the server. The data is in natural language text format, and its content is necessary for analysis on the server side.

[1047] Step 3:

[1048] The server requests the AI ​​to analyze the order details. The input data is the user's order text and menu list, which are sent as prompts to the AI ​​model. "User Order: [Order Details] Menu: [{Menu List}]. Identify the order details and respond in JSON format."

[1049] Step 4:

[1050] The generative AI model receives a prompt, parses the order details, and outputs them in a specific format (JSON format). This output includes the identifier, name, and price of each ordered item.

[1051] Step 5:

[1052] The server receives the response from the generated AI model and verifies that the output is in the correct format. If it is invalid, it generates an error message and sends it to the terminal to notify the user; otherwise, it proceeds to the next step.

[1053] Step 6:

[1054] The server sends correctly formatted output to the sentiment analysis engine to identify the user's emotions. The input data is the text of the order details, and the sentiment recognition engine analyzes the text to output the emotional state (e.g., joy, anger, sadness, etc.).

[1055] Step 7:

[1056] The server receives the sentiment analysis results and adjusts its response to the user based on the analysis. For example, if the user is happy, it generates a response that will make them even happier. Specific example: "Certainly. I'm glad! Your order is a cafe latte and cheesecake."

[1057] Step 8:

[1058] The server sends the adjusted response to the terminal. This response reflects the analysis results from the generative AI model and the sentiment analysis engine.

[1059] Step 9:

[1060] The terminal displays or verbally communicates the response received from the server to the user. This allows the user to review the information and take additional actions (such as placing an additional order) as needed.

[1061] Step 10:

[1062] If the user places an additional order, the process returns to step 1 and repeats. Once the final order is confirmed, a termination command is sent from the terminal to the server.

[1063] Step 11:

[1064] The server calculates the total amount based on the list of final order items and sends the result to the terminal. This notifies the user, and the order process is completed.

[1065] This series of steps results in a system that analyzes the user's natural language order and provides the most appropriate response based on their emotions.

[1066] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1067] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1068] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1069] [Third Embodiment]

[1070] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1071] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1072] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1073] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1074] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1075] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1076] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1077] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1078] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1079] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1080] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1081] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1082] This invention relates to an order processing system that uses generative AI to analyze user in-store orders and reduce labor costs. The embodiments for carrying out this invention will be described in detail below.

[1083] System Overview

[1084] System configuration:

[1085] This system primarily consists of three elements: servers, terminals, and users.

[1086] Server: The server sets the API key for the generative AI and manages the list of menu items. It also analyzes user input and outputs the order details in a specific format.

[1087] Terminal: The terminal provides an interface for receiving orders from users in natural language and communicates with the server to process the orders.

[1088] User: The user enters their order via a terminal and receives a response from the system.

[1089] Program Description

[1090] server:

[1091] The server performs the following specific actions:

[1092] 1. API Key Configuration: The server configures API keys for generative AI (such as OpenAI) to enable the use of specific AI engines.

[1093] 2. Menu Item Management: The server maintains a list of menu items that users can order. This list includes the item's ID, name, and price.

[1094] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[1095] Specific example:

[1096] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[1097] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[1098] Terminal:

[1099] The terminal performs the following specific actions:

[1100] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[1101] 2. Sending the order details: The terminal sends the received order details to the server.

[1102] Specific example:

[1103] When a user types "Please add some french fries," the terminal sends this input to the server.

[1104] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[1105] User:

[1106] The user will perform the following specific operations through this system.

[1107] 1. Order Input: The user enters their order details in natural language.

[1108] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1109] This system allows users to place orders using natural language, which are then analyzed in real time by generative AI, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[1110] The following describes the processing flow.

[1111] Step 1:

[1112] Server: Set up the OpenAI API key and prepare the AI ​​engine to be used (e.g., "text-davinci-003"). Also, define a list of menu items and store the ID, name, and price of each item.

[1113] Step 2:

[1114] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[1115] Step 3:

[1116] User: Enter the items you want to order in natural language. For example, "I'd like one hamburger and a soda."

[1117] Step 4:

[1118] Terminal: Receives user input and sends the input to the server.

[1119] Step 5:

[1120] Server: Receives user input and generates prompts for the AI ​​to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1121] Step 6:

[1122] Server: Sends the generated prompt to the AI ​​engine and waits for the analysis results.

[1123] Step 7:

[1124] AI: Analyzes user input and returns order details as JSON data. For example, it returns in the format {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[1125] Step 8:

[1126] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[1127] Step 9:

[1128] Server: Notifies the user that an order has been added. For example, it might notify them with a message like, "A hamburger has been added," or "A soda has been added."

[1129] Step 10:

[1130] Terminal: Receives notifications from the server and displays them to the user.

[1131] Step 11:

[1132] User: Enter any additional orders as needed. For example, "Please add one more order of french fries."

[1133] Step 12:

[1134] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[1135] Step 13:

[1136] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[1137] Step 14:

[1138] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[1139] Step 15:

[1140] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[1141] Step 16:

[1142] System: At this point, the order process is complete and the program terminates.

[1143] Through the steps described above, this system can automatically process user orders, thereby reducing labor costs and improving efficiency.

[1144] (Example 1)

[1145] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1146] Traditional order processing systems involve numerous manual processes, such as receiving and confirming orders and calculating total amounts, which not only reduces efficiency but also increases the likelihood of human error. Furthermore, the lack of technology to automatically analyze and accurately process user orders results in slow order processing.

[1147] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1148] In this invention, the server includes means for analyzing natural language orders from users using a generative artificial intelligence model and outputting the orders in a specific format; means for verifying the output and adding the order details to a database based on the output; means for receiving user input via a terminal and sending the input to the server; means for the server to send a prompt message to the generative artificial intelligence model and receive a response from the AI; means for verifying the response and, if in the correct format, reflecting it in the order processing system; means for displaying the response and error messages on the terminal; and means for calculating the total amount and notifying the user. This enables efficient and accurate analysis of natural language orders from users and allows for rapid order processing.

[1149] A "generative artificial intelligence model" is a general term for artificial intelligence algorithms and systems that analyze natural language input from users and convert it into a specific format.

[1150] "Natural language" refers to the language that humans use on a daily basis, and is not a specific programming language, but rather language that arises naturally.

[1151] A "prompt" is input text used to provide instructions or information to an artificial intelligence model.

[1152] A "server" refers to a computer or system that provides specific services or functions within a network, and in this invention, it plays the role of analyzing orders using a generative AI model.

[1153] A "terminal" refers to a device or interface used by users to input information, and its role is to receive order details and communicate with the server.

[1154] A "database" is a general term for a system or software used to efficiently store, retrieve, and manage data, and in this invention, it is used to store menu items and order details.

[1155] An "order processing system" is a general term for a system used to manage and process orders from users.

[1156] An "error message" is a message sent to inform the user of the reason why a system or program is not functioning correctly.

[1157] "Response" refers to the result or reply that a generative artificial intelligence model generates based on user input.

[1158] "Total amount" refers to the sum of the prices of all items ordered by the user.

[1159] System Overview

[1160] This invention is a system that uses a generative AI model to analyze a user's order in natural language and processes the order efficiently and accurately without human intervention. This invention mainly consists of three elements: a server, a terminal, and a user.

[1161] Server Configuration and Roles

[1162] The server sets the API key for the generated AI model and manages the menu items. Specifically, it performs the following processes:

[1163] 1. API key settings:

[1164] The server configures the API key for the generated AI model (e.g., OpenAI) and makes the specific AI engine available. The API key is saved in a configuration file and read when the server starts, thereby setting the API key in the application.

[1165] 2. Managing menu items:

[1166] The server stores a list of menu items that users can order in a database (e.g., MySQL). This list includes the item's identifier, name, and price.

[1167] Allow users to add, update, and delete menu items as needed.

[1168] 3. Analysis of user input:

[1169] It receives the user's order details in natural language, generates a prompt, and sends it to the AI ​​model. For example, if the user enters "I'd like a hamburger and a soda," it will generate a prompt like the following.

[1170] Example prompt: "User wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it as a JSON with its ID."

[1171] 4. AI response processing:

[1172] The server receives the response from the generated AI model and verifies that it is in the correct format (JSON format). If it is correct, it is reflected in the order processing system; otherwise, an error message is generated.

[1173] Device configuration and role

[1174] The terminal provides an interface for receiving order input from users and communicates with the server to process the orders.

[1175] 1. Accepting user input:

[1176] A text input field is displayed on the terminal, allowing the user to enter their order details in natural language.

[1177] 2. Submit your order:

[1178] The order details entered by the user in the text input field are sent to the server.

[1179] 3. Display of response:

[1180] The analysis results and error messages received from the server are displayed to the user.

[1181] User roles

[1182] Users enter their orders using natural language via a terminal and confirm the system's response.

[1183] 1. Enter your order:

[1184] The user enters their order details into a text input field on the device. For example, they might type, "I'd like a hamburger and a soda."

[1185] 2. Order confirmation and completion:

[1186] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[1187] This system allows users to place orders using natural language, which are then analyzed in real time by a generative AI model, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[1188] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1189] Step 1:

[1190] API key settings (server)

[1191] The server configures the API key for the generated AI model. Specifically, it saves the API key in a configuration file and reads this file when the server starts up, thereby setting the API key in the application.

[1192] Input: Configuration file (e.g., config.json)

[1193] Specific operation: Read the API key from the configuration file and enable access to the AI ​​engine.

[1194] Output: Generative AI model available for use.

[1195] Step 2:

[1196] Menu item management (server)

[1197] The server stores a list of menu items in a database. This list includes the item's identifier, name, and price. It allows for the addition, updating, and deletion of menu items as needed.

[1198] Input: Menu item data (e.g., item identifier, name, price)

[1199] Specific actions: Save, update, and delete menu items in the database.

[1200] Output: Latest menu list

[1201] Step 3:

[1202] User input reception (terminal)

[1203] The terminal provides an interface for receiving order input from users.

[1204] Input: User's order in natural language (e.g., "I'd like a hamburger and a soda, please")

[1205] Specific action: The user enters the order details into a text input field.

[1206] Output: User input

[1207] Step 4:

[1208] Sending user input (terminal)

[1209] The terminal sends the order details entered by the user to the server.

[1210] Input: User input (e.g., "I'd like a hamburger and a soda, please")

[1211] Specific action: Send the input content to the server in JSON format.

[1212] Output: Sending order details to the server

[1213] Step 5:

[1214] User input analysis prompt generation (server)

[1215] The server generates prompt messages to parse the order details received from the user.

[1216] Input: User input and menu list

[1217] Specific operation: Generate a prompt message in the format "Items the user wants to order: (User input). Menu items are as follows: (Menu list). Identify the order and return it in JSON format including the ID."

[1218] Output: Generated prompt message

[1219] Example of a specific prompt: "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1220] Step 6:

[1221] Sending prompts to the generative artificial intelligence model (server)

[1222] The server sends the generated prompt message to the artificial intelligence model.

[1223] Input: Prompt message

[1224] Specific operation: Send a prompt message to the API endpoint of the AI ​​model that generates prompts.

[1225] Output: Response from the generative AI model

[1226] Step 7:

[1227] AI response processing (server)

[1228] The server receives the response from the generated AI model and verifies that it is in the correct format. If it is abnormal, it generates an error message.

[1229] Input: Response from the generative AI model (in JSON format)

[1230] Specific actions: Check the format of the response and, if correct, reflect it in the order processing system. If incorrect, generate an error message.

[1231] Output: Correct order data or error message

[1232] Step 8:

[1233] Response display (terminal)

[1234] The terminal displays the analysis results and error messages received from the server to the user.

[1235] Input: Response from the server (analysis result or error message)

[1236] Specific action: The response content is displayed on the device's screen.

[1237] Output: Order result or error message displayed to the user

[1238] Step 9:

[1239] Order confirmation and completion (user)

[1240] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[1241] Input: Order result or error message displayed on the device

[1242] Specific actions: The user reviews and modifies the order details, and places additional orders or cancels the order.

[1243] Output: Final confirmed order or additional order

[1244] (Application Example 1)

[1245] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1246] Conventional order processing systems have problems efficiently and accurately analyzing user orders in natural language, checking inventory in real time, and confirming order processing. Furthermore, they are prone to human error and processing delays, leading to high labor costs.

[1247] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1248] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user. It also includes means for comparing the order details with an inventory database to check for availability, and means for confirming the order or reordering based on the confirmed inventory status. As a result, when a user places an order in natural language, the order details are analyzed by a generative AI, and inventory is checked in real time, efficient and accurate order processing becomes possible.

[1249] A "specific artificial intelligence engine" is a software engine with advanced algorithms designed to analyze natural language input from users and output it in a specific format.

[1250] "Ordering in natural language" refers to orders placed by users using everyday language, and is a free format that does not require a specific format or protocol.

[1251] "Outputting in a specific format" means that the order details analyzed by the AI ​​are output in a structured format that can be used in subsequent processing, such as JSON format.

[1252] "Adding order details to a list" refers to adding the analyzed order details to a specific database or list structure and managing them in conjunction with other orders.

[1253] "Calculate the total amount and notify the user" means adding up the prices of each item in the order that has been added to the list and informing the user of the total amount.

[1254] An "inventory database" is a database that manages inventory information for products in stores and warehouses, recording the current inventory status of each product.

[1255] "Inventory check" is the process of verifying whether an ordered item exists in the inventory database and confirming that it is in stock.

[1256] "Order confirmation" refers to the formal processing of an order after verifying that the user's order details match the inventory status and are feasible.

[1257] "Suggesting a reorder" refers to suggesting alternative products or encouraging users to place another order when inventory is insufficient.

[1258] A "generative AI model" is an artificial intelligence model that analyzes input text or audio, understands its meaning, and generates appropriate responses or data.

[1259] A "prompt statement" is an input statement used to instruct a generative AI model on what kind of analysis to perform, and it includes specific questions and instructions.

[1260] This invention relates to a system that uses generative AI to analyze a user's order in natural language and efficiently process the order details. The embodiments for carrying out this invention will be described in detail below.

[1261] System Overview

[1262] This system primarily consists of three elements: servers, terminals, and users. Specifically, the following processes are performed:

[1263] server

[1264] The server is responsible for setting the API key for the generative AI and analyzing the user's order details. Specifically, it uses the following hardware and software:

[1265] Hardware: Cloud servers or on-premises servers

[1266] Software: Python, Flask, OpenAI's GPT-4 API

[1267] The server will perform the following steps:

[1268] 1. Setting the API key for the generative AI: The server first sets the API key for the generative AI to make the AI ​​engine available for use.

[1269] 2. Analysis of user input: The system receives orders from the user in natural language, generates prompt sentences for analysis, and sends them to the generative AI.

[1270] 3. Processing AI responses: Analyze the response from the AI ​​and output the order details in JSON format.

[1271] 4. Inventory check: The server checks the order details against the inventory database to confirm whether the item is in stock.

[1272] 5. Confirmation and Notification: Notify the user of confirmed order details and inventory status.

[1273] For example, if a user types "I'd like a hamburger and a soda," the server will generate a prompt message like this:

[1274] "The items the user wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format, including the ID."

[1275] terminal

[1276] The terminal provides an interface for receiving order input from users in natural language. Specifically, it performs the following processes:

[1277] Hardware: Smartphone (iOS or Android)

[1278] Software: Mobile applications

[1279] 1. Accepting user input: The terminal provides a screen for accepting order input from the user in natural language.

[1280] 2. Sending the order details: The terminal sends the order details received from the user to the server.

[1281] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[1282] User

[1283] The user will perform the following specific operations through this system.

[1284] 1. Order Input: The user enters their order details in natural language.

[1285] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1286] Specific example

[1287] When a user enters "I'd like a hamburger and a soda" into a smartphone application, the order is sent to a server. The server generates a prompt for a generative AI model and sends it to the AI ​​engine. The AI ​​analyzes the order and returns a response in JSON format. The server then checks the inventory database to confirm availability. The confirmed results are returned to the device and displayed to the user.

[1288] In this way, the present invention enables users to place orders using natural language, analyze the order details in real time, check inventory levels, and ultimately achieve efficient and error-free order processing.

[1289] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1290] Step 1:

[1291] The user enters their order using natural language.

[1292] Specific operation:

[1293] The user enters their order using a smartphone application. Example: "I'd like a hamburger and a soda, please."

[1294] input:

[1295] A user's order in natural language.

[1296] output:

[1297] The order text displayed in the input field on the smartphone app.

[1298] Step 2:

[1299] The terminal sends the order details to the server.

[1300] Specific operation:

[1301] The terminal sends user input to the server in text format. Specifically, it is sent as an HTTP request.

[1302] input:

[1303] A natural language order from the user.

[1304] output:

[1305] Order data received by the server.

[1306] Step 3:

[1307] The server generates prompt text for the generative AI.

[1308] Specific operation:

[1309] The server generates prompts for the generative AI based on the user's order. Example: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1310] input:

[1311] User order data sent to the server.

[1312] output:

[1313] A prompt message to send to a generative AI.

[1314] Step 4:

[1315] The server sends a prompt message to the generative AI and receives a response.

[1316] Specific operation:

[1317] The server sends prompt messages to a generative AI (e.g., GPT-4) and receives analysis results from the AI. These analysis results are order data in JSON format.

[1318] input:

[1319] A prompt message for a generative AI.

[1320] output:

[1321] Response with order data in JSON format.

[1322] Step 5:

[1323] The server parses the order data in JSON format and compares it with the inventory database.

[1324] Specific operation:

[1325] The server analyzes the received order data and obtains the ID of each item. Next, it compares this with the inventory database to check for availability.

[1326] input:

[1327] Order data in JSON format.

[1328] output:

[1329] Inventory check results.

[1330] Step 6:

[1331] The server confirms the order or suggests reordering based on inventory status.

[1332] Specific operation:

[1333] Based on inventory levels, the server generates confirmed orders and suggestions for reorders for items that are out of stock.

[1334] input:

[1335] Inventory check results and order data in JSON format.

[1336] output:

[1337] Order confirmation information and reorder suggestion information.

[1338] Step 7:

[1339] The server sends order confirmation information and reorder suggestion information to the terminal.

[1340] Specific operation:

[1341] The server returns order confirmation information and a suggestion to reorder to the terminal as an HTTP response.

[1342] input:

[1343] Order confirmation information and reorder suggestion information.

[1344] output:

[1345] Order confirmation information and reorder suggestion information received by the terminal.

[1346] Step 8:

[1347] The terminal displays order confirmation information and reorder suggestion information to the user.

[1348] Specific operation:

[1349] The terminal displays the received information on the user interface, informing the user of the order confirmation details and suggestions for reordering.

[1350] input:

[1351] Order confirmation information and reorder suggestion information received from the server.

[1352] output:

[1353] Order confirmation information and reorder suggestion information displayed on the user's smartphone screen.

[1354] In this way, user orders in natural language are analyzed by generative AI, resulting in efficient and accurate order processing.

[1355] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1356] This invention relates to an order processing system that analyzes a user's in-store order using a generative AI and an emotion engine, and further recognizes the user's emotions to optimize the response. The embodiments for carrying out this invention will be described in detail below.

[1357] System Overview

[1358] System configuration:

[1359] This system primarily consists of three elements: servers, terminals, and users.

[1360] Server: The server sets the API keys for the generative AI and the emotion recognition engine, enabling the use of each engine. It also manages the list of menu items, analyzes user input, and outputs order details in a specific format.

[1361] Terminal: The terminal provides an interface for receiving orders from users in natural language, and communicates with the server to process orders and perform sentiment analysis.

[1362] User: The user enters their order via a terminal and receives responses and recommendations from the system.

[1363] Program Description

[1364] server:

[1365] The server performs the following specific actions:

[1366] 1. API Key Configuration: The server configures the API keys for the generative AI and emotion recognition engines, enabling the use of each engine.

[1367] 2. Menu Item Management: The server maintains a list of menu items that users can order, and this list includes the item's ID, name, and price.

[1368] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[1369] Specific example:

[1370] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[1371] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[1372] 5. Emotion Recognition: If the response is in the correct format, the user's input is also sent to the emotion recognition engine to analyze the user's emotions.

[1373] Specific example:

[1374] When a user types "I'd like a hamburger and a soda," the emotion recognition engine identifies the user's emotional state (e.g., joy, anger, sadness, etc.) from their input.

[1375] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the response content and displayed messages.

[1376] Terminal:

[1377] The terminal performs the following specific actions:

[1378] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[1379] 2. Sending the order details: The terminal sends the received order details to the server.

[1380] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[1381] User:

[1382] The user will perform the following specific operations through this system.

[1383] 1. Order Input: The user enters their order details in natural language.

[1384] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1385] This system allows users to place orders using natural language. A generative AI analyzes the input to identify and add order details in real time, ultimately calculating the total amount and notifying the user. Furthermore, by utilizing an emotion recognition engine, the system can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[1386] The following describes the processing flow.

[1387] Step 1:

[1388] Server: Configure API keys for OpenAI and the emotion recognition engine, making each engine available. Also, define a list of menu items available to the user, and maintain the ID, name, and price of each item.

[1389] Step 2:

[1390] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[1391] Step 3:

[1392] User: Enter the items you want to order in natural language. For example, enter "I'd like a hamburger and a soda, please."

[1393] Step 4:

[1394] Terminal: Receives user input and sends the input to the server.

[1395] Step 5:

[1396] Server: Receives user input and generates prompts for the generative AI to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1397] Step 6:

[1398] Server: Sends the generated prompt to the generative AI and waits for the analysis results.

[1399] Step 7:

[1400] AI: Analyzes user input and returns order details as JSON data. For example, it returns {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[1401] Step 8:

[1402] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[1403] Step 9:

[1404] Server: Based on the added order, it sends a prompt to the emotion recognition engine to recognize the user's emotions.

[1405] Step 10:

[1406] Emotion recognition engine: Identifies emotions from user input and returns the result to the server. For example, it identifies the user's emotional state, such as "happy" or "angry."

[1407] Step 11:

[1408] Server: Based on the results of sentiment analysis, it generates messages to provide appropriate responses and suggestions. For example, if the user is "happy," it generates a message such as "Thank you for your order! Have a great day!"

[1409] Step 12:

[1410] Server: Sends the generated response message to the terminal.

[1411] Step 13:

[1412] Terminal: Displays messages received from the server to the user.

[1413] Step 14:

[1414] User: Enter any additional orders as needed. For example, "Please add some french fries."

[1415] Step 15:

[1416] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[1417] Step 16:

[1418] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[1419] Step 17:

[1420] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[1421] Step 18:

[1422] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[1423] Step 19:

[1424] System: At this point, the order process is complete and the program terminates.

[1425] Through the steps described above, this system can automatically process user orders and further improve customer satisfaction by analyzing user emotions and providing optimal responses.

[1426] (Example 2)

[1427] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1428] In modern restaurants and retail businesses, efficiently processing and analyzing customer orders is a challenge. Furthermore, understanding customer emotions and providing appropriate responses based on those emotions is essential for improving customer satisfaction. Existing systems often fail to adequately analyze order content and recognize emotions, potentially leading to a diminished user experience. There is also a need for more efficient order processing and reduced labor costs.

[1429] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1430] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format; means for confirming the output and adding the order details to a list based on the output; means for calculating the total amount and notifying the user; means for generating a prompt for a generative artificial intelligence engine and obtaining the engine's response; means for sending the user's input to an emotion recognition engine and analyzing the user's emotions; and means for adjusting the response content based on the emotion analysis results. This makes it possible to efficiently analyze a user's order in natural language and provide an appropriate response in real time. Furthermore, by providing a response that corresponds to the user's emotional state, it is expected that customer satisfaction will improve.

[1431] A "specific artificial intelligence engine" is software or hardware that has the function of analyzing user input data and outputting it in a specific format.

[1432] "Natural language" refers to the words and sentences that humans use on a daily basis, in a format that can be understood and analyzed by machines.

[1433] A "prompt" is input data used to give instructions or questions to an artificial intelligence engine, and is intended to guide its analysis and response.

[1434] An "emotion recognition engine" is software or hardware that analyzes user input data to identify emotional states (e.g., joy, anger, sadness, etc.).

[1435] A "specific format" is a format that includes the ID, name, and price of the ordered items, allowing the system to clearly identify the order.

[1436] A "list" is a collection of order details, containing detailed information about each ordered item.

[1437] The "total amount" refers to the total price of all items ordered by the user and is displayed to the user.

[1438] A "generative artificial intelligence engine" refers to an artificial intelligence model that can generate the optimal response to a user's prompt.

[1439] "Response content" refers to the reactions or messages that the system generates in response to user input, and is the information presented to the user.

[1440] "Adjustment" refers to modifying or changing the response content based on the results of the user's sentiment analysis, with the aim of increasing user satisfaction.

[1441] This invention relates to a system in which a user inputs an order in natural language, and the order content is analyzed and optimized using a generative artificial intelligence engine and an emotion recognition engine. This system consists of three main elements: a server, a terminal, and a user.

[1442] Server Role

[1443] The server primarily performs the following processes:

[1444] 1. API key settings:

[1445] The server sets the API keys for the generative artificial intelligence engine and the emotion recognition engine, and makes these engines available for use.

[1446] Hardware and software used: Cloud storage and API management services.

[1447] 2. Managing menu items:

[1448] The server manages a list of menu items that users can order. This list includes the item's ID, name, and price.

[1449] Hardware and software used: Database management system.

[1450] Specific example: The menu list includes entries such as "ID: 1, Name: Hamburger, Price: 500 yen", "ID: 2, Name: Soda, Price: 150 yen", etc.

[1451] 3. Analysis of user input:

[1452] The system receives order details entered by the user in natural language and generates prompts for the generative artificial intelligence engine.

[1453] Example: If a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu options are as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1454] 4. Inquiries to generative artificial intelligence engines:

[1455] The server sends a prompt to the generative artificial intelligence engine to analyze the order details.

[1456] Hardware and software used: Cloud-based generative artificial intelligence API.

[1457] 5. Processing AI responses:

[1458] The server receives the response from the generative artificial intelligence engine, verifies that it is in the correct format (e.g., JSON format), and returns an error message if it is invalid.

[1459] Specific example: Verify that the response is in the format "{"items": [{"id": 1, "name": "hamburger"}, {"id": 2, "name": "soda"}]}".

[1460] 6. Emotion recognition:

[1461] The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions.

[1462] Hardware and software used: Cloud-based emotion recognition API.

[1463] Specific example: If a user types "I'd like a hamburger and a soda," the emotion recognition engine detects "joy."

[1464] 7. Adjusting the response:

[1465] Based on the results of the emotion analysis, the response content and displayed messages are adjusted.

[1466] Specific example: If the user expresses "joy," respond with, "Thank you! I'll bring your hamburger and soda right away."

[1467] Terminal role

[1468] The terminal receives order input from the user and provides an interface for communicating with the server.

[1469] 1. Accepting user input:

[1470] The terminal accepts order input from users in natural language.

[1471] Hardware and software to be used: Touchscreen input device and speech recognition software.

[1472] 2. Submit your order:

[1473] The terminal sends the received order details to the server.

[1474] 3. Display of emotion analysis results:

[1475] The terminal displays the analysis results and sentiment analysis results received from the server to the user.

[1476] Hardware and software to be used: Display unit and display software.

[1477] User roles

[1478] Users place orders through this system.

[1479] 1. Enter your order:

[1480] Users enter their order details in natural language.

[1481] Example: Enter "I'd like a hamburger and a soda."

[1482] 2. Order confirmation and completion:

[1483] The user reviews their order, places additional orders as needed, and finally enters "Finish" to complete the ordering process.

[1484] Overall picture of the operation

[1485] This system allows users to place orders using natural language, and a generative artificial intelligence engine analyzes the input to identify the order details in real time. Furthermore, by utilizing an emotion recognition engine, it can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[1486] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1487] Step 1:

[1488] User-generated order entry using natural language

[1489] Input: The user enters the order details in natural language.

[1490] Specific operation: The user uses the input interface of the in-store terminal to enter their order details in natural language, such as "I'd like a hamburger and a soda, please."

[1491] Output: Order data in natural language input.

[1492] Step 2:

[1493] Sending order details via terminal

[1494] Input: Order details entered by the user (in natural language).

[1495] Specific operation: The terminal sends the natural language order data entered by the user to the server.

[1496] Output: Order data sent to the server.

[1497] Step 3:

[1498] Server-driven prompt generation

[1499] Input: Order data in natural language sent from the terminal.

[1500] Specific operation: The server generates a prompt to request analysis from the generative artificial intelligence engine. It generates a prompt that says, "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON including the ID."

[1501] Output: The prompt text to send to the generative artificial intelligence engine.

[1502] Step 4:

[1503] Server queries to the generative artificial intelligence engine

[1504] Input: The generated prompt message.

[1505] Specific operation: The server sends a prompt message to the generative artificial intelligence engine, which then analyzes the order details.

[1506] Output: Analysis results returned by the generative artificial intelligence engine (data identifying the order details).

[1507] Step 5:

[1508] Server-based verification and analysis of AI responses.

[1509] Input: Analysis results returned from a generative artificial intelligence engine.

[1510] Specific operation: The server checks whether the response from the generative artificial intelligence engine is in the correct format (e.g., JSON format). If it is invalid, it generates an error message and returns it to the user.

[1511] Output: Order details data in the correct format, or an error message.

[1512] Step 6:

[1513] Server queries emotion recognition engine

[1514] Input: User's order text.

[1515] Specific operation: The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions. For example, when a user enters "I'd like a hamburger and a soda, please," the server sends this text to the emotion recognition engine.

[1516] Output: Emotion analysis results from the emotion recognition engine (e.g., joy, anger, sadness, etc.).

[1517] Step 7:

[1518] Server-side adjustment of response content

[1519] Input: Sentiment analysis results from the emotion recognition engine, and correctly formatted order data.

[1520] Specific operation: The server adjusts the response based on the results of the sentiment analysis. For example, if the user is feeling "joyful," the response message will be "Thank you! I'll bring your hamburger and soda right away."

[1521] Output: Optimized response content.

[1522] Step 8:

[1523] Server sends response data to terminal

[1524] Input: Optimized response content.

[1525] Specific operation: The server sends the optimized response to the terminal.

[1526] Output: Response data sent to the terminal.

[1527] Step 9:

[1528] Displaying the response from the terminal

[1529] Input: Response data sent from the server.

[1530] Specific operation: The terminal displays the received response on its screen and informs the user of the result.

[1531] Output: The response message displayed on the terminal's screen.

[1532] (Application Example 2)

[1533] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1534] Currently, many stores rely on manual order entry by staff, posing challenges to improving order efficiency and customer experience. Furthermore, providing appropriate responses tailored to customer emotions is difficult, limiting the potential for increased customer satisfaction. Therefore, there is a need for a system that automatically handles natural language order taking, provides optimal product suggestions based on those orders, and offers responses that take customer emotions into consideration.

[1535] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for analyzing a natural language order from a user using a specific generation technology engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user, means for identifying emotions from the user's input using an emotion analysis engine, and means for adjusting the response to the user based on the results of the emotion analysis. This not only automates the analysis and processing of natural language orders, but also enables responses that respond to the user's emotions, thereby improving customer satisfaction and operational efficiency.

[1536] A "generative technology engine" is an artificial intelligence engine that analyzes a user's order in natural language and outputs it in a specific format.

[1537] An "emotion analysis engine" is an artificial intelligence engine that identifies emotions from user input and outputs the analysis results.

[1538] "Specific format" refers to a format that includes the identifier, name, and price of the ordered items.

[1539] "Means for adding order details to a list" refers to a processing method or apparatus for adding analyzed order details to an existing order list.

[1540] "Means for calculating the total amount" refers to a processing method or apparatus for calculating the total amount by summing up the prices of the listed order items.

[1541] "Means for adjusting the content of responses to the user" refers to a processing method or apparatus for appropriately changing the content of responses to the user based on the results of the sentiment analysis engine.

[1542] This invention relates to a system that uses a generation technology engine and an emotion analysis engine to analyze a user's natural language order, recognize the user's emotional state, and provide an optimal response. This system mainly consists of three elements: a server, a terminal, and a user.

[1543] Server configuration and functionality

[1544] The server has the following roles:

[1545] 1. Setting API keys for the generative technology engine and sentiment analysis engine: The server sets the API keys for the generative technology engine and sentiment analysis engine to enable the use of each engine.

[1546] 2. Menu Item Management: The server manages a list of menu items that users can order, and this list includes the identifier, name, and price of each item.

[1547] 3. Parsing user input: The system receives order details entered by the user in natural language and generates prompt sentences for parsing using a generation technology engine.

[1548] Example: If a user enters "Please recommend a coffee and a cake," the server will generate a prompt like this: "User order: Please recommend a coffee and a cake. Menu: [{menu list}]. Identify the order and respond in JSON format."

[1549] 4. AI response processing: The server receives the response from the generation technology engine and verifies that it is in the correct format (JSON format). If it is invalid, it returns an error message to the user.

[1550] 5. Emotion Recognition: The server sends the user's input text to the emotion analysis engine, which then analyzes the user's emotions.

[1551] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the content of the response to the user.

[1552] Device configuration and functions

[1553] The terminal is for use by store staff and performs the following tasks:

[1554] 1. Receiving user input: The terminal provides an interface for receiving order input from the user in natural language.

[1555] 2. Sending the order details: The terminal sends the received order details to the server.

[1556] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[1557] User roles

[1558] The user performs the following actions:

[1559] 1. Order Input: The user enters their order details in natural language.

[1560] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1561] Overall system flow

[1562] To implement this invention, the server, terminal, and user must each fulfill their respective roles and connect and communicate seamlessly. By using a generative technology engine and an emotion analysis engine in combination, a system is built that analyzes the user's natural language input and automatically provides responses that correspond to their emotions. This makes it possible to achieve real-time order processing and high customer satisfaction.

[1563] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1564] Step 1:

[1565] The terminal accepts orders from users via voice or text input using natural language. The input data consists of the user's order details, which serves as the starting point for processing the entire system.

[1566] Step 2:

[1567] The terminal sends the received order details to the server. The data is in natural language text format, and its content is necessary for analysis on the server side.

[1568] Step 3:

[1569] The server requests the AI ​​to analyze the order details. The input data is the user's order text and menu list, which are sent as prompts to the AI ​​model. "User Order: [Order Details] Menu: [{Menu List}]. Identify the order details and respond in JSON format."

[1570] Step 4:

[1571] The generative AI model receives a prompt, parses the order details, and outputs them in a specific format (JSON format). This output includes the identifier, name, and price of each ordered item.

[1572] Step 5:

[1573] The server receives the response from the generated AI model and verifies that the output is in the correct format. If it is invalid, it generates an error message and sends it to the terminal to notify the user; otherwise, it proceeds to the next step.

[1574] Step 6:

[1575] The server sends correctly formatted output to the sentiment analysis engine to identify the user's emotions. The input data is the text of the order details, and the sentiment recognition engine analyzes the text to output the emotional state (e.g., joy, anger, sadness, etc.).

[1576] Step 7:

[1577] The server receives the sentiment analysis results and adjusts its response to the user based on the analysis. For example, if the user is happy, it generates a response that will make them even happier. Specific example: "Certainly. I'm glad! Your order is a cafe latte and cheesecake."

[1578] Step 8:

[1579] The server sends the adjusted response to the terminal. This response reflects the analysis results from the generative AI model and the sentiment analysis engine.

[1580] Step 9:

[1581] The terminal displays or verbally communicates the response received from the server to the user. This allows the user to review the information and take additional actions (such as placing an additional order) as needed.

[1582] Step 10:

[1583] If the user places an additional order, the process returns to step 1 and repeats. Once the final order is confirmed, a termination command is sent from the terminal to the server.

[1584] Step 11:

[1585] The server calculates the total amount based on the list of final order items and sends the result to the terminal. This notifies the user, and the order process is completed.

[1586] This series of steps results in a system that analyzes the user's natural language order and provides the most appropriate response based on their emotions.

[1587] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1588] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1590] [Fourth Embodiment]

[1591] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1592] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1593] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1594] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1595] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1597] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1598] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1599] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1600] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1601] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1602] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1603] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1604] This invention relates to an order processing system that uses generative AI to analyze user in-store orders and reduce labor costs. The embodiments for carrying out this invention will be described in detail below.

[1605] System Overview

[1606] System configuration:

[1607] This system primarily consists of three elements: servers, terminals, and users.

[1608] Server: The server sets the API key for the generative AI and manages the list of menu items. It also analyzes user input and outputs the order details in a specific format.

[1609] Terminal: The terminal provides an interface for receiving orders from users in natural language and communicates with the server to process the orders.

[1610] User: The user enters their order via a terminal and receives a response from the system.

[1611] Program Description

[1612] server:

[1613] The server performs the following specific actions:

[1614] 1. API Key Configuration: The server configures API keys for generative AI (such as OpenAI) to enable the use of specific AI engines.

[1615] 2. Menu Item Management: The server maintains a list of menu items that users can order. This list includes the item's ID, name, and price.

[1616] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[1617] Specific example:

[1618] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[1619] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[1620] Terminal:

[1621] The terminal performs the following specific actions:

[1622] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[1623] 2. Sending the order details: The terminal sends the received order details to the server.

[1624] Specific example:

[1625] When a user types "Please add some french fries," the terminal sends this input to the server.

[1626] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[1627] User:

[1628] The user will perform the following specific operations through this system.

[1629] 1. Order Input: The user enters their order details in natural language.

[1630] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1631] This system allows users to place orders using natural language, which are then analyzed in real time by generative AI, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[1632] The following describes the processing flow.

[1633] Step 1:

[1634] Server: Set up the OpenAI API key and prepare the AI ​​engine to be used (e.g., "text-davinci-003"). Also, define a list of menu items and store the ID, name, and price of each item.

[1635] Step 2:

[1636] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[1637] Step 3:

[1638] User: Enter the items you want to order in natural language. For example, "I'd like one hamburger and a soda."

[1639] Step 4:

[1640] Terminal: Receives user input and sends the input to the server.

[1641] Step 5:

[1642] Server: Receives user input and generates prompts for the AI ​​to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1643] Step 6:

[1644] Server: Sends the generated prompt to the AI ​​engine and waits for the analysis results.

[1645] Step 7:

[1646] AI: Analyzes user input and returns order details as JSON data. For example, it returns in the format {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[1647] Step 8:

[1648] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[1649] Step 9:

[1650] Server: Notifies the user that an order has been added. For example, it might notify them with a message like, "A hamburger has been added," or "A soda has been added."

[1651] Step 10:

[1652] Terminal: Receives notifications from the server and displays them to the user.

[1653] Step 11:

[1654] User: Enter any additional orders as needed. For example, "Please add one more order of french fries."

[1655] Step 12:

[1656] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[1657] Step 13:

[1658] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[1659] Step 14:

[1660] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[1661] Step 15:

[1662] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[1663] Step 16:

[1664] System: At this point, the order process is complete and the program terminates.

[1665] Through the steps described above, this system can automatically process user orders, thereby reducing labor costs and improving efficiency.

[1666] (Example 1)

[1667] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1668] Traditional order processing systems involve numerous manual processes, such as receiving and confirming orders and calculating total amounts, which not only reduces efficiency but also increases the likelihood of human error. Furthermore, the lack of technology to automatically analyze and accurately process user orders results in slow order processing.

[1669] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1670] In this invention, the server includes means for analyzing natural language orders from users using a generative artificial intelligence model and outputting the orders in a specific format; means for verifying the output and adding the order details to a database based on the output; means for receiving user input via a terminal and sending the input to the server; means for the server to send a prompt message to the generative artificial intelligence model and receive a response from the AI; means for verifying the response and, if in the correct format, reflecting it in the order processing system; means for displaying the response and error messages on the terminal; and means for calculating the total amount and notifying the user. This enables efficient and accurate analysis of natural language orders from users and allows for rapid order processing.

[1671] A "generative artificial intelligence model" is a general term for artificial intelligence algorithms and systems that analyze natural language input from users and convert it into a specific format.

[1672] "Natural language" refers to the language that humans use on a daily basis, and is not a specific programming language, but rather language that arises naturally.

[1673] A "prompt" is input text used to provide instructions or information to an artificial intelligence model.

[1674] A "server" refers to a computer or system that provides specific services or functions within a network, and in this invention, it plays the role of analyzing orders using a generative AI model.

[1675] A "terminal" refers to a device or interface used by users to input information, and its role is to receive order details and communicate with the server.

[1676] A "database" is a general term for a system or software used to efficiently store, retrieve, and manage data, and in this invention, it is used to store menu items and order details.

[1677] An "order processing system" is a general term for a system used to manage and process orders from users.

[1678] An "error message" is a message sent to inform the user of the reason why a system or program is not functioning correctly.

[1679] "Response" refers to the result or reply that a generative artificial intelligence model generates based on user input.

[1680] "Total amount" refers to the sum of the prices of all items ordered by the user.

[1681] System Overview

[1682] This invention is a system that uses a generative AI model to analyze a user's order in natural language and processes the order efficiently and accurately without human intervention. This invention mainly consists of three elements: a server, a terminal, and a user.

[1683] Server Configuration and Roles

[1684] The server sets the API key for the generated AI model and manages the menu items. Specifically, it performs the following processes:

[1685] 1. API key settings:

[1686] The server configures the API key for the generated AI model (e.g., OpenAI) and makes the specific AI engine available. The API key is saved in a configuration file and read when the server starts, thereby setting the API key in the application.

[1687] 2. Managing menu items:

[1688] The server stores a list of menu items that users can order in a database (e.g., MySQL). This list includes the item's identifier, name, and price.

[1689] Allow users to add, update, and delete menu items as needed.

[1690] 3. Analysis of user input:

[1691] It receives the user's order details in natural language, generates a prompt, and sends it to the AI ​​model. For example, if the user enters "I'd like a hamburger and a soda," it will generate a prompt like the following.

[1692] Example prompt: "User wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it as a JSON with its ID."

[1693] 4. AI response processing:

[1694] The server receives the response from the generated AI model and verifies that it is in the correct format (JSON format). If it is correct, it is reflected in the order processing system; otherwise, an error message is generated.

[1695] Device configuration and role

[1696] The terminal provides an interface for receiving order input from users and communicates with the server to process the orders.

[1697] 1. Accepting user input:

[1698] A text input field is displayed on the terminal, allowing the user to enter their order details in natural language.

[1699] 2. Submit your order:

[1700] The order details entered by the user in the text input field are sent to the server.

[1701] 3. Display of response:

[1702] The analysis results and error messages received from the server are displayed to the user.

[1703] User roles

[1704] Users enter their orders using natural language via a terminal and confirm the system's response.

[1705] 1. Enter your order:

[1706] The user enters their order details into a text input field on the device. For example, they might type, "I'd like a hamburger and a soda."

[1707] 2. Order confirmation and completion:

[1708] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[1709] This system allows users to place orders using natural language, which are then analyzed in real time by a generative AI model, ensuring accurate order identification and processing. This improves efficiency compared to traditional manual order processing, reduces human error, and is expected to lower labor costs.

[1710] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1711] Step 1:

[1712] API key settings (server)

[1713] The server configures the API key for the generated AI model. Specifically, it saves the API key in a configuration file and reads this file when the server starts up, thereby setting the API key in the application.

[1714] Input: Configuration file (e.g., config.json)

[1715] Specific operation: Read the API key from the configuration file and enable access to the AI ​​engine.

[1716] Output: Generative AI model available for use.

[1717] Step 2:

[1718] Menu item management (server)

[1719] The server stores a list of menu items in a database. This list includes the item's identifier, name, and price. It allows for the addition, updating, and deletion of menu items as needed.

[1720] Input: Menu item data (e.g., item identifier, name, price)

[1721] Specific actions: Save, update, and delete menu items in the database.

[1722] Output: Latest menu list

[1723] Step 3:

[1724] User input reception (terminal)

[1725] The terminal provides an interface for receiving order input from users.

[1726] Input: User's order in natural language (e.g., "I'd like a hamburger and a soda, please")

[1727] Specific action: The user enters the order details into a text input field.

[1728] Output: User input

[1729] Step 4:

[1730] Sending user input (terminal)

[1731] The terminal sends the order details entered by the user to the server.

[1732] Input: User input (e.g., "I'd like a hamburger and a soda, please")

[1733] Specific action: Send the input content to the server in JSON format.

[1734] Output: Sending order details to the server

[1735] Step 5:

[1736] User input analysis prompt generation (server)

[1737] The server generates prompt messages to parse the order details received from the user.

[1738] Input: User input and menu list

[1739] Specific operation: Generate a prompt message in the format "Items the user wants to order: (User input). Menu items are as follows: (Menu list). Identify the order and return it in JSON format including the ID."

[1740] Output: Generated prompt message

[1741] Example of a specific prompt: "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1742] Step 6:

[1743] Sending prompts to the generative artificial intelligence model (server)

[1744] The server sends the generated prompt message to the artificial intelligence model.

[1745] Input: Prompt message

[1746] Specific operation: Send a prompt message to the API endpoint of the AI ​​model that generates prompts.

[1747] Output: Response from the generative AI model

[1748] Step 7:

[1749] AI response processing (server)

[1750] The server receives the response from the generated AI model and verifies that it is in the correct format. If it is abnormal, it generates an error message.

[1751] Input: Response from the generative AI model (in JSON format)

[1752] Specific actions: Check the format of the response and, if correct, reflect it in the order processing system. If incorrect, generate an error message.

[1753] Output: Correct order data or error message

[1754] Step 8:

[1755] Response display (terminal)

[1756] The terminal displays the analysis results and error messages received from the server to the user.

[1757] Input: Response from the server (analysis result or error message)

[1758] Specific action: The response content is displayed on the device's screen.

[1759] Output: Order result or error message displayed to the user

[1760] Step 9:

[1761] Order confirmation and completion (user)

[1762] The user reviews their order and places additional orders as needed. Finally, they type "Finish" to complete the ordering process.

[1763] Input: Order result or error message displayed on the device

[1764] Specific actions: The user reviews and modifies the order details, and places additional orders or cancels the order.

[1765] Output: Final confirmed order or additional order

[1766] (Application Example 1)

[1767] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1768] Conventional order processing systems have problems efficiently and accurately analyzing user orders in natural language, checking inventory in real time, and confirming order processing. Furthermore, they are prone to human error and processing delays, leading to high labor costs.

[1769] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1770] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user. It also includes means for comparing the order details with an inventory database to check for availability, and means for confirming the order or reordering based on the confirmed inventory status. As a result, when a user places an order in natural language, the order details are analyzed by a generative AI, and inventory is checked in real time, efficient and accurate order processing becomes possible.

[1771] A "specific artificial intelligence engine" is a software engine with advanced algorithms designed to analyze natural language input from users and output it in a specific format.

[1772] "Ordering in natural language" refers to orders placed by users using everyday language, and is a free format that does not require a specific format or protocol.

[1773] "Outputting in a specific format" means that the order details analyzed by the AI ​​are output in a structured format that can be used in subsequent processing, such as JSON format.

[1774] "Adding order details to a list" refers to adding the analyzed order details to a specific database or list structure and managing them in conjunction with other orders.

[1775] "Calculate the total amount and notify the user" means adding up the prices of each item in the order that has been added to the list and informing the user of the total amount.

[1776] An "inventory database" is a database that manages inventory information for products in stores and warehouses, recording the current inventory status of each product.

[1777] "Inventory check" is the process of verifying whether an ordered item exists in the inventory database and confirming that it is in stock.

[1778] "Order confirmation" refers to the formal processing of an order after verifying that the user's order details match the inventory status and are feasible.

[1779] "Suggesting a reorder" refers to suggesting alternative products or encouraging users to place another order when inventory is insufficient.

[1780] A "generative AI model" is an artificial intelligence model that analyzes input text or audio, understands its meaning, and generates appropriate responses or data.

[1781] A "prompt statement" is an input statement used to instruct a generative AI model on what kind of analysis to perform, and it includes specific questions and instructions.

[1782] This invention relates to a system that uses generative AI to analyze a user's order in natural language and efficiently process the order details. The embodiments for carrying out this invention will be described in detail below.

[1783] System Overview

[1784] This system primarily consists of three elements: servers, terminals, and users. Specifically, the following processes are performed:

[1785] server

[1786] The server is responsible for setting the API key for the generative AI and analyzing the user's order details. Specifically, it uses the following hardware and software:

[1787] Hardware: Cloud servers or on-premises servers

[1788] Software: Python, Flask, OpenAI's GPT-4 API

[1789] The server will perform the following steps:

[1790] 1. Setting the API key for the generative AI: The server first sets the API key for the generative AI to make the AI ​​engine available for use.

[1791] 2. Analysis of user input: The system receives orders from the user in natural language, generates prompt sentences for analysis, and sends them to the generative AI.

[1792] 3. Processing AI responses: Analyze the response from the AI ​​and output the order details in JSON format.

[1793] 4. Inventory check: The server checks the order details against the inventory database to confirm whether the item is in stock.

[1794] 5. Confirmation and Notification: Notify the user of confirmed order details and inventory status.

[1795] For example, if a user types "I'd like a hamburger and a soda," the server will generate a prompt message like this:

[1796] "The items the user wants to order: a hamburger and a soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON format, including the ID."

[1797] terminal

[1798] The terminal provides an interface for receiving order input from users in natural language. Specifically, it performs the following processes:

[1799] Hardware: Smartphone (iOS or Android)

[1800] Software: Mobile applications

[1801] 1. Accepting user input: The terminal provides a screen for accepting order input from the user in natural language.

[1802] 2. Sending the order details: The terminal sends the order details received from the user to the server.

[1803] 3. Display of response: The terminal displays the analysis results and error messages received from the server to the user.

[1804] User

[1805] The user will perform the following specific operations through this system.

[1806] 1. Order Input: The user enters their order details in natural language.

[1807] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1808] Specific example

[1809] When a user enters "I'd like a hamburger and a soda" into a smartphone application, the order is sent to a server. The server generates a prompt for a generative AI model and sends it to the AI ​​engine. The AI ​​analyzes the order and returns a response in JSON format. The server then checks the inventory database to confirm availability. The confirmed results are returned to the device and displayed to the user.

[1810] In this way, the present invention enables users to place orders using natural language, analyze the order details in real time, check inventory levels, and ultimately achieve efficient and error-free order processing.

[1811] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1812] Step 1:

[1813] The user enters their order using natural language.

[1814] Specific operation:

[1815] The user enters their order using a smartphone application. Example: "I'd like a hamburger and a soda, please."

[1816] input:

[1817] A user's order in natural language.

[1818] output:

[1819] The order text displayed in the input field on the smartphone app.

[1820] Step 2:

[1821] The terminal sends the order details to the server.

[1822] Specific operation:

[1823] The terminal sends user input to the server in text format. Specifically, it is sent as an HTTP request.

[1824] input:

[1825] A natural language order from the user.

[1826] output:

[1827] Order data received by the server.

[1828] Step 3:

[1829] The server generates prompt text for the generative AI.

[1830] Specific operation:

[1831] The server generates prompts for the generative AI based on the user's order. Example: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1832] input:

[1833] User order data sent to the server.

[1834] output:

[1835] A prompt message to send to a generative AI.

[1836] Step 4:

[1837] The server sends a prompt message to the generative AI and receives a response.

[1838] Specific operation:

[1839] The server sends prompt messages to a generative AI (e.g., GPT-4) and receives analysis results from the AI. These analysis results are order data in JSON format.

[1840] input:

[1841] A prompt message for a generative AI.

[1842] output:

[1843] Response with order data in JSON format.

[1844] Step 5:

[1845] The server parses the order data in JSON format and compares it with the inventory database.

[1846] Specific operation:

[1847] The server analyzes the received order data and obtains the ID of each item. Next, it compares this with the inventory database to check for availability.

[1848] input:

[1849] Order data in JSON format.

[1850] output:

[1851] Inventory check results.

[1852] Step 6:

[1853] The server confirms the order or suggests reordering based on inventory status.

[1854] Specific operation:

[1855] Based on inventory levels, the server generates confirmed orders and suggestions for reorders for items that are out of stock.

[1856] input:

[1857] Inventory check results and order data in JSON format.

[1858] output:

[1859] Order confirmation information and reorder suggestion information.

[1860] Step 7:

[1861] The server sends order confirmation information and reorder suggestion information to the terminal.

[1862] Specific operation:

[1863] The server returns order confirmation information and a suggestion to reorder to the terminal as an HTTP response.

[1864] input:

[1865] Order confirmation information and reorder suggestion information.

[1866] output:

[1867] Order confirmation information and reorder suggestion information received by the terminal.

[1868] Step 8:

[1869] The terminal displays order confirmation information and reorder suggestion information to the user.

[1870] Specific operation:

[1871] The terminal displays the received information on the user interface, informing the user of the order confirmation details and suggestions for reordering.

[1872] input:

[1873] Order confirmation information and reorder suggestion information received from the server.

[1874] output:

[1875] Order confirmation information and reorder suggestion information displayed on the user's smartphone screen.

[1876] In this way, user orders in natural language are analyzed by generative AI, resulting in efficient and accurate order processing.

[1877] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1878] This invention relates to an order processing system that analyzes a user's in-store order using a generative AI and an emotion engine, and further recognizes the user's emotions to optimize the response. The embodiments for carrying out this invention will be described in detail below.

[1879] System Overview

[1880] System configuration:

[1881] This system primarily consists of three elements: servers, terminals, and users.

[1882] Server: The server sets the API keys for the generative AI and the emotion recognition engine, enabling the use of each engine. It also manages the list of menu items, analyzes user input, and outputs order details in a specific format.

[1883] Terminal: The terminal provides an interface for receiving orders from users in natural language, and communicates with the server to process orders and perform sentiment analysis.

[1884] User: The user enters their order via a terminal and receives responses and recommendations from the system.

[1885] Program Description

[1886] server:

[1887] The server performs the following specific actions:

[1888] 1. API Key Configuration: The server configures the API keys for the generative AI and emotion recognition engines, enabling the use of each engine.

[1889] 2. Menu Item Management: The server maintains a list of menu items that users can order, and this list includes the item's ID, name, and price.

[1890] 3. Analysis of user input: The system receives order details entered by the user in natural language and generates prompts for analysis for the generative AI.

[1891] Specific example:

[1892] For example, if a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON with the ID."

[1893] 4. AI response processing: The server receives the response from the AI, verifies that it is in the correct format (JSON format), and returns an error message to the user if it is invalid.

[1894] 5. Emotion Recognition: If the response is in the correct format, the user's input is also sent to the emotion recognition engine to analyze the user's emotions.

[1895] Specific example:

[1896] When a user types "I'd like a hamburger and a soda," the emotion recognition engine identifies the user's emotional state (e.g., joy, anger, sadness, etc.) from their input.

[1897] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the response content and displayed messages.

[1898] Terminal:

[1899] The terminal performs the following specific actions:

[1900] 1. Receiving user input: The terminal provides an interface for receiving order input from users in natural language.

[1901] 2. Sending the order details: The terminal sends the received order details to the server.

[1902] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[1903] User:

[1904] The user will perform the following specific operations through this system.

[1905] 1. Order Input: The user enters their order details in natural language.

[1906] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[1907] This system allows users to place orders using natural language. A generative AI analyzes the input to identify and add order details in real time, ultimately calculating the total amount and notifying the user. Furthermore, by utilizing an emotion recognition engine, the system can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[1908] The following describes the processing flow.

[1909] Step 1:

[1910] Server: Configure API keys for OpenAI and the emotion recognition engine, making each engine available. Also, define a list of menu items available to the user, and maintain the ID, name, and price of each item.

[1911] Step 2:

[1912] Terminal: Starts the program and displays the message to the user: "Please place your order. Type 'Finish' to end the order."

[1913] Step 3:

[1914] User: Enter the items you want to order in natural language. For example, enter "I'd like a hamburger and a soda, please."

[1915] Step 4:

[1916] Terminal: Receives user input and sends the input to the server.

[1917] Step 5:

[1918] Server: Receives user input and generates prompts for the generative AI to analyze. For example, it might create a prompt like this: "Items the user wants to order: Hamburger and soda. Menu is as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1919] Step 6:

[1920] Server: Sends the generated prompt to the generative AI and waits for the analysis results.

[1921] Step 7:

[1922] AI: Analyzes user input and returns order details as JSON data. For example, it returns {"id": 1, "name": "Hamburger", "price": 500} and {"id": 3, "name": "Soda", "price": 200}.

[1923] Step 8:

[1924] Server: Receives the response from the AI ​​and checks if it is in JSON format. If the analysis result is accurate, adds the order details to the list.

[1925] Step 9:

[1926] Server: Based on the added order, it sends a prompt to the emotion recognition engine to recognize the user's emotions.

[1927] Step 10:

[1928] Emotion recognition engine: Identifies emotions from user input and returns the result to the server. For example, it identifies the user's emotional state, such as "happy" or "angry."

[1929] Step 11:

[1930] Server: Based on the results of sentiment analysis, it generates messages to provide appropriate responses and suggestions. For example, if the user is "happy," it generates a message such as "Thank you for your order! Have a great day!"

[1931] Step 12:

[1932] Server: Sends the generated response message to the terminal.

[1933] Step 13:

[1934] Terminal: Displays messages received from the server to the user.

[1935] Step 14:

[1936] User: Enter any additional orders as needed. For example, "Please add some french fries."

[1937] Step 15:

[1938] Terminal: Sends user input to the server again. This process repeats until "Finish" is entered.

[1939] Step 16:

[1940] User: Once you have finally completed your order, type "Finish" to end the ordering process.

[1941] Step 17:

[1942] Server: After all orders have been placed, calculate the total amount for the items in the order list.

[1943] Step 18:

[1944] Terminal: Displays the total amount to the user and concludes with the message, "Your order total is 700 yen. Thank you!"

[1945] Step 19:

[1946] System: At this point, the order process is complete and the program terminates.

[1947] Through the steps described above, this system can automatically process user orders and further improve customer satisfaction by analyzing user emotions and providing optimal responses.

[1948] (Example 2)

[1949] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1950] In modern restaurants and retail businesses, efficiently processing and analyzing customer orders is a challenge. Furthermore, understanding customer emotions and providing appropriate responses based on those emotions is essential for improving customer satisfaction. Existing systems often fail to adequately analyze order content and recognize emotions, potentially leading to a diminished user experience. There is also a need for more efficient order processing and reduced labor costs.

[1951] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1952] In this invention, the server includes means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format; means for confirming the output and adding the order details to a list based on the output; means for calculating the total amount and notifying the user; means for generating a prompt for a generative artificial intelligence engine and obtaining the engine's response; means for sending the user's input to an emotion recognition engine and analyzing the user's emotions; and means for adjusting the response content based on the emotion analysis results. This makes it possible to efficiently analyze a user's order in natural language and provide an appropriate response in real time. Furthermore, by providing a response that corresponds to the user's emotional state, it is expected that customer satisfaction will improve.

[1953] A "specific artificial intelligence engine" is software or hardware that has the function of analyzing user input data and outputting it in a specific format.

[1954] "Natural language" refers to the words and sentences that humans use on a daily basis, in a format that can be understood and analyzed by machines.

[1955] A "prompt" is input data used to give instructions or questions to an artificial intelligence engine, and is intended to guide its analysis and response.

[1956] An "emotion recognition engine" is software or hardware that analyzes user input data to identify emotional states (e.g., joy, anger, sadness, etc.).

[1957] A "specific format" is a format that includes the ID, name, and price of the ordered items, allowing the system to clearly identify the order.

[1958] A "list" is a collection of order details, containing detailed information about each ordered item.

[1959] The "total amount" refers to the total price of all items ordered by the user and is displayed to the user.

[1960] A "generative artificial intelligence engine" refers to an artificial intelligence model that can generate the optimal response to a user's prompt.

[1961] "Response content" refers to the reactions or messages that the system generates in response to user input, and is the information presented to the user.

[1962] "Adjustment" refers to modifying or changing the response content based on the results of the user's sentiment analysis, with the aim of increasing user satisfaction.

[1963] This invention relates to a system in which a user inputs an order in natural language, and the order content is analyzed and optimized using a generative artificial intelligence engine and an emotion recognition engine. This system consists of three main elements: a server, a terminal, and a user.

[1964] Server Role

[1965] The server primarily performs the following processes:

[1966] 1. API key settings:

[1967] The server sets the API keys for the generative artificial intelligence engine and the emotion recognition engine, and makes these engines available for use.

[1968] Hardware and software used: Cloud storage and API management services.

[1969] 2. Managing menu items:

[1970] The server manages a list of menu items that users can order. This list includes the item's ID, name, and price.

[1971] Hardware and software used: Database management system.

[1972] Specific example: The menu list includes entries such as "ID: 1, Name: Hamburger, Price: 500 yen", "ID: 2, Name: Soda, Price: 150 yen", etc.

[1973] 3. Analysis of user input:

[1974] The system receives order details entered by the user in natural language and generates prompts for the generative artificial intelligence engine.

[1975] Example: If a user enters "I'd like a hamburger and a soda," the server will generate a prompt like this: "Items the user wants to order: Hamburger and soda. Menu options are as follows: [{menu list}]. Identify the order and return it in JSON format including the ID."

[1976] 4. Inquiries to generative artificial intelligence engines:

[1977] The server sends a prompt to the generative artificial intelligence engine to analyze the order details.

[1978] Hardware and software used: Cloud-based generative artificial intelligence API.

[1979] 5. Processing AI responses:

[1980] The server receives the response from the generative artificial intelligence engine, verifies that it is in the correct format (e.g., JSON format), and returns an error message if it is invalid.

[1981] Specific example: Verify that the response is in the format "{"items": [{"id": 1, "name": "hamburger"}, {"id": 2, "name": "soda"}]}".

[1982] 6. Emotion recognition:

[1983] The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions.

[1984] Hardware and software used: Cloud-based emotion recognition API.

[1985] Specific example: If a user types "I'd like a hamburger and a soda," the emotion recognition engine detects "joy."

[1986] 7. Adjusting the response:

[1987] Based on the results of the emotion analysis, the response content and displayed messages are adjusted.

[1988] Specific example: If the user expresses "joy," respond with, "Thank you! I'll bring your hamburger and soda right away."

[1989] Terminal role

[1990] The terminal receives order input from the user and provides an interface for communicating with the server.

[1991] 1. Accepting user input:

[1992] The terminal accepts order input from users in natural language.

[1993] Hardware and software to be used: Touchscreen input device and speech recognition software.

[1994] 2. Submit your order:

[1995] The terminal sends the received order details to the server.

[1996] 3. Display of emotion analysis results:

[1997] The terminal displays the analysis results and sentiment analysis results received from the server to the user.

[1998] Hardware and software to be used: Display unit and display software.

[1999] User roles

[2000] Users place orders through this system.

[2001] 1. Enter your order:

[2002] Users enter their order details in natural language.

[2003] Example: Enter "I'd like a hamburger and a soda."

[2004] 2. Order confirmation and completion:

[2005] The user reviews their order, places additional orders as needed, and finally enters "Finish" to complete the ordering process.

[2006] Overall picture of the operation

[2007] This system allows users to place orders using natural language, and a generative artificial intelligence engine analyzes the input to identify the order details in real time. Furthermore, by utilizing an emotion recognition engine, it can provide responses and suggestions tailored to the user's emotional state, thereby improving customer satisfaction. This is expected to reduce labor costs, streamline order processing, and enhance the user experience.

[2008] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2009] Step 1:

[2010] User-generated order entry using natural language

[2011] Input: The user enters the order details in natural language.

[2012] Specific operation: The user uses the input interface of the in-store terminal to enter their order details in natural language, such as "I'd like a hamburger and a soda, please."

[2013] Output: Order data in natural language input.

[2014] Step 2:

[2015] Sending order details via terminal

[2016] Input: Order details entered by the user (in natural language).

[2017] Specific operation: The terminal sends the natural language order data entered by the user to the server.

[2018] Output: Order data sent to the server.

[2019] Step 3:

[2020] Server-driven prompt generation

[2021] Input: Order data in natural language sent from the terminal.

[2022] Specific operation: The server generates a prompt to request analysis from the generative artificial intelligence engine. It generates a prompt that says, "Items the user wants to order: Hamburger and soda. The menu is as follows: [{menu list}]. Identify the order and return it in JSON including the ID."

[2023] Output: The prompt text to send to the generative artificial intelligence engine.

[2024] Step 4:

[2025] Server queries to the generative artificial intelligence engine

[2026] Input: The generated prompt message.

[2027] Specific operation: The server sends a prompt message to the generative artificial intelligence engine, which then analyzes the order details.

[2028] Output: Analysis results returned by the generative artificial intelligence engine (data identifying the order details).

[2029] Step 5:

[2030] Server-based verification and analysis of AI responses.

[2031] Input: Analysis results returned from a generative artificial intelligence engine.

[2032] Specific operation: The server checks whether the response from the generative artificial intelligence engine is in the correct format (e.g., JSON format). If it is invalid, it generates an error message and returns it to the user.

[2033] Output: Order details data in the correct format, or an error message.

[2034] Step 6:

[2035] Server queries emotion recognition engine

[2036] Input: User's order text.

[2037] Specific operation: The server sends the user's input text to the emotion recognition engine, which then analyzes the user's emotions. For example, when a user enters "I'd like a hamburger and a soda, please," the server sends this text to the emotion recognition engine.

[2038] Output: Emotion analysis results from the emotion recognition engine (e.g., joy, anger, sadness, etc.).

[2039] Step 7:

[2040] Server-side adjustment of response content

[2041] Input: Sentiment analysis results from the emotion recognition engine, and correctly formatted order data.

[2042] Specific operation: The server adjusts the response based on the results of the sentiment analysis. For example, if the user is feeling "joyful," the response message will be "Thank you! I'll bring your hamburger and soda right away."

[2043] Output: Optimized response content.

[2044] Step 8:

[2045] Server sends response data to terminal

[2046] Input: Optimized response content.

[2047] Specific operation: The server sends the optimized response to the terminal.

[2048] Output: Response data sent to the terminal.

[2049] Step 9:

[2050] Displaying the response from the terminal

[2051] Input: Response data sent from the server.

[2052] Specific operation: The terminal displays the received response on its screen and informs the user of the result.

[2053] Output: The response message displayed on the terminal's screen.

[2054] (Application Example 2)

[2055] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2056] Currently, many stores rely on manual order entry by staff, posing challenges to improving order efficiency and customer experience. Furthermore, providing appropriate responses tailored to customer emotions is difficult, limiting the potential for increased customer satisfaction. Therefore, there is a need for a system that automatically handles natural language order taking, provides optimal product suggestions based on those orders, and offers responses that take customer emotions into consideration.

[2057] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for analyzing a natural language order from a user using a specific generation technology engine and outputting the order in a specific format, means for confirming the output and adding the order details to a list based on the output, means for calculating the total amount and notifying the user, means for identifying emotions from the user's input using an emotion analysis engine, and means for adjusting the response to the user based on the results of the emotion analysis. This not only automates the analysis and processing of natural language orders, but also enables responses that respond to the user's emotions, thereby improving customer satisfaction and operational efficiency.

[2058] A "generative technology engine" is an artificial intelligence engine that analyzes a user's order in natural language and outputs it in a specific format.

[2059] An "emotion analysis engine" is an artificial intelligence engine that identifies emotions from user input and outputs the analysis results.

[2060] "Specific format" refers to a format that includes the identifier, name, and price of the ordered items.

[2061] "Means for adding order details to a list" refers to a processing method or apparatus for adding analyzed order details to an existing order list.

[2062] "Means for calculating the total amount" refers to a processing method or apparatus for calculating the total amount by summing up the prices of the listed order items.

[2063] "Means for adjusting the content of responses to the user" refers to a processing method or apparatus for appropriately changing the content of responses to the user based on the results of the sentiment analysis engine.

[2064] This invention relates to a system that uses a generation technology engine and an emotion analysis engine to analyze a user's natural language order, recognize the user's emotional state, and provide an optimal response. This system mainly consists of three elements: a server, a terminal, and a user.

[2065] Server configuration and functionality

[2066] The server has the following roles:

[2067] 1. Setting API keys for the generative technology engine and sentiment analysis engine: The server sets the API keys for the generative technology engine and sentiment analysis engine to enable the use of each engine.

[2068] 2. Menu Item Management: The server manages a list of menu items that users can order, and this list includes the identifier, name, and price of each item.

[2069] 3. Parsing user input: The system receives order details entered by the user in natural language and generates prompt sentences for parsing using a generation technology engine.

[2070] Example: If a user enters "Please recommend a coffee and a cake," the server will generate a prompt like this: "User order: Please recommend a coffee and a cake. Menu: [{menu list}]. Identify the order and respond in JSON format."

[2071] 4. AI response processing: The server receives the response from the generation technology engine and verifies that it is in the correct format (JSON format). If it is invalid, it returns an error message to the user.

[2072] 5. Emotion Recognition: The server sends the user's input text to the emotion analysis engine, which then analyzes the user's emotions.

[2073] 6. Adjusting response content: Based on the results of sentiment analysis, adjust the content of the response to the user.

[2074] Device configuration and functions

[2075] The terminal is for use by store staff and performs the following tasks:

[2076] 1. Receiving user input: The terminal provides an interface for receiving order input from the user in natural language.

[2077] 2. Sending the order details: The terminal sends the received order details to the server.

[2078] 3. Display of sentiment analysis results: The terminal displays the analysis results received from the server and the sentiment analysis results to the user, and provides suggestions and messages appropriate to the situation.

[2079] User roles

[2080] The user performs the following actions:

[2081] 1. Order Input: The user enters their order details in natural language.

[2082] 2. Order Confirmation and Completion: The user reviews their order and places additional orders as needed. Once the order is complete, they type "Complete" to finish the ordering process.

[2083] Overall system flow

[2084] To implement this invention, the server, terminal, and user must each fulfill their respective roles and connect and communicate seamlessly. By using a generative technology engine and an emotion analysis engine in combination, a system is built that analyzes the user's natural language input and automatically provides responses that correspond to their emotions. This makes it possible to achieve real-time order processing and high customer satisfaction.

[2085] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2086] Step 1:

[2087] The terminal accepts orders from users via voice or text input using natural language. The input data consists of the user's order details, which serves as the starting point for processing the entire system.

[2088] Step 2:

[2089] The terminal sends the received order details to the server. The data is in natural language text format, and its content is necessary for analysis on the server side.

[2090] Step 3:

[2091] The server requests the AI ​​to analyze the order details. The input data is the user's order text and menu list, which are sent as prompts to the AI ​​model. "User Order: [Order Details] Menu: [{Menu List}]. Identify the order details and respond in JSON format."

[2092] Step 4:

[2093] The generative AI model receives a prompt, parses the order details, and outputs them in a specific format (JSON format). This output includes the identifier, name, and price of each ordered item.

[2094] Step 5:

[2095] The server receives the response from the generated AI model and verifies that the output is in the correct format. If it is invalid, it generates an error message and sends it to the terminal to notify the user; otherwise, it proceeds to the next step.

[2096] Step 6:

[2097] The server sends correctly formatted output to the sentiment analysis engine to identify the user's emotions. The input data is the text of the order details, and the sentiment recognition engine analyzes the text to output the emotional state (e.g., joy, anger, sadness, etc.).

[2098] Step 7:

[2099] The server receives the sentiment analysis results and adjusts its response to the user based on the analysis. For example, if the user is happy, it generates a response that will make them even happier. Specific example: "Certainly. I'm glad! Your order is a cafe latte and cheesecake."

[2100] Step 8:

[2101] The server sends the adjusted response to the terminal. This response reflects the analysis results from the generative AI model and the sentiment analysis engine.

[2102] Step 9:

[2103] The terminal displays or verbally communicates the response received from the server to the user. This allows the user to review the information and take additional actions (such as placing an additional order) as needed.

[2104] Step 10:

[2105] If the user places an additional order, the process returns to step 1 and repeats. Once the final order is confirmed, a termination command is sent from the terminal to the server.

[2106] Step 11:

[2107] The server calculates the total amount based on the list of final order items and sends the result to the terminal. This notifies the user, and the order process is completed.

[2108] This series of steps results in a system that analyzes the user's natural language order and provides the most appropriate response based on their emotions.

[2109] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2110] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2111] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2112] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2113] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2114] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2115] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2116] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2117] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2118] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2119] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2120] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2121] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2122] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2123] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2124] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2125] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2126] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2127] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2128] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2129] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[2130] The following is further disclosed regarding the embodiments described above.

[2131] (Claim 1)

[2132] A means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format,

[2133] Means for checking the output and adding the order details to the list based on the output,

[2134] A means of calculating the total amount and notifying the user,

[2135] A system that includes this.

[2136] (Claim 2)

[2137] The system according to claim 1, wherein the output in the specified format includes the ID, name, and price of the ordered items.

[2138] (Claim 3)

[2139] The system according to claim 1, comprising means for responding to multiple user inputs in natural language and dynamically updating the total amount.

[2140] "Example 1"

[2141] (Claim 1)

[2142] A means for analyzing orders from users in natural language using a generative artificial intelligence model and outputting those orders in a specific format,

[2143] A means for confirming the output and adding the order details to the database based on the output,

[2144] A means of receiving user input via a terminal and sending that input to a server,

[2145] A means by which the server sends a prompt message to the generated artificial intelligence model and receives a response from the AI,

[2146] A means for confirming the response and, if it is in the correct format, reflecting it in the order processing system,

[2147] A means for displaying the response or error message on the terminal,

[2148] A means of calculating the total amount and notifying the user,

[2149] A system that includes this.

[2150] (Claim 2)

[2151] The system according to claim 1, wherein the output in the specified format includes an identifier, name, and price of an ordered item.

[2152] (Claim 3)

[2153] The system according to claim 1, comprising means for responding to multiple user inputs in natural language and dynamically updating the total amount.

[2154] "Application Example 1"

[2155] (Claim 1)

[2156] A means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format,

[2157] Means for checking the output and adding the order details to the list based on the output,

[2158] A means of calculating the total amount and notifying the user,

[2159] A method for checking the availability of stock by comparing order details with the inventory database,

[2160] Based on the confirmed inventory status, a means to suggest confirming or reordering the order,

[2161] A system that includes this.

[2162] (Claim 2)

[2163] The system according to claim 1, wherein the output in the specified format includes an identifier, name, and price of an ordered item.

[2164] (Claim 3)

[2165] The system according to claim 1, comprising means for responding to multiple user inputs in natural language and dynamically updating the total amount.

[2166] "Example 2 of combining an emotion engine"

[2167] (Claim 1)

[2168] A means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format,

[2169] Means for checking the output and adding the order details to the list based on the output,

[2170] A means of calculating the total amount and notifying the user,

[2171] A means for generating prompts for a generative artificial intelligence engine and obtaining the engine's response,

[2172] A means of sending user input to an emotion recognition engine and analyzing the user's emotions,

[2173] A means of adjusting the response content based on the emotion analysis results,

[2174] A system that includes this.

[2175] (Claim 2)

[2176] The system according to claim 1, wherein the output in the specified format includes the ID, name, and price of the ordered items.

[2177] (Claim 3)

[2178] The system according to claim 1, comprising means for responding to multiple user inputs in natural language and dynamically updating the total amount.

[2179] "Application example 2 when combining with an emotional engine"

[2180] (Claim 1)

[2181] A means for analyzing a user's order in natural language using a specific generation technology engine and outputting the order in a specific format,

[2182] Means for checking the output and adding the order details to the list based on the output,

[2183] A means of calculating the total amount and notifying the user,

[2184] A means of identifying emotions from user input using an emotion analysis engine,

[2185] A means for adjusting the content of the response to the user based on the results of the emotion analysis,

[2186] A system that includes this.

[2187] (Claim 2)

[2188] The system according to claim 1, wherein the output in the specified format includes an identifier, name, and price of the ordered items.

[2189] (Claim 3)

[2190] The system according to claim 1, comprising means for responding to multiple user inputs in natural language and dynamically updating the total amount. [Explanation of Symbols]

[2191] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for analyzing a user's order in natural language using a specific artificial intelligence engine and outputting the order in a specific format, Means for checking the output and adding the order details to the list based on the output, A means of calculating the total amount and notifying the user, A system that includes this.

2. The system according to claim 1, wherein the output in the specified format includes the ID, name, and price of the ordered items.

3. The system according to claim 1, comprising means for responding to multiple user inputs in natural language and dynamically updating the total amount.

Images