System

Abstract

To provide a system.SOLUTION: A system includes first means for acquiring order contents regarding a user's meal, second means for recognizing the user's emotions from the user's facial expressions and tone of voice using an emotion recognition engine, third means for generating a first prompt sentence for outputting a message to be presented to the user based on the user's emotions and text data indicating the order contents, fourth means for inputting the generated first prompt sentence into a generative AI model to generate the message, and fifth means for presenting the generated message to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional customer service work is dependent on human labor, and labor shortages and reductions in working hours are issues. In addition, vending machines are also required to have more advanced customer service functions. [Means for solving the problem]

[0005] By developing an application using generative AI, it will supplement prompts according to the purpose and call the APIs of other services, linking with a cash register app to automate cash register operations that involve customer service at restaurants and other establishments. It will also realize a vending machine that serves customers. This will automate customer service operations that do not depend on human labor and improve the customer service functions of vending machines. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16]FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention is an application that uses generative AI. This application generates an appropriate prompt in response to an input from a user, and calls the API of another service based on the prompt. For example, when a user inputs "I would like to order pizza," the application generates a prompt "What kind of pizza would you like to order?" and calls an API that places a pizza order based on the user's response.

[0029] "Example 2"

[0030] Furthermore, as one embodiment of the present invention, there is a link with a cash register app. An application using generative AI links with a cash register app to automate cash register operations that involve customer service. For example, to automate ordering at a restaurant, the order details from the user are sent to the cash register app, and the cash register app rings up the cash based on the order details.

[0031] "Example 3"

[0032] As one embodiment of the present invention, there is a vending machine that serves customers. An application using generative AI works with the vending machine and provides products in response to requests from users. For example, when a user inputs "I want to buy coffee," the application sends an instruction to the vending machine, and the vending machine provides coffee.

[0033] The process flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Receive input from the user. For example, "I want to order pizza."

[0036] Step 2: Use generative AI to generate an appropriate prompt, for example, "What kind of pizza would you like to order?"

[0037] Step 3: Call the API of another service based on the user's answer. For example, if the user selects "Margherita," call the API to order a Margherita.

[0038] "Example 2"

[0039] Step 1: Receive an order from a user. For example, receive an "order for a hamburger" from a restaurant.

[0040] Step 2: Use generative AI to send the order details to the cash register app.

[0041] Step 3: The cashier app processes the order based on the details of the order, for example calculating the price of a burger and generating a receipt.

[0042] "Example 3"

[0043] Step 1: Receive a request from a user. For example, "I want to buy coffee."

[0044] Step 2: Use generative AI to send instructions to the vending machine, for example to dispense coffee.

[0045] Step 3: The vending machine dispenses the product as instructed, e.g., coffee.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0048] In conventional systems, when a user uses a specific service, the process of generating appropriate prompts and calling APIs of other services based on the user's input is often done manually, which is inefficient. In addition, it is difficult to generate appropriate prompts in response to user input, which leads to a poor user experience. Furthermore, there is a demand for automating cash register operations and vending machine customer service in restaurants and other establishments, but this is difficult to achieve with current systems.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative artificial intelligence, a means for receiving input from a user, a means for generating a prompt using a generative artificial intelligence model, a means for presenting the generated prompt to the user, a means for receiving a response from the user, a means for calling an application programming interface of another service, a means for presenting a response of the application programming interface to the user, and a means for linking with a cash register application. This makes it possible to automatically generate an appropriate prompt based on a user's input, to streamline the process of calling the API of another service, and to improve the user experience. In addition, it is possible to automate cash register operations that involve customer service in restaurants and the like, and customer service at vending machines.

[0050] "Generative artificial intelligence" is an artificial intelligence technique for generating appropriate prompts and responses based on user input.

[0051] An "application" is a software program designed to perform a specific task or function.

[0052] "Means for receiving input from a user" refers to a function that provides an interface for a user to enter input in the form of text, voice, or the like.

[0053] A "generative artificial intelligence model" is an algorithm or data model for generating prompts and responses based on user input.

[0054] A "prompt" is a question or instruction that prompts the user for their next action or input.

[0055] An "application programming interface" is an interface that allows different software systems to communicate with each other.

[0056] A "cash register application" is software for managing transactions related to the sale of goods or the provision of services.

[0057] "User experience" is a concept that refers to the overall satisfaction and ease of use when a user uses a particular system or service.

[0058] A "vending machine" is an automated machine that enables a user to purchase goods.

[0059] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0060] The present invention is a system for realizing an application using generative artificial intelligence. A specific embodiment of this system will be described below.

[0061] System Configuration

[0062] The system is composed of three main elements: a server, a terminal, and a user. The server generates prompts using a generative artificial intelligence model and calls the application programming interfaces (APIs) of other services based on user input. The terminal provides an interface for users to enter input and communicates with the server. Users access the system through their terminals and input the required information.

[0063] Hardware and Software Used

[0064] The server is a computer system equipped with a high-performance processor and sufficient memory. A generative artificial intelligence model (e.g., GPT-3 (registered trademark) by OpenAI (registered trademark)) is installed on the server. The terminals are devices such as smartphones, tablets, or personal computers, and access the server through a web browser or a dedicated application. Users use these terminals to input data into the system.

[0065] Data processing and calculation

[0066] The server receives input from the user, passes the input to a generative artificial intelligence model to generate an appropriate prompt, which is then presented to the user again and receives a response from the user. The server then calls the appropriate API based on the user's response and presents the response to the user.

[0067] As a concrete example, the following code shows what happens when a user enters "I would like to order pizza."

[0068] 1. A user accesses the application using a terminal and types, "I want to order a pizza."

[0069] 2. The server uses a generative artificial intelligence model (GPT-3) to generate a prompt: “What kind of pizza would you like to order?”

[0070] 3. The server presents this prompt to the user.

[0071] 4. The user answers "Margherita."

[0072] 5. The server calls the API for ordering pizza and sends the user's order details.

[0073] 6. The server receives the response from the API and displays a message to the user saying, "Your order has been completed. Your order will be delivered within 30 minutes."

[0074] Example prompt:

[0075] User input: "I want to order pizza."

[0076] Generated prompt: "What kind of pizza would you like to order?"

[0077] In this way, the server, terminals, and users can work together to efficiently fulfill user requests. This system improves the user experience and makes it possible to automate cash register operations and vending machine operations that involve customer service in restaurants and other establishments.

[0078] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0079] Step 1:

[0080] A user accesses the application using a terminal, types "I would like to order pizza" in the text box, and clicks the submit button.

[0081] Input: User text input "I want to order pizza"

[0082] Output: User input data is sent to the server

[0083] Step 2:

[0084] The server receives the HTTP POST request and extracts the user input text from the request body.

[0085] Input: User input data

[0086] Output: Extracted text data "I want to order pizza"

[0087] Step 3:

[0088] The server passes the extracted text data to a generative artificial intelligence model (e.g., GPT-3) to generate an appropriate prompt.

[0089] Input: Extracted text data "I want to order pizza"

[0090] Data processing: A generative artificial intelligence model analyzes the text data and generates appropriate prompts.

[0091] Output: Generated prompt "What kind of pizza would you like to order?"

[0092] Step 4:

[0093] The server sends the generated prompt to the user's terminal as an HTTP response, and displays it on the user's screen.

[0094] Input: Generated prompt "What kind of pizza would you like to order?"

[0095] Output: The prompt that is displayed on the user's terminal.

[0096] Step 5:

[0097] The user responds with "Margherita" to the displayed prompt and clicks the submit button again.

[0098] Input: User text input "Margherita"

[0099] Output: User answer data is sent to the server

[0100] Step 6:

[0101] The server again receives the HTTP POST request and extracts the user's answer from the request body.

[0102] Input: User response data

[0103] Output: Extracted text data "Margherita"

[0104] Step 7:

[0105] The server calls the appropriate API (e.g., a pizza ordering API) based on the user's answers and submits the user's order.

[0106] Input: Extracted text data "Margherita"

[0107] Data processing: generating and sending API requests

[0108] Output: An order request to the API.

[0109] Step 8:

[0110] The server receives the response from the API and sends the response to the user's device as an HTTP response. The user's screen displays the message, "Your order has been completed. Your order will be delivered within 30 minutes."

[0111] Input: Response data from the API

[0112] Output: A message displayed on the user's device: "Your order is complete. Your order will be delivered in 30 minutes."

[0113] In this way, the server, the terminal, and the user can work together to efficiently fulfill the user's requests.

[0114] (Application example 1)

[0115] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] Conventional food delivery systems require a lot of effort for users to place an order, and the operation is complicated, especially for first-time users. In addition, there is a lack of suggestions that take into account the user's preferences and past order history, making it difficult to improve the user experience. Furthermore, there is also an issue that it is not possible to generate appropriate prompts when placing an order, which means that it takes a long time for users to enter the necessary information.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0118] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to a purpose, a means for calling an API of another service, a means for generating an appropriate prompt based on a user's input, a means for making optimal suggestions based on the user's preferences and past order history, and a means for completing an order based on the suggestions. This allows the user to easily place an order, and makes suggestions that take into account the user's preferences and past order history, improving the user experience. In addition, by generating an appropriate prompt, the user can quickly input the necessary information.

[0119] "Generative AI" is an artificial intelligence technique that generates appropriate responses or prompts based on user input.

[0120] An "application role-playing vehicle" is software or a system designed to provide a specific function or service.

[0121] A "supplementary prompt" is a feature that provides additional information or questions in response to the user's input.

[0122] "Means of calling the API of other services" refers to a function that uses an API (Application Programming Interface) to connect with external services or systems.

[0123] "Means for generating appropriate prompts based on user input" refers to a function that generates questions or instructions to elicit the next required information based on the information entered by the user.

[0124] "A means of making optimal suggestions based on a user's preferences and past order history" is a function that analyzes a user's past behavior and preferences and suggests optimal products and services based on that.

[0125] The "means for completing an order based on a suggestion" is a function for actually confirming an order and completing the process based on a suggestion selected by a user.

[0126] The following system is conceivable as a mode for implementing this invention. The system includes means for fulfilling the role of an application using generative AI, means for supplementing prompts according to the purpose, means for calling the API of other services, means for generating appropriate prompts based on user input, means for making optimal suggestions based on the user's preferences and past order history, and means for completing the order based on the suggestions.

[0127] Hardware and Software Used

[0128] Hardware: Smartphone

[0129] Software: OpenAI API, food delivery service API, requests library

[0130] Data processing and calculation

[0131] 1. User input: The user inputs the food they want into the smartphone app.

[0132] 2. Prompt Generation: The server uses the OpenAI API to generate an appropriate prompt based on the user's input.

[0133] 3. User Answers: The user answers the generated prompt.

[0134] 4. Generate order details: The server generates the order details including the user's answers.

[0135] 5. API call: The server calls the food delivery service's API and completes the order.

[0136] Examples

[0137] When a user types "I'd like to order pizza," the server uses the OpenAI API to generate a prompt: "What kind of pizza would you like to order?" If the user responds with "Margherita," the server calls the food delivery service's API to complete the order.

[0138] Examples of prompt statements

[0139] The user types, "I want to order pizza." Generate an appropriate prompt.

[0140] In this way, we have realized a system that allows users to easily place orders by generating appropriate prompts based on user input and calling the API of the food delivery service. This improves the user experience by making suggestions that take into account the user's preferences and past order history. In addition, generating appropriate prompts allows users to quickly enter the necessary information.

[0141] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0142] Step 1:

[0143] The user inputs the food they want to eat into a smartphone app.

[0144] Input: The user types, "I would like to order pizza."

[0145] Output: The user's input data.

[0146] Specific behavior: The user enters "I would like to order pizza" into the app's text input field and presses the submit button.

[0147] Step 2:

[0148] The server uses the OpenAI API to generate appropriate prompts based on the user's input.

[0149] Input: User input data: "I want to order pizza."

[0150] Output: Generated prompt "What kind of pizza would you like to order?".

[0151] Specific behavior: The server sends the user's input data to the OpenAI API and receives the prompt returned by the API.

[0152] Step 3:

[0153] The user answers the generated prompts.

[0154] Input: Generated prompt "What kind of pizza would you like to order?"

[0155] Output: The user's answer, "Margherita".

[0156] Specific behavior: The user types "Margherita" into the app's text input field and presses the send button.

[0157] Step 4:

[0158] The server generates order details including the user's answers.

[0159] Input: The user's answer, "Margherita".

[0160] Output: Order details (e.g. { "Cuisine": "Pizza", "Type": "Margherita"}).

[0161] What happens: The server builds order details based on the user's answers.

[0162] Step 5:

[0163] The server calls the food delivery service's API and completes the order.

[0164] Input: Order details data.

[0165] Output: Order completed response from the food delivery service.

[0166] What happens: The server sends the order details to the food delivery service's API and receives a response indicating the order was successfully completed.

[0167] In this way, we have created a system that allows users to easily place orders by generating appropriate prompts based on user input and calling the API of a food delivery service.

[0168] Example 2

[0169] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] Conventional cash register systems required the user to input an order and have it reflected on the register manually, which was time-consuming and labor-intensive. There was also the possibility of incorrect input of order details and delays in processing, making efficient operation difficult. Furthermore, in tasks involving customer service, the burden on staff was heavy, and there was a risk of the quality of service declining.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0172] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling the API of other services, a means for receiving order details entered by a user, a means for analyzing the received order details using a generative AI model, a means for sending the analysis results to a cash register app, and a means for the cash register app to ring up the order based on the analysis results. This automates the process from ordering to ringing up the order, enabling efficient and accurate order processing.

[0173] "Generative AI" is a type of artificial intelligence that uses natural language processing and machine learning techniques to analyze user input and generate appropriate responses and data.

[0174] A "prompt" is an instruction or question entered into a generative AI to elicit a particular response or action.

[0175] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[0176] A "cash register app" is application software used to calculate fees associated with the sale of goods or the provision of services, process payments, issue receipts, and so on.

[0177] A "User" is a person who utilizes the system to enter an order.

[0178] "Order details" refers to detailed information about the product or service that the user wishes to purchase.

[0179] "Analysis" refers to the process in which the generative AI understands the user's order and extracts specific items and information.

[0180] "Structured data" is data that is organized according to a particular format or structure, such as JSON.

[0181] "Cash registering" refers to the task of calculating and processing payments associated with the sale of goods or the provision of services.

[0182] "Automation" refers to enabling systems or machines to operate automatically, without human intervention.

[0183] This invention is a system that automates the process from ordering to ringing up a cash register by linking a cash register app with an application that uses generative AI. A specific embodiment of this system is described below.

[0184] System Configuration

[0185] server

[0186] The server is responsible for analyzing the order contents from users using a generative AI model. Specifically, the server uses the following hardware and software:

[0187] Hardware: A server machine with a powerful processor, sufficient memory, and a fast network connection.

[0188] Software: Generative AI models (e.g., GPT-4 (registered trademark)), natural language processing libraries, libraries for calling APIs

[0189] The server receives the order details sent by the user and analyzes them using a generative AI model. The analysis results are generated as structured data in JSON format and sent to the cash register app.

[0190] Terminal

[0191] A terminal is a device that allows a user to input an order. Specifically, this includes a smartphone or tablet. An order application is installed on the terminal, and the terminal has the function of allowing a user to input order details and send them to a server.

[0192] User

[0193] A user is a person who uses the system to place an order. The user uses a terminal to open an order application and inputs the order details. For example, the user inputs "I'd like to order a hamburger and a coke."

[0194] Examples

[0195] Example 1: Ordering at a restaurant

[0196] A user enters "I'd like to order a hamburger and a coke" into an application at a restaurant. The application sends this order to a server. The server uses a generative AI model (e.g. GPT-4) to analyze the order and extracts the specific items "hamburger" and "coke." This information is sent to a cash register app, which automatically rings up the order, calculates the charge, and issues a receipt.

[0197] Examples of prompt statements

[0198] "When a user places an order at a restaurant, they use an application to input the order details. When the user types, 'I'd like a hamburger and a coke,' the application sends this information to a server. The server uses a generative AI model (e.g. GPT-4) to analyze the order and extract the specific items, 'hamburger' and 'coke.' This information is then sent to a cash register app, which automatically records the order, calculates the charge, and issues a receipt."

[0199] In this way, the server, terminal, and user all play their respective roles in the system, automating the process from ordering to ringing up the cash register. This system improves the efficiency and accuracy of order processing.

[0200] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0201] Step 1:

[0202] The user enters an order

[0203] The user opens an ordering application installed on a smartphone or tablet. He / she enters "I would like to order a hamburger and a cola" into the application's input form and presses the send button. The input data is in text format and includes the order details.

[0204] Step 2:

[0205] The terminal sends the order details to the server

[0206] The terminal sends the order details entered by the user to the server as text data. At this time, the data is sent using an HTTP POST request. The input is the text data entered by the user, and the output is the order details sent to the server.

[0207] Step 3:

[0208] The server receives the order.

[0209] The server receives the order details sent from the terminal. The received data is in text format and contains the order details. The input is the text data sent from the terminal, and the output is the received order details.

[0210] Step 4:

[0211] The server parses the order

[0212] The server inputs the received order details into a generative AI model (e.g., GPT-4). The generative AI model analyzes the order details and extracts the specific items "hamburger" and "cola." The input is the text data of the received order details, and the output is structured data (e.g., JSON format) as the analysis result.

[0213] Step 5:

[0214] The server sends the analysis results to the cash register app.

[0215] The server sends the JSON data obtained as a result of the analysis to the cash register app using an HTTP POST request. The input is the JSON data of the analysis result, and the output is the data sent to the cash register app.

[0216] Step 6:

[0217] The cash register app processes the order.

[0218] The cash register app analyzes the JSON data received from the server and calculates the charge based on the order details. For example, if a "hamburger" costs 500 yen and a "cola" costs 200 yen, the total amount is 700 yen. The cash register app displays this amount and issues a receipt. The input is the JSON data received from the server, and the output is the calculated charge and the issued receipt.

[0219] In this way, the specific actions and data flow are clearly defined at each step, automating the process from ordering to ringing up the cash register.

[0220] (Application example 2)

[0221] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0222] The traditional ordering and cash register process in restaurants is often done manually, which is time-consuming and labor-intensive. There is also a high risk of ordering mistakes and cash register errors. This can lead to lower customer satisfaction and poor business efficiency. Furthermore, there is concern that the burden on staff increases during peak times, leading to a decline in the quality of service.

[0223] The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register app, a means for analyzing the contents of a user's order, a means for sending the analysis results to the cash register app, and a means for prompting the user to make payment. This automates the ordering and cash register process, making it possible to improve business efficiency and customer satisfaction.

[0224] "Generative AI" is a type of artificial intelligence that generates new data and information based on user input.

[0225] An "application" is a software program designed to accomplish a particular purpose.

[0226] A "prompt" is an instruction or question that prompts the user for input.

[0227] "API" stands for Application Program Interface, an interface for exchanging data and functions between different software systems.

[0228] A "cash register app" is a software application used to operate the cash register and process payments.

[0229] A "user" is someone who uses a system or application.

[0230] "Order Details" refers to detailed information about the products or services a user wishes to purchase or use.

[0231] "Analysis" is the process of examining data or information in detail to understand its meaning and structure.

[0232] "Transmission" is the act of sending data or information from one place to another.

[0233] "Payment" is the act of giving money in exchange for goods or services.

[0234] As an embodiment of the present invention, we provide a system that automates the ordering and cash register process in a restaurant. The system includes an application that uses generative AI, a means for supplementing prompts, a means for calling APIs of other services, a means for linking with a cash register app, a means for analyzing the contents of a user's order, a means for sending the analysis results to the cash register app, and a means for prompting the user to make payment.

[0235] Hardware and Software Used

[0236] Hardware: Smartphone

[0237] Software: Python (registered trademark), generative AI API, cash register app API

[0238] System Operation

[0239] 1. Your order:

[0240] A user uses a smartphone app to select an item from a menu. For example, the user selects "hamburger."

[0241] 2. Order Analysis:

[0242] The smartphone app sends the selected order details to the generative AI API and obtains the analysis results. The generative AI understands the order details and converts them into an appropriate format.

[0243] 3. Send to the cash register app:

[0244] The analysis results are sent to the cash register app API, which then rings up the order based on the received order details.

[0245] 4. Payment prompts:

[0246] Based on the response from the cash register app, the smartphone app prompts the user to make the payment, and the user completes the payment using their smartphone.

[0247] Examples

[0248] For example, if a user selects "hamburger" on a smartphone app, the generative AI analyzes the order and sends an order for "one hamburger" to the cash register app. The cash register app receives the order, records it, and prompts the user to pay.

[0249] Examples of prompt statements

[0250] A user selects "Hamburger" on a smartphone app. The generative AI analyzes this order and sends it to a cash register app. The cash register app receives the order, records it, and prompts the user to pay.

[0251] In this way, automating the ordering and cashiering process can improve operational efficiency and customer satisfaction.

[0252] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0253] Step 1:

[0254] A user launches a smartphone app and selects a product from the menu. The input is the product selected by the user (e.g., "hamburger"), and the output is information about the selected product. In concrete terms, the selection is completed when the user taps the product on the app's menu screen.

[0255] Step 2:

[0256] The terminal sends the selected product information to the generative AI API. The input is the selected product information, and the output is the analysis result by the generative AI. Specifically, the terminal sends the product information in JSON format to the generative AI API as a POST request.

[0257] Step 3:

[0258] The server receives the analysis results from the generative AI API. The input is the response data from the generative AI API, and the output is the analyzed order details. Specifically, the server parses the API response and extracts the order details.

[0259] Step 4:

[0260] The server sends the parsed order details to the cash register app API. The input is the parsed order details, and the output is the response from the cash register app API. Specifically, the server sends the order details in JSON format to the cash register app API as a POST request.

[0261] Step 5:

[0262] The cash register application will perform cash register operations based on the order details. The input is the order details sent from the server, and the output is the cash register completion status. In concrete terms, the cash register application will receive the order details, record them in an internal database, and generate a receipt.

[0263] Step 6:

[0264] The cash register app returns the completion status of the cash register to the server. The input is the completion status of the cash register, and the output is the response data to the server. In concrete terms, the cash register app sends the completion status to the server in JSON format.

[0265] Step 7:

[0266] The server receives the completion status of the cash register and sends a message to the terminal to prompt payment. The input is the completion status of the cash register and the output is a message to prompt payment. In concrete terms, the server checks the completion status, generates a payment message and sends it to the terminal.

[0267] Step 8:

[0268] The terminal displays a message prompting the user to make a payment. The input is a payment message from the server, and the output is the display to the user. As a specific operation, the terminal displays the payment message to the user in a pop-up format.

[0269] Example 3

[0270] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0271] Conventional vending machines and cash register systems have difficulty responding flexibly to user requests, and the automation of specific product provision and services has been limited. In addition, there has been a lack of systems that can accurately understand user requests and provide appropriate products based on those requests. This has led to a demand for improved user experience and operational efficiency.

[0272] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0273] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for analyzing user requests, a means for sending instructions to the vending machine based on the analysis results, and a means for the vending machine to provide products. This makes it possible to flexibly and accurately provide products in response to user requests.

[0274] "Generative AI" is an artificial intelligence technology that analyzes user input and generates appropriate responses or instructions through natural language processing.

[0275] An "application" is a software program designed to accomplish a particular purpose.

[0276] A "prompt" is text data or instructions input to a generative AI.

[0277] An "API" is an interface for exchanging functions and data between different software systems.

[0278] A "user request" is a specific request or instruction given by a user to the system.

[0279] "Analysis" is the process of understanding and extracting meaning from input data.

[0280] An "instruction" is a command to perform a particular action or process.

[0281] A "vending machine" is a mechanical device that automatically dispenses merchandise selected by a user.

[0282] "Product" refers to any goods or services offered by a vending machine.

[0283] This invention is a system in which a vending machine provides products in response to a user's request by utilizing an application using generative AI. A specific embodiment of this system will be described below.

[0284] First, a user uses a terminal such as a smartphone or tablet to input a request to the vending machine. For example, the user inputs, "I would like to buy coffee." This request is then sent from the terminal to the server.

[0285] The server uses a generative AI model (e.g., OpenAI's GPT-4) to analyze the user's request. The generative AI model uses natural language processing techniques to understand the user's request and generate appropriate instructions. These instructions are converted into a format that is sent to the vending machine.

[0286] Specifically, the server inputs the following prompt sentence into the generative AI model:

[0287] The user types, "I want to buy coffee." Generate an instruction for the vending machine to provide the coffee.

[0288] The generative AI model would parse this prompt and generate instructions like this:

[0289] Sends a command to the vending machine to dispense coffee.

[0290] The server transmits the instruction to the vending machine. The vending machine operates based on the received instruction and operates the internal machinery to provide coffee. The user can take out the coffee from the receiving slot of the vending machine.

[0291] This system uses a generative AI model to flexibly and accurately respond to user requests. In addition, by having the server have the means to call the APIs of other services, it becomes possible to provide a wider variety of services and products.

[0292] In this way, a system is realized in which an application using generative AI operates a vending machine in response to a user request and provides a product. The flow of the specific process in the third embodiment will be described with reference to FIG.

[0293] Step 1:

[0294] The user enters a request

[0295] A user inputs a request to a vending machine using a terminal such as a smartphone or tablet. For example, the user inputs "I want to buy coffee." This input is saved as text data in the terminal. The input data is the text "I want to buy coffee."

[0296] Step 2:

[0297] The device sends a request to the server

[0298] The terminal transmits the request input by the user to the server via the Internet. At this time, the request is transmitted as text data. The input data is the text "I want to buy coffee," and the output data is the text data transmitted to the server.

[0299] Step 3:

[0300] The server parses the request

[0301] The server inputs the received request into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model processes the request in natural language and generates appropriate instructions. The input data is the text "I want to buy coffee," and the output data is the instruction "Send an instruction to the vending machine to dispense coffee."

[0302] Step 4:

[0303] The server sends instructions to the vending machine

[0304] The server sends the generated instruction to the vending machine. This instruction is converted into a format that the vending machine can understand. For example, it is sent in a specific command format (such as JSON format). The input data is the instruction "Send an instruction to the vending machine to serve coffee", and the output data is the command sent to the vending machine.

[0305] Step 5:

[0306] Vending machines provide products

[0307] The vending machine operates based on instructions received from the server, specifically by operating an internal machine to dispense coffee. A user can dispense the coffee from a receiving slot of the vending machine. The input data are the commands sent to the vending machine, and the output data is the dispensed coffee.

[0308] (Application example 3)

[0309] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] Conventional vending machines and cash register systems required users to operate them manually when purchasing products, which resulted in low convenience. In addition, in stores that involved customer service, the burden on staff was heavy and efficient operation was difficult. Furthermore, there was a lack of systems that could flexibly respond to user requests, and there was a need to improve customer satisfaction.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for performing the role of an application using a generative AI, a means for supplementing a prompt according to a purpose, a means for calling an API of another service, a means for linking with a cash register application, a means for sending instructions to a vending machine that provides products in response to a user's request, a means for generating instructions using a generative AI model, and a means for providing a user interface using a smartphone. This not only allows users to easily purchase products using a smartphone, but also reduces the burden on store staff and enables efficient store operation. In addition, the use of a generative AI model makes it possible to flexibly respond to user requests and improve customer satisfaction.

[0312] "Generative AI" is an artificial intelligence technology that generates natural language and instructions based on user input.

[0313] A "means for fulfilling the role of an application" is software or a program for providing a specific function or service.

[0314] A "means for supplementing prompts" is a function that complements the user's input and generates appropriate instructions or questions.

[0315] "Means of calling the API of other services" refers to a function that uses an API (Application Programming Interface) to connect with external services or systems.

[0316] "Means of linking with a cash register app" refers to a function that links with the store's cash register system to handle payments and inventory management.

[0317] The "means for sending instructions to a vending machine that provides a product in response to a user's request" is a function for sending instructions to the vending machine to provide a product based on the user's purchase request.

[0318] "Means for generating instructions using a generative AI model" refers to a function that uses a generative AI model to generate specific instructions in response to a user's request.

[0319] "Means for providing a user interface using a smartphone" refers to a function for providing an interface that can be operated by a user using a smartphone.

[0320] A system for implementing this invention includes means for performing the role of an application using generative AI, means for supplementing prompts according to the purpose, means for calling APIs of other services, means for linking with a cash register app, means for sending instructions to a vending machine which provides products in response to a user request, means for generating instructions using a generative AI model, and means for providing a user interface using a smartphone.

[0321] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate natural language and instructions based on user input. When a user inputs on their smartphone that they would like to purchase an item, the server receives the input and sends a prompt to the generative AI model. An example of a prompt sentence is, "The user has input that they would like to purchase a coffee. Please send instructions to the vending machine."

[0322] The generative AI model generates appropriate instructions based on the prompts and returns the instructions to the server. The server receives the generated instructions and calls the APIs of other services to send the instructions to the vending machine. The vending machine follows the instructions and provides the product to the user.

[0323] As a concrete example, a user opens a smartphone app and types, "I want to buy coffee." The server receives this input and sends the following prompt to the generative AI model:

[0324] "The user has indicated that they would like to purchase coffee. Please send instructions to the vending machine."

[0325] Based on this prompt, the generative AI model generates the instruction, "Send an instruction to the vending machine to serve coffee." The server receives this instruction and calls the vending machine's API to send the instruction. The vending machine follows this instruction and serves coffee to the user.

[0326] This system not only allows users to easily purchase products using their smartphones, but also reduces the burden on store staff and enables efficient store operations. In addition, the use of generative AI models allows for flexible response to user requests, improving customer satisfaction.

[0327] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0328] Step 1:

[0329] A user starts the application on a smartphone terminal and inputs the product he or she wants to purchase. The input product name is "coffee."

[0330] Step 2:

[0331] The terminal transmits the user's input to the server. The input data is text data for "coffee."

[0332] Step 3:

[0333] The server sends a prompt to the generative AI model based on the received user input data. The prompt text is "The user has input that he / she would like to buy coffee. Please send instructions to the vending machine."

[0334] Step 4:

[0335] The generative AI model analyzes the prompt and generates the appropriate instruction, which is "Send an instruction to the vending machine to serve coffee."

[0336] Step 5:

[0337] The server analyzes the instructions received from the generative AI model and generates data to call the vending machine's API. The input data are the generated instructions, and the output data are the data for the API call.

[0338] Step 6:

[0339] The server calls the vending machine's API and sends the generated data. As a result of the API call, the vending machine dispenses the coffee.

[0340] Step 7:

[0341] The vending machine provides coffee to the user based on instructions received from the server. The product provided is "coffee."

[0342] Step 8:

[0343] The server notifies the user that the product has been served. The content of the notification is "Coffee has been served."

[0344] Step 9:

[0345] The terminal receives the notification from the server and displays it to the user, saying "Coffee has been served."

[0346] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0347] "Example 1"

[0348] One embodiment of the present invention is a system that includes an emotion engine that recognizes a user's emotions. The system receives input from a user and generates an appropriate prompt using a generative AI. It also calls APIs of other services. The emotion engine then recognizes the user's emotions and supplements the prompt according to the emotions. For example, if a user inputs "I would like to order pizza" and the emotion engine recognizes the user's emotion as "joy," the generative AI generates a prompt such as "Congratulations! What kind of pizza would you like to order?"

[0349] "Example 2"

[0350] Another embodiment of the present invention is a system that automates cash register operations involving customer service based on the user's emotions recognized by an emotion engine. This system receives the order details from the user and sends the order details to a cash register app using generative AI. The emotion engine then recognizes the user's emotions and automates cash register operations according to those emotions. For example, if a user "orders a hamburger" and the emotion engine recognizes the user's emotion as "anger," the cash register app displays a message such as "Sorry. We will quickly prepare the hamburger."

[0351] "Example 3"

[0352] As yet another embodiment of the present invention, there is a system for realizing a vending machine that serves customers based on the user's emotions recognized by an emotion engine. This system receives requests from a user and sends instructions to the vending machine using a generative AI. The emotion engine then recognizes the user's emotions, and the vending machine provides products according to those emotions. For example, if a user requests "I want to buy coffee" and the emotion engine recognizes the user's emotion as "sadness," the vending machine will display a message such as "Are you okay? Enjoy your coffee."

[0353] The process flow of each embodiment will be described below.

[0354] "Example 1"

[0355] Step 1: Receive input from the user.

[0356] Step 2: Use generative AI to generate appropriate prompts.

[0357] Step 3: Call the API of other services.

[0358] Step 4: Recognize the user's emotions using the emotion engine.

[0359] Step 5: Complement the prompt with the emotion you identify.

[0360] "Example 2"

[0361] Step 1: Receive the order from the user.

[0362] Step 2: Use generative AI to send the order details to the cash register app.

[0363] Step 3: Recognize the user's emotions using the emotion engine.

[0364] Step 4: Automate cashiering based on the recognized emotions.

[0365] "Example 3"

[0366] Step 1: Receive a request from a user.

[0367] Step 2: Use generative AI to send instructions to the vending machine.

[0368] Step 3: Recognize the user's emotions using the emotion engine.

[0369] Step 4: The vending machine will dispense a product based on the emotion it recognizes.

[0370] Example 1

[0371] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0372] Conventional systems have problems in that it is difficult to generate appropriate prompts in response to user input, and they are unable to respond in a way that takes into account the user's emotions. Furthermore, when calling APIs for other services, flexible responses based on the user's emotions are required, but there is a lack of means to achieve this.

[0373] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0374] In this invention, the server includes a means for receiving input from a user, a means for generating an appropriate prompt sentence using a generative AI model, a means for recognizing the user's emotion, a means for supplementing the prompt sentence based on the recognized emotion, and a means for calling an API of another service. This makes it possible to generate an appropriate prompt for the user's input and to flexibly respond according to the emotion.

[0375] The "means for receiving input from a user" is a function that provides an interface for a user to input information such as text or voice to the system through a terminal.

[0376] "Means for generating appropriate prompt sentences using a generative AI model" refers to a function that automatically creates prompt sentences containing appropriate questions or instructions using a generative AI model based on user input.

[0377] "Means for recognizing user emotions" refers to a function that uses an algorithm or engine to analyze emotions from the user's input or voice and identify those emotions.

[0378] The "means for supplementing the prompt sentence based on the recognized emotion" is a function that takes the user's emotion into account and adds additional information or an expression according to the emotion to the generated prompt sentence.

[0379] The "means for calling the API of other services" refers to a function for calling the API of external services or systems based on the generated prompt text or the user's response, and performing the required operations or data acquisition.

[0380] "Means for linking with a cash register app" refers to a function for communicating with a cash register application and sending and receiving data and linking operations.

[0381] This invention is configured mainly with an application using a generative AI model. A specific embodiment of this system will be described below.

[0382] System Configuration

[0383] The system mainly uses the following hardware and software:

[0384] Hardware: Devices used by users (smartphones, tablets, PCs, etc.), servers

[0385] Software: Generative AI models (e.g. GPT-3), emotion recognition engines (e.g. emotion analysis API), API call functions, cash register applications

[0386] Program Processing

[0387] 1. Receiving user input

[0388] A user inputs text or voice to the system through a terminal. For example, the user inputs "I would like to order pizza."

[0389] 2. Prompt Generation

[0390] The server uses a generative AI model (e.g., GPT-3) to generate an appropriate prompt for the user's input, such as "What kind of pizza would you like to order?"

[0391] 3. Emotion recognition

[0392] The server uses an emotion recognition engine (e.g., emotion analysis API) to recognize emotions from the user's input. For example, it recognizes the emotion "joy" from the user's input "I want to order pizza."

[0393] 4. Prompt Supplement

[0394] The server supplements the generated prompt sentence based on the recognized emotion, i.e., "Congratulations! What kind of pizza would you like to order?"

[0395] 5. API Calls

[0396] When the user answers the prompts, the server calls APIs of other services based on the answer. For example, if the user answers "Margherita," the server calls a pizza ordering API to order a Margherita pizza.

[0397] Examples

[0398] Receiving user input

[0399] The user enters "I would like to order pizza" into the terminal and presses the send button.

[0400] Prompt Generation

[0401] The server uses a generative AI model (e.g. GPT-3) to generate a prompt such as "What kind of pizza would you like to order?"

[0402] emotion recognition

[0403] The server calls an emotion recognition engine (e.g., emotion analysis API) and recognizes the emotion “joy” from the user’s input “I want to order pizza.”

[0404] Prompt Supplement

[0405] The server again uses the generative AI model to generate a follow-up prompt: "Congratulations! What kind of pizza would you like to order?"

[0406] API Call

[0407] The user responds with "Margherita," and the server calls the pizza ordering API to order a Margherita pizza.

[0408] In this way, a system is realized that generates appropriate prompts in response to user input and responds flexibly according to emotions.

[0409] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0410] Step 1:

[0411] Receiving user input

[0412] Specific action: The user enters "I would like to order pizza" into the input field of the terminal and presses the send button.

[0413] Input: User text input "I want to order pizza".

[0414] Output: The user's input data is sent to the server.

[0415] Step 2:

[0416] Prompt Generation

[0417] What it does: The server uses a generative AI model (e.g. GPT-3) to generate an appropriate prompt in response to the user's input.

[0418] Input: User input data: "I want to order pizza."

[0419] Data processing: The generative AI model analyzes the user's input and generates an appropriate prompt.

[0420] Output: The generated prompt sentence: "What kind of pizza would you like to order?".

[0421] Step 3:

[0422] emotion recognition

[0423] Specific operation: The server uses an emotion recognition engine (e.g., emotion analysis API) to recognize emotions from the user's input.

[0424] Input: User input data: "I want to order pizza."

[0425] Data processing: An emotion recognition engine analyzes user input and identifies emotions.

[0426] Output: The perceived emotion "joy".

[0427] Step 4:

[0428] Prompt Supplement

[0429] Specific behavior: The server supplements the generated prompt sentence based on the emotion recognized by the server.

[0430] Input: The generated prompt sentence "What kind of pizza would you like to order?" and the recognized emotion "Joy".

[0431] Data processing: The generative AI model adds supplementary sentences based on the sentiment.

[0432] Output: The augmented prompt, "Congratulations! What kind of pizza would you like to order?".

[0433] Step 5:

[0434] API Call

[0435] Specific behavior: The user answers prompts, and the server calls the API of other services based on the answers.

[0436] Input: The user's answer, "Margherita".

[0437] Data processing: The server analyzes the user's answers and generates a request to call the pizza ordering API.

[0438] Output: A request is sent to the pizza ordering API, ordering a Margherita pizza.

[0439] (Application example 1)

[0440] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0441] In conventional food delivery systems, the ordering process proceeds without considering the user's emotions, resulting in a uniform user experience and making it difficult to provide services that meet the needs and emotions of individual users. In addition, the ordering process can be cumbersome because appropriate prompts are not generated when the user places an order.

[0442] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to a purpose, a means for calling APIs of other services, a means including an emotion engine for recognizing the user's emotions, a means for generating prompts according to the user's emotions, and a means for linking with a food delivery service. This makes it possible to generate appropriate prompts according to the user's emotions and to smoothly progress the food delivery ordering process.

[0443] "Generative AI" is an artificial intelligence technology that generates appropriate prompts based on user input and calls the APIs of other services.

[0444] A "prompt" is an instruction that presents the user with the next action or choice to make.

[0445] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[0446] An "emotion engine" is a technology that recognizes emotions from user input and actions and generates appropriate responses based on those emotions.

[0447] A "food delivery service" is a service that allows users to order food online and have it delivered to a specified location.

[0448] "User emotion" refers to the psychological state or feelings that a user exhibits through their input and actions.

[0449] "Supplementing a prompt" means generating supplemental instructions to provide additional information or options to the user in response to their input.

[0450] "Integration" means that different systems or services work together to share information and functions.

[0451] The system for implementing this invention is configured by linking a generative AI, an emotion engine, and an API for a food delivery service. A specific embodiment of the system is shown below.

[0452] System Configuration

[0453] The system is composed of a terminal that receives user input, a server that runs the generative AI, a server that runs the emotion engine, and a server that calls the API of the food delivery service. The terminal is a device such as a smartphone or tablet, and provides an interface for users to place orders.

[0454] Program Processing

[0455] The server performs the process in the following procedure.

[0456] 1. Receive user input:

[0457] A user inputs "I would like to order pizza" through a terminal. This input is sent to the server.

[0458] 2. Emotion Recognition:

[0459] The server uses an emotion engine to recognize emotions from the user's input. For example, the user's input is recognized as indicating the emotion "happiness."

[0460] 3. Generate prompt:

[0461] The server uses generative AI to generate prompts based on the user's emotions, for example, "Congratulations! What kind of pizza would you like to order?"

[0462] 4. Receiving user responses:

[0463] The user answers "Margherita" via the terminal. This answer is sent back to the server.

[0464] 5. Call the food delivery API:

[0465] The server calls the API of the food delivery service to process the user's order, and the order data is sent to the API to complete the order.

[0466] Hardware and Software Used

[0467] Devices: Smartphones, tablets

[0468] Servers: A server that runs the generative AI, a server that runs the emotion engine, and a server that calls the API of a food delivery service.

[0469] Software: Generative AI models, emotion engines, food delivery service APIs

[0470] Examples

[0471] If a user types "I'd like to order sushi" and the emotion engine recognizes this as "surprise," the generative AI will generate a prompt saying, "Great choice! What kind of sushi would you like to order?"

[0472] Examples of prompt statements

[0473] "I want to order pizza."

[0474] "Congratulations! What kind of pizza would you like to order?"

[0475] "I want to order sushi."

[0476] "Great choice! What type of sushi would you like to order?"

[0477] In this way, appropriate prompts can be generated based on the user's emotions, ensuring a smooth food delivery ordering process.

[0478] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0479] Step 1:

[0480] A user inputs "I want to order pizza" through a terminal. This input is sent from the terminal to the server. The input data is in text format and represents the user's order request.

[0481] Step 2:

[0482] The server uses an emotion engine to recognize emotions from user input. The input data is the user's text input, which the emotion engine analyzes and outputs emotions such as "joy." The emotion engine uses natural language processing technology to classify the emotion of the text.

[0483] Step 3:

[0484] The server uses a generative AI to generate a prompt based on the user's emotion. The input data is the user's emotion and text input, and the generative AI outputs a prompt such as "Congratulations! What kind of pizza would you like to order?" The generative AI uses a pre-trained model to generate the appropriate prompt.

[0485] Step 4:

[0486] The user answers "Margherita" through the terminal. This answer is sent from the terminal to the server. The input data is a text that represents the type of pizza selected by the user.

[0487] Step 5:

[0488] The server calls the API of the food delivery service to process the user's order. The input data is the type and quantity of pizza selected by the user, and the server sends this to the API of the food delivery service. The API receives the order data and processes the order.

[0489] Step 6:

[0490] The server receives the order confirmation from the food delivery service and notifies the user that the order has been completed. The input data is the order confirmation from the food delivery service, and the server converts it into a message to notify the user and sends it to the terminal. The user receives the notification of the order completion on the terminal.

[0491] Example 2

[0492] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0493] Conventional cash register systems process orders without considering the user's emotions, which can result in poor customer service quality. In addition, it is difficult to respond flexibly to the user's emotions, making it a challenge to improve user satisfaction, especially in situations involving customer service such as restaurants.

[0494] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0495] In this invention, the server includes a means for performing the role of an application using generative artificial intelligence, a means for supplementing prompts according to a purpose, a means for calling an application programming interface of another service, a means for linking with a cash register application, a means for recognizing a user's emotion, a means for generating a message based on the recognized emotion, and a means for sending the generated message to the cash register application. This enables flexible responses according to the user's emotion, improving the quality of customer service.

[0496] "Generative artificial intelligence" is an artificial intelligence technology that generates text and data using natural language processing and machine learning.

[0497] A "prompt" is an instruction or question input to a generative artificial intelligence to obtain a specific output.

[0498] An "application programming interface" is an interface for exchanging data and functions between different software systems.

[0499] A "cash register application" is software for managing point-of-sale information and supports product sales and inventory management.

[0500] "Means for recognizing a user's emotions" refers to technology that analyzes a user's facial expressions and tone of voice to identify their emotional state.

[0501] "Means for generating a message" refers to technology that uses generative artificial intelligence to create an appropriate text message based on specific input.

[0502] A "message sending means" is a technique for forwarding a generated message to another system or application.

[0503] This invention is a system that realizes flexible responses according to the emotions of a user by linking an application using generative artificial intelligence with a cash register application. Specific embodiments of this system are described below.

[0504] Hardware and Software Used

[0505] server

[0506] The server provides the computational resources to run the generative artificial intelligence model. Specifically, it uses the following software and hardware:

[0507] Generative artificial intelligence models: such as OpenAI's GPT-4

[0508] Application Programming Interface: Use RESTful APIs to integrate with other services

[0509] Database: A database for storing user order history and emotion data (e.g. MySQL (registered trademark))

[0510] Terminal

[0511] The terminal is a device for capturing the user's voice and facial expressions and recognizing emotions. Specifically, the following hardware and software are used:

[0512] Microphone: Captures the user's voice

[0513] Camera: Captures the user's facial expressions

[0514] Voice recognition software: Google® Speech-to-Text, etc.

[0515] Emotion recognition engine: Microsoft(R) Azure(R) Emotion API, etc.

[0516] Cash register application

[0517] The cash register application is software that processes user orders and displays appropriate messages. Specifically, it uses the following software:

[0518] POS system: Square POS, etc.

[0519] System Operation

[0520] User Orders

[0521] A user goes to a restaurant and says, "I'd like to order a hamburger." The device uses a microphone to capture the user's voice and converts the voice into text data using Google Speech-to-Text. This text data contains the content, "I'd like to order a hamburger."

[0522] Emotion recognition

[0523] The device uses a camera and microphone to capture the user's facial expressions and tone of voice. Using Microsoft Azure's Emotion API, the captured data is analyzed and the user's emotion is recognized as "anger." For example, if the user's voice is high-pitched and their facial expression is grim, the emotion engine will determine that the emotion is "anger."

[0524] Sending your order

[0525] The terminal sends a prompt sentence to the generative AI model. An example of a prompt sentence is "The user ordered a hamburger. The emotion engine recognized the user's emotion as anger. Please generate a message to be displayed in the cash register app." Based on this prompt sentence, the generative AI model generates the message "I'm sorry. I'll prepare the hamburger quickly."

[0526] Receiving and sending generated messages

[0527] The server receives messages generated from the generative artificial intelligence model. The server sends the received messages to the cash register application. At this time, the server converts the message format into a format that the cash register application can understand.

[0528] Displaying messages

[0529] The cash register application displays the message received from the server. Specifically, the cash register application displays the message "Sorry, we'll hurry up and prepare your burger" on its screen. This message reassures the user that their order will be fulfilled promptly.

[0530] This system enables flexible responses according to the user's emotions, improving the quality of customer service.

[0531] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0532] Step 1:

[0533] A user is at a restaurant and says, "I want to order a hamburger."

[0534] Input: User's voice

[0535] Specific behavior: The device uses the microphone to capture the user's voice.

[0536] Output: Captured audio data

[0537] Step 2:

[0538] The terminal converts the captured voice data into text data.

[0539] Input: Audio data

[0540] Specific operation: The device uses voice recognition software (e.g. Google Speech-to-Text) to convert voice data into text data.

[0541] Output: Text data "I want to order a hamburger"

[0542] Step 3:

[0543] The device captures the user's facial expressions and tone of voice to recognise emotions.

[0544] Input: User's facial expression data, voice tone data

[0545] How it works: The device uses the camera and microphone to capture the user's facial expressions and tone of voice, and then analyzes them using an emotion recognition engine (e.g. Microsoft Azure's Emotion API).

[0546] Output: User emotion (e.g. "Anger")

[0547] Step 4:

[0548] The terminal sends a prompt sentence to the generative artificial intelligence model.

[0549] Input: Text data, user sentiment

[0550] Specific operation: The terminal sends the following prompt to the generative artificial intelligence model: "The user ordered a hamburger. The emotion engine recognized the user's emotion as anger. Please generate a message to be displayed in the cash register app."

[0551] Output: The prompt sent to the generative AI model

[0552] Step 5:

[0553] The generative artificial intelligence model generates a message based on the prompt sentence.

[0554] Input: Prompt statement

[0555] How it works: A generative artificial intelligence model (e.g., OpenAI's GPT-4) analyzes the prompt and generates an appropriate message.

[0556] Output: "Sorry, I'll hurry up and prepare the burger."

[0557] Step 6:

[0558] The server receives the generated message and sends it to the cash register application.

[0559] Input: The generated message

[0560] Specific operation: The server receives messages generated from the generative artificial intelligence model, converts them into a format that the cash register application can understand, and sends them.

[0561] Output: Message sent to the cash register application

[0562] Step 7:

[0563] The cash register application displays the received message.

[0564] Input: The message sent by the server

[0565] Specific operation: The cash register application displays the received message on the screen.

[0566] Output: The message "Sorry, I'll hurry up and prepare the burger" is displayed on the screen.

[0567] In this way, it becomes possible to respond flexibly to the user's emotions, thereby improving the quality of customer service.

[0568] (Application example 2)

[0569] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0570] Conventional cash register systems were unable to respond to the user's emotions, resulting in poor customer service quality. In addition, because order details were processed manually, there were issues with inefficiency and the likelihood of mistakes occurring. This resulted in problems such as lower customer satisfaction and a worsening operational efficiency of the store.

[0571] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0572] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register application, a means for recognizing the user's emotions using an emotion engine, and a means for generating messages according to the user's emotions. This allows for appropriate responses according to the user's emotions, improving the quality of customer service. Furthermore, automating the processing of order details allows for efficient operation and reduces the occurrence of errors.

[0573] "Generative AI" is an artificial intelligence technology that generates natural language sentences, images, etc. based on user input.

[0574] A "prompt" is an input sentence or instruction given to a generative AI to obtain a specific output.

[0575] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[0576] A "cash register app" is application software used to process product sales and orders in a store.

[0577] An "emotion engine" is a technology for recognizing emotions from a user's voice, facial expressions, etc.

[0578] "User emotion" refers to the psychological state or feelings exhibited by a user.

[0579] "Means for generating a message" refers to technology for generating an appropriate response sentence depending on the user's emotions and situation.

[0580] The system for implementing this invention is composed of a server including a generative AI, an emotion engine, a cash register application, and an API calling function, and a terminal operated by a user. The specific configuration and operation of the system are described below.

[0581] The server plays the role of an application that uses generative AI. Generative AI is an artificial intelligence technology that generates natural language text and images based on user input. When a user enters an order through a terminal, the contents are sent to the server. The server analyzes the order contents and sends appropriate prompts to the generative AI.

[0582] The server recognizes the user's emotions using an emotion engine. An emotion engine is a technology for recognizing emotions from the user's voice, facial expressions, etc. When the user enters an order, the voice and image data are sent to the emotion engine, which analyzes the user's emotions.

[0583] The server processes the order details in cooperation with the cash register app. The cash register app is application software for selling products and processing orders in stores. The server sends the order details to the cash register app's API and automates the order processing.

[0584] The server generates a message according to the user's emotions. Generative AI is a technology for generating appropriate responses according to the user's emotions and circumstances. For example, if a user "orders a hamburger" and the emotion engine recognizes the user's emotion as "anger," the generative AI will generate a message saying, "Sorry. We'll quickly prepare the hamburger."

[0585] As a concrete example, if a user orders a hamburger using a smartphone app and the emotion engine recognizes the user's emotion as "anger," the server will use generative AI to generate a message saying, "We're sorry. We'll prepare the hamburger quickly," and display it on the user's device.

[0586] Examples of prompts for a generative AI model include:

[0587] "Generate a message for when the user is angry."

[0588] In this way, the system can respond appropriately according to the user's emotions and improve the quality of customer service. In addition, automating the processing of order details enables efficient operation and reduces the occurrence of mistakes.

[0589] The flow of the identification process in the application example 2 will be described with reference to FIG.

[0590] Step 1:

[0591] A user enters an order through a terminal.

[0592] Input: A user uses a smartphone app to input the order details (e.g., hamburger).

[0593] Data processing: The terminal converts the order details into a data format such as JSON.

[0594] Output: The transformed order data is sent to the server.

[0595] Step 2:

[0596] The server receives and analyzes the order.

[0597] Input: The order data sent from the terminal.

[0598] Data processing: The server analyzes the order data and extracts the order details.

[0599] Output: The parsed order details are sent to the generative AI.

[0600] Step 3:

[0601] The server recognizes the user's emotions using an emotion engine.

[0602] Input: User's voice and image data.

[0603] Data processing: The emotion engine analyzes voice and image data to recognize the user's emotions.

[0604] Output: The recognized emotion data is sent to the server.

[0605] Step 4:

[0606] The server generates the prompt using generative AI.

[0607] Input: Parsed order details and recognized emotion data.

[0608] Data processing: The generative AI generates a prompt sentence and then generates an appropriate response sentence.

[0609] Output: The generated response is sent to the server.

[0610] Step 5:

[0611] The server calls the API of the cash register app and processes the order details.

[0612] Input: The parsed order.

[0613] Data processing: The server calls the API of the cash register app and sends the order details.

[0614] Output: The order is processed by the cash register app.

[0615] Step 6:

[0616] The server sends the generated response to the user's terminal.

[0617] Input: The generated response sentence.

[0618] Data processing: The server sends the response to the user's terminal.

[0619] Output: The response is displayed on the user's terminal.

[0620] Step 7:

[0621] The user checks the response on the terminal.

[0622] Input: The response sent by the server.

[0623] Data processing: The terminal displays the response text.

[0624] Output: The user confirms the response.

[0625] Example 3

[0626] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0627] Conventional vending machines and customer service systems can provide products according to user requests, but they have the problem of being unable to respond to the user's emotions. This limits the user experience and makes it difficult to provide personalized services. In addition, because the system does not take the user's emotions into account, it is difficult to improve user satisfaction.

[0628] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a means for performing the role of an application using a generative AI, a means for supplementing a prompt according to a purpose, a means for calling an API of another service, a means for receiving and analyzing a user request, a means for generating instructions using a generative AI model, a means for sending instructions to a vending machine, a means for recognizing a user's emotion using an emotion engine, a means for generating a message based on the recognized emotion, a means for sending a message to the vending machine, and a means for the vending machine to provide a product and display a message. This makes it possible not only to provide a product according to a user's request, but also to provide a personalized response according to the user's emotion.

[0629] "Generative AI" is an artificial intelligence technology that generates natural language text or instructions based on user input.

[0630] A "means for fulfilling the role of an application" is software or a program for providing a specific function or service.

[0631] A "means for supplementing prompts" is a function that adds appropriate supplementary information or instructions to the user's input.

[0632] "Means of calling the API of other services" refers to a function that uses the API provided by that service in order to link with external services or systems.

[0633] The "means for receiving and analyzing a user's request" is a function for receiving input from a user and analyzing and understanding the content of that input.

[0634] "Means for generating instructions using a generative AI model" refers to a function that uses generative AI to create specific instructions based on user requests.

[0635] The "means for transmitting instructions to the vending machine" refers to a communication function for transmitting the generated instructions to the vending machine.

[0636] "Means for recognizing a user's emotions using an emotion engine" refers to a technology that analyzes a user's facial expressions, voice, etc. to identify their emotions.

[0637] The "means for generating a message based on the recognized emotion" is a function for creating an appropriate message according to the user's emotion.

[0638] The "means for transmitting a message to a vending machine" is a communication function for transmitting the generated message to the vending machine.

[0639] The "means for the vending machine to provide a product and display a message" refers to the function of the vending machine to provide a product to a user and display a message on the display.

[0640] This invention is a system that links a vending machine with an application that uses generative AI, provides products according to user requests, and responds according to emotions using an emotion engine. A specific embodiment of this system is described below.

[0641] Hardware and Software Used

[0642] Hardware: Vending machines, user devices (smartphones and tablets)

[0643] Software: Generative AI models (e.g. GPT-4), emotion engines (e.g. Affectiva®)

[0644] Data processing and calculation

[0645] User Request Input

[0646] The user uses the terminal to enter a request to the vending machine, for example "I would like to buy a coffee." The terminal transmits this request to the server.

[0647] The server receives and analyzes the request

[0648] The server receives the user's request and analyzes it. As a result of the analysis, it realizes that the user is requesting to "buy coffee."

[0649] Use of generative AI models

[0650] The server uses a generative AI model (e.g., GPT-4) to generate instructions based on the user's request, such as "serve coffee."

[0651] Sending instructions to vending machines

[0652] The server sends the generated instructions to the vending machine, which receives the instructions and proceeds to the next step.

[0653] Product provision through vending machines

[0654] The vending machine follows the received instructions to provide coffee to the user, specifically, by dispensing a cup of coffee.

[0655] Using the Emotion Engine

[0656] The server uses an emotion engine (e.g., Affectiva) to recognize the user's emotion. It analyzes the user's facial expressions and voice data and recognizes the emotion as "sadness."

[0657] Message generation based on emotions

[0658] Based on the results of the emotion engine, the server uses a generative AI model to generate an appropriate message, for example, "Are you okay? Enjoy your coffee."

[0659] Sending messages to vending machines

[0660] The server generates a message and sends it to the vending machine, which receives it and displays it on its display.

[0661] Examples

[0662] User request: The user types "I want to buy coffee" into the terminal.

[0663] Generative AI instructions: The server generates the instruction "serve coffee" and sends it to the vending machine.

[0664] Emotion engine recognition: The emotion engine recognizes the user's emotion as "sadness."

[0665] Emotion-based response: A vending machine displays the message "Are you OK? Enjoy your coffee."

[0666] Examples of prompt statements

[0667] Prompt: "If the user requests 'I want to buy coffee' and the emotion engine identifies the user's emotion as 'sad', what message should be displayed?"

[0668] This system provides a more personalized customer service experience by providing products according to the user's request and responding according to the user's emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0669] Step 1:

[0670] The user uses the terminal to enter a request to the vending machine, for example "I would like to buy a coffee." The terminal transmits this request to the server.

[0671] Input: User request (e.g. "I want to buy a coffee")

[0672] Output: The request data sent to the server.

[0673] Step 2:

[0674] The server receives the user's request and analyzes it. As a result of the analysis, it realizes that the user is requesting to "buy coffee."

[0675] Input: Request data sent from the terminal

[0676] Output: The parsed request (e.g. "serve coffee")

[0677] Step 3:

[0678] The server uses a generative AI model (e.g., GPT-4) to generate instructions based on the user's request, such as "serve coffee."

[0679] Input: Parsed request content

[0680] Output: The generated instructions (e.g. "serve coffee")

[0681] Step 4:

[0682] The server sends the generated instructions to the vending machine, which receives the instructions and proceeds to the next step.

[0683] Input: Generated instructions

[0684] Output: Instructions sent to the vending machine

[0685] Step 5:

[0686] The vending machine follows the received instructions to provide coffee to the user, specifically, by dispensing a cup of coffee.

[0687] Input: Instructions sent by the server

[0688] Output: The product provided (e.g. coffee)

[0689] Step 6:

[0690] The server uses an emotion engine (e.g., Affectiva) to recognize the user's emotion. It analyzes the user's facial expressions and voice data and recognizes the emotion as "sadness."

[0691] Input: User facial and voice data

[0692] Output: The recognized emotion (e.g. "sadness")

[0693] Step 7:

[0694] Based on the results of the emotion engine, the server uses a generative AI model to generate an appropriate message, for example, "Are you okay? Enjoy your coffee."

[0695] Input: Perceived emotion

[0696] Output: The generated message (e.g. "Are you OK? Enjoy your coffee").

[0697] Step 8:

[0698] The server generates a message and sends it to the vending machine, which receives it and displays it on its display.

[0699] Input: The generated message

[0700] Output: Message sent to the vending machine

[0701] Step 9:

[0702] The vending machine displays the message received from the server on its display. The user sees the message "Are you OK? Enjoy your coffee."

[0703] Input: The message sent by the server.

[0704] Output: The message displayed on the display

[0705] (Application example 3)

[0706] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0707] Conventional vending machines and customer service systems can provide products according to user requests, but they have the problem of being unable to respond to the user's emotions. This can limit the user experience and reduce satisfaction. In addition, because they cannot provide customer service based on emotions, it is difficult to respond flexibly to the user's needs.

[0708] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register application, a means for recognizing the user's emotions by an emotion engine, a means for displaying a message according to the user's emotions, and a means for sending instructions to a vending machine. This makes it possible to not only provide products according to the user's requests, but also to provide flexible customer service based on the user's emotions.

[0709] "Generative AI" refers to artificial intelligence that generates appropriate responses or instructions based on user requests or input.

[0710] A "prompt" is the additional information or instructions that a generative AI needs to generate an appropriate response.

[0711] An "API" is an interface for exchanging functions and data between different software systems.

[0712] A "cash register app" is an application used to sell products and make payments in a store.

[0713] An "emotion engine" is a technology that recognizes emotions from a user's facial expressions, tone of voice, etc.

[0714] A "vending machine" is a machine that automatically provides products selected by a user.

[0715] "Message display" refers to a function that provides information to the user in text or audio.

[0716] "Means of sending instructions" refers to the methods or technologies that a generative AI uses to instruct a vending machine or other device to operate or perform certain actions.

[0717] The following system configuration will be described as an embodiment of the present invention.

[0718] System Configuration

[0719] Hardware

[0720] 1. Server: A server for running the generative AI model and emotion engine.

[0721] 2. Vending Machine: An automatic machine for providing goods.

[0722] 3. User device: A device operated by the user, such as a smartphone or tablet.

[0723] 4. Camera: A camera to capture the user's facial expressions.

[0724] software

[0725] 1. Generative AI model: An artificial intelligence model that generates appropriate responses or instructions based on user requests.

[0726] 2. Emotion engine: Software that recognizes emotions by analyzing a user's facial expressions and tone of voice.

[0727] 3. Vending Machine API: An interface for sending instructions to the vending machine and dispensing products.

[0728] 4. Text-to-Speech Library: A library for playing audio messages to the user.

[0729] 5. OpenCV(Registered Trademark): An image processing library for processing the user's facial images.

[0730] Processing flow

[0731] 1. User request input: The user inputs a request, such as "I want to buy coffee," using a smartphone or tablet.

[0732] 2. Analysis by generative AI: A generative AI model on the server analyzes the user's request and sends instructions to the vending machine.

[0733] 3. Emotion recognition: The user's facial image captured by the camera is processed using OpenCV, and the emotion engine recognizes the user's emotions.

[0734] 4. Offering the product and displaying a message: The vending machine offers the product and plays an appropriate message aloud using the Text-to-Speech library based on the result of the emotion engine.

[0735] Examples

[0736] When a user types "I want to buy coffee" into their smartphone and shows their face to the camera, the system recognizes the user's emotion as "sadness" and plays an audible message saying, "Are you okay? Enjoy your coffee."

[0737] Examples of prompt statements

[0738] When a user types "I want to buy coffee" and shows their face to the camera, the system recognizes the user's emotion and displays an appropriate message: if the emotion is "sadness," it displays "Are you OK? Enjoy your coffee."

[0739] In this way, it is possible to provide products that meet the user's needs and provide flexible customer service based on emotions.

[0740] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0741] Step 1:

[0742] A user inputs a request such as "I want to buy coffee" using a smartphone or tablet. The input request is sent to the server as text data.

[0743] Step 2:

[0744] The server uses a generative AI model to analyze the user's request. The input text data is input as a prompt into the generative AI model, and data is generated as the analysis result to send instructions to the vending machine. The generated data is sent to the vending machine API.

[0745] Step 3:

[0746] The server receives the user's face image captured by the camera. The received image data is converted to grayscale using OpenCV and resized to 48x48. The resized image data is input to the emotion engine.

[0747] Step 4:

[0748] The server recognizes the user's emotion using an emotion engine. The resized image data is input to the emotion engine, and an emotion label is output as an emotion recognition result. The emotion label is data indicating the user's emotion.

[0749] Step 5:

[0750] The server generates an appropriate message based on the emotion recognition result. If the emotion label is "sadness", the server uses a Text-to-Speech library to generate a message such as "Are you okay? Enjoy your coffee" as voice data. The generated voice data is sent to the user device.

[0751] Step 6:

[0752] The vending machine dispenses products based on instructions received from the server, analysing the instruction data and performing operations to dispense the product requested by the user (coffee in this case).

[0753] Step 7:

[0754] The user terminal plays the voice data received from the server. By playing the voice data, a message is provided to the user saying, "Are you OK? Enjoy your coffee."

[0755] In this way, it is possible to provide products that meet the user's needs and to provide flexible customer service based on the user's emotions.

[0756] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0757] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0758] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0759] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0760] [Second embodiment]

[0761] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0762] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0763] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0764] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0765] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0766] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0767] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0768] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0769] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0770] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0771] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0772] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0773] "Example 1"

[0774] One embodiment of the present invention is an application that uses generative AI. This application generates an appropriate prompt in response to an input from a user, and calls the API of another service based on the prompt. For example, when a user inputs "I would like to order pizza," the application generates a prompt "What kind of pizza would you like to order?" and calls an API that places a pizza order based on the user's response.

[0775] "Example 2"

[0776] Furthermore, as one embodiment of the present invention, there is a link with a cash register app. An application using generative AI links with a cash register app to automate cash register operations that involve customer service. For example, to automate ordering at a restaurant, the order details from the user are sent to the cash register app, and the cash register app rings up the cash based on the order details.

[0777] "Example 3"

[0778] As one embodiment of the present invention, there is a vending machine that serves customers. An application using generative AI works with the vending machine and provides products in response to requests from users. For example, when a user inputs "I want to buy coffee," the application sends an instruction to the vending machine, and the vending machine provides coffee.

[0779] The process flow of each embodiment will be described below.

[0780] "Example 1"

[0781] Step 1: Receive input from the user. For example, "I want to order pizza."

[0782] Step 2: Use generative AI to generate an appropriate prompt, for example, "What kind of pizza would you like to order?"

[0783] Step 3: Call the API of another service based on the user's answer. For example, if the user selects "Margherita," call the API to order a Margherita.

[0784] "Example 2"

[0785] Step 1: Receive an order from a user. For example, receive an "order for a hamburger" from a restaurant.

[0786] Step 2: Use generative AI to send the order details to the cash register app.

[0787] Step 3: The cashier app processes the order based on the details of the order, for example calculating the price of a burger and generating a receipt.

[0788] "Example 3"

[0789] Step 1: Receive a request from a user. For example, "I want to buy coffee."

[0790] Step 2: Use generative AI to send instructions to the vending machine, for example to dispense coffee.

[0791] Step 3: The vending machine dispenses the product as instructed, e.g., coffee.

[0792] Example 1

[0793] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0794] In conventional systems, when a user uses a specific service, the process of generating appropriate prompts and calling APIs of other services based on the user's input is often done manually, which is inefficient. In addition, it is difficult to generate appropriate prompts in response to user input, which leads to a poor user experience. Furthermore, there is a demand for automating cash register operations and vending machine customer service in restaurants and other establishments, but this is difficult to achieve with current systems.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative artificial intelligence, a means for receiving input from a user, a means for generating a prompt using a generative artificial intelligence model, a means for presenting the generated prompt to the user, a means for receiving a response from the user, a means for calling an application programming interface of another service, a means for presenting a response of the application programming interface to the user, and a means for linking with a cash register application. This makes it possible to automatically generate an appropriate prompt based on a user's input, to streamline the process of calling the API of another service, and to improve the user experience. In addition, it is possible to automate cash register operations that involve customer service in restaurants and the like, and customer service at vending machines.

[0796] "Generative artificial intelligence" is an artificial intelligence technique for generating appropriate prompts and responses based on user input.

[0797] An "application" is a software program designed to perform a specific task or function.

[0798] "Means for receiving input from a user" refers to a function that provides an interface for a user to enter input in the form of text, voice, or the like.

[0799] A "generative artificial intelligence model" is an algorithm or data model for generating prompts and responses based on user input.

[0800] A "prompt" is a question or instruction that prompts the user for their next action or input.

[0801] An "application programming interface" is an interface that allows different software systems to communicate with each other.

[0802] A "cash register application" is software for managing transactions related to the sale of goods or the provision of services.

[0803] "User experience" is a concept that refers to the overall satisfaction and ease of use when a user uses a particular system or service.

[0804] A "vending machine" is an automated machine that enables a user to purchase goods.

[0805] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0806] The present invention is a system for realizing an application using generative artificial intelligence. A specific embodiment of this system will be described below.

[0807] System Configuration

[0808] The system is composed of three main elements: a server, a terminal, and a user. The server generates prompts using a generative artificial intelligence model and calls the application programming interfaces (APIs) of other services based on user input. The terminal provides an interface for users to enter input and communicates with the server. Users access the system through their terminals and input the required information.

[0809] Hardware and Software Used

[0810] The server is a computer system equipped with a high-performance processor and sufficient memory. A generative artificial intelligence model (e.g., OpenAI's GPT-3) is installed on the server. The terminals are devices such as smartphones, tablets, or personal computers, and access the server through a web browser or a dedicated application. Users use these terminals to provide input to the system.

[0811] Data processing and calculation

[0812] The server receives input from the user, passes the input to a generative artificial intelligence model to generate an appropriate prompt, which is then presented to the user again and receives a response from the user. The server then calls the appropriate API based on the user's response and presents the response to the user.

[0813] As a concrete example, the following code shows what happens when a user enters "I would like to order pizza."

[0814] 1. A user accesses the application using a terminal and types, "I want to order a pizza."

[0815] 2. The server uses a generative artificial intelligence model (GPT-3) to generate a prompt: “What kind of pizza would you like to order?”

[0816] 3. The server presents this prompt to the user.

[0817] 4. The user answers "Margherita."

[0818] 5. The server calls the API for ordering pizza and sends the user's order details.

[0819] 6. The server receives the response from the API and displays a message to the user saying, "Your order has been completed. Your order will be delivered within 30 minutes."

[0820] Example prompt:

[0821] User input: "I want to order pizza."

[0822] Generated prompt: "What kind of pizza would you like to order?"

[0823] In this way, the server, terminals, and users can work together to efficiently fulfill user requests. This system improves the user experience and makes it possible to automate cash register operations and vending machine operations that involve customer service in restaurants and other establishments.

[0824] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0825] Step 1:

[0826] A user accesses the application using a terminal, types "I would like to order pizza" in the text box, and clicks the submit button.

[0827] Input: User text input "I want to order pizza"

[0828] Output: User input data is sent to the server

[0829] Step 2:

[0830] The server receives the HTTP POST request and extracts the user input text from the request body.

[0831] Input: User input data

[0832] Output: Extracted text data "I want to order pizza"

[0833] Step 3:

[0834] The server passes the extracted text data to a generative artificial intelligence model (e.g., GPT-3) to generate an appropriate prompt.

[0835] Input: Extracted text data "I want to order pizza"

[0836] Data processing: A generative artificial intelligence model analyzes the text data and generates appropriate prompts.

[0837] Output: Generated prompt "What kind of pizza would you like to order?"

[0838] Step 4:

[0839] The server sends the generated prompt to the user's terminal as an HTTP response, causing it to be displayed on the user's screen.

[0840] Input: Generated prompt "What kind of pizza would you like to order?"

[0841] Output: The prompt that is displayed on the user's terminal.

[0842] Step 5:

[0843] The user responds with "Margherita" to the displayed prompt and clicks the submit button again.

[0844] Input: User text input "Margherita"

[0845] Output: User answer data is sent to the server

[0846] Step 6:

[0847] The server again receives the HTTP POST request and extracts the user's answer from the request body.

[0848] Input: User response data

[0849] Output: Extracted text data "Margherita"

[0850] Step 7:

[0851] The server calls the appropriate API (e.g., a pizza ordering API) based on the user's answers and submits the user's order.

[0852] Input: Extracted text data "Margherita"

[0853] Data processing: generating and sending API requests

[0854] Output: An order request to the API.

[0855] Step 8:

[0856] The server receives the response from the API and sends the response to the user's device as an HTTP response. The user's screen displays the message, "Your order has been completed. Your order will be delivered within 30 minutes."

[0857] Input: Response data from the API

[0858] Output: A message displayed on the user's device: "Your order is complete. Your order will be delivered in 30 minutes."

[0859] In this way, the server, the terminal, and the user can work together to efficiently fulfill the user's requests.

[0860] (Application example 1)

[0861] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0862] Conventional food delivery systems require a lot of effort for users to place an order, and the operation is complicated, especially for first-time users. In addition, there is a lack of suggestions that take into account the user's preferences and past order history, making it difficult to improve the user experience. Furthermore, there is also an issue that it is not possible to generate appropriate prompts when placing an order, which means that it takes a long time for users to enter the necessary information.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0864] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to a purpose, a means for calling an API of another service, a means for generating an appropriate prompt based on a user's input, a means for making optimal suggestions based on the user's preferences and past order history, and a means for completing an order based on the suggestions. This allows the user to easily place an order, and makes suggestions that take into account the user's preferences and past order history, improving the user experience. In addition, by generating an appropriate prompt, the user can quickly input the necessary information.

[0865] "Generative AI" is an artificial intelligence technique that generates appropriate responses or prompts based on user input.

[0866] An "application role-playing vehicle" is software or a system designed to provide a specific function or service.

[0867] A "supplementary prompt" is a feature that provides additional information or questions in response to the user's input.

[0868] "Means of calling the API of other services" refers to a function that uses an API (Application Programming Interface) to connect with external services or systems.

[0869] "A means for generating appropriate prompts based on user input" is a function that generates questions or instructions to elicit the next required information based on the information entered by the user.

[0870] "A means of making optimal suggestions based on a user's preferences and past order history" is a function that analyzes a user's past behavior and preferences and suggests optimal products and services based on that.

[0871] The "means for completing an order based on a suggestion" is a function for actually confirming an order and completing the process based on a suggestion selected by a user.

[0872] The following system is conceivable as a mode for implementing this invention. The system includes means for fulfilling the role of an application using generative AI, means for supplementing prompts according to the purpose, means for calling the API of other services, means for generating appropriate prompts based on user input, means for making optimal suggestions based on the user's preferences and past order history, and means for completing the order based on the suggestions.

[0873] Hardware and Software Used

[0874] Hardware: Smartphone

[0875] Software: OpenAI API, food delivery service API, requests library

[0876] Data processing and calculation

[0877] 1. User input: The user inputs the food they want into the smartphone app.

[0878] 2. Prompt Generation: The server uses the OpenAI API to generate an appropriate prompt based on the user's input.

[0879] 3. User Answers: The user answers the generated prompt.

[0880] 4. Generate order details: The server generates the order details including the user's answers.

[0881] 5. API call: The server calls the food delivery service's API and completes the order.

[0882] Examples

[0883] When a user types "I'd like to order pizza," the server uses the OpenAI API to generate a prompt: "What kind of pizza would you like to order?" If the user responds with "Margherita," the server calls the food delivery service's API to complete the order.

[0884] Examples of prompt statements

[0885] The user types, "I want to order pizza." Generate an appropriate prompt.

[0886] In this way, we have realized a system that allows users to easily place orders by generating appropriate prompts based on user input and calling the API of the food delivery service. This improves the user experience by making suggestions that take into account the user's preferences and past order history. In addition, generating appropriate prompts allows users to quickly enter the necessary information.

[0887] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0888] Step 1:

[0889] The user inputs the food they want to eat into a smartphone app.

[0890] Input: The user types, "I would like to order pizza."

[0891] Output: The user's input data.

[0892] Specific behavior: The user enters "I would like to order pizza" into the app's text input field and presses the submit button.

[0893] Step 2:

[0894] The server uses the OpenAI API to generate appropriate prompts based on the user's input.

[0895] Input: User input data: "I want to order pizza."

[0896] Output: Generated prompt "What kind of pizza would you like to order?".

[0897] Specific behavior: The server sends the user's input data to the OpenAI API and receives the prompt returned by the API.

[0898] Step 3:

[0899] The user answers the generated prompts.

[0900] Input: Generated prompt "What kind of pizza would you like to order?"

[0901] Output: The user's answer, "Margherita".

[0902] Specific behavior: The user types "Margherita" into the app's text input field and presses the send button.

[0903] Step 4:

[0904] The server generates order details including the user's answers.

[0905] Input: The user's answer, "Margherita".

[0906] Output: Order details (e.g. { "Cuisine": "Pizza", "Type": "Margherita"}).

[0907] What happens: The server builds order details based on the user's answers.

[0908] Step 5:

[0909] The server calls the food delivery service's API and completes the order.

[0910] Input: Order details data.

[0911] Output: Order completed response from the food delivery service.

[0912] What happens: The server sends the order details to the food delivery service's API and receives a response indicating the order was successfully completed.

[0913] In this way, we have created a system that allows users to easily place orders by generating appropriate prompts based on user input and calling the API of a food delivery service.

[0914] Example 2

[0915] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0916] Conventional cash register systems required the user to input an order and have it reflected on the register manually, which was time-consuming and labor-intensive. There was also the possibility of incorrect input of order details and delays in processing, making efficient operation difficult. Furthermore, in tasks involving customer service, the burden on staff was heavy, and there was a risk of the quality of service declining.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0918] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling the API of other services, a means for receiving order details entered by a user, a means for analyzing the received order details using a generative AI model, a means for sending the analysis results to a cash register app, and a means for the cash register app to ring up the order based on the analysis results. This automates the process from ordering to ringing up the order, enabling efficient and accurate order processing.

[0919] "Generative AI" is a type of artificial intelligence that uses natural language processing and machine learning techniques to analyze user input and generate appropriate responses and data.

[0920] A "prompt" is an instruction or question entered into a generative AI to elicit a particular response or action.

[0921] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[0922] A "cash register app" is application software used to calculate fees associated with the sale of goods or the provision of services, process payments, issue receipts, and so on.

[0923] A "User" is a person who utilizes the system to enter an order.

[0924] "Order details" refers to detailed information about the product or service that the user wishes to purchase.

[0925] "Analysis" refers to the process in which the generative AI understands the user's order and extracts specific items and information.

[0926] "Structured data" is data that is organized according to a particular format or structure, such as JSON.

[0927] "Cash registering" refers to the task of calculating and processing payments associated with the sale of goods or the provision of services.

[0928] "Automation" refers to enabling systems or machines to operate automatically, without human intervention.

[0929] This invention is a system that automates the process from ordering to ringing up a cash register by linking a cash register app with an application that uses generative AI. A specific embodiment of this system is described below.

[0930] System Configuration

[0931] server

[0932] The server is responsible for analyzing the order contents from users using a generative AI model. Specifically, the server uses the following hardware and software:

[0933] Hardware: A server machine with a powerful processor, sufficient memory, and a fast network connection.

[0934] Software: Generative AI models (e.g., GPT-4), natural language processing libraries, libraries for calling APIs

[0935] The server receives the order details sent by the user and analyzes them using a generative AI model. The analysis results are generated as structured data in JSON format and sent to the cash register app.

[0936] Terminal

[0937] A terminal is a device that allows a user to input an order. Specifically, this includes a smartphone or tablet. An order application is installed on the terminal, and the terminal has the function of allowing a user to input order details and send them to a server.

[0938] User

[0939] A user is a person who uses the system to place an order. The user uses a terminal to open an order application and inputs the order details. For example, the user inputs "I'd like to order a hamburger and a coke."

[0940] Examples

[0941] Example 1: Ordering at a restaurant

[0942] A user enters "I'd like to order a hamburger and a coke" into an application at a restaurant. The application sends this order to a server. The server uses a generative AI model (e.g. GPT-4) to analyze the order and extracts the specific items "hamburger" and "coke." This information is sent to a cash register app, which automatically rings up the order, calculates the charge, and issues a receipt.

[0943] Examples of prompt statements

[0944] "When a user places an order at a restaurant, they use an application to input the order details. When the user types, 'I'd like a hamburger and a coke,' the application sends this information to a server. The server uses a generative AI model (e.g. GPT-4) to analyze the order and extract the specific items, 'hamburger' and 'coke.' This information is then sent to a cash register app, which automatically records the order, calculates the charge, and issues a receipt."

[0945] In this way, the server, terminal, and user all play their respective roles in the system, automating the process from ordering to ringing up the cash register. This system improves the efficiency and accuracy of order processing.

[0946] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0947] Step 1:

[0948] The user enters an order

[0949] The user opens an ordering application installed on a smartphone or tablet. He / she enters "I would like to order a hamburger and a cola" into the application's input form and presses the send button. The input data is in text format and includes the order details.

[0950] Step 2:

[0951] The terminal sends the order details to the server

[0952] The terminal sends the order details entered by the user to the server as text data. At this time, the data is sent using an HTTP POST request. The input is the text data entered by the user, and the output is the order details sent to the server.

[0953] Step 3:

[0954] The server receives the order.

[0955] The server receives the order details sent from the terminal. The received data is in text format and contains the order details. The input is the text data sent from the terminal, and the output is the received order details.

[0956] Step 4:

[0957] The server parses the order

[0958] The server inputs the received order details into a generative AI model (e.g., GPT-4). The generative AI model analyzes the order details and extracts the specific items "hamburger" and "cola." The input is the text data of the received order details, and the output is structured data (e.g., JSON format) as the analysis result.

[0959] Step 5:

[0960] The server sends the analysis results to the cash register app.

[0961] The server sends the JSON data obtained as a result of the analysis to the cash register app using an HTTP POST request. The input is the JSON data of the analysis result, and the output is the data sent to the cash register app.

[0962] Step 6:

[0963] The cash register app processes the order.

[0964] The cash register app analyzes the JSON data received from the server and calculates the charge based on the order details. For example, if a "hamburger" costs 500 yen and a "cola" costs 200 yen, the total amount is 700 yen. The cash register app displays this amount and issues a receipt. The input is the JSON data received from the server, and the output is the calculated charge and the issued receipt.

[0965] In this way, the specific actions and data flow are clearly defined at each step, automating the process from ordering to ringing up the cash register.

[0966] (Application example 2)

[0967] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0968] The traditional ordering and cash register process in restaurants is often done manually, which is time-consuming and labor-intensive. There is also a high risk of ordering mistakes and cash register errors. This can lead to lower customer satisfaction and poor business efficiency. Furthermore, there is concern that the burden on staff increases during peak times, leading to a decline in the quality of service.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register app, a means for analyzing the contents of a user's order, a means for sending the analysis results to the cash register app, and a means for prompting the user to make payment. This automates the ordering and cash register process, making it possible to improve business efficiency and customer satisfaction.

[0970] "Generative AI" is a type of artificial intelligence that generates new data and information based on user input.

[0971] An "application" is a software program designed to accomplish a particular purpose.

[0972] A "prompt" is an instruction or question that prompts the user for input.

[0973] "API" stands for Application Program Interface, an interface for exchanging data and functions between different software systems.

[0974] A "cash register app" is a software application used to operate the cash register and process payments.

[0975] A "user" is someone who uses a system or application.

[0976] "Order Details" refers to detailed information about the products or services a user wishes to purchase or use.

[0977] "Analysis" is the process of examining data or information in detail to understand its meaning and structure.

[0978] "Transmission" is the act of sending data or information from one place to another.

[0979] "Payment" is the act of giving money in exchange for goods or services.

[0980] As an embodiment of the present invention, we provide a system that automates the ordering and cash register process in a restaurant. The system includes an application that uses generative AI, a means for supplementing prompts, a means for calling APIs of other services, a means for linking with a cash register app, a means for analyzing the contents of a user's order, a means for sending the analysis results to the cash register app, and a means for prompting the user to make payment.

[0981] Hardware and Software Used

[0982] Hardware: Smartphone

[0983] Software: Python, generative AI API, cash register app API

[0984] System Operation

[0985] 1. Your order:

[0986] A user uses a smartphone app to select an item from a menu. For example, the user selects "hamburger."

[0987] 2. Order Analysis:

[0988] The smartphone app sends the selected order details to the generative AI API and obtains the analysis results. The generative AI understands the order details and converts them into an appropriate format.

[0989] 3. Send to the cash register app:

[0990] The analysis results are sent to the cash register app API, which then rings up the order based on the received order details.

[0991] 4. Payment prompts:

[0992] Based on the response from the cash register app, the smartphone app prompts the user to make the payment, and the user completes the payment using their smartphone.

[0993] Examples

[0994] For example, if a user selects "hamburger" on a smartphone app, the generative AI analyzes the order and sends an order for "one hamburger" to the cash register app. The cash register app receives the order, records it, and prompts the user to pay.

[0995] Examples of prompt statements

[0996] A user selects "Hamburger" on a smartphone app. The generative AI analyzes this order and sends it to a cash register app. The cash register app receives the order, records it, and prompts the user to pay.

[0997] In this way, automating the ordering and cashiering process can improve operational efficiency and customer satisfaction.

[0998] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0999] Step 1:

[1000] A user launches a smartphone app and selects a product from the menu. The input is the product selected by the user (e.g., "hamburger"), and the output is information about the selected product. In concrete terms, the selection is completed when the user taps the product on the app's menu screen.

[1001] Step 2:

[1002] The terminal sends the selected product information to the generative AI API. The input is the selected product information, and the output is the analysis result by the generative AI. Specifically, the terminal sends the product information in JSON format to the generative AI API as a POST request.

[1003] Step 3:

[1004] The server receives the analysis results from the generative AI API. The input is the response data from the generative AI API, and the output is the analyzed order details. Specifically, the server parses the API response and extracts the order details.

[1005] Step 4:

[1006] The server sends the parsed order details to the cash register app API. The input is the parsed order details, and the output is the response from the cash register app API. Specifically, the server sends the order details in JSON format to the cash register app API as a POST request.

[1007] Step 5:

[1008] The cash register application will perform cash register operations based on the order details. The input is the order details sent from the server, and the output is the cash register completion status. In concrete terms, the cash register application will receive the order details, record them in an internal database, and generate a receipt.

[1009] Step 6:

[1010] The cash register app returns the completion status of the cash register to the server. The input is the completion status of the cash register, and the output is the response data to the server. In concrete terms, the cash register app sends the completion status to the server in JSON format.

[1011] Step 7:

[1012] The server receives the completion status of the cash register and sends a message to the terminal to prompt payment. The input is the completion status of the cash register and the output is a message to prompt payment. In concrete terms, the server checks the completion status, generates a payment message and sends it to the terminal.

[1013] Step 8:

[1014] The terminal displays a message prompting the user to make a payment. The input is a payment message from the server, and the output is the display to the user. As a specific operation, the terminal displays the payment message to the user in a pop-up format.

[1015] Example 3

[1016] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1017] Conventional vending machines and cash register systems have difficulty responding flexibly to user requests, and the automation of specific product provision and services has been limited. In addition, there has been a lack of systems that can accurately understand user requests and provide appropriate products based on those requests. This has led to a demand for improved user experience and operational efficiency.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1019] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for analyzing user requests, a means for sending instructions to the vending machine based on the analysis results, and a means for the vending machine to provide products. This makes it possible to flexibly and accurately provide products in response to user requests.

[1020] "Generative AI" is an artificial intelligence technology that analyzes user input and generates appropriate responses or instructions through natural language processing.

[1021] An "application" is a software program designed to accomplish a particular purpose.

[1022] A "prompt" is text data or instructions input to a generative AI.

[1023] An "API" is an interface for exchanging functions and data between different software systems.

[1024] A "user request" is a specific request or instruction given by a user to the system.

[1025] "Analysis" is the process of understanding and extracting meaning from input data.

[1026] An "instruction" is a command to perform a particular action or process.

[1027] A "vending machine" is a mechanical device that automatically dispenses merchandise selected by a user.

[1028] "Product" refers to any goods or services offered by a vending machine.

[1029] This invention is a system in which a vending machine provides products in response to a user's request by utilizing an application using generative AI. A specific embodiment of this system will be described below.

[1030] First, a user uses a terminal such as a smartphone or tablet to input a request to the vending machine. For example, the user inputs, "I would like to buy coffee." This request is then sent from the terminal to the server.

[1031] The server uses a generative AI model (e.g., OpenAI's GPT-4) to analyze the user's request. The generative AI model uses natural language processing techniques to understand the user's request and generate appropriate instructions. These instructions are converted into a format that is sent to the vending machine.

[1032] Specifically, the server inputs the following prompt sentence into the generative AI model:

[1033] The user types, "I want to buy coffee." Generate an instruction for the vending machine to provide the coffee.

[1034] The generative AI model would parse this prompt and generate instructions like this:

[1035] Sends a command to the vending machine to dispense coffee.

[1036] The server transmits the instruction to the vending machine. The vending machine operates based on the received instruction and operates the internal machinery to provide coffee. The user can take out the coffee from the receiving slot of the vending machine.

[1037] This system uses a generative AI model to flexibly and accurately respond to user requests. In addition, by having the server have the means to call the APIs of other services, it becomes possible to provide a wider variety of services and products.

[1038] In this way, a system is realized in which an application using generative AI operates a vending machine in response to a user request and provides a product. The flow of the specific process in the third embodiment will be described with reference to FIG.

[1039] Step 1:

[1040] The user enters a request

[1041] A user inputs a request to a vending machine using a terminal such as a smartphone or tablet. For example, the user inputs "I want to buy coffee." This input is saved as text data in the terminal. The input data is the text "I want to buy coffee."

[1042] Step 2:

[1043] The device sends a request to the server

[1044] The terminal transmits the request input by the user to the server via the Internet. At this time, the request is transmitted as text data. The input data is the text "I want to buy coffee," and the output data is the text data transmitted to the server.

[1045] Step 3:

[1046] The server parses the request

[1047] The server inputs the received request into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model processes the request in natural language and generates appropriate instructions. The input data is the text "I want to buy coffee," and the output data is the instruction "Send an instruction to the vending machine to dispense coffee."

[1048] Step 4:

[1049] The server sends instructions to the vending machine

[1050] The server sends the generated instruction to the vending machine. This instruction is converted into a format that the vending machine can understand. For example, it is sent in a specific command format (such as JSON format). The input data is the instruction "Send an instruction to the vending machine to serve coffee", and the output data is the command sent to the vending machine.

[1051] Step 5:

[1052] Vending machines provide products

[1053] The vending machine operates based on instructions received from the server, specifically by operating an internal machine to dispense coffee. A user can dispense the coffee from a receiving slot of the vending machine. The input data are the commands sent to the vending machine, and the output data is the dispensed coffee.

[1054] (Application example 3)

[1055] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1056] Conventional vending machines and cash register systems required users to operate them manually when purchasing products, which resulted in low convenience. In addition, in stores that involved customer service, the burden on staff was heavy and efficient operation was difficult. Furthermore, there was a lack of systems that could flexibly respond to user requests, and there was a need to improve customer satisfaction.

[1057] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for performing the role of an application using a generative AI, a means for supplementing a prompt according to a purpose, a means for calling an API of another service, a means for linking with a cash register application, a means for sending instructions to a vending machine that provides products in response to a user's request, a means for generating instructions using a generative AI model, and a means for providing a user interface using a smartphone. This not only allows users to easily purchase products using a smartphone, but also reduces the burden on store staff and enables efficient store operation. In addition, the use of a generative AI model makes it possible to flexibly respond to user requests and improve customer satisfaction.

[1058] "Generative AI" is an artificial intelligence technology that generates natural language and instructions based on user input.

[1059] A "means for fulfilling the role of an application" is software or a program for providing a specific function or service.

[1060] A "means for supplementing prompts" is a function that complements the user's input and generates appropriate instructions or questions.

[1061] "Means of calling the API of other services" refers to a function that uses an API (Application Programming Interface) to connect with external services or systems.

[1062] "Means of linking with a cash register app" refers to a function that links with the store's cash register system to handle payments and inventory management.

[1063] The "means for sending instructions to a vending machine that provides a product in response to a user's request" is a function for sending instructions to the vending machine to provide a product based on the user's purchase request.

[1064] "Means for generating instructions using a generative AI model" refers to a function that uses a generative AI model to generate specific instructions in response to a user's request.

[1065] "Means for providing a user interface using a smartphone" refers to a function for providing an interface that can be operated by a user using a smartphone.

[1066] A system for implementing this invention includes means for performing the role of an application using generative AI, means for supplementing prompts according to the purpose, means for calling APIs of other services, means for linking with a cash register app, means for sending instructions to a vending machine which provides products in response to a user request, means for generating instructions using a generative AI model, and means for providing a user interface using a smartphone.

[1067] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate natural language and instructions based on user input. When a user inputs on their smartphone that they would like to purchase an item, the server receives the input and sends a prompt to the generative AI model. An example of a prompt sentence is, "The user has input that they would like to purchase a coffee. Please send instructions to the vending machine."

[1068] The generative AI model generates appropriate instructions based on the prompts and returns the instructions to the server. The server receives the generated instructions and calls the APIs of other services to send the instructions to the vending machine. The vending machine follows the instructions and provides the product to the user.

[1069] As a concrete example, a user opens a smartphone app and types, "I want to buy coffee." The server receives this input and sends the following prompt to the generative AI model:

[1070] "The user has indicated that they would like to purchase coffee. Please send instructions to the vending machine."

[1071] Based on this prompt, the generative AI model generates the instruction, "Send an instruction to the vending machine to serve coffee." The server receives this instruction and calls the vending machine's API to send the instruction. The vending machine follows this instruction and serves coffee to the user.

[1072] This system not only allows users to easily purchase products using their smartphones, but also reduces the burden on store staff and enables efficient store operations. In addition, the use of generative AI models allows for flexible response to user requests, improving customer satisfaction.

[1073] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1074] Step 1:

[1075] A user starts the application on a smartphone terminal and inputs the product he or she wants to purchase. The input product name is "coffee."

[1076] Step 2:

[1077] The terminal transmits the user's input to the server. The input data is text data for "coffee."

[1078] Step 3:

[1079] The server sends a prompt to the generative AI model based on the received user input data. The prompt text is "The user has input that he / she would like to buy coffee. Please send instructions to the vending machine."

[1080] Step 4:

[1081] The generative AI model analyzes the prompt and generates the appropriate instruction, which is "Send an instruction to the vending machine to serve coffee."

[1082] Step 5:

[1083] The server analyzes the instructions received from the generative AI model and generates data to call the vending machine's API. The input data are the generated instructions, and the output data are the data for the API call.

[1084] Step 6:

[1085] The server calls the vending machine's API and sends the generated data. As a result of the API call, the vending machine dispenses the coffee.

[1086] Step 7:

[1087] The vending machine provides coffee to the user based on instructions received from the server. The product provided is "coffee."

[1088] Step 8:

[1089] The server notifies the user that the product has been served. The content of the notification is "Coffee has been served."

[1090] Step 9:

[1091] The terminal receives the notification from the server and displays it to the user, saying "Coffee has been served."

[1092] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1093] "Example 1"

[1094] One embodiment of the present invention is a system that includes an emotion engine that recognizes a user's emotions. The system receives input from a user and generates an appropriate prompt using a generative AI. It also calls APIs of other services. The emotion engine then recognizes the user's emotions and supplements the prompt according to the emotions. For example, if a user inputs "I would like to order pizza" and the emotion engine recognizes the user's emotion as "joy," the generative AI generates a prompt such as "Congratulations! What kind of pizza would you like to order?"

[1095] "Example 2"

[1096] Another embodiment of the present invention is a system that automates cash register operations involving customer service based on the user's emotions recognized by an emotion engine. This system receives the order details from the user and sends the order details to a cash register app using generative AI. The emotion engine then recognizes the user's emotions and automates cash register operations according to those emotions. For example, if a user "orders a hamburger" and the emotion engine recognizes the user's emotion as "anger," the cash register app displays a message such as "Sorry. We will quickly prepare the hamburger."

[1097] "Example 3"

[1098] As yet another embodiment of the present invention, there is a system for realizing a vending machine that serves customers based on the user's emotions recognized by an emotion engine. This system receives requests from a user and sends instructions to the vending machine using a generative AI. The emotion engine then recognizes the user's emotions, and the vending machine provides products according to those emotions. For example, if a user requests "I want to buy coffee" and the emotion engine recognizes the user's emotion as "sadness," the vending machine will display a message such as "Are you okay? Enjoy your coffee."

[1099] The process flow of each embodiment will be described below.

[1100] "Example 1"

[1101] Step 1: Receive input from the user.

[1102] Step 2: Use generative AI to generate appropriate prompts.

[1103] Step 3: Call the API of other services.

[1104] Step 4: Recognize the user's emotions using the emotion engine.

[1105] Step 5: Complement the prompt with the emotion you identify.

[1106] "Example 2"

[1107] Step 1: Receive the order from the user.

[1108] Step 2: Use generative AI to send the order details to the cash register app.

[1109] Step 3: Recognize the user's emotions using the emotion engine.

[1110] Step 4: Automate cashiering based on the recognized emotions.

[1111] "Example 3"

[1112] Step 1: Receive a request from a user.

[1113] Step 2: Use generative AI to send instructions to the vending machine.

[1114] Step 3: Recognize the user's emotions using the emotion engine.

[1115] Step 4: The vending machine will dispense a product based on the emotion it recognizes.

[1116] Example 1

[1117] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1118] Conventional systems have problems in that it is difficult to generate appropriate prompts in response to user input, and they are unable to respond in a way that takes into account the user's emotions. Furthermore, when calling APIs for other services, flexible responses based on the user's emotions are required, but there is a lack of means to achieve this.

[1119] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1120] In this invention, the server includes a means for receiving input from a user, a means for generating an appropriate prompt sentence using a generative AI model, a means for recognizing the user's emotion, a means for supplementing the prompt sentence based on the recognized emotion, and a means for calling an API of another service. This makes it possible to generate an appropriate prompt for the user's input and to flexibly respond according to the emotion.

[1121] The "means for receiving input from a user" is a function that provides an interface for a user to input information such as text or voice to the system through a terminal.

[1122] "Means for generating appropriate prompt sentences using a generative AI model" refers to a function that automatically creates prompt sentences containing appropriate questions or instructions using a generative AI model based on user input.

[1123] "Means for recognizing user emotions" refers to a function that uses an algorithm or engine to analyze emotions from the user's input or voice and identify those emotions.

[1124] The "means for supplementing the prompt sentence based on the recognized emotion" is a function that takes the user's emotion into account and adds additional information or an expression according to the emotion to the generated prompt sentence.

[1125] The "means for calling the API of other services" refers to a function for calling the API of external services or systems based on the generated prompt text or the user's response, and performing the required operations or data acquisition.

[1126] "Means for linking with a cash register app" refers to a function for communicating with a cash register application and sending and receiving data and linking operations.

[1127] This invention is configured mainly with an application using a generative AI model. A specific embodiment of this system will be described below.

[1128] System Configuration

[1129] The system mainly uses the following hardware and software:

[1130] Hardware: Devices used by users (smartphones, tablets, PCs, etc.), servers

[1131] Software: Generative AI models (e.g. GPT-3), emotion recognition engines (e.g. emotion analysis API), API call functions, cash register applications

[1132] Program Processing

[1133] 1. Receiving user input

[1134] A user inputs text or voice to the system through a terminal. For example, the user inputs "I would like to order pizza."

[1135] 2. Prompt Generation

[1136] The server uses a generative AI model (e.g., GPT-3) to generate an appropriate prompt for the user's input, such as "What kind of pizza would you like to order?"

[1137] 3. Emotion recognition

[1138] The server uses an emotion recognition engine (e.g., emotion analysis API) to recognize emotions from the user's input. For example, it recognizes the emotion "joy" from the user's input "I want to order pizza."

[1139] 4. Prompt Supplement

[1140] The server supplements the generated prompt sentence based on the recognized emotion, i.e., "Congratulations! What kind of pizza would you like to order?"

[1141] 5. API Calls

[1142] When the user answers the prompts, the server calls APIs of other services based on the answer. For example, if the user answers "Margherita," the server calls a pizza ordering API to order a Margherita pizza.

[1143] Examples

[1144] Receiving user input

[1145] The user enters "I would like to order pizza" into the terminal and presses the send button.

[1146] Prompt Generation

[1147] The server uses a generative AI model (e.g. GPT-3) to generate a prompt such as "What kind of pizza would you like to order?"

[1148] emotion recognition

[1149] The server calls an emotion recognition engine (e.g., emotion analysis API) and recognizes the emotion “joy” from the user’s input “I want to order pizza.”

[1150] Prompt Supplement

[1151] The server again uses the generative AI model to generate a follow-up prompt: "Congratulations! What kind of pizza would you like to order?"

[1152] API Call

[1153] The user responds with "Margherita," and the server calls the pizza ordering API to order a Margherita pizza.

[1154] In this way, a system is realized that generates appropriate prompts in response to user input and responds flexibly according to emotions.

[1155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1156] Step 1:

[1157] Receiving user input

[1158] Specific action: The user enters "I would like to order pizza" into the input field of the terminal and presses the send button.

[1159] Input: User text input "I want to order pizza".

[1160] Output: The user's input data is sent to the server.

[1161] Step 2:

[1162] Prompt Generation

[1163] What it does: The server uses a generative AI model (e.g. GPT-3) to generate an appropriate prompt in response to the user's input.

[1164] Input: User input data: "I want to order pizza."

[1165] Data processing: The generative AI model analyzes the user's input and generates an appropriate prompt.

[1166] Output: The generated prompt sentence: "What kind of pizza would you like to order?".

[1167] Step 3:

[1168] emotion recognition

[1169] Specific operation: The server uses an emotion recognition engine (e.g., emotion analysis API) to recognize emotions from the user's input.

[1170] Input: User input data: "I want to order pizza."

[1171] Data processing: An emotion recognition engine analyzes user input and identifies emotions.

[1172] Output: The perceived emotion "joy".

[1173] Step 4:

[1174] Prompt Supplement

[1175] Specific behavior: The server supplements the generated prompt sentence based on the emotion recognized by the server.

[1176] Input: The generated prompt sentence "What kind of pizza would you like to order?" and the recognized emotion "Joy".

[1177] Data processing: The generative AI model adds supplementary sentences based on the sentiment.

[1178] Output: The augmented prompt, "Congratulations! What kind of pizza would you like to order?".

[1179] Step 5:

[1180] API Call

[1181] Specific behavior: The user answers prompts, and the server calls the API of other services based on the answers.

[1182] Input: The user's answer, "Margherita".

[1183] Data processing: The server analyzes the user's answers and generates a request to call the pizza ordering API.

[1184] Output: A request is sent to the pizza ordering API, ordering a Margherita pizza.

[1185] (Application example 1)

[1186] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1187] In conventional food delivery systems, the ordering process proceeds without considering the user's emotions, resulting in a uniform user experience and making it difficult to provide services that meet the needs and emotions of individual users. In addition, the ordering process can be cumbersome because appropriate prompts are not generated when the user places an order.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to a purpose, a means for calling APIs of other services, a means including an emotion engine for recognizing the user's emotions, a means for generating prompts according to the user's emotions, and a means for linking with a food delivery service. This makes it possible to generate appropriate prompts according to the user's emotions and to smoothly progress the food delivery ordering process.

[1189] "Generative AI" is an artificial intelligence technology that generates appropriate prompts based on user input and calls the APIs of other services.

[1190] A "prompt" is an instruction that presents the user with the next action or choice to make.

[1191] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[1192] An "emotion engine" is a technology that recognizes emotions from user input and actions and generates appropriate responses based on those emotions.

[1193] A "food delivery service" is a service that allows users to order food online and have it delivered to a specified location.

[1194] "User emotion" refers to the psychological state or feelings that a user exhibits through their input and actions.

[1195] "Supplementing a prompt" means generating supplemental instructions to provide additional information or options to the user in response to their input.

[1196] "Integration" means that different systems or services work together to share information and functions.

[1197] The system for implementing this invention is configured by linking a generative AI, an emotion engine, and an API for a food delivery service. A specific embodiment of the system is shown below.

[1198] System Configuration

[1199] The system is composed of a terminal that receives user input, a server that runs the generative AI, a server that runs the emotion engine, and a server that calls the API of the food delivery service. The terminal is a device such as a smartphone or tablet, and provides an interface for users to place orders.

[1200] Program Processing

[1201] The server performs the process in the following procedure.

[1202] 1. Receive user input:

[1203] A user inputs "I would like to order pizza" through a terminal. This input is sent to the server.

[1204] 2. Emotion Recognition:

[1205] The server uses an emotion engine to recognize emotions from the user's input. For example, the user's input is recognized as indicating the emotion "happiness."

[1206] 3. Generate prompt:

[1207] The server uses generative AI to generate prompts based on the user's emotions, for example, "Congratulations! What kind of pizza would you like to order?"

[1208] 4. Receiving user responses:

[1209] The user answers "Margherita" via the terminal. This answer is sent back to the server.

[1210] 5. Call the food delivery API:

[1211] The server calls the API of the food delivery service to process the user's order, and the order data is sent to the API to complete the order.

[1212] Hardware and Software Used

[1213] Devices: Smartphones, tablets

[1214] Servers: A server that runs the generative AI, a server that runs the emotion engine, and a server that calls the API of a food delivery service.

[1215] Software: Generative AI models, emotion engines, food delivery service APIs

[1216] Examples

[1217] If a user types "I'd like to order sushi" and the emotion engine recognizes this as "surprise," the generative AI will generate a prompt saying, "Great choice! What kind of sushi would you like to order?"

[1218] Examples of prompt statements

[1219] "I want to order pizza."

[1220] "Congratulations! What kind of pizza would you like to order?"

[1221] "I want to order sushi."

[1222] "Great choice! What type of sushi would you like to order?"

[1223] In this way, appropriate prompts can be generated based on the user's emotions, ensuring a smooth food delivery ordering process.

[1224] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1225] Step 1:

[1226] A user inputs "I want to order pizza" through a terminal. This input is sent from the terminal to the server. The input data is in text format and represents the user's order request.

[1227] Step 2:

[1228] The server uses an emotion engine to recognize emotions from user input. The input data is the user's text input, which the emotion engine analyzes and outputs emotions such as "joy." The emotion engine uses natural language processing technology to classify the emotion of the text.

[1229] Step 3:

[1230] The server uses a generative AI to generate a prompt based on the user's emotion. The input data is the user's emotion and text input, and the generative AI outputs a prompt such as "Congratulations! What kind of pizza would you like to order?" The generative AI uses a pre-trained model to generate the appropriate prompt.

[1231] Step 4:

[1232] The user answers "Margherita" through the terminal. This answer is sent from the terminal to the server. The input data is a text that represents the type of pizza selected by the user.

[1233] Step 5:

[1234] The server calls the API of the food delivery service to process the user's order. The input data is the type and quantity of pizza selected by the user, and the server sends this to the API of the food delivery service. The API receives the order data and processes the order.

[1235] Step 6:

[1236] The server receives the order confirmation from the food delivery service and notifies the user that the order has been completed. The input data is the order confirmation from the food delivery service, and the server converts it into a message to notify the user and sends it to the terminal. The user receives the notification of the order completion on the terminal.

[1237] Example 2

[1238] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1239] Conventional cash register systems process orders without considering the user's emotions, which can result in poor customer service quality. In addition, it is difficult to respond flexibly to the user's emotions, making it a challenge to improve user satisfaction, especially in situations involving customer service such as restaurants.

[1240] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1241] In this invention, the server includes a means for performing the role of an application using generative artificial intelligence, a means for supplementing prompts according to a purpose, a means for calling an application programming interface of another service, a means for linking with a cash register application, a means for recognizing a user's emotion, a means for generating a message based on the recognized emotion, and a means for sending the generated message to the cash register application. This enables flexible responses according to the user's emotion, improving the quality of customer service.

[1242] "Generative artificial intelligence" is an artificial intelligence technology that generates text and data using natural language processing and machine learning.

[1243] A "prompt" is an instruction or question input to a generative artificial intelligence to obtain a specific output.

[1244] An "application programming interface" is an interface for exchanging data and functions between different software systems.

[1245] A "cash register application" is software for managing point-of-sale information and supports product sales and inventory management.

[1246] "Means for recognizing a user's emotions" refers to technology that analyzes a user's facial expressions and tone of voice to identify their emotional state.

[1247] "Means for generating a message" refers to technology that uses generative artificial intelligence to create an appropriate text message based on specific input.

[1248] A "message sending means" is a technique for forwarding a generated message to another system or application.

[1249] This invention is a system that realizes flexible responses according to the emotions of a user by linking an application using generative artificial intelligence with a cash register application. Specific embodiments of this system are described below.

[1250] Hardware and Software Used

[1251] server

[1252] The server provides the computational resources to run the generative artificial intelligence model. Specifically, it uses the following software and hardware:

[1253] Generative artificial intelligence models: such as OpenAI's GPT-4

[1254] Application Programming Interface: Use RESTful APIs to integrate with other services

[1255] Database: A database to store user order history and emotion data (e.g. MySQL)

[1256] Terminal

[1257] The terminal is a device for capturing the user's voice and facial expressions and recognizing emotions. Specifically, the following hardware and software are used:

[1258] Microphone: Captures the user's voice

[1259] Camera: Captures the user's facial expressions

[1260] Voice recognition software: Google Speech-to-Text, etc.

[1261] Emotion recognition engine: Microsoft Azure's Emotion API, etc.

[1262] Cash register application

[1263] The cash register application is software that processes user orders and displays appropriate messages. Specifically, it uses the following software:

[1264] POS system: Square POS, etc.

[1265] System Operation

[1266] User Orders

[1267] A user goes to a restaurant and says, "I'd like to order a hamburger." The device uses a microphone to capture the user's voice and converts the voice into text data using Google Speech-to-Text. This text data contains the content, "I'd like to order a hamburger."

[1268] Emotion recognition

[1269] The device uses a camera and microphone to capture the user's facial expressions and tone of voice. Using Microsoft Azure's Emotion API, the captured data is analyzed and the user's emotion is recognized as "anger." For example, if the user's voice is high-pitched and their facial expression is grim, the emotion engine will determine that the emotion is "anger."

[1270] Sending your order

[1271] The terminal sends a prompt sentence to the generative AI model. An example of a prompt sentence is "The user ordered a hamburger. The emotion engine recognized the user's emotion as anger. Please generate a message to be displayed in the cash register app." Based on this prompt sentence, the generative AI model generates the message "I'm sorry. I'll prepare the hamburger quickly."

[1272] Receiving and sending generated messages

[1273] The server receives messages generated from the generative artificial intelligence model. The server sends the received messages to the cash register application. At this time, the server converts the message format into a format that the cash register application can understand.

[1274] Displaying messages

[1275] The cash register application displays the message received from the server. Specifically, the cash register application displays the message "Sorry, we'll hurry up and prepare your burger" on its screen. This message reassures the user that their order will be fulfilled promptly.

[1276] This system enables flexible responses according to the user's emotions, improving the quality of customer service.

[1277] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1278] Step 1:

[1279] A user is at a restaurant and says, "I want to order a hamburger."

[1280] Input: User's voice

[1281] Specific behavior: The device uses the microphone to capture the user's voice.

[1282] Output: Captured audio data

[1283] Step 2:

[1284] The terminal converts the captured voice data into text data.

[1285] Input: Audio data

[1286] Specific operation: The device uses voice recognition software (e.g. Google Speech-to-Text) to convert voice data into text data.

[1287] Output: Text data "I want to order a hamburger"

[1288] Step 3:

[1289] The device captures the user's facial expressions and tone of voice to recognise emotions.

[1290] Input: User's facial expression data, voice tone data

[1291] How it works: The device uses the camera and microphone to capture the user's facial expressions and tone of voice, and then analyzes them using an emotion recognition engine (e.g. Microsoft Azure's Emotion API).

[1292] Output: User emotion (e.g. "Anger")

[1293] Step 4:

[1294] The terminal sends a prompt sentence to the generative artificial intelligence model.

[1295] Input: Text data, user sentiment

[1296] Specific operation: The terminal sends the following prompt to the generative artificial intelligence model: "The user ordered a hamburger. The emotion engine recognized the user's emotion as anger. Please generate a message to be displayed in the cash register app."

[1297] Output: The prompt sent to the generative AI model

[1298] Step 5:

[1299] The generative artificial intelligence model generates a message based on the prompt sentence.

[1300] Input: Prompt statement

[1301] How it works: A generative artificial intelligence model (e.g., OpenAI's GPT-4) analyzes the prompt and generates an appropriate message.

[1302] Output: "Sorry, I'll hurry up and prepare the burger."

[1303] Step 6:

[1304] The server receives the generated message and sends it to the cash register application.

[1305] Input: The generated message

[1306] Specific operation: The server receives messages generated from the generative artificial intelligence model, converts them into a format that the cash register application can understand, and sends them.

[1307] Output: Message sent to the cash register application

[1308] Step 7:

[1309] The cash register application displays the received message.

[1310] Input: The message sent by the server

[1311] Specific operation: The cash register application displays the received message on the screen.

[1312] Output: The message "Sorry, I'll hurry up and prepare the burger" is displayed on the screen.

[1313] In this way, it becomes possible to respond flexibly to the user's emotions, thereby improving the quality of customer service.

[1314] (Application example 2)

[1315] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1316] Conventional cash register systems were unable to respond to the user's emotions, resulting in poor customer service quality. In addition, because order details were processed manually, there were issues with inefficiency and the likelihood of mistakes occurring. This resulted in problems such as lower customer satisfaction and a worsening operational efficiency of the store.

[1317] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1318] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register application, a means for recognizing the user's emotions using an emotion engine, and a means for generating messages according to the user's emotions. This allows for appropriate responses according to the user's emotions, improving the quality of customer service. Furthermore, automating the processing of order details allows for efficient operation and reduces the occurrence of errors.

[1319] "Generative AI" is an artificial intelligence technology that generates natural language sentences, images, etc. based on user input.

[1320] A "prompt" is an input sentence or instruction given to a generative AI to obtain a specific output.

[1321] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[1322] A "cash register app" is application software used to process product sales and orders in a store.

[1323] An "emotion engine" is a technology for recognizing emotions from a user's voice, facial expressions, etc.

[1324] "User emotion" refers to the psychological state or feelings exhibited by a user.

[1325] "Means for generating a message" refers to technology for generating an appropriate response sentence depending on the user's emotions and situation.

[1326] The system for implementing this invention is composed of a server including a generative AI, an emotion engine, a cash register application, and an API calling function, and a terminal operated by a user. The specific configuration and operation of the system are described below.

[1327] The server plays the role of an application that uses generative AI. Generative AI is an artificial intelligence technology that generates natural language text and images based on user input. When a user enters an order through a terminal, the contents are sent to the server. The server analyzes the order contents and sends appropriate prompts to the generative AI.

[1328] The server recognizes the user's emotions using an emotion engine. An emotion engine is a technology for recognizing emotions from the user's voice, facial expressions, etc. When the user enters an order, the voice and image data are sent to the emotion engine, which analyzes the user's emotions.

[1329] The server processes the order details in cooperation with the cash register app. The cash register app is application software for selling products and processing orders in stores. The server sends the order details to the cash register app's API and automates the order processing.

[1330] The server generates a message according to the user's emotions. Generative AI is a technology for generating appropriate responses according to the user's emotions and circumstances. For example, if a user "orders a hamburger" and the emotion engine recognizes the user's emotion as "anger," the generative AI will generate a message saying, "Sorry. We'll quickly prepare the hamburger."

[1331] As a concrete example, if a user orders a hamburger using a smartphone app and the emotion engine recognizes the user's emotion as "anger," the server will use generative AI to generate a message saying, "We're sorry. We'll prepare the hamburger quickly," and display it on the user's device.

[1332] Examples of prompts for a generative AI model include:

[1333] "Generate a message for when the user is angry."

[1334] In this way, the system can respond appropriately according to the user's emotions and improve the quality of customer service. In addition, automating the processing of order details enables efficient operation and reduces the occurrence of mistakes.

[1335] The flow of the identification process in the application example 2 will be described with reference to FIG.

[1336] Step 1:

[1337] A user enters an order through a terminal.

[1338] Input: A user uses a smartphone app to input the order details (e.g., hamburger).

[1339] Data processing: The terminal converts the order details into a data format such as JSON.

[1340] Output: The transformed order data is sent to the server.

[1341] Step 2:

[1342] The server receives and analyzes the order.

[1343] Input: The order data sent from the terminal.

[1344] Data processing: The server analyzes the order data and extracts the order details.

[1345] Output: The parsed order details are sent to the generative AI.

[1346] Step 3:

[1347] The server recognizes the user's emotions using an emotion engine.

[1348] Input: User's voice and image data.

[1349] Data processing: The emotion engine analyzes voice and image data to recognize the user's emotions.

[1350] Output: The recognized emotion data is sent to the server.

[1351] Step 4:

[1352] The server generates the prompt using generative AI.

[1353] Input: Parsed order details and recognized emotion data.

[1354] Data processing: The generative AI generates a prompt sentence and then generates an appropriate response sentence.

[1355] Output: The generated response is sent to the server.

[1356] Step 5:

[1357] The server calls the API of the cash register app and processes the order details.

[1358] Input: The parsed order.

[1359] Data processing: The server calls the API of the cash register app and sends the order details.

[1360] Output: The order is processed by the cash register app.

[1361] Step 6:

[1362] The server sends the generated response to the user's terminal.

[1363] Input: The generated response sentence.

[1364] Data processing: The server sends the response to the user's terminal.

[1365] Output: The response is displayed on the user's terminal.

[1366] Step 7:

[1367] The user checks the response on the terminal.

[1368] Input: The response sent by the server.

[1369] Data processing: The terminal displays the response text.

[1370] Output: The user confirms the response.

[1371] Example 3

[1372] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1373] Conventional vending machines and customer service systems can provide products according to user requests, but they have the problem of being unable to respond to the user's emotions. This limits the user experience and makes it difficult to provide personalized services. In addition, because the system does not take the user's emotions into account, it is difficult to improve user satisfaction.

[1374] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a means for performing the role of an application using a generative AI, a means for supplementing a prompt according to a purpose, a means for calling an API of another service, a means for receiving and analyzing a user request, a means for generating instructions using a generative AI model, a means for sending instructions to a vending machine, a means for recognizing a user's emotion using an emotion engine, a means for generating a message based on the recognized emotion, a means for sending a message to the vending machine, and a means for the vending machine to provide a product and display a message. This makes it possible not only to provide a product according to a user's request, but also to provide a personalized response according to the user's emotion.

[1375] "Generative AI" is an artificial intelligence technology that generates natural language text or instructions based on user input.

[1376] A "means for fulfilling the role of an application" is software or a program for providing a specific function or service.

[1377] A "means for supplementing prompts" is a function that adds appropriate supplementary information or instructions to the user's input.

[1378] "Means of calling the API of other services" refers to a function that uses the API provided by that service in order to link with external services or systems.

[1379] The "means for receiving and analyzing a user's request" is a function for receiving input from a user and analyzing and understanding the content of that input.

[1380] "Means for generating instructions using a generative AI model" refers to a function that uses generative AI to create specific instructions based on user requests.

[1381] The "means for transmitting instructions to the vending machine" refers to a communication function for transmitting the generated instructions to the vending machine.

[1382] "Means for recognizing a user's emotions using an emotion engine" refers to a technology that analyzes a user's facial expressions, voice, etc. to identify their emotions.

[1383] The "means for generating a message based on the recognized emotion" is a function for creating an appropriate message according to the user's emotion.

[1384] The "means for transmitting a message to a vending machine" is a communication function for transmitting the generated message to the vending machine.

[1385] The "means for the vending machine to provide a product and display a message" refers to the function of the vending machine to provide a product to a user and display a message on the display.

[1386] This invention is a system that links a vending machine with an application that uses generative AI, provides products according to user requests, and responds according to emotions using an emotion engine. A specific embodiment of this system is described below.

[1387] Hardware and Software Used

[1388] Hardware: Vending machines, user devices (smartphones and tablets)

[1389] Software: Generative AI models (e.g. GPT-4), emotion engines (e.g. Affectiva)

[1390] Data processing and calculation

[1391] User Request Input

[1392] The user uses the terminal to enter a request to the vending machine, for example "I would like to buy a coffee." The terminal transmits this request to the server.

[1393] The server receives and analyzes the request

[1394] The server receives the user's request and analyzes it. As a result of the analysis, it realizes that the user is requesting to "buy coffee."

[1395] Use of generative AI models

[1396] The server uses a generative AI model (e.g., GPT-4) to generate instructions based on the user's request, such as "serve coffee."

[1397] Sending instructions to vending machines

[1398] The server sends the generated instructions to the vending machine, which receives the instructions and proceeds to the next step.

[1399] Product provision through vending machines

[1400] The vending machine follows the received instructions to provide coffee to the user, specifically, by dispensing a cup of coffee.

[1401] Using the Emotion Engine

[1402] The server uses an emotion engine (e.g., Affectiva) to recognize the user's emotion. It analyzes the user's facial expressions and voice data and recognizes the emotion as "sadness."

[1403] Message generation based on emotions

[1404] Based on the results of the emotion engine, the server uses a generative AI model to generate an appropriate message, for example, "Are you okay? Enjoy your coffee."

[1405] Sending messages to vending machines

[1406] The server generates a message and sends it to the vending machine, which receives it and displays it on its display.

[1407] Examples

[1408] User request: The user types "I want to buy coffee" into the terminal.

[1409] Generative AI instructions: The server generates the instruction "serve coffee" and sends it to the vending machine.

[1410] Emotion engine recognition: The emotion engine recognizes the user's emotion as "sadness."

[1411] Emotion-based response: A vending machine displays the message "Are you OK? Enjoy your coffee."

[1412] Examples of prompt statements

[1413] Prompt: "If the user requests 'I want to buy coffee' and the emotion engine identifies the user's emotion as 'sad', what message should be displayed?"

[1414] This system provides a more personalized customer service experience by providing products according to the user's request and responding according to the user's emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1415] Step 1:

[1416] The user uses the terminal to enter a request to the vending machine, for example "I would like to buy a coffee." The terminal transmits this request to the server.

[1417] Input: User request (e.g. "I want to buy a coffee")

[1418] Output: The request data sent to the server.

[1419] Step 2:

[1420] The server receives the user's request and analyzes it. As a result of the analysis, it realizes that the user is requesting to "buy coffee."

[1421] Input: Request data sent from the terminal

[1422] Output: The parsed request (e.g. "serve coffee")

[1423] Step 3:

[1424] The server uses a generative AI model (e.g., GPT-4) to generate instructions based on the user's request, such as "serve coffee."

[1425] Input: Parsed request content

[1426] Output: The generated instructions (e.g. "serve coffee")

[1427] Step 4:

[1428] The server sends the generated instructions to the vending machine, which receives the instructions and proceeds to the next step.

[1429] Input: Generated instructions

[1430] Output: Instructions sent to the vending machine

[1431] Step 5:

[1432] The vending machine follows the received instructions to provide coffee to the user, specifically, by dispensing a cup of coffee.

[1433] Input: Instructions sent by the server

[1434] Output: The product provided (e.g. coffee)

[1435] Step 6:

[1436] The server uses an emotion engine (e.g., Affectiva) to recognize the user's emotion. It analyzes the user's facial expressions and voice data and recognizes the emotion as "sadness."

[1437] Input: User facial and voice data

[1438] Output: The recognized emotion (e.g. "sadness")

[1439] Step 7:

[1440] Based on the results of the emotion engine, the server uses a generative AI model to generate an appropriate message, for example, "Are you okay? Enjoy your coffee."

[1441] Input: Perceived emotion

[1442] Output: The generated message (e.g. "Are you OK? Enjoy your coffee").

[1443] Step 8:

[1444] The server generates a message and sends it to the vending machine, which receives it and displays it on its display.

[1445] Input: The generated message

[1446] Output: Message sent to the vending machine

[1447] Step 9:

[1448] The vending machine displays the message received from the server on its display. The user sees the message "Are you OK? Enjoy your coffee."

[1449] Input: The message sent by the server.

[1450] Output: The message displayed on the display

[1451] (Application example 3)

[1452] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1453] Conventional vending machines and customer service systems can provide products according to user requests, but they have the problem of being unable to respond to the user's emotions. This can limit the user experience and reduce satisfaction. In addition, because they cannot provide customer service based on emotions, it is difficult to respond flexibly to the user's needs.

[1454] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register application, a means for recognizing the user's emotions by an emotion engine, a means for displaying a message according to the user's emotions, and a means for sending instructions to a vending machine. This makes it possible to not only provide products according to the user's requests, but also to provide flexible customer service based on the user's emotions.

[1455] "Generative AI" refers to artificial intelligence that generates appropriate responses or instructions based on user requests or input.

[1456] A "prompt" is the additional information or instructions that a generative AI needs to generate an appropriate response.

[1457] An "API" is an interface for exchanging functions and data between different software systems.

[1458] A "cash register app" is an application used to sell products and make payments in a store.

[1459] An "emotion engine" is a technology that recognizes emotions from a user's facial expressions, tone of voice, etc.

[1460] A "vending machine" is a machine that automatically provides products selected by a user.

[1461] "Message display" refers to a function that provides information to the user in text or audio.

[1462] "Means of sending instructions" refers to the methods or technologies that a generative AI uses to instruct a vending machine or other device to operate or perform certain actions.

[1463] The following system configuration will be described as an embodiment of the present invention.

[1464] System Configuration

[1465] Hardware

[1466] 1. Server: A server for running the generative AI model and emotion engine.

[1467] 2. Vending Machine: An automatic machine for providing goods.

[1468] 3. User device: A device operated by the user, such as a smartphone or tablet.

[1469] 4. Camera: A camera to capture the user's facial expressions.

[1470] software

[1471] 1. Generative AI model: An artificial intelligence model that generates appropriate responses or instructions based on user requests.

[1472] 2. Emotion engine: Software that recognizes emotions by analyzing a user's facial expressions and tone of voice.

[1473] 3. Vending Machine API: An interface for sending instructions to the vending machine and dispensing products.

[1474] 4. Text-to-Speech Library: A library for playing audio messages to the user.

[1475] 5. OpenCV: An image processing library for processing the user's facial images.

[1476] Processing flow

[1477] 1. User request input: The user inputs a request, such as "I want to buy coffee," using a smartphone or tablet.

[1478] 2. Analysis by generative AI: A generative AI model on the server analyzes the user's request and sends instructions to the vending machine.

[1479] 3. Emotion recognition: The user's facial image captured by the camera is processed using OpenCV, and the emotion engine recognizes the user's emotions.

[1480] 4. Offering the product and displaying a message: The vending machine offers the product and plays an appropriate message aloud using the Text-to-Speech library based on the result of the emotion engine.

[1481] Examples

[1482] When a user types "I want to buy coffee" into their smartphone and shows their face to the camera, the system recognizes the user's emotion as "sadness" and plays an audible message saying, "Are you okay? Enjoy your coffee."

[1483] Examples of prompt statements

[1484] When a user types "I want to buy coffee" and shows their face to the camera, the system recognizes the user's emotion and displays an appropriate message: if the emotion is "sadness," it displays "Are you OK? Enjoy your coffee."

[1485] In this way, it is possible to provide products that meet the user's needs and provide flexible customer service based on emotions.

[1486] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1487] Step 1:

[1488] A user inputs a request such as "I want to buy coffee" using a smartphone or tablet. The input request is sent to the server as text data.

[1489] Step 2:

[1490] The server uses a generative AI model to analyze the user's request. The input text data is input as a prompt into the generative AI model, and data is generated as the analysis result to send instructions to the vending machine. The generated data is sent to the vending machine API.

[1491] Step 3:

[1492] The server receives the user's face image captured by the camera. The received image data is converted to grayscale using OpenCV and resized to 48x48. The resized image data is input to the emotion engine.

[1493] Step 4:

[1494] The server recognizes the user's emotion using an emotion engine. The resized image data is input to the emotion engine, and an emotion label is output as an emotion recognition result. The emotion label is data indicating the user's emotion.

[1495] Step 5:

[1496] The server generates an appropriate message based on the emotion recognition result. If the emotion label is "sadness", the server uses a Text-to-Speech library to generate a message such as "Are you okay? Enjoy your coffee" as voice data. The generated voice data is sent to the user device.

[1497] Step 6:

[1498] The vending machine dispenses products based on instructions received from the server, analysing the instruction data and performing operations to dispense the product requested by the user (coffee in this case).

[1499] Step 7:

[1500] The user terminal plays the voice data received from the server. By playing the voice data, a message is provided to the user saying, "Are you OK? Enjoy your coffee."

[1501] In this way, it is possible to provide products that meet the user's needs and to provide flexible customer service based on the user's emotions.

[1502] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1503] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1504] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1505] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1506] [Third embodiment]

[1507] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1508] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1509] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1510] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1511] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1512] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1513] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1514] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1515] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1516] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1517] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1518] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1519] "Example 1"

[1520] One embodiment of the present invention is an application that uses generative AI. This application generates an appropriate prompt in response to an input from a user, and calls the API of another service based on the prompt. For example, when a user inputs "I would like to order pizza," the application generates a prompt "What kind of pizza would you like to order?" and calls an API that places a pizza order based on the user's response.

[1521] "Example 2"

[1522] Furthermore, as one embodiment of the present invention, there is a link with a cash register app. An application using generative AI links with a cash register app to automate cash register operations that involve customer service. For example, to automate ordering at a restaurant, the order details from the user are sent to the cash register app, and the cash register app rings up the cash based on the order details.

[1523] "Example 3"

[1524] As one embodiment of the present invention, there is a vending machine that serves customers. An application using generative AI works with the vending machine and provides products in response to requests from users. For example, when a user inputs "I want to buy coffee," the application sends an instruction to the vending machine, and the vending machine provides coffee.

[1525] The process flow of each embodiment will be described below.

[1526] "Example 1"

[1527] Step 1: Receive input from the user. For example, "I want to order pizza."

[1528] Step 2: Use generative AI to generate an appropriate prompt, for example, "What kind of pizza would you like to order?"

[1529] Step 3: Call the API of another service based on the user's answer. For example, if the user selects "Margherita," call the API to order a Margherita.

[1530] "Example 2"

[1531] Step 1: Receive an order from a user. For example, receive an "order for a hamburger" from a restaurant.

[1532] Step 2: Use generative AI to send the order details to the cash register app.

[1533] Step 3: The cashier app processes the order based on the details of the order, for example calculating the price of a burger and generating a receipt.

[1534] "Example 3"

[1535] Step 1: Receive a request from a user. For example, "I want to buy coffee."

[1536] Step 2: Use generative AI to send instructions to the vending machine, for example to dispense coffee.

[1537] Step 3: The vending machine dispenses the product as instructed, e.g., coffee.

[1538] Example 1

[1539] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1540] In conventional systems, when a user uses a specific service, the process of generating appropriate prompts and calling APIs of other services based on the user's input is often done manually, which is inefficient. In addition, it is difficult to generate appropriate prompts in response to user input, which leads to a poor user experience. Furthermore, there is a demand for automating cash register operations and vending machine customer service in restaurants and other establishments, but this is difficult to achieve with current systems.

[1541] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative artificial intelligence, a means for receiving input from a user, a means for generating a prompt using a generative artificial intelligence model, a means for presenting the generated prompt to the user, a means for receiving a response from the user, a means for calling an application programming interface of another service, a means for presenting a response of the application programming interface to the user, and a means for linking with a cash register application. This makes it possible to automatically generate an appropriate prompt based on a user's input, to streamline the process of calling the API of another service, and to improve the user experience. In addition, it is possible to automate cash register operations that involve customer service in restaurants and the like, and customer service at vending machines.

[1542] "Generative artificial intelligence" is an artificial intelligence technique for generating appropriate prompts and responses based on user input.

[1543] An "application" is a software program designed to perform a specific task or function.

[1544] "Means for receiving input from a user" refers to a function that provides an interface for a user to enter input in the form of text, voice, or the like.

[1545] A "generative artificial intelligence model" is an algorithm or data model for generating prompts and responses based on user input.

[1546] A "prompt" is a question or instruction that prompts the user for their next action or input.

[1547] An "application programming interface" is an interface that allows different software systems to communicate with each other.

[1548] A "cash register application" is software for managing transactions related to the sale of goods or the provision of services.

[1549] "User experience" is a concept that refers to the overall satisfaction and ease of use when a user uses a particular system or service.

[1550] A "vending machine" is an automated machine that enables a user to purchase goods.

[1551] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1552] The present invention is a system for realizing an application using generative artificial intelligence. A specific embodiment of this system will be described below.

[1553] System Configuration

[1554] The system is composed of three main elements: a server, a terminal, and a user. The server generates prompts using a generative artificial intelligence model and calls the application programming interfaces (APIs) of other services based on user input. The terminal provides an interface for users to enter input and communicates with the server. Users access the system through their terminals and input the required information.

[1555] Hardware and Software Used

[1556] The server is a computer system equipped with a high-performance processor and sufficient memory. A generative artificial intelligence model (e.g., OpenAI's GPT-3) is installed on the server. The terminals are devices such as smartphones, tablets, or personal computers, and access the server through a web browser or a dedicated application. Users use these terminals to provide input to the system.

[1557] Data processing and calculation

[1558] The server receives input from the user, passes the input to a generative artificial intelligence model to generate an appropriate prompt, which is then presented to the user again and receives a response from the user. The server then calls the appropriate API based on the user's response and presents the response to the user.

[1559] As a concrete example, the following code shows what happens when a user enters "I would like to order pizza."

[1560] 1. A user accesses the application using a terminal and types, "I want to order a pizza."

[1561] 2. The server uses a generative artificial intelligence model (GPT-3) to generate a prompt: “What kind of pizza would you like to order?”

[1562] 3. The server presents this prompt to the user.

[1563] 4. The user answers "Margherita."

[1564] 5. The server calls the API for ordering pizza and sends the user's order details.

[1565] 6. The server receives the response from the API and displays a message to the user saying, "Your order has been completed. Your order will be delivered within 30 minutes."

[1566] Example prompt:

[1567] User input: "I want to order pizza."

[1568] Generated prompt: "What kind of pizza would you like to order?"

[1569] In this way, the server, terminals, and users can work together to efficiently fulfill user requests. This system improves the user experience and makes it possible to automate cash register operations and vending machine operations that involve customer service in restaurants and other establishments.

[1570] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1571] Step 1:

[1572] A user accesses the application using a terminal, types "I would like to order pizza" in the text box, and clicks the submit button.

[1573] Input: User text input "I want to order pizza"

[1574] Output: User input data is sent to the server

[1575] Step 2:

[1576] The server receives the HTTP POST request and extracts the user input text from the request body.

[1577] Input: User input data

[1578] Output: Extracted text data "I want to order pizza"

[1579] Step 3:

[1580] The server passes the extracted text data to a generative artificial intelligence model (e.g., GPT-3) to generate an appropriate prompt.

[1581] Input: Extracted text data "I want to order pizza"

[1582] Data processing: A generative artificial intelligence model analyzes the text data and generates appropriate prompts.

[1583] Output: Generated prompt "What kind of pizza would you like to order?"

[1584] Step 4:

[1585] The server sends the generated prompt to the user's terminal as an HTTP response, and displays it on the user's screen.

[1586] Input: Generated prompt "What kind of pizza would you like to order?"

[1587] Output: The prompt that is displayed on the user's terminal.

[1588] Step 5:

[1589] The user responds with "Margherita" to the displayed prompt and clicks the submit button again.

[1590] Input: User text input "Margherita"

[1591] Output: User answer data is sent to the server

[1592] Step 6:

[1593] The server again receives the HTTP POST request and extracts the user's answer from the request body.

[1594] Input: User response data

[1595] Output: Extracted text data "Margherita"

[1596] Step 7:

[1597] The server calls the appropriate API (e.g., a pizza ordering API) based on the user's answers and submits the user's order.

[1598] Input: Extracted text data "Margherita"

[1599] Data processing: generating and sending API requests

[1600] Output: An order request to the API.

[1601] Step 8:

[1602] The server receives the response from the API and sends the response to the user's device as an HTTP response. The user's screen displays the message, "Your order has been completed. Your order will be delivered within 30 minutes."

[1603] Input: Response data from the API

[1604] Output: A message displayed on the user's device: "Your order is complete. Your order will be delivered in 30 minutes."

[1605] In this way, the server, the terminal, and the user can work together to efficiently fulfill the user's requests.

[1606] (Application example 1)

[1607] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1608] Conventional food delivery systems require a lot of effort for users to place an order, and the operation is complicated, especially for first-time users. In addition, there is a lack of suggestions that take into account the user's preferences and past order history, making it difficult to improve the user experience. Furthermore, there is also an issue that it is not possible to generate appropriate prompts when placing an order, which means that it takes a long time for users to enter the necessary information.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1610] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to a purpose, a means for calling an API of another service, a means for generating an appropriate prompt based on a user's input, a means for making optimal suggestions based on the user's preferences and past order history, and a means for completing an order based on the suggestions. This allows the user to easily place an order, and makes suggestions that take into account the user's preferences and past order history, improving the user experience. In addition, by generating an appropriate prompt, the user can quickly input the necessary information.

[1611] "Generative AI" is an artificial intelligence technique that generates appropriate responses or prompts based on user input.

[1612] An "application role-playing vehicle" is software or a system designed to provide a specific function or service.

[1613] A "supplementary prompt" is a feature that provides additional information or questions in response to the user's input.

[1614] "Means of calling the API of other services" refers to a function that uses an API (Application Programming Interface) to connect with external services or systems.

[1615] "A means for generating appropriate prompts based on user input" is a function that generates questions or instructions to elicit the next required information based on the information entered by the user.

[1616] "A means of making optimal suggestions based on a user's preferences and past order history" is a function that analyzes a user's past behavior and preferences and suggests optimal products and services based on that.

[1617] The "means for completing an order based on a suggestion" is a function for actually confirming an order and completing the process based on a suggestion selected by a user.

[1618] The following system is conceivable as a mode for implementing this invention. The system includes means for fulfilling the role of an application using generative AI, means for supplementing prompts according to the purpose, means for calling the API of other services, means for generating appropriate prompts based on user input, means for making optimal suggestions based on the user's preferences and past order history, and means for completing the order based on the suggestions.

[1619] Hardware and Software Used

[1620] Hardware: Smartphone

[1621] Software: OpenAI API, food delivery service API, requests library

[1622] Data processing and calculation

[1623] 1. User input: The user inputs the food they want into the smartphone app.

[1624] 2. Prompt Generation: The server uses the OpenAI API to generate an appropriate prompt based on the user's input.

[1625] 3. User Answers: The user answers the generated prompt.

[1626] 4. Generate order details: The server generates the order details including the user's answers.

[1627] 5. API call: The server calls the food delivery service's API and completes the order.

[1628] Examples

[1629] When a user types "I'd like to order pizza," the server uses the OpenAI API to generate a prompt: "What kind of pizza would you like to order?" If the user responds with "Margherita," the server calls the food delivery service's API to complete the order.

[1630] Examples of prompt statements

[1631] The user types, "I want to order pizza." Generate an appropriate prompt.

[1632] In this way, we have realized a system that allows users to easily place orders by generating appropriate prompts based on user input and calling the API of the food delivery service. This improves the user experience by making suggestions that take into account the user's preferences and past order history. In addition, generating appropriate prompts allows users to quickly enter the necessary information.

[1633] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1634] Step 1:

[1635] The user inputs the food they want to eat into a smartphone app.

[1636] Input: The user types, "I would like to order pizza."

[1637] Output: The user's input data.

[1638] Specific behavior: The user enters "I would like to order pizza" into the app's text input field and presses the submit button.

[1639] Step 2:

[1640] The server uses the OpenAI API to generate appropriate prompts based on the user's input.

[1641] Input: User input data: "I want to order pizza."

[1642] Output: Generated prompt "What kind of pizza would you like to order?".

[1643] Specific behavior: The server sends the user's input data to the OpenAI API and receives the prompt returned by the API.

[1644] Step 3:

[1645] The user answers the generated prompts.

[1646] Input: Generated prompt "What kind of pizza would you like to order?"

[1647] Output: The user's answer, "Margherita".

[1648] Specific behavior: The user types "Margherita" into the app's text input field and presses the send button.

[1649] Step 4:

[1650] The server generates order details including the user's answers.

[1651] Input: The user's answer, "Margherita".

[1652] Output: Order details (e.g. { "Cuisine": "Pizza", "Type": "Margherita"}).

[1653] What happens: The server builds order details based on the user's answers.

[1654] Step 5:

[1655] The server calls the food delivery service's API and completes the order.

[1656] Input: Order details data.

[1657] Output: Order completed response from the food delivery service.

[1658] What happens: The server sends the order details to the food delivery service's API and receives a response indicating the order was successfully completed.

[1659] In this way, we have created a system that allows users to easily place orders by generating appropriate prompts based on user input and calling the API of a food delivery service.

[1660] Example 2

[1661] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1662] Conventional cash register systems required the user to input an order and have it reflected on the register manually, which was time-consuming and labor-intensive. There was also the possibility of incorrect input of order details and delays in processing, making efficient operation difficult. Furthermore, in tasks involving customer service, the burden on staff was heavy, and there was a risk of the quality of service declining.

[1663] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1664] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling the API of other services, a means for receiving order details entered by a user, a means for analyzing the received order details using a generative AI model, a means for sending the analysis results to a cash register app, and a means for the cash register app to ring up the order based on the analysis results. This automates the process from ordering to ringing up the order, enabling efficient and accurate order processing.

[1665] "Generative AI" is a type of artificial intelligence that uses natural language processing and machine learning techniques to analyze user input and generate appropriate responses and data.

[1666] A "prompt" is an instruction or question entered into a generative AI to elicit a particular response or action.

[1667] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[1668] A "cash register app" is application software used to calculate fees associated with the sale of goods or the provision of services, process payments, issue receipts, and so on.

[1669] A "User" is a person who utilizes the system to enter an order.

[1670] "Order details" refers to detailed information about the product or service that the user wishes to purchase.

[1671] "Analysis" refers to the process in which the generative AI understands the user's order and extracts specific items and information.

[1672] "Structured data" is data that is organized according to a particular format or structure, such as JSON.

[1673] "Cash registering" refers to the task of calculating and processing payments associated with the sale of goods or the provision of services.

[1674] "Automation" refers to enabling systems or machines to operate automatically, without human intervention.

[1675] This invention is a system that automates the process from ordering to ringing up a cash register by linking a cash register app with an application that uses generative AI. A specific embodiment of this system is described below.

[1676] System Configuration

[1677] server

[1678] The server is responsible for analyzing the order contents from users using a generative AI model. Specifically, the server uses the following hardware and software:

[1679] Hardware: A server machine with a powerful processor, sufficient memory, and a fast network connection.

[1680] Software: Generative AI models (e.g., GPT-4), natural language processing libraries, libraries for calling APIs

[1681] The server receives the order details sent by the user and analyzes them using a generative AI model. The analysis results are generated as structured data in JSON format and sent to the cash register app.

[1682] Terminal

[1683] A terminal is a device that allows a user to input an order. Specifically, this includes a smartphone or tablet. An order application is installed on the terminal, and the terminal has the function of allowing a user to input order details and send them to a server.

[1684] User

[1685] A user is a person who uses the system to place an order. The user uses a terminal to open an order application and inputs the order details. For example, the user inputs "I'd like to order a hamburger and a coke."

[1686] Examples

[1687] Example 1: Ordering at a restaurant

[1688] A user enters "I'd like to order a hamburger and a coke" into an application at a restaurant. The application sends this order to a server. The server uses a generative AI model (e.g. GPT-4) to analyze the order and extracts the specific items "hamburger" and "coke." This information is sent to a cash register app, which automatically rings up the order, calculates the charge, and issues a receipt.

[1689] Examples of prompt statements

[1690] "When a user places an order at a restaurant, they use an application to input the order details. When the user types, 'I'd like a hamburger and a coke,' the application sends this information to a server. The server uses a generative AI model (e.g. GPT-4) to analyze the order and extract the specific items, 'hamburger' and 'coke.' This information is then sent to a cash register app, which automatically records the order, calculates the charge, and issues a receipt."

[1691] In this way, the server, terminal, and user all play their respective roles in the system, automating the process from ordering to ringing up the cash register. This system improves the efficiency and accuracy of order processing.

[1692] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1693] Step 1:

[1694] The user enters an order

[1695] The user opens an ordering application installed on a smartphone or tablet. He / she enters "I would like to order a hamburger and a cola" into the application's input form and presses the send button. The input data is in text format and includes the order details.

[1696] Step 2:

[1697] The terminal sends the order details to the server

[1698] The terminal sends the order details entered by the user to the server as text data. At this time, the data is sent using an HTTP POST request. The input is the text data entered by the user, and the output is the order details sent to the server.

[1699] Step 3:

[1700] The server receives the order.

[1701] The server receives the order details sent from the terminal. The received data is in text format and contains the order details. The input is the text data sent from the terminal, and the output is the received order details.

[1702] Step 4:

[1703] The server parses the order

[1704] The server inputs the received order details into a generative AI model (e.g., GPT-4). The generative AI model analyzes the order details and extracts the specific items "hamburger" and "cola." The input is the text data of the received order details, and the output is structured data (e.g., JSON format) as the analysis result.

[1705] Step 5:

[1706] The server sends the analysis results to the cash register app.

[1707] The server sends the JSON data obtained as a result of the analysis to the cash register app using an HTTP POST request. The input is the JSON data of the analysis result, and the output is the data sent to the cash register app.

[1708] Step 6:

[1709] The cash register app processes the order.

[1710] The cash register app analyzes the JSON data received from the server and calculates the charge based on the order details. For example, if a "hamburger" costs 500 yen and a "cola" costs 200 yen, the total amount is 700 yen. The cash register app displays this amount and issues a receipt. The input is the JSON data received from the server, and the output is the calculated charge and the issued receipt.

[1711] In this way, the specific actions and data flow are clearly defined at each step, automating the process from ordering to ringing up the cash register.

[1712] (Application example 2)

[1713] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1714] The traditional ordering and cash register process in restaurants is often done manually, which is time-consuming and labor-intensive. There is also a high risk of ordering mistakes and cash register errors. This can lead to lower customer satisfaction and poor business efficiency. Furthermore, there is concern that the burden on staff increases during peak times, leading to a decline in the quality of service.

[1715] The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register app, a means for analyzing the contents of a user's order, a means for sending the analysis results to the cash register app, and a means for prompting the user to make payment. This automates the ordering and cash register process, making it possible to improve business efficiency and customer satisfaction.

[1716] "Generative AI" is a type of artificial intelligence that generates new data and information based on user input.

[1717] An "application" is a software program designed to accomplish a particular purpose.

[1718] A "prompt" is an instruction or question that prompts the user for input.

[1719] "API" stands for Application Program Interface, an interface for exchanging data and functions between different software systems.

[1720] A "cash register app" is a software application used to operate the cash register and process payments.

[1721] A "user" is someone who uses a system or application.

[1722] "Order Details" refers to detailed information about the products or services a user wishes to purchase or use.

[1723] "Analysis" is the process of examining data or information in detail to understand its meaning and structure.

[1724] "Transmission" is the act of sending data or information from one place to another.

[1725] "Payment" is the act of giving money in exchange for goods or services.

[1726] As an embodiment of the present invention, we provide a system that automates the ordering and cash register process in a restaurant. The system includes an application that uses generative AI, a means for supplementing prompts, a means for calling APIs of other services, a means for linking with a cash register app, a means for analyzing the contents of a user's order, a means for sending the analysis results to the cash register app, and a means for prompting the user to make payment.

[1727] Hardware and Software Used

[1728] Hardware: Smartphone

[1729] Software: Python, generative AI API, cash register app API

[1730] System Operation

[1731] 1. Your order:

[1732] A user uses a smartphone app to select an item from a menu. For example, the user selects "hamburger."

[1733] 2. Order Analysis:

[1734] The smartphone app sends the selected order details to the generative AI API and obtains the analysis results. The generative AI understands the order details and converts them into an appropriate format.

[1735] 3. Send to the cash register app:

[1736] The analysis results are sent to the cash register app API, which then rings up the order based on the received order details.

[1737] 4. Payment prompts:

[1738] Based on the response from the cash register app, the smartphone app prompts the user to make the payment, and the user completes the payment using their smartphone.

[1739] Examples

[1740] For example, if a user selects "hamburger" on a smartphone app, the generative AI analyzes the order and sends an order for "one hamburger" to the cash register app. The cash register app receives the order, records it, and prompts the user to pay.

[1741] Examples of prompt statements

[1742] A user selects "Hamburger" on a smartphone app. The generative AI analyzes this order and sends it to a cash register app. The cash register app receives the order, records it, and prompts the user to pay.

[1743] In this way, automating the ordering and cashiering process can improve operational efficiency and customer satisfaction.

[1744] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1745] Step 1:

[1746] A user launches a smartphone app and selects a product from the menu. The input is the product selected by the user (e.g., "hamburger"), and the output is information about the selected product. In concrete terms, the selection is completed when the user taps the product on the app's menu screen.

[1747] Step 2:

[1748] The terminal sends the selected product information to the generative AI API. The input is the selected product information, and the output is the analysis result by the generative AI. Specifically, the terminal sends the product information in JSON format to the generative AI API as a POST request.

[1749] Step 3:

[1750] The server receives the analysis results from the generative AI API. The input is the response data from the generative AI API, and the output is the analyzed order details. Specifically, the server parses the API response and extracts the order details.

[1751] Step 4:

[1752] The server sends the parsed order details to the cash register app API. The input is the parsed order details, and the output is the response from the cash register app API. Specifically, the server sends the order details in JSON format to the cash register app API as a POST request.

[1753] Step 5:

[1754] The cash register application will perform cash register operations based on the order details. The input is the order details sent from the server, and the output is the cash register completion status. In concrete terms, the cash register application will receive the order details, record them in an internal database, and generate a receipt.

[1755] Step 6:

[1756] The cash register app returns the completion status of the cash register to the server. The input is the completion status of the cash register, and the output is the response data to the server. In concrete terms, the cash register app sends the completion status to the server in JSON format.

[1757] Step 7:

[1758] The server receives the completion status of the cash register and sends a message to the terminal to prompt payment. The input is the completion status of the cash register and the output is a message to prompt payment. In concrete terms, the server checks the completion status, generates a payment message and sends it to the terminal.

[1759] Step 8:

[1760] The terminal displays a message prompting the user to make a payment. The input is a payment message from the server, and the output is the display to the user. As a specific operation, the terminal displays the payment message to the user in a pop-up format.

[1761] Example 3

[1762] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1763] Conventional vending machines and cash register systems have difficulty responding flexibly to user requests, and the automation of specific product provision and services has been limited. In addition, there has been a lack of systems that can accurately understand user requests and provide appropriate products based on those requests. This has led to a demand for improved user experience and operational efficiency.

[1764] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1765] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for analyzing user requests, a means for sending instructions to the vending machine based on the analysis results, and a means for the vending machine to provide products. This makes it possible to flexibly and accurately provide products in response to user requests.

[1766] "Generative AI" is an artificial intelligence technology that analyzes user input and generates appropriate responses or instructions through natural language processing.

[1767] An "application" is a software program designed to accomplish a particular purpose.

[1768] A "prompt" is text data or instructions input to a generative AI.

[1769] An "API" is an interface for exchanging functions and data between different software systems.

[1770] A "user request" is a specific request or instruction given by a user to the system.

[1771] "Analysis" is the process of understanding and extracting meaning from input data.

[1772] An "instruction" is a command to perform a particular action or process.

[1773] A "vending machine" is a mechanical device that automatically dispenses merchandise selected by a user.

[1774] "Product" refers to any goods or services offered by a vending machine.

[1775] This invention is a system in which a vending machine provides products in response to a user's request by utilizing an application using generative AI. A specific embodiment of this system will be described below.

[1776] First, a user uses a terminal such as a smartphone or tablet to input a request to the vending machine. For example, the user inputs, "I would like to buy coffee." This request is then sent from the terminal to the server.

[1777] The server uses a generative AI model (e.g., OpenAI's GPT-4) to analyze the user's request. The generative AI model uses natural language processing techniques to understand the user's request and generate appropriate instructions. These instructions are converted into a format that is sent to the vending machine.

[1778] Specifically, the server inputs the following prompt sentence into the generative AI model:

[1779] The user types, "I want to buy coffee." Generate an instruction for the vending machine to provide the coffee.

[1780] The generative AI model would parse this prompt and generate instructions like this:

[1781] Sends a command to the vending machine to dispense coffee.

[1782] The server transmits the instruction to the vending machine. The vending machine operates based on the received instruction and operates the internal machinery to provide coffee. The user can take out the coffee from the receiving slot of the vending machine.

[1783] This system uses a generative AI model to flexibly and accurately respond to user requests. In addition, by having the server have the means to call the APIs of other services, it becomes possible to provide a wider variety of services and products.

[1784] In this way, a system is realized in which an application using generative AI operates a vending machine in response to a user request and provides a product. The flow of the specific process in the third embodiment will be described with reference to FIG.

[1785] Step 1:

[1786] The user enters a request

[1787] A user inputs a request to a vending machine using a terminal such as a smartphone or tablet. For example, the user inputs "I want to buy coffee." This input is saved as text data in the terminal. The input data is the text "I want to buy coffee."

[1788] Step 2:

[1789] The device sends a request to the server

[1790] The terminal transmits the request input by the user to the server via the Internet. At this time, the request is transmitted as text data. The input data is the text "I want to buy coffee," and the output data is the text data transmitted to the server.

[1791] Step 3:

[1792] The server parses the request

[1793] The server inputs the received request into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model processes the request in natural language and generates appropriate instructions. The input data is the text "I want to buy coffee," and the output data is the instruction "Send an instruction to the vending machine to dispense coffee."

[1794] Step 4:

[1795] The server sends instructions to the vending machine

[1796] The server sends the generated instruction to the vending machine. This instruction is converted into a format that the vending machine can understand. For example, it is sent in a specific command format (such as JSON format). The input data is the instruction "Send an instruction to the vending machine to serve coffee", and the output data is the command sent to the vending machine.

[1797] Step 5:

[1798] Vending machines provide products

[1799] The vending machine operates based on instructions received from the server, specifically by operating an internal machine to dispense coffee. A user can dispense the coffee from a receiving slot of the vending machine. The input data are the commands sent to the vending machine, and the output data is the dispensed coffee.

[1800] (Application example 3)

[1801] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1802] Conventional vending machines and cash register systems required users to operate them manually when purchasing products, which resulted in low convenience. In addition, in stores that involved customer service, the burden on staff was heavy and efficient operation was difficult. Furthermore, there was a lack of systems that could flexibly respond to user requests, and there was a need to improve customer satisfaction.

[1803] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for performing the role of an application using a generative AI, a means for supplementing a prompt according to a purpose, a means for calling an API of another service, a means for linking with a cash register application, a means for sending instructions to a vending machine that provides products in response to a user's request, a means for generating instructions using a generative AI model, and a means for providing a user interface using a smartphone. This not only allows users to easily purchase products using a smartphone, but also reduces the burden on store staff and enables efficient store operation. In addition, the use of a generative AI model makes it possible to flexibly respond to user requests and improve customer satisfaction.

[1804] "Generative AI" is an artificial intelligence technology that generates natural language and instructions based on user input.

[1805] A "means for fulfilling the role of an application" is software or a program for providing a specific function or service.

[1806] A "means for supplementing prompts" is a function that complements the user's input and generates appropriate instructions or questions.

[1807] "Means of calling the API of other services" refers to a function that uses an API (Application Programming Interface) to connect with external services or systems.

[1808] "Means of linking with a cash register app" refers to a function that links with the store's cash register system to handle payments and inventory management.

[1809] The "means for sending instructions to a vending machine that provides a product in response to a user's request" is a function for sending instructions to the vending machine to provide a product based on the user's purchase request.

[1810] "Means for generating instructions using a generative AI model" refers to a function that uses a generative AI model to generate specific instructions in response to a user's request.

[1811] "Means for providing a user interface using a smartphone" refers to a function for providing an interface that can be operated by a user using a smartphone.

[1812] A system for implementing this invention includes means for performing the role of an application using generative AI, means for supplementing prompts according to the purpose, means for calling APIs of other services, means for linking with a cash register app, means for sending instructions to a vending machine which provides products in response to a user request, means for generating instructions using a generative AI model, and means for providing a user interface using a smartphone.

[1813] The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate natural language and instructions based on user input. When a user inputs on their smartphone that they would like to purchase an item, the server receives the input and sends a prompt to the generative AI model. An example of a prompt sentence is, "The user has input that they would like to purchase a coffee. Please send instructions to the vending machine."

[1814] The generative AI model generates appropriate instructions based on the prompts and returns the instructions to the server. The server receives the generated instructions and calls the APIs of other services to send the instructions to the vending machine. The vending machine follows the instructions and provides the product to the user.

[1815] As a concrete example, a user opens a smartphone app and types, "I want to buy coffee." The server receives this input and sends the following prompt to the generative AI model:

[1816] "The user has indicated that they would like to purchase coffee. Please send instructions to the vending machine."

[1817] Based on this prompt, the generative AI model generates the instruction, "Send an instruction to the vending machine to serve coffee." The server receives this instruction and calls the vending machine's API to send the instruction. The vending machine follows this instruction and serves coffee to the user.

[1818] This system not only allows users to easily purchase products using their smartphones, but also reduces the burden on store staff and enables efficient store operations. In addition, the use of generative AI models allows for flexible response to user requests, improving customer satisfaction.

[1819] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1820] Step 1:

[1821] A user starts the application on a smartphone terminal and inputs the product he or she wants to purchase. The input product name is "coffee."

[1822] Step 2:

[1823] The terminal transmits the user's input to the server. The input data is text data for "coffee."

[1824] Step 3:

[1825] The server sends a prompt to the generative AI model based on the received user input data. The prompt text is "The user has input that he / she would like to buy coffee. Please send instructions to the vending machine."

[1826] Step 4:

[1827] The generative AI model analyzes the prompt and generates the appropriate instruction, which is "Send an instruction to the vending machine to serve coffee."

[1828] Step 5:

[1829] The server analyzes the instructions received from the generative AI model and generates data to call the vending machine's API. The input data are the generated instructions, and the output data are the data for the API call.

[1830] Step 6:

[1831] The server calls the vending machine's API and sends the generated data. As a result of the API call, the vending machine dispenses the coffee.

[1832] Step 7:

[1833] The vending machine provides coffee to the user based on instructions received from the server. The product provided is "coffee."

[1834] Step 8:

[1835] The server notifies the user that the product has been served. The content of the notification is "Coffee has been served."

[1836] Step 9:

[1837] The terminal receives the notification from the server and displays it to the user, saying "Coffee has been served."

[1838] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1839] "Example 1"

[1840] One embodiment of the present invention is a system that includes an emotion engine that recognizes a user's emotions. The system receives input from a user and generates an appropriate prompt using a generative AI. It also calls APIs of other services. The emotion engine then recognizes the user's emotions and supplements the prompt according to the emotions. For example, if a user inputs "I would like to order pizza" and the emotion engine recognizes the user's emotion as "joy," the generative AI generates a prompt such as "Congratulations! What kind of pizza would you like to order?"

[1841] "Example 2"

[1842] Another embodiment of the present invention is a system that automates cash register operations involving customer service based on the user's emotions recognized by an emotion engine. This system receives the order details from the user and sends the order details to a cash register app using generative AI. The emotion engine then recognizes the user's emotions and automates cash register operations according to those emotions. For example, if a user "orders a hamburger" and the emotion engine recognizes the user's emotion as "anger," the cash register app displays a message such as "Sorry. We will quickly prepare the hamburger."

[1843] "Example 3"

[1844] As yet another embodiment of the present invention, there is a system for realizing a vending machine that serves customers based on the user's emotions recognized by an emotion engine. This system receives requests from a user and sends instructions to the vending machine using a generative AI. The emotion engine then recognizes the user's emotions, and the vending machine provides products according to those emotions. For example, if a user requests "I want to buy coffee" and the emotion engine recognizes the user's emotion as "sadness," the vending machine will display a message such as "Are you okay? Enjoy your coffee."

[1845] The process flow of each embodiment will be described below.

[1846] "Example 1"

[1847] Step 1: Receive input from the user.

[1848] Step 2: Use generative AI to generate appropriate prompts.

[1849] Step 3: Call the API of other services.

[1850] Step 4: Recognize the user's emotions using the emotion engine.

[1851] Step 5: Complement the prompt with the emotion you identify.

[1852] "Example 2"

[1853] Step 1: Receive the order from the user.

[1854] Step 2: Use generative AI to send the order details to the cash register app.

[1855] Step 3: Recognize the user's emotions using the emotion engine.

[1856] Step 4: Automate cashiering based on the recognized emotions.

[1857] "Example 3"

[1858] Step 1: Receive a request from a user.

[1859] Step 2: Use generative AI to send instructions to the vending machine.

[1860] Step 3: Recognize the user's emotions using the emotion engine.

[1861] Step 4: The vending machine will dispense a product based on the emotion it recognizes.

[1862] Example 1

[1863] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1864] Conventional systems have problems in that it is difficult to generate appropriate prompts in response to user input, and they are unable to respond in a way that takes into account the user's emotions. Furthermore, when calling APIs for other services, flexible responses based on the user's emotions are required, but there is a lack of means to achieve this.

[1865] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1866] In this invention, the server includes a means for receiving input from a user, a means for generating an appropriate prompt sentence using a generative AI model, a means for recognizing the user's emotion, a means for supplementing the prompt sentence based on the recognized emotion, and a means for calling an API of another service. This makes it possible to generate an appropriate prompt for the user's input and to flexibly respond according to the emotion.

[1867] The "means for receiving input from a user" is a function that provides an interface for a user to input information such as text or voice to the system through a terminal.

[1868] "Means for generating appropriate prompt sentences using a generative AI model" refers to a function that automatically creates prompt sentences containing appropriate questions or instructions using a generative AI model based on user input.

[1869] "Means for recognizing user emotions" refers to a function that uses an algorithm or engine to analyze emotions from the user's input or voice and identify those emotions.

[1870] The "means for supplementing the prompt sentence based on the recognized emotion" is a function that takes the user's emotion into account and adds additional information or an expression according to the emotion to the generated prompt sentence.

[1871] The "means for calling the API of other services" refers to a function for calling the API of external services or systems based on the generated prompt text or the user's response, and performing the required operations or data acquisition.

[1872] "Means for linking with a cash register app" refers to a function for communicating with a cash register application and sending and receiving data and linking operations.

[1873] This invention is configured mainly with an application using a generative AI model. A specific embodiment of this system will be described below.

[1874] System Configuration

[1875] The system mainly uses the following hardware and software:

[1876] Hardware: Devices used by users (smartphones, tablets, PCs, etc.), servers

[1877] Software: Generative AI models (e.g. GPT-3), emotion recognition engines (e.g. emotion analysis API), API call functions, cash register applications

[1878] Program Processing

[1879] 1. Receiving user input

[1880] A user inputs text or voice to the system through a terminal. For example, the user inputs "I would like to order pizza."

[1881] 2. Prompt Generation

[1882] The server uses a generative AI model (e.g., GPT-3) to generate an appropriate prompt for the user's input, such as "What kind of pizza would you like to order?"

[1883] 3. Emotion recognition

[1884] The server uses an emotion recognition engine (e.g., emotion analysis API) to recognize emotions from the user's input. For example, it recognizes the emotion "joy" from the user's input "I want to order pizza."

[1885] 4. Prompt Supplement

[1886] The server supplements the generated prompt sentence based on the recognized emotion, i.e., "Congratulations! What kind of pizza would you like to order?"

[1887] 5. API Calls

[1888] When the user answers the prompts, the server calls APIs of other services based on the answer. For example, if the user answers "Margherita," the server calls a pizza ordering API to order a Margherita pizza.

[1889] Examples

[1890] Receiving user input

[1891] The user enters "I would like to order pizza" into the terminal and presses the send button.

[1892] Prompt Generation

[1893] The server uses a generative AI model (e.g. GPT-3) to generate a prompt such as "What kind of pizza would you like to order?"

[1894] emotion recognition

[1895] The server calls an emotion recognition engine (e.g., emotion analysis API) and recognizes the emotion “joy” from the user’s input “I want to order pizza.”

[1896] Prompt Supplement

[1897] The server again uses the generative AI model to generate a follow-up prompt: "Congratulations! What kind of pizza would you like to order?"

[1898] API Call

[1899] The user responds with "Margherita," and the server calls the pizza ordering API to order a Margherita pizza.

[1900] In this way, a system is realized that generates appropriate prompts in response to user input and responds flexibly according to emotions.

[1901] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1902] Step 1:

[1903] Receiving user input

[1904] Specific action: The user enters "I would like to order pizza" into the input field of the terminal and presses the send button.

[1905] Input: User text input "I want to order pizza".

[1906] Output: The user's input data is sent to the server.

[1907] Step 2:

[1908] Prompt Generation

[1909] What it does: The server uses a generative AI model (e.g. GPT-3) to generate an appropriate prompt in response to the user's input.

[1910] Input: User input data: "I want to order pizza."

[1911] Data processing: The generative AI model analyzes the user's input and generates an appropriate prompt.

[1912] Output: The generated prompt sentence: "What kind of pizza would you like to order?".

[1913] Step 3:

[1914] emotion recognition

[1915] Specific operation: The server uses an emotion recognition engine (e.g., emotion analysis API) to recognize emotions from the user's input.

[1916] Input: User input data: "I want to order pizza."

[1917] Data processing: An emotion recognition engine analyzes user input and identifies emotions.

[1918] Output: The perceived emotion "joy".

[1919] Step 4:

[1920] Prompt Supplement

[1921] Specific behavior: The server supplements the generated prompt sentence based on the emotion recognized by the server.

[1922] Input: The generated prompt sentence "What kind of pizza would you like to order?" and the recognized emotion "Joy".

[1923] Data processing: The generative AI model adds supplementary sentences based on the sentiment.

[1924] Output: The augmented prompt, "Congratulations! What kind of pizza would you like to order?".

[1925] Step 5:

[1926] API Call

[1927] Specific behavior: The user answers prompts, and the server calls the API of other services based on the answers.

[1928] Input: The user's answer, "Margherita".

[1929] Data processing: The server analyzes the user's answers and generates a request to call the pizza ordering API.

[1930] Output: A request is sent to the pizza ordering API, ordering a Margherita pizza.

[1931] (Application example 1)

[1932] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1933] In conventional food delivery systems, the ordering process proceeds without considering the user's emotions, resulting in a uniform user experience and making it difficult to provide services that meet the needs and emotions of individual users. In addition, the ordering process can be cumbersome because appropriate prompts are not generated when the user places an order.

[1934] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to a purpose, a means for calling APIs of other services, a means including an emotion engine for recognizing the user's emotions, a means for generating prompts according to the user's emotions, and a means for linking with a food delivery service. This makes it possible to generate appropriate prompts according to the user's emotions and to smoothly progress the food delivery ordering process.

[1935] "Generative AI" is an artificial intelligence technology that generates appropriate prompts based on user input and calls the APIs of other services.

[1936] A "prompt" is an instruction that presents the user with the next action or choice to make.

[1937] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[1938] An "emotion engine" is a technology that recognizes emotions from user input and actions and generates appropriate responses based on those emotions.

[1939] A "food delivery service" is a service that allows users to order food online and have it delivered to a specified location.

[1940] "User emotion" refers to the psychological state or feelings that a user exhibits through their input and actions.

[1941] "Supplementing a prompt" means generating supplemental instructions to provide additional information or options to the user in response to their input.

[1942] "Integration" means that different systems or services work together to share information and functions.

[1943] The system for implementing this invention is configured by linking a generative AI, an emotion engine, and an API for a food delivery service. A specific embodiment of the system is shown below.

[1944] System Configuration

[1945] The system is composed of a terminal that receives user input, a server that runs the generative AI, a server that runs the emotion engine, and a server that calls the API of the food delivery service. The terminal is a device such as a smartphone or tablet, and provides an interface for users to place orders.

[1946] Program Processing

[1947] The server performs the process in the following procedure.

[1948] 1. Receive user input:

[1949] A user inputs "I would like to order pizza" through a terminal. This input is sent to the server.

[1950] 2. Emotion Recognition:

[1951] The server uses an emotion engine to recognize emotions from the user's input. For example, the user's input is recognized as indicating the emotion "happiness."

[1952] 3. Generate prompt:

[1953] The server uses generative AI to generate prompts based on the user's emotions, for example, "Congratulations! What kind of pizza would you like to order?"

[1954] 4. Receiving user responses:

[1955] The user answers "Margherita" via the terminal. This answer is sent back to the server.

[1956] 5. Call the food delivery API:

[1957] The server calls the API of the food delivery service to process the user's order, and the order data is sent to the API to complete the order.

[1958] Hardware and Software Used

[1959] Devices: Smartphones, tablets

[1960] Servers: A server that runs the generative AI, a server that runs the emotion engine, and a server that calls the API of a food delivery service.

[1961] Software: Generative AI models, emotion engines, food delivery service APIs

[1962] Examples

[1963] If a user types "I'd like to order sushi" and the emotion engine recognizes this as "surprise," the generative AI will generate a prompt saying, "Great choice! What kind of sushi would you like to order?"

[1964] Examples of prompt statements

[1965] "I want to order pizza."

[1966] "Congratulations! What kind of pizza would you like to order?"

[1967] "I want to order sushi."

[1968] "Great choice! What type of sushi would you like to order?"

[1969] In this way, appropriate prompts can be generated based on the user's emotions, ensuring a smooth food delivery ordering process.

[1970] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1971] Step 1:

[1972] A user inputs "I want to order pizza" through a terminal. This input is sent from the terminal to the server. The input data is in text format and represents the user's order request.

[1973] Step 2:

[1974] The server uses an emotion engine to recognize emotions from user input. The input data is the user's text input, which the emotion engine analyzes and outputs emotions such as "joy." The emotion engine uses natural language processing technology to classify the emotion of the text.

[1975] Step 3:

[1976] The server uses a generative AI to generate a prompt based on the user's emotion. The input data is the user's emotion and text input, and the generative AI outputs a prompt such as "Congratulations! What kind of pizza would you like to order?" The generative AI uses a pre-trained model to generate the appropriate prompt.

[1977] Step 4:

[1978] The user answers "Margherita" through the terminal. This answer is sent from the terminal to the server. The input data is a text that represents the type of pizza selected by the user.

[1979] Step 5:

[1980] The server calls the API of the food delivery service to process the user's order. The input data is the type and quantity of pizza selected by the user, and the server sends this to the API of the food delivery service. The API receives the order data and processes the order.

[1981] Step 6:

[1982] The server receives the order confirmation from the food delivery service and notifies the user that the order has been completed. The input data is the order confirmation from the food delivery service, and the server converts it into a message to notify the user and sends it to the terminal. The user receives the notification of the order completion on the terminal.

[1983] Example 2

[1984] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1985] Conventional cash register systems process orders without considering the user's emotions, which can result in poor customer service quality. In addition, it is difficult to respond flexibly to the user's emotions, making it a challenge to improve user satisfaction, especially in situations involving customer service such as restaurants.

[1986] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1987] In this invention, the server includes a means for performing the role of an application using generative artificial intelligence, a means for supplementing prompts according to a purpose, a means for calling an application programming interface of another service, a means for linking with a cash register application, a means for recognizing a user's emotion, a means for generating a message based on the recognized emotion, and a means for sending the generated message to the cash register application. This enables flexible responses according to the user's emotion, improving the quality of customer service.

[1988] "Generative artificial intelligence" is an artificial intelligence technology that generates text and data using natural language processing and machine learning.

[1989] A "prompt" is an instruction or question input to a generative artificial intelligence to obtain a specific output.

[1990] An "application programming interface" is an interface for exchanging data and functions between different software systems.

[1991] A "cash register application" is software for managing point-of-sale information and supports product sales and inventory management.

[1992] "Means for recognizing a user's emotions" refers to technology that analyzes a user's facial expressions and tone of voice to identify their emotional state.

[1993] "Means for generating a message" refers to technology that uses generative artificial intelligence to create an appropriate text message based on specific input.

[1994] A "message sending means" is a technique for forwarding a generated message to another system or application.

[1995] This invention is a system that realizes flexible responses according to the emotions of a user by linking an application using generative artificial intelligence with a cash register application. Specific embodiments of this system are described below.

[1996] Hardware and Software Used

[1997] server

[1998] The server provides the computational resources to run the generative artificial intelligence model. Specifically, it uses the following software and hardware:

[1999] Generative artificial intelligence models: such as OpenAI's GPT-4

[2000] Application Programming Interface: Use RESTful APIs to integrate with other services

[2001] Database: A database to store user order history and emotion data (e.g. MySQL)

[2002] Terminal

[2003] The terminal is a device for capturing the user's voice and facial expressions and recognizing emotions. Specifically, the following hardware and software are used:

[2004] Microphone: Captures the user's voice

[2005] Camera: Captures the user's facial expressions

[2006] Voice recognition software: Google Speech-to-Text, etc.

[2007] Emotion recognition engine: Microsoft Azure's Emotion API, etc.

[2008] Cash register application

[2009] The cash register application is software that processes user orders and displays appropriate messages. Specifically, it uses the following software:

[2010] POS system: Square POS, etc.

[2011] System Operation

[2012] User Orders

[2013] A user goes to a restaurant and says, "I'd like to order a hamburger." The device uses a microphone to capture the user's voice and converts the voice into text data using Google Speech-to-Text. This text data contains the content, "I'd like to order a hamburger."

[2014] Emotion recognition

[2015] The device uses a camera and microphone to capture the user's facial expressions and tone of voice. Using Microsoft Azure's Emotion API, the captured data is analyzed and the user's emotion is recognized as "anger." For example, if the user's voice is high-pitched and their facial expression is grim, the emotion engine will determine that the emotion is "anger."

[2016] Sending your order

[2017] The terminal sends a prompt sentence to the generative AI model. An example of a prompt sentence is "The user ordered a hamburger. The emotion engine recognized the user's emotion as anger. Please generate a message to be displayed in the cash register app." Based on this prompt sentence, the generative AI model generates the message "I'm sorry. I'll prepare the hamburger quickly."

[2018] Receiving and sending generated messages

[2019] The server receives messages generated from the generative artificial intelligence model. The server sends the received messages to the cash register application. At this time, the server converts the message format into a format that the cash register application can understand.

[2020] Displaying messages

[2021] The cash register application displays the message received from the server. Specifically, the cash register application displays the message "Sorry, we'll hurry up and prepare your burger" on its screen. This message reassures the user that their order will be fulfilled promptly.

[2022] This system enables flexible responses according to the user's emotions, improving the quality of customer service.

[2023] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2024] Step 1:

[2025] A user is at a restaurant and says, "I want to order a hamburger."

[2026] Input: User's voice

[2027] Specific behavior: The device uses the microphone to capture the user's voice.

[2028] Output: Captured audio data

[2029] Step 2:

[2030] The terminal converts the captured voice data into text data.

[2031] Input: Audio data

[2032] Specific operation: The device uses voice recognition software (e.g. Google Speech-to-Text) to convert voice data into text data.

[2033] Output: Text data "I want to order a hamburger"

[2034] Step 3:

[2035] The device captures the user's facial expressions and tone of voice to recognise emotions.

[2036] Input: User's facial expression data, voice tone data

[2037] How it works: The device uses the camera and microphone to capture the user's facial expressions and tone of voice, and then analyzes them using an emotion recognition engine (e.g. Microsoft Azure's Emotion API).

[2038] Output: User emotion (e.g. "Anger")

[2039] Step 4:

[2040] The terminal sends a prompt sentence to the generative artificial intelligence model.

[2041] Input: Text data, user sentiment

[2042] Specific operation: The terminal sends the following prompt to the generative artificial intelligence model: "The user ordered a hamburger. The emotion engine recognized the user's emotion as anger. Please generate a message to be displayed in the cash register app."

[2043] Output: The prompt sent to the generative AI model

[2044] Step 5:

[2045] The generative artificial intelligence model generates a message based on the prompt sentence.

[2046] Input: Prompt statement

[2047] How it works: A generative artificial intelligence model (e.g., OpenAI's GPT-4) analyzes the prompt and generates an appropriate message.

[2048] Output: "Sorry, I'll hurry up and prepare the burger."

[2049] Step 6:

[2050] The server receives the generated message and sends it to the cash register application.

[2051] Input: The generated message

[2052] Specific operation: The server receives messages generated from the generative artificial intelligence model, converts them into a format that the cash register application can understand, and sends them.

[2053] Output: Message sent to the cash register application

[2054] Step 7:

[2055] The cash register application displays the received message.

[2056] Input: The message sent by the server

[2057] Specific operation: The cash register application displays the received message on the screen.

[2058] Output: The message "Sorry, I'll hurry up and prepare the burger" is displayed on the screen.

[2059] In this way, it becomes possible to respond flexibly to the user's emotions, thereby improving the quality of customer service.

[2060] (Application example 2)

[2061] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[2062] Conventional cash register systems were unable to respond to the user's emotions, resulting in poor customer service quality. In addition, because order details were processed manually, there were issues with inefficiency and the likelihood of mistakes occurring. This resulted in problems such as lower customer satisfaction and a worsening operational efficiency of the store.

[2063] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2064] In this invention, the server includes a means for performing the role of an application using generative AI, a means for supplementing prompts according to the purpose, a means for calling APIs of other services, a means for linking with a cash register application, a means for recognizing the user's emotions using an emotion engine, and a means for generating messages according to the user's emotions. This allows for appropriate responses according to the user's emotions, improving the quality of customer service. Furthermore, automating the processing of order details allows for efficient operation and reduces the occurrence of errors.

[2065] "Generative AI" is an artificial intelligence technology that generates natural language sentences, images, etc. based on user input.

[2066] A "prompt" is an input sentence or instruction given to a generative AI to obtain a specific output.

[2067] "API" stands for Application Programming Interface, an interface for exchanging functions and data between different software programs.

[2068] A "cash register app" is application software used to process product sales and orders in a store.

[2069] An "emotion engine" is a technology for recognizing emotions from a user's voice, facial expressions, etc.

[2070] "User emotion" refers to the psychological state or feelings exhibited by a user.

[2071] "Means for generating a message" refers to technology for generating an appropriate response sentence depending on the user's emotions and situation.

[2072] The system for implementing this invention is composed of a server including a generative AI, an emotion engine, a cash register application, and an API calling function, and a terminal operated by a user. The specific configuration and operation of the system are described below.

[2073] The server plays the role of an application that uses generative AI. Generative AI is an artificial intelligence technology that generates natural language text and images based on user input. When a user enters an order through a terminal, the contents are sent to the server. The server analyzes the order contents and sends appropriate prompts to the generative AI.

[2074] The server recognizes the user's emotions using an emotion engine. An emotion engine is a technology for recognizing emotions from the user's voice, facial expressions, etc. When the user enters an order, the voice and image data are sent to the emotion engine, which analyzes the user's emotions.

[2075] The server processes the order details in cooperation with the cash register app. The cash register app is application software for selling products and processing orders in stores. The server sends the order details to the cash register app's API and automates the order processing.

[2076] The server generates a message according to the user's emotions. Generative AI is a technology for generating appropriate responses according to the user's emotions and circumstances. For example, if a user "orders a hamburger" and the emotion engine recognizes the user's emotion as "anger," the generative AI will generate a message saying, "Sorry. We'll quickly prepare the hamburger."

[2077] As a concrete example, if a user orders a hamburger using a smartphone app and the emotion engine recognizes the user's emotion as "anger," the server will use generative AI to generate a message saying, "We're sorry. We'll prepare the hamburger quickly," and display it on the user's device.

[2078] Examples of prompts for a generative AI model include:

[2079] "Generate a message for when the user is angry."

[2080] In this way, the system can respond appropriately according to the user's emotions and improve the quality of customer service. In addition, autom...

Claims

1. A first means for acquiring details of a user's meal order; A second means for recognizing the emotion of the user from the facial expression and tone of voice of the user using an emotion recognition engine; a third means for generating a first prompt sentence for outputting a message to be presented to the user based on the text data indicating the order content and the user's emotion; a fourth means for inputting the generated first prompt sentence into a generative AI model to generate the message; a fifth means for presenting the generated message to the user; A system including:

2. The fifth means displays the generated message on a screen of a cash register application, a terminal of the user, or a display of a vending machine to present the message to the user. The system of claim 1 .

3. The fourth means generates the message including an apology when the emotion of the user is recognized as anger, and generates the message including an appreciation when the emotion of the user is recognized as sadness.

3. A system according to claim 1 or 2.

Images