system

Abstract

Provide a system. [Solution] an input means by which a user can input conditions; a transmission means for transmitting the conditions input by the input means to a server; a selection means for selecting a dish to be proposed by the server based on the conditions; a display means for presenting the dishes selected by the selection means to the user; A selection means for allowing a user to select an order from the presented dishes; and an order sending means for sending the order selected by the selection means to a server.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is common for restaurant users to spend a lot of time selecting their order from a menu. This not only reduces user satisfaction but also negatively impacts restaurant turnover. It also reduces restaurant operational efficiency because restaurant staff are unable to concentrate on other tasks while waiting for users to order. A system that solves these problems and speeds up and streamlines the user ordering process is needed. [Means for solving the problem]

[0005] The present invention provides a system that includes an input means for allowing a user to input conditions, a transmission means for transmitting the conditions input by the input means to a server, a selection means for the server to select dishes to suggest based on the conditions, a display means for presenting the dishes selected by the selection means to the user, a selection means for the user to select an order from the presented dishes, and an order transmission means for transmitting the order selected by the selection means to the server. The server also analyzes the conditions using natural language processing and selects the optimal dish based on the analysis results, thereby suggesting a dish that is tailored to the user. This system speeds up the user's ordering process and improves restaurant turnover and operational efficiency. Furthermore, the server automatically notifies the restaurant of the selected order, thereby reducing the workload on staff.

[0006] "Input means" refers to a device or interface that a user uses to input order conditions.

[0007] "Transmission means" refers to means including functions and protocols for transmitting input order conditions to the server.

[0008] The "selection means" refers to the algorithm or processing means by which the server selects the optimal menu based on the input conditions.

[0009] "Display means" refers to a device or interface that visually presents the list of suggested dishes to the user.

[0010] The "selection means" refers to an operating means for the user to select a desired dish from the proposed dishes.

[0011] The "order sending means" refers to a means including functions and protocols for sending the dishes selected by the user to the server.

[0012] "Natural language processing" refers to the technology that enables computers to understand and analyze human language.

[0013] "Analysis means" refers to the technology or algorithm that the server uses to analyze the input conditions from the user and understand their contents.

[0014] "Notification means" refers to the function by which the server notifies the store of the final order details. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs order conditions, selects dishes suggested by a server based on those conditions, and finally confirms the order. The program processing of this system is explained in detail below.

[0037] System Overview:

[0038] 1. User Input

[0039] A user launches a restaurant's application using a device such as a smartphone or tablet. The device displays a menu, and the user inputs their preferences, allergies, and other criteria. For example, they can specify criteria such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[0040] 2. Conditional Sending

[0041] The terminal transmits the conditions entered by the user to the server, which receives information to select dishes based on the user's preferences.

[0042] 3. Server Analysis

[0043] The server analyzes the received criteria using natural language processing technology. Specifically, it extracts keywords from the words and phrases entered by the user and determines the characteristics of the food based on them. For example, the keyword "spicy" can be used to search for spicy food.

[0044] 4. Food selection

[0045] Based on the analysis results, the server selects dishes from the menu that match the user's criteria. The server searches the menu items in the database and generates a list of dishes that match the criteria. For example, it might select "Mapo Tofu" or "Curry" as a "spicy dish."

[0046] 5. Submit your proposal

[0047] The server sends the list of selected dishes to the terminal, allowing the user to see dishes that match their criteria.

[0048] 6. User Display

[0049] The device displays the suggestions sent from the server to the user, allowing the user to choose the dish they like from the suggestions.

[0050] 7. Order Selection

[0051] The user selects the desired dish from the list of suggestions and confirms the order. The selected dish is sent to the server via the terminal by pressing the send button.

[0052] 8. Submitting your order

[0053] The terminal sends the order details selected by the user to the server, which then receives the final order details.

[0054] 9. Notification to stores

[0055] The server notifies the restaurant of the final order details, which are then passed on to the restaurant's management system, and staff begin preparing the food.

[0056] This system allows users to easily input their order requirements, and the server analyzes and selects the items, allowing them to receive prompt and accurate suggestions. Ultimately, this has the effect of streamlining the user's ordering process, improving store turnover and operational efficiency.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] A user launches a restaurant application and displays the menu. The device sends a request to the server, retrieves the menu list, and displays it on the screen.

[0060] Step 2:

[0061] The user enters order conditions, such as "spicy food," "vegetarian," or "gluten-free," using text boxes, drop-down lists, and check boxes.

[0062] Step 3:

[0063] The device sends the entered conditions to the server. When the user enters the conditions and presses the "Send" button, the device sends the conditions to the server in a data format such as JSON.

[0064] Step 4:

[0065] The server analyzes the received conditions using natural language processing. Specifically, it tokenizes the conditions and extracts keywords. It generates analysis results based on keywords such as "spicy" or "gluten-free."

[0066] Step 5:

[0067] Based on the analysis results, the server searches for matching dishes on the menu. It searches the server's database and generates a list of dishes that match the criteria. For "spicy food," it will select dishes such as "mapo tofu" or "curry."

[0068] Step 6:

[0069] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[0070] Step 7:

[0071] The device displays the suggested dishes to the user. Suggested dishes such as "Mapo Tofu" and "Curry" are displayed in a list format on the screen.

[0072] Step 8:

[0073] The user selects the dish they wish to order from the suggested dishes by tapping on the desired dish and pressing the "Order" button.

[0074] Step 9:

[0075] The terminal sends the final order details to the server, where the selected dishes are encoded in JSON format or similar and sent to the server.

[0076] Step 10:

[0077] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0078] In this way, users can place orders efficiently and stores can respond quickly.

[0079] Example 1

[0080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0081] When ordering at a restaurant, a system is needed that allows users to quickly select the appropriate dish based on their preferences and allergies. Restaurants also need to efficiently receive orders and serve food smoothly. With conventional systems, communication between users and restaurants can be time-consuming, potentially resulting in mismatched orders. A new system is needed to solve these problems.

[0082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0083] In this invention, the server includes input means for allowing a user to input conditions, transmission means for transmitting the conditions input by the input means to the server, selection means for selecting dishes to be proposed by the server based on the conditions, display means for presenting the dishes selected by the selection means to the user, selection means for the user to select an order from the presented dishes, order transmission means for transmitting the order selected by the selection means to the server, and notification means for notifying the store's management system of the order details. This allows the user to easily input conditions, and the server to quickly analyze and select dishes, enabling an efficient ordering process.

[0084] "User" refers to a person who uses a restaurant ordering system.

[0085] "Input means" refers to a device or interface for a user to input conditions.

[0086] "Transmission means" refers to a communication means for transmitting data entered by a user to a server.

[0087] "Server" refers to a computer system that receives, processes, and analyzes data sent by users.

[0088] "Selection means" refers to the function by which the server selects dishes to suggest based on the user's conditions.

[0089] "Display means" refers to a device or interface for visually presenting the dishes selected by the server to the user.

[0090] The "selection means" refers to the means by which the user selects the desired order from the displayed dishes.

[0091] "Order sending means" refers to a communication means for sending an order selected by a user to a server.

[0092] "Notification means" refers to the function by which the server notifies the store's management system of the final order details.

[0093] "Natural language processing" refers to a technology in which a server analyzes user input data and extracts appropriate keywords and phrases.

[0094] This system allows users to efficiently place orders at restaurants and is configured as follows.

[0095] First, a user launches a restaurant ordering application on a device such as a smartphone or tablet. The application displays a menu, and the user uses an input device to enter their preferences and allergies. The input device includes text boxes and options, allowing users to enter specific preferences such as "spicy food," "vegetarian," or "gluten-free."

[0096] Next, the terminal sends the conditions entered by the user to the server. The transmission method is HTTP protocol, and the data is sent in JSON format. Specifically, when the user presses the "Send" button, the terminal sends the input data to the server.

[0097] The server uses Python natural language processing libraries (e.g., nltk or spaCy) to analyze the received data. Through analysis, the server extracts keywords from the user's input and selects dishes based on the criteria. Analyzed keywords include "spicy," "vegetarian," and "gluten-free."

[0098] Next, the server uses a selection method to search for food data in a database (MySQL (registered trademark) or PostgreSQL). Using this selection method, the server lists dishes that meet the user's criteria. For example, dishes that meet the criteria of "spicy" and "gluten-free" might be selected as "Mapo Tofu" or "Spicy Gluten-Free Salad."

[0099] Once the list of selected dishes is complete, the server sends it to the device in JSON format. The device then uses a display to present the listed dishes to the user. The display uses the Flutter (registered trademark) or React Native framework, and UI components such as list views are used to visually present the list to the user.

[0100] When the user selects a desired dish from the displayed list, the user uses a selection means. Specifically, the user taps the name of the dish (e.g., "Mapo Tofu") and presses the "Confirm Order" button to confirm the selection.

[0101] The terminal then transmits the order details confirmed by the selection means to the server, again using the HTTP protocol and JSON format.

[0102] Finally, the server notifies the restaurant's management system of the final order details using a REST API, allowing restaurant staff to begin preparing the food.

[0103] As a concrete example, consider the case where a user launches an application and enters the conditions "spicy food" and "gluten-free." The device sends these conditions to the server, which analyzes them and selects "mapo tofu" and "spicy gluten-free salad" to create a list. The device displays this list to the user, who selects "mapo tofu" and confirms the order. The server then notifies the restaurant of the final order details, and food preparation begins.

[0104] Example prompt for a generative AI model:

[0105] Please explain a system that streamlines ordering at restaurants. This system has a function that allows users to input their preferences and allergies, and the server analyzes these and suggests dishes. Please explain the following process in detail.

[0106] 1. User Input

[0107] 2. Conditional Sending

[0108] 3. Server Analysis

[0109] 4. Food selection

[0110] 5. Submit your proposal

[0111] 6. User Display

[0112] 7. Order Selection

[0113] 8. Submitting your order

[0114] 9. Notification to stores

[0115] The system uses a Python natural language processing library and a MySQL database. For example, include a case where a user inputs "spicy food" and "gluten-free" and finally selects "mapo tofu."

[0116] In this way, the present invention allows the user to easily input requirements and the server to quickly analyze and select, thereby enabling an efficient ordering process.

[0117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0118] Step 1:

[0119] A user launches a restaurant ordering application using a smartphone or tablet device.

[0120] Input: Launch application

[0121] Output: Display the menu screen

[0122] Specific behavior: The application displays a menu screen on the device and allows the user to enter conditions.

[0123] Step 2:

[0124] The user inputs their preferences and allergies.

[0125] Input: User criteria (spicy food, vegetarian, gluten-free, etc.)

[0126] Output: Confirmation of input data

[0127] Specific behavior: The user enters conditions in the text boxes or options and clicks the "Submit" button.

[0128] Step 3:

[0129] The terminal transmits the input conditions to the server.

[0130] Input: User input data (JSON format)

[0131] Output: Notification of completion of transmission to the server

[0132] Specific operation: The terminal generates an HTTP request and sends the input data in JSON format to the server.

[0133] Step 4:

[0134] The server analyzes the received conditions.

[0135] Input: JSON data submitted

[0136] Output: Extracted keywords

[0137] What it does: The server uses a Python natural language processing library (such as nltk or spaCy) to extract keywords from the conditions.

[0138] Step 5:

[0139] The server selects dishes based on the analyzed keywords.

[0140] Input: Extracted keywords

[0141] Output: A list of suitable dishes

[0142] Specific operation: The server uses a database management system (MySQL or PostgreSQL) to search for dishes that match the keywords and generate a list.

[0143] Step 6:

[0144] The server sends the selected dish list to the terminal.

[0145] Input: List of selected dishes (JSON format)

[0146] Output: Notification of completion of transmission to the terminal

[0147] Specific behavior: The server generates an HTTP response and sends the list of dishes in JSON format to the device.

[0148] Step 7:

[0149] The terminal displays the received recipe list to the user.

[0150] Input: A list of dishes sent from the server

[0151] Output: what is displayed to the user

[0152] What it does: The device uses the Flutter and React Native frameworks to display a list of dishes on a user interface.

[0153] Step 8:

[0154] The user selects an order from the displayed list of dishes.

[0155] Input: User's food selection

[0156] Output: Confirmed order data

[0157] Specific operation: The user taps the desired dish and presses the "Confirm order" button.

[0158] Step 9:

[0159] The terminal sends the confirmed order to the server.

[0160] Input: Confirmed order data (JSON format)

[0161] Output: Notification of completion of transmission to the server

[0162] Specific operation: The terminal sends the order data in JSON format to the server as an HTTP request.

[0163] Step 10:

[0164] The server notifies the store's management system of the final order details.

[0165] Input: Confirmed order data

[0166] Output: Notification to store management system completed

[0167] Specific operation: The server notifies the store management system of the order data via the REST API.

[0168] (Application example 1)

[0169] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] In today's restaurants and food delivery services, it is difficult for users to select appropriate dishes taking into account individual preferences, allergies, and other factors. It is also difficult for restaurants and delivery services to provide menus that reflect the different preferences of each user, which can result in a decrease in user satisfaction. Furthermore, when a user simply wants to reorder a dish they have ordered in the past, it is inefficient because it requires time and effort to manually search for the dish. A system that solves these problems is needed.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0172] In this invention, the server includes an analysis means that analyzes the user's input criteria using natural language processing and calculates cosine similarity to select the dish that most closely matches the user's criteria, a suggestion means that generates a list of dishes selected by the analysis means and suggests them to the user, and an order history management means that makes it easy to place repeat orders based on past order history.This allows users to quickly and accurately select and order dishes based on their individual preferences and allergy information.Furthermore, utilizing past order history makes repeat orders easier, improving user satisfaction.

[0173] "Input means" refers to a device or interface that allows a user to input conditions.

[0174] The "transmission means" refers to a device or program that transmits the conditions input by the input means to the server.

[0175] The "selection means" refers to a device or program that allows the server to select dishes to propose based on the above conditions.

[0176] The "display means" refers to a device or program that presents the dishes selected by the selection means to the user.

[0177] The "selection means" refers to a device or program that allows the user to select an order from the presented dishes.

[0178] The "order sending means" refers to a device or program that sends the order selected by the selection means to the server.

[0179] "Analysis means" refers to a device or program that analyzes the conditions entered by the user based on natural language processing, calculates the cosine similarity, and selects the dish that most closely matches the user's conditions.

[0180] The "suggestion means" refers to a device or program for generating a list of dishes selected by the analysis means and suggesting the list to the user.

[0181] "Order history management means" refers to a device or program that allows repeat orders to be easily placed based on past order history.

[0182] "Natural language processing" refers to the technology that enables computers to understand and analyze human language.

[0183] "Cosine similarity" refers to a method for calculating the similarity between vectorized data.

[0184] A specific system for carrying out the invention has the following configuration.

[0185] First, a user launches the application using their device, such as a smartphone or tablet. The application provides input means for the user to enter their preferences and allergy information. Input means include text boxes and check boxes. This input information is then sent from the device to the server.

[0186] Next, the server receives the user's request information. It then uses natural language processing technology to analyze the user's input. Specifically, it uses Python's Scikit-learn library to vectorize the text and calculate the cosine similarity. Based on this analysis, it selects the dish from the menu that most closely matches the user's request.

[0187] The server sends a list of selected dishes to the terminal. The terminal displays this list to the user. The user selects the dishes they like from the suggestions and confirms the order. The user's selection information is sent back to the server, which then confirms the final order.

[0188] Furthermore, this system includes an order history management means, which allows users to easily place repeat orders based on the history information of dishes they have ordered in the past.

[0189] Specific examples of prompt sentences for implementing the present invention are shown below: The user inputs prompt sentences such as "Please suggest spicy and vegetarian dishes" or "I'm a vegetarian and I like spicy dishes. Please tell me your recommendations."

[0190] This system allows users to efficiently and accurately select and order dishes, enabling restaurants and food delivery services to improve user satisfaction.

[0191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0192] Step 1:

[0193] A user launches an application using a device such as a smartphone or tablet. The user then enters their preferences and allergy information within the application. This information is entered using text boxes and check boxes.

[0194] Input: User criteria (e.g. "spicy food", "vegetarian", "gluten-free", etc.)

[0195] Output: User condition data

[0196] Step 2:

[0197] The user's condition data is sent from the device to the server. This information is sent to the server using an HTTP request.

[0198] Input: User condition data

[0199] Output: Condition data sent to the server

[0200] Step 3:

[0201] The server then uses natural language processing technology to analyze the received user requirements. This analysis involves vectorizing the text data using the Scikit-learn library, and then calculating cosine similarity to extract dishes from the menu that match the user's requirements.

[0202] Input: Condition data sent to the server

[0203] Output: A list of dishes that match the user's criteria

[0204] Step 4:

[0205] The server sends a list of selected dishes to the device, and returns this information to the device in an HTTP response.

[0206] Input: A list of dishes that match the user's criteria

[0207] Output: Recipe list sent to the device

[0208] Step 5:

[0209] The terminal displays the list of dishes received from the server to the user, who then checks the list and selects the dish they wish to order.

[0210] Input: Recipe list sent to device

[0211] Output: User-selected dishes

[0212] Step 6:

[0213] The user's selected dish data is sent from the device to the server, again using an HTTP request.

[0214] Input: User-selected dish data

[0215] Output: Final order data sent to the server

[0216] Step 7:

[0217] The server receives the final order data and confirms the order. The order details are saved on the server and sent to the restaurant. The order details are then sent to the restaurant's management system, and the staff begins preparing the food.

[0218] Input: Final order data sent to the server

[0219] Output: Order details notified to the store

[0220] Step 8:

[0221] The server manages the order history and makes this information available to users when they place repeat orders in the future. An interface is provided to make it easy to place repeat orders based on past order history.

[0222] Input: User's past order data

[0223] Output: Historical data available for repeat orders

[0224] This allows users to efficiently select dishes and confirm their orders, and by utilizing past order history, they can easily place repeat orders.

[0225] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0226] This invention relates to combining an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which a user inputs order conditions, selects dishes suggested by a server based on the conditions and the user's emotions, and finally confirms the order.

[0227] System Overview:

[0228] 1. User Input

[0229] A user launches a restaurant's application on a device such as a smartphone or tablet, and the menu is displayed. The device then sends a request to the server, which retrieves the menu list and displays it on the screen. The user then inputs preferences, allergies, and other conditions. For example, they can specify conditions such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[0230] 2. Emotion recognition

[0231] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. For example, it uses facial recognition technology and voice analysis technology to identify the user's emotional state, such as whether they are happy, excited, relaxed, etc.

[0232] 3. Sending conditions and emotions

[0233] The device sends the input conditions and recognized emotions to the server, which simultaneously sends the user's conditions and emotion information to the server, providing the data that will be used as the basis for analysis.

[0234] 4. Server Analysis

[0235] The server analyzes the order conditions and emotional data it receives using natural language processing and emotion analysis technology. Specifically, it extracts keywords from the conditions and determines the user's emotional state from the emotional data. For example, if a user requests "spicy food" and is "excited," it will prioritize suggesting spicy food.

[0236] 5. Food selection

[0237] Based on the analysis results, the server searches for the dish from the menu that best suits the conditions and emotions. It searches the menu items in the database and generates a list of dishes according to the conditions and emotions. For example, for a user who likes "spicy food" and is "excited," it selects "super spicy mapo tofu" or "spicy curry."

[0238] 6. Submit your proposal

[0239] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[0240] 7. User Presentation

[0241] The device displays the suggestions sent from the server to the user. The suggested dishes are displayed in a list format on the screen. For example, dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are presented.

[0242] 8. Order Selection

[0243] The user selects the desired dish from the suggested dishes, taps on the desired dish, and presses the "Order" button.

[0244] 9. Submitting your order

[0245] The device sends the selected order details to the server, which then encodes the selected dishes in JSON format and sends them to the server.

[0246] 10. Notice to Stores

[0247] The server notifies the restaurant of the final order details, which are then transmitted to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0248] In this way, by combining and analyzing user input and emotional information, more accurate and personalized dish suggestions can be made, thereby increasing user satisfaction and store operational efficiency.

[0249] The processing flow will be explained below.

[0250] Step 1:

[0251] A user launches a restaurant application on their device, which sends a request to the server, downloads a menu list, and displays it on the screen.

[0252] Step 2:

[0253] The user inputs ordering conditions, such as "spicy food" or "vegetarian food preferred," into text boxes and check boxes.

[0254] Step 3:

[0255] The device recognizes the user's facial expressions and voice using an emotion engine. The device's camera captures the user's facial expressions and the microphone records the user's voice. The emotion engine analyzes this data to identify the user's emotional state (e.g., happy, excited, relaxed).

[0256] Step 4:

[0257] The terminal transmits the input conditions and the recognized emotion data to the server, thereby providing the user's order conditions and emotion information to the server.

[0258] Step 5:

[0259] The server analyzes the received conditions and emotional data. It uses natural language processing technology to analyze the order conditions and simultaneously analyzes the emotional data to determine the user's emotional state. For example, it analyzes the emotional state of "spicy food" and "excited."

[0260] Step 6:

[0261] The server selects dishes based on the conditions and emotional data. From the menu in the database, it selects "spicy dishes" that are suitable for "excited" users. For example, it suggests "super spicy mapo tofu" or "spicy curry."

[0262] Step 7:

[0263] The server sends the list of selected dishes to the device, which then encodes the list in an appropriate data format (e.g., JSON) and sends it to the device.

[0264] Step 8:

[0265] The device displays the suggested dishes to the user. Dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are displayed in a list format on the screen.

[0266] Step 9:

[0267] The user selects the desired dish from the suggested dishes and presses the "Order" button. For example, the user selects and orders "Super Spicy Mapo Tofu."

[0268] Step 10:

[0269] The terminal sends the selected order details to the server. The dishes selected by the user are encoded in a data format and sent to the server.

[0270] Step 11:

[0271] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0272] In this way, the combined analysis of user input conditions and emotional information leads to more personalized suggestions and an improved user experience.

[0273] Example 2

[0274] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] Conventional restaurant ordering systems mainly provide suggestions based on the user's preferences, but lack personalized suggestions that take into account the user's emotional state. There is a need for a method to improve satisfaction by providing dish suggestions that adapt to the user's emotions.

[0276] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means by which the user can input conditions, a transmission means for transmitting the conditions input by the input means and the user's emotional information to the server, a selection means by which the server selects dishes to suggest based on the conditions and emotional information, a display means for presenting the dishes selected by the selection means to the user, a selection means by which the user selects an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration.

[0277] "Input means" refers to a device or interface for inputting desired conditions and information by the user.

[0278] The "transmission means" is a device or system that transmits input information or data to a server via a network.

[0279] The "selection means" is a device or algorithm that performs the process of selecting appropriate proposals based on the conditions and information received by the server.

[0280] The "display means" is a device or interface that visually shows the proposal selected by the selection means to the user.

[0281] The "selection means" is a device or system that allows the user to select a desired item from the displayed suggestions.

[0282] The "order sending means" is a device or system that sends the order details selected by the user to the server.

[0283] "Natural language processing" is a technology that enables computers to understand, analyze, and respond to human language.

[0284] "Emotion analysis" is a technology for identifying a user's emotional state from their facial expressions and voice.

[0285] The "notification means" is a device or system that allows the server to transmit the order details to the store.

[0286] This invention combines an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs their order conditions, selects dishes suggested by the server based on those conditions and the user's emotions, and finally confirms the order.

[0287] Basic system configuration

[0288] This system is composed of a user input means, a transmission means, a selection means, a display means, a selection means, and an order transmission means. Each means will be explained in detail below.

[0289] User input method

[0290] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu. The device then sends an HTTP request to the server to retrieve the menu list. This retrieval process includes retrieving menu information from a MySQL database, encoding it in JSON format, and sending it to the device. The retrieved menu is then displayed to the user on the device using a UI created with React Native or Flutter.

[0291] Users input their preferences, allergies, and other conditions via text boxes and radio buttons on the terminal.

[0292] emotion recognition means

[0293] As the user enters their conditions, the device's camera and microphone capture their facial expressions and voice. This allows the system to recognize emotions using Google's (registered trademark) Face API, Amazon's Rekognition, Microsoft's (registered trademark) AI services, etc. This emotion recognition identifies the user's emotional state, such as whether they are happy, excited, or relaxed.

[0294] For example, if a user smiles while looking at a menu, the camera captures that smile and each AI service recognizes the emotion as "enjoying."

[0295] Server analysis and selection method

[0296] The entered conditions and recognized emotion information are encoded in JSON format by the device and sent to the server. The server receives this data and analyzes it using natural language processing (NLP) and emotion analysis techniques. Specifically, it uses libraries such as NLTK and spaCy to extract keywords from the conditions and determine the user's emotional state from the emotion data.

[0297] For example, the server receives data such as "spicy food" and "enjoying it," and determines that "spicy food" matches "super spicy mapo tofu" or "spicy curry."

[0298] The server then selects the dish from the menu that best suits the criteria and emotion, using a MySQL query to generate a list of dishes based on the criteria and emotion.

[0299] Proposal and ordering methods

[0300] The selected suggestions are encoded in JSON format and sent to the device as an HTTP response. The device then displays the received suggestions to the user. A list of appropriate dishes is displayed on the screen using a UI created with React Native or Flutter.

[0301] The user selects the desired dish from the presented dishes and presses the "Order" button. The selected order details are also encoded in JSON format and sent to the server.

[0302] Finally, the server notifies the store's order management system (e.g., a POS system) of the order details, which then confirms the order and begins cooking.

[0303] Specific examples and prompts for the generative AI model

[0304] For example, if a user requests "spicy food" and their emotion is recognized as "enjoying it," the system will suggest dishes such as "super spicy mapo tofu" or "spicy curry."

[0305] Prompt sentence for generative AI model

[0306] Example 1: "If the user selects a spicy dish and facial analysis indicates that they are enjoying it, please recommend a dish."

[0307] Example 2: "If a user selects gluten-free as their allergy information and a happy expression is recognized, suggest appropriate dishes."

[0308] As a result, personalized recipe suggestions that take into account the user's emotional state become possible, thereby improving user satisfaction.

[0309] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0310] Step 1:

[0311] User Input

[0312] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu.

[0313] Specific operation: When a user opens the app, the device sends an HTTP request (e.g., GET / menu) to the server to retrieve a menu list. The server retrieves menu information from the MySQL database and sends it to the device in JSON format. Input is done through text boxes and check boxes. Input contents include conditions such as "spicy food," "vegetarian," and "gluten-free."

[0314] Input: User criteria (e.g. "spicy food" or "vegetarian")

[0315] Output: Menu list displayed on the device

[0316] Step 2:

[0317] emotion recognition

[0318] The device captures the user's facial expressions and voice using a camera and microphone.

[0319] How it works: The device uses Google's Face API, Amazon's Rekognition, and Microsoft's AI services to analyze the user's emotions, recognizing emotions such as "enjoyed" or "excited" based on the user's facial expressions and voice patterns.

[0320] Input: User's facial expressions and voice data

[0321] Output: Parsed emotion data (e.g., "having fun")

[0322] Step 3:

[0323] Sending conditions and emotions

[0324] The device sends the input conditions and recognized emotions to the server in JSON format.

[0325] Specific operation: The device sends a request to the POST / analyze endpoint: {"conditions": ["spicy", "vegetarian"], "emotion": "enjoying"}.

[0326] Input: User condition and emotion data

[0327] Output: Data sent to the server

[0328] Step 4:

[0329] Server Analysis

[0330] The server analyzes the order conditions and emotional data received using natural language processing (NLP) and emotion analysis technology.

[0331] Specific operation: The server uses libraries such as NLTK and spaCy to extract keywords from the conditions and identify the user's emotions from the emotional data. For example, if the conditions are "spicy food" and "enjoying it," the server will extract "super spicy mapo tofu" and "spicy curry" as matching candidates.

[0332] Input: User condition and emotion data

[0333] Output: Parsed list of recommended dishes

[0334] Step 5:

[0335] Food selection

[0336] Based on the analysis results, the server selects the dish from the restaurant's menu that best suits the conditions and emotions.

[0337] What it does: The server runs the query SELECT FROM menu WHERE spicy='true' AND category='vegetarian' against the MySQL database to extract dishes that fit the criteria, generating a list of appropriate dishes.

[0338] Input: Parsed conditions and emotion data

[0339] Output: A list of recommended dishes based on criteria and sentiment

[0340] Step 6:

[0341] Submit your proposal

[0342] The server encodes the list of selected dishes in JSON format and sends it to the terminal as an HTTP response.

[0343] Specific operation: For example, generate JSON data {"dishes": ["Super spicy mapo tofu", "spicy curry"]} and send it to the terminal.

[0344] Input: Recommended dish list

[0345] Output: Suggestion sent to device

[0346] Step 7:

[0347] What the user sees

[0348] The terminal displays the received proposal to the user.

[0349] Specific operation: A list of dishes is displayed on the device screen through a UI created with React Native or Flutter. For example, "Super Spicy Mapo Tofu" and "Spicy Curry" are listed.

[0350] Input: Proposal received from the server

[0351] Output: A list of dishes displayed to the user

[0352] Step 8:

[0353] Order Selection

[0354] The user selects the desired dish from the suggested dishes and presses the "Order" button.

[0355] Specific operation: The user taps on "Super Spicy Mapo Tofu" and presses the "Order" button to confirm the selection.

[0356] Input: User's food selection

[0357] Output: Selected order details

[0358] Step 9:

[0359] Sending order details

[0360] The terminal encodes the selected order details in JSON format and sends them to the server.

[0361] Specific operation: The terminal sends a request to the POST / order endpoint: {"order": "Super spicy mapo tofu"}.

[0362] Input: Selected order details

[0363] Output: Order data sent to the server

[0364] Step 10:

[0365] Notification to stores

[0366] The server notifies the store's order management system of the final order details.

[0367] Specific operation: The server sends a request to the store's order management API: {"order": "Super spicy mapo tofu"}. The store then confirms the order and begins cooking.

[0368] Input: User's order details

[0369] Output: Order details sent to the store

[0370] (Application example 2)

[0371] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0372] Conventional restaurant and food delivery systems only consider user preferences and allergies when suggesting dishes, which means they are unable to suggest dishes that take into account the user's emotional state. This makes it difficult for users to select an appropriate dish that matches their current mood and emotions. This can lead to a decrease in user satisfaction, so there is a need for a suggestion system that incorporates emotions.

[0373] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0374] In this invention, the server includes an input means for allowing a user to input conditions and emotions, a transmission means for transmitting the conditions and emotions input by the input means to the server, a selection means for the server to select dishes to suggest based on the conditions and emotions, a display means for presenting the dishes selected by the selection means to the user, a selection means for the user to select an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration, thereby increasing user satisfaction.

[0375] "Input means" refers to a device or interface for a user to input conditions and emotions.

[0376] The "transmission means" is a device or function for transmitting the conditions and emotions input by the input means to the server.

[0377] The "selection means" is a device or function that allows the server to select the most suitable dish based on the above conditions and emotions.

[0378] The "display means" is a device or function for presenting the dish selected by the selection means to the user.

[0379] The "selection means" is a device or interface that allows the user to select a desired dish from the dishes presented.

[0380] The "order sending means" is a device or function for sending the order selected by the selection means to the server.

[0381] The "emotion recognition means" is a device or function for analyzing the user's facial expressions and voice to recognize the user's emotional state.

[0382] The "notification means" is a device or function that allows the server to notify the store of the final order details.

[0383] This invention is a system that allows users to efficiently use restaurants and food delivery services, and by combining an emotion recognition engine, suggests dishes based on the user's emotional state. This system includes input means, transmission means, selection means, display means, selection means, order transmission means, emotion recognition means, and notification means.

[0384] 1. Overall system configuration

[0385] The main components of the system are as follows:

[0386] The user inputs conditions and emotions from a device such as a smartphone or tablet.

[0387] The terminal transmits the input data to the server.

[0388] The server analyzes the user's conditions and emotions and selects the most suitable dish.

[0389] The terminal presents the selected dishes to the user, and the user selects an order.

[0390] The terminal transmits the selected order to the server, which then notifies the store.

[0391] 2. Hardware and Software Used

[0392] Input methods: smartphone or tablet touchscreen, text box, selection options.

[0393] Emotion recognition means: smartphone or tablet camera, microphone, and facial recognition and voice analysis software via EmotionEngine (e.g., OpenCV or Google Cloud Speech-to-Text API).

[0394] Transmission method: Send data to the server (in JSON format via HTTP request) over an internet connection.

[0395] Selection method: In-server natural language processing and sentiment analysis algorithms.

[0396] Display medium: Smartphone or tablet display.

[0397] Options: Smartphone or tablet touchscreen interface.

[0398] Order sending method: Notification from the server to the store system (HTTP request or WebSocket communication).

[0399] Notification method: Store order management system.

[0400] 3. Example of the process

[0401] First, the user uses a smartphone application to input their preferred food preferences (e.g., "spicy food" or "vegetarian"). Next, the device's camera and microphone are used to recognize and analyze the user's emotions. This allows the system to identify the user's emotional state, such as "excited" or "relaxed."

[0402] The above data is sent to the server, which then performs natural language processing and sentiment analysis based on the data. For example, if the user requests "spicy food" and is in an "excited" state, spicy dishes (such as "super spicy mapo tofu" or "spicy curry") will be suggested. The suggested dishes are displayed on the smartphone application, and the user can select the desired dish and confirm the order.

[0403] 4. Examples of prompts

[0404] Below are some specific examples of input prompts for generative AI models.

[0405] User input criteria: spicy food, vegetarian

[0406] User sentiment: Excited

[0407] Generate suggested dishes.

[0408] Using this prompt, the AI ​​model can generate optimal recipe suggestions based on the user's emotions and cooking requirements, thereby suggesting dishes that match the user's current emotions and providing a highly satisfying user experience.

[0409] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0410] Step 1:

[0411] The user launches the app on their smartphone or tablet and enters the food preferences (e.g., "spicy food," "vegetarian," etc.).

[0412] Input: User condition input (text box, choice, etc.)

[0413] Output: Data of the entered conditions (JSON format)

[0414] Step 2:

[0415] The device's camera and microphone are used to collect the user's emotions, which are then analyzed by an emotion recognition engine (EmotionEngine).

[0416] Input: Camera video, microphone audio

[0417] Output: Recognized emotion data (e.g., "excited," "relaxed," etc.)

[0418] Step 3:

[0419] The terminal transmits the input conditions and the recognized emotion data to the server.

[0420] Input: Condition data, emotion data

[0421] Output: Request sent to the server (JSON format data)

[0422] Step 4:

[0423] Based on the conditions and emotional data received by the server, natural language processing and emotional analysis are performed to select the most suitable dish.

[0424] Input: Condition data, emotion data

[0425] Output: List of selected dishes (JSON format)

[0426] Step 5:

[0427] The server sends a list of selected dishes to the terminal.

[0428] Input: List of selected dishes

[0429] Output: A list of dishes sent to the device (JSON format)

[0430] Step 6:

[0431] The terminal displays the list of dishes sent from the server to the user, who then selects the dish they want from the list.

[0432] Input: list of dishes

[0433] Output: User's selected dish

[0434] Step 7:

[0435] The terminal transmits the dish selected by the user to the server.

[0436] Input: Selected dish data (JSON format)

[0437] Output: Order data sent to the server

[0438] Step 8:

[0439] The server notifies the restaurant of the final order, and the restaurant begins cooking based on the order.

[0440] Input: Order data

[0441] Output: Order notified to the store

[0442] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0443] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0444] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0445] [Second embodiment]

[0446] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0447] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0448] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0449] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0450] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0451] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0452] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0453] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0454] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0455] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0456] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0457] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0458] The present invention relates to a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs order conditions, selects dishes suggested by a server based on those conditions, and finally confirms the order. The program processing of this system is explained in detail below.

[0459] System Overview:

[0460] 1. User Input

[0461] A user launches a restaurant's application using a device such as a smartphone or tablet. The device displays a menu, and the user inputs their preferences, allergies, and other criteria. For example, they can specify criteria such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[0462] 2. Conditional Sending

[0463] The terminal transmits the conditions entered by the user to the server, which receives information to select dishes based on the user's preferences.

[0464] 3. Server Analysis

[0465] The server analyzes the received criteria using natural language processing technology. Specifically, it extracts keywords from the words and phrases entered by the user and determines the characteristics of the food based on them. For example, the keyword "spicy" can be used to search for spicy food.

[0466] 4. Food selection

[0467] Based on the analysis results, the server selects dishes from the menu that match the user's criteria. The server searches the menu items in the database and generates a list of dishes that match the criteria. For example, it might select "Mapo Tofu" or "Curry" as a "spicy dish."

[0468] 5. Submit your proposal

[0469] The server sends the list of selected dishes to the terminal, allowing the user to see dishes that match their criteria.

[0470] 6. User Display

[0471] The device displays the suggestions sent from the server to the user, allowing the user to choose the dish they like from the suggestions.

[0472] 7. Order Selection

[0473] The user selects the desired dish from the list of suggestions and confirms the order. The selected dish is sent to the server via the terminal by pressing the send button.

[0474] 8. Submitting your order

[0475] The terminal sends the order details selected by the user to the server, which then receives the final order details.

[0476] 9. Notification to stores

[0477] The server notifies the restaurant of the final order details, which are then passed on to the restaurant's management system, and staff begin preparing the food.

[0478] This system allows users to easily input their order requirements, and the server analyzes and selects the items, allowing them to receive prompt and accurate suggestions. Ultimately, this has the effect of streamlining the user's ordering process, improving store turnover and operational efficiency.

[0479] The processing flow will be explained below.

[0480] Step 1:

[0481] A user launches a restaurant application and displays the menu. The device sends a request to the server, retrieves the menu list, and displays it on the screen.

[0482] Step 2:

[0483] The user enters order conditions, such as "spicy food," "vegetarian," or "gluten-free," using text boxes, drop-down lists, and check boxes.

[0484] Step 3:

[0485] The device sends the entered conditions to the server. When the user enters the conditions and presses the "Send" button, the device sends the conditions to the server in a data format such as JSON.

[0486] Step 4:

[0487] The server analyzes the received conditions using natural language processing. Specifically, it tokenizes the conditions and extracts keywords. It generates analysis results based on keywords such as "spicy" or "gluten-free."

[0488] Step 5:

[0489] Based on the analysis results, the server searches for matching dishes on the menu. It searches the server's database and generates a list of dishes that match the criteria. For "spicy food," it will select dishes such as "mapo tofu" or "curry."

[0490] Step 6:

[0491] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[0492] Step 7:

[0493] The device displays the suggested dishes to the user. Suggested dishes such as "Mapo Tofu" and "Curry" are displayed in a list format on the screen.

[0494] Step 8:

[0495] The user selects the dish they wish to order from the suggested dishes by tapping on the desired dish and pressing the "Order" button.

[0496] Step 9:

[0497] The terminal sends the final order details to the server, where the selected dishes are encoded in JSON format or similar and sent to the server.

[0498] Step 10:

[0499] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0500] In this way, users can place orders efficiently and stores can respond quickly.

[0501] Example 1

[0502] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0503] When ordering at a restaurant, a system is needed that allows users to quickly select the appropriate dish based on their preferences and allergies. Restaurants also need to efficiently receive orders and serve food smoothly. With conventional systems, communication between users and restaurants can be time-consuming, potentially resulting in mismatched orders. A new system is needed to solve these problems.

[0504] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0505] In this invention, the server includes input means for allowing a user to input conditions, transmission means for transmitting the conditions input by the input means to the server, selection means for selecting dishes to be proposed by the server based on the conditions, display means for presenting the dishes selected by the selection means to the user, selection means for the user to select an order from the presented dishes, order transmission means for transmitting the order selected by the selection means to the server, and notification means for notifying the store's management system of the order details. This allows the user to easily input conditions, and the server to quickly analyze and select dishes, enabling an efficient ordering process.

[0506] "User" refers to a person who uses a restaurant ordering system.

[0507] "Input means" refers to a device or interface for a user to input conditions.

[0508] "Transmission means" refers to a communication means for transmitting data entered by a user to a server.

[0509] "Server" refers to a computer system that receives, processes, and analyzes data sent by users.

[0510] "Selection means" refers to the function by which the server selects dishes to suggest based on the user's conditions.

[0511] "Display means" refers to a device or interface for visually presenting the dishes selected by the server to the user.

[0512] The "selection means" refers to the means by which the user selects the desired order from the displayed dishes.

[0513] "Order sending means" refers to a communication means for sending an order selected by a user to a server.

[0514] "Notification means" refers to the function by which the server notifies the store's management system of the final order details.

[0515] "Natural language processing" refers to a technology in which a server analyzes user input data and extracts appropriate keywords and phrases.

[0516] This system allows users to efficiently place orders at restaurants and is configured as follows.

[0517] First, a user launches a restaurant ordering application on a device such as a smartphone or tablet. The application displays a menu, and the user uses an input device to enter their preferences and allergies. The input device includes text boxes and options, allowing users to enter specific preferences such as "spicy food," "vegetarian," or "gluten-free."

[0518] Next, the terminal sends the conditions entered by the user to the server. The transmission method is HTTP protocol, and the data is sent in JSON format. Specifically, when the user presses the "Send" button, the terminal sends the input data to the server.

[0519] The server uses Python natural language processing libraries (e.g., nltk or spaCy) to analyze the received data. Through analysis, the server extracts keywords from the user's input and selects dishes based on the criteria. Analyzed keywords include "spicy," "vegetarian," and "gluten-free."

[0520] Next, the server uses a selection method to search for food data in a database (MySQL or PostgreSQL). Using this selection method, the server lists dishes that match the user's criteria. For example, dishes that match the criteria of "spicy" and "gluten-free" might include "Mapo Tofu" and "Spicy Gluten-Free Salad."

[0521] Once the list of selected dishes is complete, the server sends it to the device in JSON format. The device then uses a display mechanism to present the list to the user. The display mechanism uses the Flutter or React Native framework, and UI components such as list views are used to visually present the list to the user.

[0522] When the user selects a desired dish from the displayed list, the user uses a selection means. Specifically, the user taps the name of the dish (e.g., "Mapo Tofu") and presses the "Confirm Order" button to confirm the selection.

[0523] The terminal then transmits the order details confirmed by the selection means to the server, again using the HTTP protocol and JSON format.

[0524] Finally, the server notifies the restaurant's management system of the final order details using a REST API, allowing restaurant staff to begin preparing the food.

[0525] As a concrete example, consider the case where a user launches an application and enters the conditions "spicy food" and "gluten-free." The device sends these conditions to the server, which analyzes them and selects "mapo tofu" and "spicy gluten-free salad" to create a list. The device displays this list to the user, who selects "mapo tofu" and confirms the order. The server then notifies the restaurant of the final order details, and food preparation begins.

[0526] Example prompt for a generative AI model:

[0527] Please explain a system that streamlines ordering at restaurants. This system has a function that allows users to input their preferences and allergies, and the server analyzes these and suggests dishes. Please explain the following process in detail.

[0528] 1. User Input

[0529] 2. Conditional Sending

[0530] 3. Server Analysis

[0531] 4. Food selection

[0532] 5. Submit your proposal

[0533] 6. User Display

[0534] 7. Order Selection

[0535] 8. Submitting your order

[0536] 9. Notification to stores

[0537] The system uses a Python natural language processing library and a MySQL database. For example, include a case where a user inputs "spicy food" and "gluten-free" and finally selects "mapo tofu."

[0538] In this way, the present invention allows the user to easily input requirements and the server to quickly analyze and select, thereby enabling an efficient ordering process.

[0539] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0540] Step 1:

[0541] A user launches a restaurant ordering application using a smartphone or tablet device.

[0542] Input: Launch application

[0543] Output: Display the menu screen

[0544] Specific behavior: The application displays a menu screen on the device and allows the user to enter conditions.

[0545] Step 2:

[0546] The user inputs their preferences and allergies.

[0547] Input: User criteria (spicy food, vegetarian, gluten-free, etc.)

[0548] Output: Confirmation of input data

[0549] Specific behavior: The user enters conditions in the text boxes or options and clicks the "Submit" button.

[0550] Step 3:

[0551] The terminal transmits the input conditions to the server.

[0552] Input: User input data (JSON format)

[0553] Output: Notification of completion of transmission to the server

[0554] Specific operation: The terminal generates an HTTP request and sends the input data in JSON format to the server.

[0555] Step 4:

[0556] The server analyzes the received conditions.

[0557] Input: JSON data submitted

[0558] Output: Extracted keywords

[0559] What it does: The server uses a Python natural language processing library (such as nltk or spaCy) to extract keywords from the conditions.

[0560] Step 5:

[0561] The server selects dishes based on the analyzed keywords.

[0562] Input: Extracted keywords

[0563] Output: A list of suitable dishes

[0564] Specific operation: The server uses a database management system (MySQL or PostgreSQL) to search for dishes that match the keywords and generate a list.

[0565] Step 6:

[0566] The server sends the selected dish list to the terminal.

[0567] Input: List of selected dishes (JSON format)

[0568] Output: Notification of completion of transmission to the terminal

[0569] Specific behavior: The server generates an HTTP response and sends the list of dishes in JSON format to the device.

[0570] Step 7:

[0571] The terminal displays the received recipe list to the user.

[0572] Input: A list of dishes sent from the server

[0573] Output: what is displayed to the user

[0574] What it does: The device uses the Flutter and React Native frameworks to display a list of dishes on a user interface.

[0575] Step 8:

[0576] The user selects an order from the displayed list of dishes.

[0577] Input: User's food selection

[0578] Output: Confirmed order data

[0579] Specific operation: The user taps the desired dish and presses the "Confirm order" button.

[0580] Step 9:

[0581] The terminal sends the confirmed order to the server.

[0582] Input: Confirmed order data (JSON format)

[0583] Output: Notification of completion of transmission to the server

[0584] Specific operation: The terminal sends the order data in JSON format to the server as an HTTP request.

[0585] Step 10:

[0586] The server notifies the store's management system of the final order details.

[0587] Input: Confirmed order data

[0588] Output: Notification to store management system completed

[0589] Specific operation: The server notifies the store management system of the order data via the REST API.

[0590] (Application example 1)

[0591] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0592] In today's restaurants and food delivery services, it is difficult for users to select appropriate dishes taking into account individual preferences, allergies, and other factors. It is also difficult for restaurants and delivery services to provide menus that reflect the different preferences of each user, which can result in a decrease in user satisfaction. Furthermore, when a user simply wants to reorder a dish they have ordered in the past, it is inefficient because it requires time and effort to manually search for the dish. A system that solves these problems is needed.

[0593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0594] In this invention, the server includes an analysis means that analyzes the user's input criteria using natural language processing and calculates cosine similarity to select the dish that most closely matches the user's criteria, a suggestion means that generates a list of dishes selected by the analysis means and suggests them to the user, and an order history management means that makes it easy to place repeat orders based on past order history.This allows users to quickly and accurately select and order dishes based on their individual preferences and allergy information.Furthermore, utilizing past order history makes repeat orders easier, improving user satisfaction.

[0595] "Input means" refers to a device or interface that allows a user to input conditions.

[0596] The "transmission means" refers to a device or program that transmits the conditions input by the input means to the server.

[0597] The "selection means" refers to a device or program that allows the server to select dishes to propose based on the above conditions.

[0598] The "display means" refers to a device or program that presents the dishes selected by the selection means to the user.

[0599] The "selection means" refers to a device or program that allows the user to select an order from the presented dishes.

[0600] The "order sending means" refers to a device or program that sends the order selected by the selection means to the server.

[0601] "Analysis means" refers to a device or program that analyzes the conditions entered by the user based on natural language processing, calculates the cosine similarity, and selects the dish that most closely matches the user's conditions.

[0602] The "suggestion means" refers to a device or program for generating a list of dishes selected by the analysis means and suggesting the list to the user.

[0603] "Order history management means" refers to a device or program that allows repeat orders to be easily placed based on past order history.

[0604] "Natural language processing" refers to the technology that enables computers to understand and analyze human language.

[0605] "Cosine similarity" refers to a method for calculating the similarity between vectorized data.

[0606] A specific system for carrying out the invention has the following configuration.

[0607] First, a user launches the application using their device, such as a smartphone or tablet. The application provides input means for the user to enter their preferences and allergy information. Input means include text boxes and check boxes. This input information is then sent from the device to the server.

[0608] Next, the server receives the user's request information. It then uses natural language processing technology to analyze the user's input. Specifically, it uses Python's Scikit-learn library to vectorize the text and calculate the cosine similarity. Based on this analysis, it selects the dish from the menu that most closely matches the user's request.

[0609] The server sends a list of selected dishes to the terminal. The terminal displays this list to the user. The user selects the dishes they like from the suggestions and confirms the order. The user's selection information is sent back to the server, which then confirms the final order.

[0610] Furthermore, this system includes an order history management means, which allows users to easily place repeat orders based on the history information of dishes they have ordered in the past.

[0611] Specific examples of prompt sentences for implementing the present invention are shown below: The user inputs prompt sentences such as "Please suggest spicy and vegetarian dishes" or "I'm a vegetarian and I like spicy dishes. Please tell me your recommendations."

[0612] This system allows users to efficiently and accurately select and order dishes, enabling restaurants and food delivery services to improve user satisfaction.

[0613] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0614] Step 1:

[0615] A user launches an application using a device such as a smartphone or tablet. The user then enters their preferences and allergy information within the application. This information is entered using text boxes and check boxes.

[0616] Input: User criteria (e.g. "spicy food", "vegetarian", "gluten-free", etc.)

[0617] Output: User condition data

[0618] Step 2:

[0619] The user's condition data is sent from the device to the server. This information is sent to the server using an HTTP request.

[0620] Input: User condition data

[0621] Output: Condition data sent to the server

[0622] Step 3:

[0623] The server then uses natural language processing technology to analyze the received user requirements. This analysis involves vectorizing the text data using the Scikit-learn library, and then calculating cosine similarity to extract dishes from the menu that match the user's requirements.

[0624] Input: Condition data sent to the server

[0625] Output: A list of dishes that match the user's criteria

[0626] Step 4:

[0627] The server sends a list of selected dishes to the device, and returns this information to the device in an HTTP response.

[0628] Input: A list of dishes that match the user's criteria

[0629] Output: Recipe list sent to the device

[0630] Step 5:

[0631] The terminal displays the list of dishes received from the server to the user, who then checks the list and selects the dish they wish to order.

[0632] Input: Recipe list sent to device

[0633] Output: User-selected dishes

[0634] Step 6:

[0635] The user's selected dish data is sent from the device to the server, again using an HTTP request.

[0636] Input: User-selected dish data

[0637] Output: Final order data sent to the server

[0638] Step 7:

[0639] The server receives the final order data and confirms the order. The order details are saved on the server and sent to the restaurant. The order details are then sent to the restaurant's management system, and the staff begins preparing the food.

[0640] Input: Final order data sent to the server

[0641] Output: Order details notified to the store

[0642] Step 8:

[0643] The server manages the order history and makes this information available to users when they place repeat orders in the future. An interface is provided to make it easy to place repeat orders based on past order history.

[0644] Input: User's past order data

[0645] Output: Historical data available for repeat orders

[0646] This allows users to efficiently select dishes and confirm their orders, and by utilizing past order history, they can easily place repeat orders.

[0647] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0648] This invention relates to combining an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which a user inputs order conditions, selects dishes suggested by a server based on the conditions and the user's emotions, and finally confirms the order.

[0649] System Overview:

[0650] 1. User Input

[0651] A user launches a restaurant's application on a device such as a smartphone or tablet, and the menu is displayed. The device then sends a request to the server, which retrieves the menu list and displays it on the screen. The user then inputs preferences, allergies, and other conditions. For example, they can specify conditions such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[0652] 2. Emotion recognition

[0653] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. For example, it uses facial recognition technology and voice analysis technology to identify the user's emotional state, such as whether they are happy, excited, relaxed, etc.

[0654] 3. Sending conditions and emotions

[0655] The device sends the input conditions and recognized emotions to the server, which simultaneously sends the user's conditions and emotion information to the server, providing the data that will be used as the basis for analysis.

[0656] 4. Server Analysis

[0657] The server analyzes the order conditions and emotional data it receives using natural language processing and emotion analysis technology. Specifically, it extracts keywords from the conditions and determines the user's emotional state from the emotional data. For example, if a user requests "spicy food" and is "excited," it will prioritize suggesting spicy food.

[0658] 5. Food selection

[0659] Based on the analysis results, the server searches for the dish from the menu that best suits the conditions and emotions. It searches the menu items in the database and generates a list of dishes according to the conditions and emotions. For example, for a user who likes "spicy food" and is "excited," it selects "super spicy mapo tofu" or "spicy curry."

[0660] 6. Submit your proposal

[0661] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[0662] 7. User Presentation

[0663] The device displays the suggestions sent from the server to the user. The suggested dishes are displayed in a list format on the screen. For example, dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are presented.

[0664] 8. Order Selection

[0665] The user selects the desired dish from the suggested dishes, taps on the desired dish, and presses the "Order" button.

[0666] 9. Submitting your order

[0667] The device sends the selected order details to the server, which then encodes the selected dishes in JSON format and sends them to the server.

[0668] 10. Notice to Stores

[0669] The server notifies the restaurant of the final order details, which are then transmitted to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0670] In this way, by combining and analyzing user input and emotional information, more accurate and personalized dish suggestions can be made, thereby increasing user satisfaction and store operational efficiency.

[0671] The processing flow will be explained below.

[0672] Step 1:

[0673] A user launches a restaurant application on their device, which sends a request to the server, downloads a menu list, and displays it on the screen.

[0674] Step 2:

[0675] The user inputs ordering conditions, such as "spicy food" or "vegetarian food preferred," into text boxes and check boxes.

[0676] Step 3:

[0677] The device recognizes the user's facial expressions and voice using an emotion engine. The device's camera captures the user's facial expressions and the microphone records the user's voice. The emotion engine analyzes this data to identify the user's emotional state (e.g., happy, excited, relaxed).

[0678] Step 4:

[0679] The terminal transmits the input conditions and the recognized emotion data to the server, thereby providing the user's order conditions and emotion information to the server.

[0680] Step 5:

[0681] The server analyzes the received conditions and emotional data. It uses natural language processing technology to analyze the order conditions and simultaneously analyzes the emotional data to determine the user's emotional state. For example, it analyzes the emotional state of "spicy food" and "excited."

[0682] Step 6:

[0683] The server selects dishes based on the conditions and emotional data. From the menu in the database, it selects "spicy dishes" that are suitable for "excited" users. For example, it suggests "super spicy mapo tofu" or "spicy curry."

[0684] Step 7:

[0685] The server sends the list of selected dishes to the device, which then encodes the list in an appropriate data format (e.g., JSON) and sends it to the device.

[0686] Step 8:

[0687] The device displays the suggested dishes to the user. Dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are displayed in a list format on the screen.

[0688] Step 9:

[0689] The user selects the desired dish from the suggested dishes and presses the "Order" button. For example, the user selects and orders "Super Spicy Mapo Tofu."

[0690] Step 10:

[0691] The terminal sends the selected order details to the server. The dishes selected by the user are encoded in a data format and sent to the server.

[0692] Step 11:

[0693] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0694] In this way, the combined analysis of user input conditions and emotional information leads to more personalized suggestions and an improved user experience.

[0695] Example 2

[0696] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] Conventional restaurant ordering systems mainly provide suggestions based on the user's preferences, but lack personalized suggestions that take into account the user's emotional state. There is a need for a method to improve satisfaction by providing dish suggestions that adapt to the user's emotions.

[0698] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means by which the user can input conditions, a transmission means for transmitting the conditions input by the input means and the user's emotional information to the server, a selection means by which the server selects dishes to suggest based on the conditions and emotional information, a display means for presenting the dishes selected by the selection means to the user, a selection means by which the user selects an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration.

[0699] "Input means" refers to a device or interface for inputting desired conditions and information by the user.

[0700] The "transmission means" is a device or system that transmits input information or data to a server via a network.

[0701] The "selection means" is a device or algorithm that performs the process of selecting appropriate proposals based on the conditions and information received by the server.

[0702] The "display means" is a device or interface that visually shows the proposal selected by the selection means to the user.

[0703] The "selection means" is a device or system that allows the user to select a desired item from the displayed suggestions.

[0704] The "order sending means" is a device or system that sends the order details selected by the user to the server.

[0705] "Natural language processing" is a technology that enables computers to understand, analyze, and respond to human language.

[0706] "Emotion analysis" is a technology for identifying a user's emotional state from their facial expressions and voice.

[0707] The "notification means" is a device or system that allows the server to transmit the order details to the store.

[0708] This invention combines an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs their order conditions, selects dishes suggested by the server based on those conditions and the user's emotions, and finally confirms the order.

[0709] Basic system configuration

[0710] This system is composed of a user input means, a transmission means, a selection means, a display means, a selection means, and an order transmission means. Each means will be explained in detail below.

[0711] User input method

[0712] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu. The device then sends an HTTP request to the server to retrieve the menu list. This retrieval process includes retrieving menu information from a MySQL database, encoding it in JSON format, and sending it to the device. The retrieved menu is then displayed to the user on the device using a UI created with React Native or Flutter.

[0713] Users input their preferences, allergies, and other conditions via text boxes and radio buttons on the terminal.

[0714] emotion recognition means

[0715] As the user enters their conditions, the device's camera and microphone capture their facial expressions and voice, which are then used to recognize their emotions using Google's Face API, Amazon's Rekognition, Microsoft's AI services, and more. This emotion recognition identifies the user's emotional state, such as whether they are happy, excited, or relaxed.

[0716] For example, if a user smiles while looking at a menu, the camera captures that smile and each AI service recognizes the emotion as "enjoying."

[0717] Server analysis and selection method

[0718] The entered conditions and recognized emotion information are encoded in JSON format by the device and sent to the server. The server receives this data and analyzes it using natural language processing (NLP) and emotion analysis techniques. Specifically, it uses libraries such as NLTK and spaCy to extract keywords from the conditions and determine the user's emotional state from the emotion data.

[0719] For example, the server receives data such as "spicy food" and "enjoying it," and determines that "spicy food" matches "super spicy mapo tofu" or "spicy curry."

[0720] The server then selects the dish from the menu that best suits the criteria and emotion, using a MySQL query to generate a list of dishes based on the criteria and emotion.

[0721] Proposal and ordering methods

[0722] The selected suggestions are encoded in JSON format and sent to the device as an HTTP response. The device then displays the received suggestions to the user. A list of appropriate dishes is displayed on the screen using a UI created with React Native or Flutter.

[0723] The user selects the desired dish from the presented dishes and presses the "Order" button. The selected order details are also encoded in JSON format and sent to the server.

[0724] Finally, the server notifies the store's order management system (e.g., a POS system) of the order details, which then confirms the order and begins cooking.

[0725] Specific examples and prompts for the generative AI model

[0726] For example, if a user requests "spicy food" and their emotion is recognized as "enjoying it," the system will suggest dishes such as "super spicy mapo tofu" or "spicy curry."

[0727] Prompt sentence for generative AI model

[0728] Example 1: "If the user selects a spicy dish and facial analysis indicates that they are enjoying it, please recommend a dish."

[0729] Example 2: "If a user selects gluten-free as their allergy information and a happy expression is recognized, suggest appropriate dishes."

[0730] As a result, personalized recipe suggestions that take into account the user's emotional state become possible, thereby improving user satisfaction.

[0731] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0732] Step 1:

[0733] User Input

[0734] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu.

[0735] Specific operation: When a user opens the app, the device sends an HTTP request (e.g., GET / menu) to the server to retrieve a menu list. The server retrieves menu information from the MySQL database and sends it to the device in JSON format. Input is done through text boxes and check boxes. Input contents include conditions such as "spicy food," "vegetarian," and "gluten-free."

[0736] Input: User criteria (e.g. "spicy food" or "vegetarian")

[0737] Output: Menu list displayed on the device

[0738] Step 2:

[0739] emotion recognition

[0740] The device captures the user's facial expressions and voice using a camera and microphone.

[0741] How it works: The device uses Google's Face API, Amazon's Rekognition, and Microsoft's AI services to analyze the user's emotions, recognizing emotions such as "enjoyed" or "excited" based on the user's facial expressions and voice patterns.

[0742] Input: User's facial expressions and voice data

[0743] Output: Parsed emotion data (e.g., "having fun")

[0744] Step 3:

[0745] Sending conditions and emotions

[0746] The device sends the input conditions and recognized emotions to the server in JSON format.

[0747] Specific operation: The device sends a request to the POST / analyze endpoint: {"conditions": ["spicy", "vegetarian"], "emotion": "enjoying"}.

[0748] Input: User condition and emotion data

[0749] Output: Data sent to the server

[0750] Step 4:

[0751] Server Analysis

[0752] The server analyzes the order conditions and emotional data received using natural language processing (NLP) and emotion analysis technology.

[0753] Specific operation: The server uses libraries such as NLTK and spaCy to extract keywords from the conditions and identify the user's emotions from the emotional data. For example, if the conditions are "spicy food" and "enjoying it," the server will extract "super spicy mapo tofu" and "spicy curry" as matching candidates.

[0754] Input: User condition and emotion data

[0755] Output: Parsed list of recommended dishes

[0756] Step 5:

[0757] Food selection

[0758] Based on the analysis results, the server selects the dish from the restaurant's menu that best suits the conditions and emotions.

[0759] What it does: The server runs the query SELECT FROM menu WHERE spicy='true' AND category='vegetarian' against the MySQL database to extract dishes that fit the criteria, generating a list of appropriate dishes.

[0760] Input: Parsed conditions and emotion data

[0761] Output: A list of recommended dishes based on criteria and sentiment

[0762] Step 6:

[0763] Submit your proposal

[0764] The server encodes the list of selected dishes in JSON format and sends it to the terminal as an HTTP response.

[0765] Specific operation: For example, generate JSON data {"dishes": ["Super spicy mapo tofu", "spicy curry"]} and send it to the terminal.

[0766] Input: Recommended dish list

[0767] Output: Suggestion sent to device

[0768] Step 7:

[0769] What the user sees

[0770] The terminal displays the received proposal to the user.

[0771] Specific operation: A list of dishes is displayed on the device screen through a UI created with React Native or Flutter. For example, "Super Spicy Mapo Tofu" and "Spicy Curry" are listed.

[0772] Input: Proposal received from the server

[0773] Output: A list of dishes displayed to the user

[0774] Step 8:

[0775] Order Selection

[0776] The user selects the desired dish from the suggested dishes and presses the "Order" button.

[0777] Specific operation: The user taps on "Super Spicy Mapo Tofu" and presses the "Order" button to confirm the selection.

[0778] Input: User's food selection

[0779] Output: Selected order details

[0780] Step 9:

[0781] Sending order details

[0782] The terminal encodes the selected order details in JSON format and sends them to the server.

[0783] Specific operation: The terminal sends a request to the POST / order endpoint: {"order": "Super spicy mapo tofu"}.

[0784] Input: Selected order details

[0785] Output: Order data sent to the server

[0786] Step 10:

[0787] Notification to stores

[0788] The server notifies the store's order management system of the final order details.

[0789] Specific operation: The server sends a request to the store's order management API: {"order": "Super spicy mapo tofu"}. The store then confirms the order and begins cooking.

[0790] Input: User's order details

[0791] Output: Order details sent to the store

[0792] (Application example 2)

[0793] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] Conventional restaurant and food delivery systems only consider user preferences and allergies when suggesting dishes, which means they are unable to suggest dishes that take into account the user's emotional state. This makes it difficult for users to select an appropriate dish that matches their current mood and emotions. This can lead to a decrease in user satisfaction, so there is a need for a suggestion system that incorporates emotions.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0796] In this invention, the server includes an input means for allowing a user to input conditions and emotions, a transmission means for transmitting the conditions and emotions input by the input means to the server, a selection means for the server to select dishes to suggest based on the conditions and emotions, a display means for presenting the dishes selected by the selection means to the user, a selection means for the user to select an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration, thereby increasing user satisfaction.

[0797] "Input means" refers to a device or interface for a user to input conditions and emotions.

[0798] The "transmission means" is a device or function for transmitting the conditions and emotions input by the input means to the server.

[0799] The "selection means" is a device or function that allows the server to select the most suitable dish based on the above conditions and emotions.

[0800] The "display means" is a device or function for presenting the dish selected by the selection means to the user.

[0801] The "selection means" is a device or interface that allows the user to select a desired dish from the dishes presented.

[0802] The "order sending means" is a device or function for sending the order selected by the selection means to the server.

[0803] The "emotion recognition means" is a device or function for analyzing the user's facial expressions and voice to recognize the user's emotional state.

[0804] The "notification means" is a device or function that allows the server to notify the store of the final order details.

[0805] This invention is a system that allows users to efficiently use restaurants and food delivery services, and by combining an emotion recognition engine, suggests dishes based on the user's emotional state. This system includes input means, transmission means, selection means, display means, selection means, order transmission means, emotion recognition means, and notification means.

[0806] 1. Overall system configuration

[0807] The main components of the system are as follows:

[0808] The user inputs conditions and emotions from a device such as a smartphone or tablet.

[0809] The terminal transmits the input data to the server.

[0810] The server analyzes the user's conditions and emotions and selects the most suitable dish.

[0811] The terminal presents the selected dishes to the user, and the user selects an order.

[0812] The terminal transmits the selected order to the server, which then notifies the store.

[0813] 2. Hardware and Software Used

[0814] Input methods: smartphone or tablet touchscreen, text box, selection options.

[0815] Emotion recognition means: smartphone or tablet camera, microphone, and facial recognition and voice analysis software via EmotionEngine (e.g., OpenCV or Google Cloud Speech-to-Text API).

[0816] Transmission method: Send data to the server (in JSON format via HTTP request) over an internet connection.

[0817] Selection method: In-server natural language processing and sentiment analysis algorithms.

[0818] Display medium: Smartphone or tablet display.

[0819] Options: Smartphone or tablet touchscreen interface.

[0820] Order sending method: Notification from the server to the store system (HTTP request or WebSocket communication).

[0821] Notification method: Store order management system.

[0822] 3. Example of the process

[0823] First, the user uses a smartphone application to input their preferred food preferences (e.g., "spicy food" or "vegetarian"). Next, the device's camera and microphone are used to recognize and analyze the user's emotions. This allows the system to identify the user's emotional state, such as "excited" or "relaxed."

[0824] The above data is sent to the server, which then performs natural language processing and sentiment analysis based on the data. For example, if the user requests "spicy food" and is in an "excited" state, spicy dishes (such as "super spicy mapo tofu" or "spicy curry") will be suggested. The suggested dishes are displayed on the smartphone application, and the user can select the desired dish and confirm the order.

[0825] 4. Examples of prompts

[0826] Below are some specific examples of input prompts for generative AI models.

[0827] User input criteria: spicy food, vegetarian

[0828] User sentiment: Excited

[0829] Generate suggested dishes.

[0830] Using this prompt, the AI ​​model can generate optimal recipe suggestions based on the user's emotions and cooking requirements, thereby suggesting dishes that match the user's current emotions and providing a highly satisfying user experience.

[0831] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0832] Step 1:

[0833] The user launches the app on their smartphone or tablet and enters the food preferences (e.g., "spicy food," "vegetarian," etc.).

[0834] Input: User condition input (text box, choice, etc.)

[0835] Output: Data of the entered conditions (JSON format)

[0836] Step 2:

[0837] The device's camera and microphone are used to collect the user's emotions, which are then analyzed by an emotion recognition engine (EmotionEngine).

[0838] Input: Camera video, microphone audio

[0839] Output: Recognized emotion data (e.g., "excited," "relaxed," etc.)

[0840] Step 3:

[0841] The terminal transmits the input conditions and the recognized emotion data to the server.

[0842] Input: Condition data, emotion data

[0843] Output: Request sent to the server (JSON format data)

[0844] Step 4:

[0845] Based on the conditions and emotional data received by the server, natural language processing and emotional analysis are performed to select the most suitable dish.

[0846] Input: Condition data, emotion data

[0847] Output: List of selected dishes (JSON format)

[0848] Step 5:

[0849] The server sends a list of selected dishes to the terminal.

[0850] Input: List of selected dishes

[0851] Output: A list of dishes sent to the device (JSON format)

[0852] Step 6:

[0853] The terminal displays the list of dishes sent from the server to the user, who then selects the dish they want from the list.

[0854] Input: list of dishes

[0855] Output: User's selected dish

[0856] Step 7:

[0857] The terminal transmits the dish selected by the user to the server.

[0858] Input: Selected dish data (JSON format)

[0859] Output: Order data sent to the server

[0860] Step 8:

[0861] The server notifies the restaurant of the final order, and the restaurant begins cooking based on the order.

[0862] Input: Order data

[0863] Output: Order notified to the store

[0864] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0865] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0866] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0867] [Third embodiment]

[0868] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0869] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0870] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0871] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0872] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0873] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0874] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0875] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0876] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0877] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0878] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0879] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0880] The present invention relates to a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs order conditions, selects dishes suggested by a server based on those conditions, and finally confirms the order. The program processing of this system is explained in detail below.

[0881] System Overview:

[0882] 1. User Input

[0883] A user launches a restaurant's application using a device such as a smartphone or tablet. The device displays a menu, and the user inputs their preferences, allergies, and other criteria. For example, they can specify criteria such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[0884] 2. Conditional Sending

[0885] The terminal transmits the conditions entered by the user to the server, which receives information to select dishes based on the user's preferences.

[0886] 3. Server Analysis

[0887] The server analyzes the received criteria using natural language processing technology. Specifically, it extracts keywords from the words and phrases entered by the user and determines the characteristics of the food based on them. For example, the keyword "spicy" can be used to search for spicy food.

[0888] 4. Food selection

[0889] Based on the analysis results, the server selects dishes from the menu that match the user's criteria. The server searches the menu items in the database and generates a list of dishes that match the criteria. For example, it might select "Mapo Tofu" or "Curry" as a "spicy dish."

[0890] 5. Submit your proposal

[0891] The server sends the list of selected dishes to the terminal, allowing the user to see dishes that match their criteria.

[0892] 6. User Display

[0893] The device displays the suggestions sent from the server to the user, allowing the user to choose the dish they like from the suggestions.

[0894] 7. Order Selection

[0895] The user selects the desired dish from the list of suggestions and confirms the order. The selected dish is sent to the server via the terminal by pressing the send button.

[0896] 8. Submitting your order

[0897] The terminal sends the order details selected by the user to the server, which then receives the final order details.

[0898] 9. Notification to stores

[0899] The server notifies the restaurant of the final order details, which are then passed on to the restaurant's management system, and staff begin preparing the food.

[0900] This system allows users to easily input their order requirements, and the server analyzes and selects the items, allowing them to receive prompt and accurate suggestions. Ultimately, this has the effect of streamlining the user's ordering process, improving store turnover and operational efficiency.

[0901] The processing flow will be explained below.

[0902] Step 1:

[0903] A user launches a restaurant application and displays the menu. The device sends a request to the server, retrieves the menu list, and displays it on the screen.

[0904] Step 2:

[0905] The user enters order conditions, such as "spicy food," "vegetarian," or "gluten-free," using text boxes, drop-down lists, and check boxes.

[0906] Step 3:

[0907] The device sends the entered conditions to the server. When the user enters the conditions and presses the "Send" button, the device sends the conditions to the server in a data format such as JSON.

[0908] Step 4:

[0909] The server analyzes the received conditions using natural language processing. Specifically, it tokenizes the conditions and extracts keywords. It generates analysis results based on keywords such as "spicy" or "gluten-free."

[0910] Step 5:

[0911] Based on the analysis results, the server searches for matching dishes on the menu. It searches the server's database and generates a list of dishes that match the criteria. For "spicy food," it will select dishes such as "mapo tofu" or "curry."

[0912] Step 6:

[0913] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[0914] Step 7:

[0915] The device displays the suggested dishes to the user. Suggested dishes such as "Mapo Tofu" and "Curry" are displayed in a list format on the screen.

[0916] Step 8:

[0917] The user selects the dish they wish to order from the suggested dishes by tapping on the desired dish and pressing the "Order" button.

[0918] Step 9:

[0919] The terminal sends the final order details to the server, where the selected dishes are encoded in JSON format or similar and sent to the server.

[0920] Step 10:

[0921] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[0922] In this way, users can place orders efficiently and stores can respond quickly.

[0923] Example 1

[0924] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0925] When ordering at a restaurant, a system is needed that allows users to quickly select the appropriate dish based on their preferences and allergies. Restaurants also need to efficiently receive orders and serve food smoothly. With conventional systems, communication between users and restaurants can be time-consuming, potentially resulting in mismatched orders. A new system is needed to solve these problems.

[0926] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0927] In this invention, the server includes input means for allowing a user to input conditions, transmission means for transmitting the conditions input by the input means to the server, selection means for selecting dishes to be proposed by the server based on the conditions, display means for presenting the dishes selected by the selection means to the user, selection means for the user to select an order from the presented dishes, order transmission means for transmitting the order selected by the selection means to the server, and notification means for notifying the store's management system of the order details. This allows the user to easily input conditions, and the server to quickly analyze and select dishes, enabling an efficient ordering process.

[0928] "User" refers to a person who uses a restaurant ordering system.

[0929] "Input means" refers to a device or interface for a user to input conditions.

[0930] "Transmission means" refers to a communication means for transmitting data entered by a user to a server.

[0931] "Server" refers to a computer system that receives, processes, and analyzes data sent by users.

[0932] "Selection means" refers to the function by which the server selects dishes to suggest based on the user's conditions.

[0933] "Display means" refers to a device or interface for visually presenting the dishes selected by the server to the user.

[0934] The "selection means" refers to the means by which the user selects the desired order from the displayed dishes.

[0935] "Order sending means" refers to a communication means for sending an order selected by a user to a server.

[0936] "Notification means" refers to the function by which the server notifies the store's management system of the final order details.

[0937] "Natural language processing" refers to a technology in which a server analyzes user input data and extracts appropriate keywords and phrases.

[0938] This system allows users to efficiently place orders at restaurants and is configured as follows.

[0939] First, a user launches a restaurant ordering application on a device such as a smartphone or tablet. The application displays a menu, and the user uses an input device to enter their preferences and allergies. The input device includes text boxes and options, allowing users to enter specific preferences such as "spicy food," "vegetarian," or "gluten-free."

[0940] Next, the terminal sends the conditions entered by the user to the server. The transmission method is HTTP protocol, and the data is sent in JSON format. Specifically, when the user presses the "Send" button, the terminal sends the input data to the server.

[0941] The server uses Python natural language processing libraries (e.g., nltk or spaCy) to analyze the received data. Through analysis, the server extracts keywords from the user's input and selects dishes based on the criteria. Analyzed keywords include "spicy," "vegetarian," and "gluten-free."

[0942] Next, the server uses a selection method to search for food data in a database (MySQL or PostgreSQL). Using this selection method, the server lists dishes that match the user's criteria. For example, dishes that match the criteria of "spicy" and "gluten-free" might include "Mapo Tofu" and "Spicy Gluten-Free Salad."

[0943] Once the list of selected dishes is complete, the server sends it to the device in JSON format. The device then uses a display mechanism to present the list to the user. The display mechanism uses the Flutter or React Native framework, and UI components such as list views are used to visually present the list to the user.

[0944] When the user selects a desired dish from the displayed list, the user uses a selection means. Specifically, the user taps the name of the dish (e.g., "Mapo Tofu") and presses the "Confirm Order" button to confirm the selection.

[0945] The terminal then transmits the order details confirmed by the selection means to the server, again using the HTTP protocol and JSON format.

[0946] Finally, the server notifies the restaurant's management system of the final order details using a REST API, allowing restaurant staff to begin preparing the food.

[0947] As a concrete example, consider the case where a user launches an application and enters the conditions "spicy food" and "gluten-free." The device sends these conditions to the server, which analyzes them and selects "mapo tofu" and "spicy gluten-free salad" to create a list. The device displays this list to the user, who selects "mapo tofu" and confirms the order. The server then notifies the restaurant of the final order details, and food preparation begins.

[0948] Example prompt for a generative AI model:

[0949] Please explain a system that streamlines ordering at restaurants. This system has a function that allows users to input their preferences and allergies, and the server analyzes these and suggests dishes. Please explain the following process in detail.

[0950] 1. User Input

[0951] 2. Conditional Sending

[0952] 3. Server Analysis

[0953] 4. Food selection

[0954] 5. Submit your proposal

[0955] 6. User Display

[0956] 7. Order Selection

[0957] 8. Submitting your order

[0958] 9. Notification to stores

[0959] The system uses a Python natural language processing library and a MySQL database. For example, include a case where a user inputs "spicy food" and "gluten-free" and finally selects "mapo tofu."

[0960] In this way, the present invention allows the user to easily input requirements and the server to quickly analyze and select, thereby enabling an efficient ordering process.

[0961] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0962] Step 1:

[0963] A user launches a restaurant ordering application using a smartphone or tablet device.

[0964] Input: Launch application

[0965] Output: Display the menu screen

[0966] Specific behavior: The application displays a menu screen on the device and allows the user to enter conditions.

[0967] Step 2:

[0968] The user inputs their preferences and allergies.

[0969] Input: User criteria (spicy food, vegetarian, gluten-free, etc.)

[0970] Output: Confirmation of input data

[0971] Specific behavior: The user enters conditions in the text boxes or options and clicks the "Submit" button.

[0972] Step 3:

[0973] The terminal transmits the input conditions to the server.

[0974] Input: User input data (JSON format)

[0975] Output: Notification of completion of transmission to the server

[0976] Specific operation: The terminal generates an HTTP request and sends the input data in JSON format to the server.

[0977] Step 4:

[0978] The server analyzes the received conditions.

[0979] Input: JSON data submitted

[0980] Output: Extracted keywords

[0981] What it does: The server uses a Python natural language processing library (such as nltk or spaCy) to extract keywords from the conditions.

[0982] Step 5:

[0983] The server selects dishes based on the analyzed keywords.

[0984] Input: Extracted keywords

[0985] Output: A list of suitable dishes

[0986] Specific operation: The server uses a database management system (MySQL or PostgreSQL) to search for dishes that match the keywords and generate a list.

[0987] Step 6:

[0988] The server sends the selected dish list to the terminal.

[0989] Input: List of selected dishes (JSON format)

[0990] Output: Notification of completion of transmission to the terminal

[0991] Specific behavior: The server generates an HTTP response and sends the list of dishes in JSON format to the device.

[0992] Step 7:

[0993] The terminal displays the received recipe list to the user.

[0994] Input: A list of dishes sent from the server

[0995] Output: what is displayed to the user

[0996] What it does: The device uses the Flutter and React Native frameworks to display a list of dishes on a user interface.

[0997] Step 8:

[0998] The user selects an order from the displayed list of dishes.

[0999] Input: User's food selection

[1000] Output: Confirmed order data

[1001] Specific operation: The user taps the desired dish and presses the "Confirm order" button.

[1002] Step 9:

[1003] The terminal sends the confirmed order to the server.

[1004] Input: Confirmed order data (JSON format)

[1005] Output: Notification of completion of transmission to the server

[1006] Specific operation: The terminal sends the order data in JSON format to the server as an HTTP request.

[1007] Step 10:

[1008] The server notifies the store's management system of the final order details.

[1009] Input: Confirmed order data

[1010] Output: Notification to store management system completed

[1011] Specific operation: The server notifies the store management system of the order data via the REST API.

[1012] (Application example 1)

[1013] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1014] In today's restaurants and food delivery services, it is difficult for users to select appropriate dishes taking into account individual preferences, allergies, and other factors. It is also difficult for restaurants and delivery services to provide menus that reflect the different preferences of each user, which can result in a decrease in user satisfaction. Furthermore, when a user simply wants to reorder a dish they have ordered in the past, it is inefficient because it requires time and effort to manually search for the dish. A system that solves these problems is needed.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1016] In this invention, the server includes an analysis means that analyzes the user's input criteria using natural language processing and calculates cosine similarity to select the dish that most closely matches the user's criteria, a suggestion means that generates a list of dishes selected by the analysis means and suggests them to the user, and an order history management means that makes it easy to place repeat orders based on past order history.This allows users to quickly and accurately select and order dishes based on their individual preferences and allergy information.Furthermore, utilizing past order history makes repeat orders easier, improving user satisfaction.

[1017] "Input means" refers to a device or interface that allows a user to input conditions.

[1018] The "transmission means" refers to a device or program that transmits the conditions input by the input means to the server.

[1019] The "selection means" refers to a device or program that allows the server to select dishes to propose based on the above conditions.

[1020] The "display means" refers to a device or program that presents the dishes selected by the selection means to the user.

[1021] The "selection means" refers to a device or program that allows the user to select an order from the presented dishes.

[1022] The "order sending means" refers to a device or program that sends the order selected by the selection means to the server.

[1023] "Analysis means" refers to a device or program that analyzes the conditions entered by the user based on natural language processing, calculates the cosine similarity, and selects the dish that most closely matches the user's conditions.

[1024] The "suggestion means" refers to a device or program for generating a list of dishes selected by the analysis means and suggesting the list to the user.

[1025] "Order history management means" refers to a device or program that allows repeat orders to be easily placed based on past order history.

[1026] "Natural language processing" refers to the technology that enables computers to understand and analyze human language.

[1027] "Cosine similarity" refers to a method for calculating the similarity between vectorized data.

[1028] A specific system for carrying out the invention has the following configuration.

[1029] First, a user launches the application using their device, such as a smartphone or tablet. The application provides input means for the user to enter their preferences and allergy information. Input means include text boxes and check boxes. This input information is then sent from the device to the server.

[1030] Next, the server receives the user's request information. It then uses natural language processing technology to analyze the user's input. Specifically, it uses Python's Scikit-learn library to vectorize the text and calculate the cosine similarity. Based on this analysis, it selects the dish from the menu that most closely matches the user's request.

[1031] The server sends a list of selected dishes to the terminal. The terminal displays this list to the user. The user selects the dishes they like from the suggestions and confirms the order. The user's selection information is sent back to the server, which then confirms the final order.

[1032] Furthermore, this system includes an order history management means, which allows users to easily place repeat orders based on the history information of dishes they have ordered in the past.

[1033] Specific examples of prompt sentences for implementing the present invention are shown below: The user inputs prompt sentences such as "Please suggest spicy and vegetarian dishes" or "I'm a vegetarian and I like spicy dishes. Please tell me your recommendations."

[1034] This system allows users to efficiently and accurately select and order dishes, enabling restaurants and food delivery services to improve user satisfaction.

[1035] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1036] Step 1:

[1037] A user launches an application using a device such as a smartphone or tablet. The user then enters their preferences and allergy information within the application. This information is entered using text boxes and check boxes.

[1038] Input: User criteria (e.g. "spicy food", "vegetarian", "gluten-free", etc.)

[1039] Output: User condition data

[1040] Step 2:

[1041] The user's condition data is sent from the device to the server. This information is sent to the server using an HTTP request.

[1042] Input: User condition data

[1043] Output: Condition data sent to the server

[1044] Step 3:

[1045] The server then uses natural language processing technology to analyze the received user requirements. This analysis involves vectorizing the text data using the Scikit-learn library, and then calculating cosine similarity to extract dishes from the menu that match the user's requirements.

[1046] Input: Condition data sent to the server

[1047] Output: A list of dishes that match the user's criteria

[1048] Step 4:

[1049] The server sends a list of selected dishes to the device, and returns this information to the device in an HTTP response.

[1050] Input: A list of dishes that match the user's criteria

[1051] Output: Recipe list sent to the device

[1052] Step 5:

[1053] The terminal displays the list of dishes received from the server to the user, who then checks the list and selects the dish they wish to order.

[1054] Input: Recipe list sent to device

[1055] Output: User-selected dishes

[1056] Step 6:

[1057] The user's selected dish data is sent from the device to the server, again using an HTTP request.

[1058] Input: User-selected dish data

[1059] Output: Final order data sent to the server

[1060] Step 7:

[1061] The server receives the final order data and confirms the order. The order details are saved on the server and sent to the restaurant. The order details are then sent to the restaurant's management system, and the staff begins preparing the food.

[1062] Input: Final order data sent to the server

[1063] Output: Order details notified to the store

[1064] Step 8:

[1065] The server manages the order history and makes this information available to users when they place repeat orders in the future. An interface is provided to make it easy to place repeat orders based on past order history.

[1066] Input: User's past order data

[1067] Output: Historical data available for repeat orders

[1068] This allows users to efficiently select dishes and confirm their orders, and by utilizing past order history, they can easily place repeat orders.

[1069] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1070] This invention relates to combining an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which a user inputs order conditions, selects dishes suggested by a server based on the conditions and the user's emotions, and finally confirms the order.

[1071] System Overview:

[1072] 1. User Input

[1073] A user launches a restaurant's application on a device such as a smartphone or tablet, and the menu is displayed. The device then sends a request to the server, which retrieves the menu list and displays it on the screen. The user then inputs preferences, allergies, and other conditions. For example, they can specify conditions such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[1074] 2. Emotion recognition

[1075] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. For example, it uses facial recognition technology and voice analysis technology to identify the user's emotional state, such as whether they are happy, excited, relaxed, etc.

[1076] 3. Sending conditions and emotions

[1077] The device sends the input conditions and recognized emotions to the server, which simultaneously sends the user's conditions and emotion information to the server, providing the data that will be used as the basis for analysis.

[1078] 4. Server Analysis

[1079] The server analyzes the order conditions and emotional data it receives using natural language processing and emotion analysis technology. Specifically, it extracts keywords from the conditions and determines the user's emotional state from the emotional data. For example, if a user requests "spicy food" and is "excited," it will prioritize suggesting spicy food.

[1080] 5. Food selection

[1081] Based on the analysis results, the server searches for the dish from the menu that best suits the conditions and emotions. It searches the menu items in the database and generates a list of dishes according to the conditions and emotions. For example, for a user who likes "spicy food" and is "excited," it selects "super spicy mapo tofu" or "spicy curry."

[1082] 6. Submit your proposal

[1083] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[1084] 7. User Presentation

[1085] The device displays the suggestions sent from the server to the user. The suggested dishes are displayed in a list format on the screen. For example, dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are presented.

[1086] 8. Order Selection

[1087] The user selects the desired dish from the suggested dishes, taps on the desired dish, and presses the "Order" button.

[1088] 9. Submitting your order

[1089] The device sends the selected order details to the server, which then encodes the selected dishes in JSON format and sends them to the server.

[1090] 10. Notice to Stores

[1091] The server notifies the restaurant of the final order details, which are then transmitted to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[1092] In this way, by combining and analyzing user input and emotional information, more accurate and personalized dish suggestions can be made, thereby increasing user satisfaction and store operational efficiency.

[1093] The processing flow will be explained below.

[1094] Step 1:

[1095] A user launches a restaurant application on their device, which sends a request to the server, downloads a menu list, and displays it on the screen.

[1096] Step 2:

[1097] The user inputs ordering conditions, such as "spicy food" or "vegetarian food preferred," into text boxes and check boxes.

[1098] Step 3:

[1099] The device recognizes the user's facial expressions and voice using an emotion engine. The device's camera captures the user's facial expressions and the microphone records the user's voice. The emotion engine analyzes this data to identify the user's emotional state (e.g., happy, excited, relaxed).

[1100] Step 4:

[1101] The terminal transmits the input conditions and the recognized emotion data to the server, thereby providing the user's order conditions and emotion information to the server.

[1102] Step 5:

[1103] The server analyzes the received conditions and emotional data. It uses natural language processing technology to analyze the order conditions and simultaneously analyzes the emotional data to determine the user's emotional state. For example, it analyzes the emotional state of "spicy food" and "excited."

[1104] Step 6:

[1105] The server selects dishes based on the conditions and emotional data. From the menu in the database, it selects "spicy dishes" that are suitable for "excited" users. For example, it suggests "super spicy mapo tofu" or "spicy curry."

[1106] Step 7:

[1107] The server sends the list of selected dishes to the device, which then encodes the list in an appropriate data format (e.g., JSON) and sends it to the device.

[1108] Step 8:

[1109] The device displays the suggested dishes to the user. Dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are displayed in a list format on the screen.

[1110] Step 9:

[1111] The user selects the desired dish from the suggested dishes and presses the "Order" button. For example, the user selects and orders "Super Spicy Mapo Tofu."

[1112] Step 10:

[1113] The terminal sends the selected order details to the server. The dishes selected by the user are encoded in a data format and sent to the server.

[1114] Step 11:

[1115] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[1116] In this way, the combined analysis of user input conditions and emotional information leads to more personalized suggestions and an improved user experience.

[1117] Example 2

[1118] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1119] Conventional restaurant ordering systems mainly provide suggestions based on the user's preferences, but lack personalized suggestions that take into account the user's emotional state. There is a need for a method to improve satisfaction by providing dish suggestions that adapt to the user's emotions.

[1120] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means by which the user can input conditions, a transmission means for transmitting the conditions input by the input means and the user's emotional information to the server, a selection means by which the server selects dishes to suggest based on the conditions and emotional information, a display means for presenting the dishes selected by the selection means to the user, a selection means by which the user selects an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration.

[1121] "Input means" refers to a device or interface for inputting desired conditions and information by the user.

[1122] The "transmission means" is a device or system that transmits input information or data to a server via a network.

[1123] The "selection means" is a device or algorithm that performs the process of selecting appropriate proposals based on the conditions and information received by the server.

[1124] The "display means" is a device or interface that visually shows the proposal selected by the selection means to the user.

[1125] The "selection means" is a device or system that allows the user to select a desired item from the displayed suggestions.

[1126] The "order sending means" is a device or system that sends the order details selected by the user to the server.

[1127] "Natural language processing" is a technology that enables computers to understand, analyze, and respond to human language.

[1128] "Emotion analysis" is a technology for identifying a user's emotional state from their facial expressions and voice.

[1129] The "notification means" is a device or system that allows the server to transmit the order details to the store.

[1130] This invention combines an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs their order conditions, selects dishes suggested by the server based on those conditions and the user's emotions, and finally confirms the order.

[1131] Basic system configuration

[1132] This system is composed of a user input means, a transmission means, a selection means, a display means, a selection means, and an order transmission means. Each means will be explained in detail below.

[1133] User input method

[1134] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu. The device then sends an HTTP request to the server to retrieve the menu list. This retrieval process includes retrieving menu information from a MySQL database, encoding it in JSON format, and sending it to the device. The retrieved menu is then displayed to the user on the device using a UI created with React Native or Flutter.

[1135] Users input their preferences, allergies, and other conditions via text boxes and radio buttons on the terminal.

[1136] emotion recognition means

[1137] As the user enters their conditions, the device's camera and microphone capture their facial expressions and voice, which are then used to recognize their emotions using Google's Face API, Amazon's Rekognition, Microsoft's AI services, and more. This emotion recognition identifies the user's emotional state, such as whether they are happy, excited, or relaxed.

[1138] For example, if a user smiles while looking at a menu, the camera captures that smile and each AI service recognizes the emotion as "enjoying."

[1139] Server analysis and selection method

[1140] The entered conditions and recognized emotion information are encoded in JSON format by the device and sent to the server. The server receives this data and analyzes it using natural language processing (NLP) and emotion analysis techniques. Specifically, it uses libraries such as NLTK and spaCy to extract keywords from the conditions and determine the user's emotional state from the emotion data.

[1141] For example, the server receives data such as "spicy food" and "enjoying it," and determines that "spicy food" matches "super spicy mapo tofu" or "spicy curry."

[1142] The server then selects the dish from the menu that best suits the criteria and emotion, using a MySQL query to generate a list of dishes based on the criteria and emotion.

[1143] Proposal and ordering methods

[1144] The selected suggestions are encoded in JSON format and sent to the device as an HTTP response. The device then displays the received suggestions to the user. A list of appropriate dishes is displayed on the screen using a UI created with React Native or Flutter.

[1145] The user selects the desired dish from the presented dishes and presses the "Order" button. The selected order details are also encoded in JSON format and sent to the server.

[1146] Finally, the server notifies the store's order management system (e.g., a POS system) of the order details, which then confirms the order and begins cooking.

[1147] Specific examples and prompts for the generative AI model

[1148] For example, if a user requests "spicy food" and their emotion is recognized as "enjoying it," the system will suggest dishes such as "super spicy mapo tofu" or "spicy curry."

[1149] Prompt sentence for generative AI model

[1150] Example 1: "If the user selects a spicy dish and facial analysis indicates that they are enjoying it, please recommend a dish."

[1151] Example 2: "If a user selects gluten-free as their allergy information and a happy expression is recognized, suggest appropriate dishes."

[1152] As a result, personalized recipe suggestions that take into account the user's emotional state become possible, thereby improving user satisfaction.

[1153] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1154] Step 1:

[1155] User Input

[1156] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu.

[1157] Specific operation: When a user opens the app, the device sends an HTTP request (e.g., GET / menu) to the server to retrieve a menu list. The server retrieves menu information from the MySQL database and sends it to the device in JSON format. Input is done through text boxes and check boxes. Input contents include conditions such as "spicy food," "vegetarian," and "gluten-free."

[1158] Input: User criteria (e.g. "spicy food" or "vegetarian")

[1159] Output: Menu list displayed on the device

[1160] Step 2:

[1161] emotion recognition

[1162] The device captures the user's facial expressions and voice using a camera and microphone.

[1163] How it works: The device uses Google's Face API, Amazon's Rekognition, and Microsoft's AI services to analyze the user's emotions, recognizing emotions such as "enjoyed" or "excited" based on the user's facial expressions and voice patterns.

[1164] Input: User's facial expressions and voice data

[1165] Output: Parsed emotion data (e.g., "having fun")

[1166] Step 3:

[1167] Sending conditions and emotions

[1168] The device sends the input conditions and recognized emotions to the server in JSON format.

[1169] Specific operation: The device sends a request to the POST / analyze endpoint: {"conditions": ["spicy", "vegetarian"], "emotion": "enjoying"}.

[1170] Input: User condition and emotion data

[1171] Output: Data sent to the server

[1172] Step 4:

[1173] Server Analysis

[1174] The server analyzes the order conditions and emotional data received using natural language processing (NLP) and emotion analysis technology.

[1175] Specific operation: The server uses libraries such as NLTK and spaCy to extract keywords from the conditions and identify the user's emotions from the emotional data. For example, if the conditions are "spicy food" and "enjoying it," the server will extract "super spicy mapo tofu" and "spicy curry" as matching candidates.

[1176] Input: User condition and emotion data

[1177] Output: Parsed list of recommended dishes

[1178] Step 5:

[1179] Food selection

[1180] Based on the analysis results, the server selects the dish from the restaurant's menu that best suits the conditions and emotions.

[1181] What it does: The server runs the query SELECT FROM menu WHERE spicy='true' AND category='vegetarian' against the MySQL database to extract dishes that fit the criteria, generating a list of appropriate dishes.

[1182] Input: Parsed conditions and emotion data

[1183] Output: A list of recommended dishes based on criteria and sentiment

[1184] Step 6:

[1185] Submit your proposal

[1186] The server encodes the list of selected dishes in JSON format and sends it to the terminal as an HTTP response.

[1187] Specific operation: For example, generate JSON data {"dishes": ["Super spicy mapo tofu", "spicy curry"]} and send it to the terminal.

[1188] Input: Recommended dish list

[1189] Output: Suggestion sent to device

[1190] Step 7:

[1191] What the user sees

[1192] The terminal displays the received proposal to the user.

[1193] Specific operation: A list of dishes is displayed on the device screen through a UI created with React Native or Flutter. For example, "Super Spicy Mapo Tofu" and "Spicy Curry" are listed.

[1194] Input: Proposal received from the server

[1195] Output: A list of dishes displayed to the user

[1196] Step 8:

[1197] Order Selection

[1198] The user selects the desired dish from the suggested dishes and presses the "Order" button.

[1199] Specific operation: The user taps on "Super Spicy Mapo Tofu" and presses the "Order" button to confirm the selection.

[1200] Input: User's food selection

[1201] Output: Selected order details

[1202] Step 9:

[1203] Sending order details

[1204] The terminal encodes the selected order details in JSON format and sends them to the server.

[1205] Specific operation: The terminal sends a request to the POST / order endpoint: {"order": "Super spicy mapo tofu"}.

[1206] Input: Selected order details

[1207] Output: Order data sent to the server

[1208] Step 10:

[1209] Notification to stores

[1210] The server notifies the store's order management system of the final order details.

[1211] Specific operation: The server sends a request to the store's order management API: {"order": "Super spicy mapo tofu"}. The store then confirms the order and begins cooking.

[1212] Input: User's order details

[1213] Output: Order details sent to the store

[1214] (Application example 2)

[1215] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1216] Conventional restaurant and food delivery systems only consider user preferences and allergies when suggesting dishes, which means they are unable to suggest dishes that take into account the user's emotional state. This makes it difficult for users to select an appropriate dish that matches their current mood and emotions. This can lead to a decrease in user satisfaction, so there is a need for a suggestion system that incorporates emotions.

[1217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1218] In this invention, the server includes an input means for allowing a user to input conditions and emotions, a transmission means for transmitting the conditions and emotions input by the input means to the server, a selection means for the server to select dishes to suggest based on the conditions and emotions, a display means for presenting the dishes selected by the selection means to the user, a selection means for the user to select an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration, thereby increasing user satisfaction.

[1219] "Input means" refers to a device or interface for a user to input conditions and emotions.

[1220] The "transmission means" is a device or function for transmitting the conditions and emotions input by the input means to the server.

[1221] The "selection means" is a device or function that allows the server to select the most suitable dish based on the above conditions and emotions.

[1222] The "display means" is a device or function for presenting the dish selected by the selection means to the user.

[1223] The "selection means" is a device or interface that allows the user to select a desired dish from the dishes presented.

[1224] The "order sending means" is a device or function for sending the order selected by the selection means to the server.

[1225] The "emotion recognition means" is a device or function for analyzing the user's facial expressions and voice to recognize the user's emotional state.

[1226] The "notification means" is a device or function that allows the server to notify the store of the final order details.

[1227] This invention is a system that allows users to efficiently use restaurants and food delivery services, and by combining an emotion recognition engine, suggests dishes based on the user's emotional state. This system includes input means, transmission means, selection means, display means, selection means, order transmission means, emotion recognition means, and notification means.

[1228] 1. Overall system configuration

[1229] The main components of the system are as follows:

[1230] The user inputs conditions and emotions from a device such as a smartphone or tablet.

[1231] The terminal transmits the input data to the server.

[1232] The server analyzes the user's conditions and emotions and selects the most suitable dish.

[1233] The terminal presents the selected dishes to the user, and the user selects an order.

[1234] The terminal transmits the selected order to the server, which then notifies the store.

[1235] 2. Hardware and Software Used

[1236] Input methods: smartphone or tablet touchscreen, text box, selection options.

[1237] Emotion recognition means: smartphone or tablet camera, microphone, and facial recognition and voice analysis software via EmotionEngine (e.g., OpenCV or Google Cloud Speech-to-Text API).

[1238] Transmission method: Send data to the server (in JSON format via HTTP request) over an internet connection.

[1239] Selection method: In-server natural language processing and sentiment analysis algorithms.

[1240] Display medium: Smartphone or tablet display.

[1241] Options: Smartphone or tablet touchscreen interface.

[1242] Order sending method: Notification from the server to the store system (HTTP request or WebSocket communication).

[1243] Notification method: Store order management system.

[1244] 3. Example of the process

[1245] First, the user uses a smartphone application to input their preferred food preferences (e.g., "spicy food" or "vegetarian"). Next, the device's camera and microphone are used to recognize and analyze the user's emotions. This allows the system to identify the user's emotional state, such as "excited" or "relaxed."

[1246] The above data is sent to the server, which then performs natural language processing and sentiment analysis based on the data. For example, if the user requests "spicy food" and is in an "excited" state, spicy dishes (such as "super spicy mapo tofu" or "spicy curry") will be suggested. The suggested dishes are displayed on the smartphone application, and the user can select the desired dish and confirm the order.

[1247] 4. Examples of prompts

[1248] Below are some specific examples of input prompts for generative AI models.

[1249] User input criteria: spicy food, vegetarian

[1250] User sentiment: Excited

[1251] Generate suggested dishes.

[1252] Using this prompt, the AI ​​model can generate optimal recipe suggestions based on the user's emotions and cooking requirements, thereby suggesting dishes that match the user's current emotions and providing a highly satisfying user experience.

[1253] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1254] Step 1:

[1255] The user launches the app on their smartphone or tablet and enters the food preferences (e.g., "spicy food," "vegetarian," etc.).

[1256] Input: User condition input (text box, choice, etc.)

[1257] Output: Data of the entered conditions (JSON format)

[1258] Step 2:

[1259] The device's camera and microphone are used to collect the user's emotions, which are then analyzed by an emotion recognition engine (EmotionEngine).

[1260] Input: Camera video, microphone audio

[1261] Output: Recognized emotion data (e.g., "excited," "relaxed," etc.)

[1262] Step 3:

[1263] The terminal transmits the input conditions and the recognized emotion data to the server.

[1264] Input: Condition data, emotion data

[1265] Output: Request sent to the server (JSON format data)

[1266] Step 4:

[1267] Based on the conditions and emotional data received by the server, natural language processing and emotional analysis are performed to select the most suitable dish.

[1268] Input: Condition data, emotion data

[1269] Output: List of selected dishes (JSON format)

[1270] Step 5:

[1271] The server sends a list of selected dishes to the terminal.

[1272] Input: List of selected dishes

[1273] Output: A list of dishes sent to the device (JSON format)

[1274] Step 6:

[1275] The terminal displays the list of dishes sent from the server to the user, who then selects the dish they want from the list.

[1276] Input: list of dishes

[1277] Output: User's selected dish

[1278] Step 7:

[1279] The terminal transmits the dish selected by the user to the server.

[1280] Input: Selected dish data (JSON format)

[1281] Output: Order data sent to the server

[1282] Step 8:

[1283] The server notifies the restaurant of the final order, and the restaurant begins cooking based on the order.

[1284] Input: Order data

[1285] Output: Order notified to the store

[1286] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1287] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1288] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1289] [Fourth embodiment]

[1290] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1291] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1292] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1293] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1294] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1295] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1296] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1297] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1298] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1299] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1300] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1301] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1302] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1303] The present invention relates to a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs order conditions, selects dishes suggested by a server based on those conditions, and finally confirms the order. The program processing of this system is explained in detail below.

[1304] System Overview:

[1305] 1. User Input

[1306] A user launches a restaurant's application using a device such as a smartphone or tablet. The device displays a menu, and the user inputs their preferences, allergies, and other criteria. For example, they can specify criteria such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[1307] 2. Conditional Sending

[1308] The terminal transmits the conditions entered by the user to the server, which receives information to select dishes based on the user's preferences.

[1309] 3. Server Analysis

[1310] The server analyzes the received criteria using natural language processing technology. Specifically, it extracts keywords from the words and phrases entered by the user and determines the characteristics of the food based on them. For example, the keyword "spicy" can be used to search for spicy food.

[1311] 4. Food selection

[1312] Based on the analysis results, the server selects dishes from the menu that match the user's criteria. The server searches the menu items in the database and generates a list of dishes that match the criteria. For example, it might select "Mapo Tofu" or "Curry" as a "spicy dish."

[1313] 5. Submit your proposal

[1314] The server sends the list of selected dishes to the terminal, allowing the user to see dishes that match their criteria.

[1315] 6. User Display

[1316] The device displays the suggestions sent from the server to the user, allowing the user to choose the dish they like from the suggestions.

[1317] 7. Order Selection

[1318] The user selects the desired dish from the list of suggestions and confirms the order. The selected dish is sent to the server via the terminal by pressing the send button.

[1319] 8. Submitting your order

[1320] The terminal sends the order details selected by the user to the server, which then receives the final order details.

[1321] 9. Notification to stores

[1322] The server notifies the restaurant of the final order details, which are then passed on to the restaurant's management system, and staff begin preparing the food.

[1323] This system allows users to easily input their order requirements, and the server analyzes and selects the items, allowing them to receive prompt and accurate suggestions. Ultimately, this has the effect of streamlining the user's ordering process, improving store turnover and operational efficiency.

[1324] The processing flow will be explained below.

[1325] Step 1:

[1326] A user launches a restaurant application and displays the menu. The device sends a request to the server, retrieves the menu list, and displays it on the screen.

[1327] Step 2:

[1328] The user enters order conditions, such as "spicy food," "vegetarian," or "gluten-free," using text boxes, drop-down lists, and check boxes.

[1329] Step 3:

[1330] The device sends the entered conditions to the server. When the user enters the conditions and presses the "Send" button, the device sends the conditions to the server in a data format such as JSON.

[1331] Step 4:

[1332] The server analyzes the received conditions using natural language processing. Specifically, it tokenizes the conditions and extracts keywords. It generates analysis results based on keywords such as "spicy" or "gluten-free."

[1333] Step 5:

[1334] Based on the analysis results, the server searches for matching dishes on the menu. It searches the server's database and generates a list of dishes that match the criteria. For "spicy food," it will select dishes such as "mapo tofu" or "curry."

[1335] Step 6:

[1336] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[1337] Step 7:

[1338] The device displays the suggested dishes to the user. Suggested dishes such as "Mapo Tofu" and "Curry" are displayed in a list format on the screen.

[1339] Step 8:

[1340] The user selects the dish they wish to order from the suggested dishes by tapping on the desired dish and pressing the "Order" button.

[1341] Step 9:

[1342] The terminal sends the final order details to the server, where the selected dishes are encoded in JSON format or similar and sent to the server.

[1343] Step 10:

[1344] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[1345] In this way, users can place orders efficiently and stores can respond quickly.

[1346] Example 1

[1347] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1348] When ordering at a restaurant, a system is needed that allows users to quickly select the appropriate dish based on their preferences and allergies. Restaurants also need to efficiently receive orders and serve food smoothly. With conventional systems, communication between users and restaurants can be time-consuming, potentially resulting in mismatched orders. A new system is needed to solve these problems.

[1349] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1350] In this invention, the server includes input means for allowing a user to input conditions, transmission means for transmitting the conditions input by the input means to the server, selection means for selecting dishes to be proposed by the server based on the conditions, display means for presenting the dishes selected by the selection means to the user, selection means for the user to select an order from the presented dishes, order transmission means for transmitting the order selected by the selection means to the server, and notification means for notifying the store's management system of the order details. This allows the user to easily input conditions, and the server to quickly analyze and select dishes, enabling an efficient ordering process.

[1351] "User" refers to a person who uses a restaurant ordering system.

[1352] "Input means" refers to a device or interface for a user to input conditions.

[1353] "Transmission means" refers to a communication means for transmitting data entered by a user to a server.

[1354] "Server" refers to a computer system that receives, processes, and analyzes data sent by users.

[1355] "Selection means" refers to the function by which the server selects dishes to suggest based on the user's conditions.

[1356] "Display means" refers to a device or interface for visually presenting the dishes selected by the server to the user.

[1357] The "selection means" refers to the means by which the user selects the desired order from the displayed dishes.

[1358] "Order sending means" refers to a communication means for sending an order selected by a user to a server.

[1359] "Notification means" refers to the function by which the server notifies the store's management system of the final order details.

[1360] "Natural language processing" refers to a technology in which a server analyzes user input data and extracts appropriate keywords and phrases.

[1361] This system allows users to efficiently place orders at restaurants and is configured as follows.

[1362] First, a user launches a restaurant ordering application on a device such as a smartphone or tablet. The application displays a menu, and the user uses an input device to enter their preferences and allergies. The input device includes text boxes and options, allowing users to enter specific preferences such as "spicy food," "vegetarian," or "gluten-free."

[1363] Next, the terminal sends the conditions entered by the user to the server. The transmission method is HTTP protocol, and the data is sent in JSON format. Specifically, when the user presses the "Send" button, the terminal sends the input data to the server.

[1364] The server uses Python natural language processing libraries (e.g., nltk or spaCy) to analyze the received data. Through analysis, the server extracts keywords from the user's input and selects dishes based on the criteria. Analyzed keywords include "spicy," "vegetarian," and "gluten-free."

[1365] Next, the server uses a selection method to search for food data in a database (MySQL or PostgreSQL). Using this selection method, the server lists dishes that match the user's criteria. For example, dishes that match the criteria of "spicy" and "gluten-free" might include "Mapo Tofu" and "Spicy Gluten-Free Salad."

[1366] Once the list of selected dishes is complete, the server sends it to the device in JSON format. The device then uses a display mechanism to present the list to the user. The display mechanism uses the Flutter or React Native framework, and UI components such as list views are used to visually present the list to the user.

[1367] When the user selects a desired dish from the displayed list, the user uses a selection means. Specifically, the user taps the name of the dish (e.g., "Mapo Tofu") and presses the "Confirm Order" button to confirm the selection.

[1368] The terminal then transmits the order details confirmed by the selection means to the server, again using the HTTP protocol and JSON format.

[1369] Finally, the server notifies the restaurant's management system of the final order details using a REST API, allowing restaurant staff to begin preparing the food.

[1370] As a concrete example, consider the case where a user launches an application and enters the conditions "spicy food" and "gluten-free." The device sends these conditions to the server, which analyzes them and selects "mapo tofu" and "spicy gluten-free salad" to create a list. The device displays this list to the user, who selects "mapo tofu" and confirms the order. The server then notifies the restaurant of the final order details, and food preparation begins.

[1371] Example prompt for a generative AI model:

[1372] Please explain a system that streamlines ordering at restaurants. This system has a function that allows users to input their preferences and allergies, and the server analyzes these and suggests dishes. Please explain the following process in detail.

[1373] 1. User Input

[1374] 2. Conditional Sending

[1375] 3. Server Analysis

[1376] 4. Food selection

[1377] 5. Submit your proposal

[1378] 6. User Display

[1379] 7. Order Selection

[1380] 8. Submitting your order

[1381] 9. Notification to stores

[1382] The system uses a Python natural language processing library and a MySQL database. For example, include a case where a user inputs "spicy food" and "gluten-free" and finally selects "mapo tofu."

[1383] In this way, the present invention allows the user to easily input requirements and the server to quickly analyze and select, thereby enabling an efficient ordering process.

[1384] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1385] Step 1:

[1386] A user launches a restaurant ordering application using a smartphone or tablet device.

[1387] Input: Launch application

[1388] Output: Display the menu screen

[1389] Specific behavior: The application displays a menu screen on the device and allows the user to enter conditions.

[1390] Step 2:

[1391] The user inputs their preferences and allergies.

[1392] Input: User criteria (spicy food, vegetarian, gluten-free, etc.)

[1393] Output: Confirmation of input data

[1394] Specific behavior: The user enters conditions in the text boxes or options and clicks the "Submit" button.

[1395] Step 3:

[1396] The terminal transmits the input conditions to the server.

[1397] Input: User input data (JSON format)

[1398] Output: Notification of completion of transmission to the server

[1399] Specific operation: The terminal generates an HTTP request and sends the input data in JSON format to the server.

[1400] Step 4:

[1401] The server analyzes the received conditions.

[1402] Input: JSON data submitted

[1403] Output: Extracted keywords

[1404] What it does: The server uses a Python natural language processing library (such as nltk or spaCy) to extract keywords from the conditions.

[1405] Step 5:

[1406] The server selects dishes based on the analyzed keywords.

[1407] Input: Extracted keywords

[1408] Output: A list of suitable dishes

[1409] Specific operation: The server uses a database management system (MySQL or PostgreSQL) to search for dishes that match the keywords and generate a list.

[1410] Step 6:

[1411] The server sends the selected dish list to the terminal.

[1412] Input: List of selected dishes (JSON format)

[1413] Output: Notification of completion of transmission to the terminal

[1414] Specific behavior: The server generates an HTTP response and sends the list of dishes in JSON format to the device.

[1415] Step 7:

[1416] The terminal displays the received recipe list to the user.

[1417] Input: A list of dishes sent from the server

[1418] Output: what is displayed to the user

[1419] What it does: The device uses the Flutter and React Native frameworks to display a list of dishes on a user interface.

[1420] Step 8:

[1421] The user selects an order from the displayed list of dishes.

[1422] Input: User's food selection

[1423] Output: Confirmed order data

[1424] Specific operation: The user taps the desired dish and presses the "Confirm order" button.

[1425] Step 9:

[1426] The terminal sends the confirmed order to the server.

[1427] Input: Confirmed order data (JSON format)

[1428] Output: Notification of completion of transmission to the server

[1429] Specific operation: The terminal sends the order data in JSON format to the server as an HTTP request.

[1430] Step 10:

[1431] The server notifies the store's management system of the final order details.

[1432] Input: Confirmed order data

[1433] Output: Notification to store management system completed

[1434] Specific operation: The server notifies the store management system of the order data via the REST API.

[1435] (Application example 1)

[1436] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1437] In today's restaurants and food delivery services, it is difficult for users to select appropriate dishes taking into account individual preferences, allergies, and other factors. It is also difficult for restaurants and delivery services to provide menus that reflect the different preferences of each user, which can result in a decrease in user satisfaction. Furthermore, when a user simply wants to reorder a dish they have ordered in the past, it is inefficient because it requires time and effort to manually search for the dish. A system that solves these problems is needed.

[1438] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1439] In this invention, the server includes an analysis means that analyzes the user's input criteria using natural language processing and calculates cosine similarity to select the dish that most closely matches the user's criteria, a suggestion means that generates a list of dishes selected by the analysis means and suggests them to the user, and an order history management means that makes it easy to place repeat orders based on past order history.This allows users to quickly and accurately select and order dishes based on their individual preferences and allergy information.Furthermore, utilizing past order history makes repeat orders easier, improving user satisfaction.

[1440] "Input means" refers to a device or interface that allows a user to input conditions.

[1441] The "transmission means" refers to a device or program that transmits the conditions input by the input means to the server.

[1442] The "selection means" refers to a device or program that allows the server to select dishes to propose based on the above conditions.

[1443] The "display means" refers to a device or program that presents the dishes selected by the selection means to the user.

[1444] The "selection means" refers to a device or program that allows the user to select an order from the presented dishes.

[1445] The "order sending means" refers to a device or program that sends the order selected by the selection means to the server.

[1446] "Analysis means" refers to a device or program that analyzes the conditions entered by the user based on natural language processing, calculates the cosine similarity, and selects the dish that most closely matches the user's conditions.

[1447] The "suggestion means" refers to a device or program for generating a list of dishes selected by the analysis means and suggesting the list to the user.

[1448] "Order history management means" refers to a device or program that allows repeat orders to be easily placed based on past order history.

[1449] "Natural language processing" refers to the technology that enables computers to understand and analyze human language.

[1450] "Cosine similarity" refers to a method for calculating the similarity between vectorized data.

[1451] A specific system for carrying out the invention has the following configuration.

[1452] First, a user launches the application using their device, such as a smartphone or tablet. The application provides input means for the user to enter their preferences and allergy information. Input means include text boxes and check boxes. This input information is then sent from the device to the server.

[1453] Next, the server receives the user's request information. It then uses natural language processing technology to analyze the user's input. Specifically, it uses Python's Scikit-learn library to vectorize the text and calculate the cosine similarity. Based on this analysis, it selects the dish from the menu that most closely matches the user's request.

[1454] The server sends a list of selected dishes to the terminal. The terminal displays this list to the user. The user selects the dishes they like from the suggestions and confirms the order. The user's selection information is sent back to the server, which then confirms the final order.

[1455] Furthermore, this system includes an order history management means, which allows users to easily place repeat orders based on the history information of dishes they have ordered in the past.

[1456] Specific examples of prompt sentences for implementing the present invention are shown below: The user inputs prompt sentences such as "Please suggest spicy and vegetarian dishes" or "I'm a vegetarian and I like spicy dishes. Please tell me your recommendations."

[1457] This system allows users to efficiently and accurately select and order dishes, enabling restaurants and food delivery services to improve user satisfaction.

[1458] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1459] Step 1:

[1460] A user launches an application using a device such as a smartphone or tablet. The user then enters their preferences and allergy information within the application. This information is entered using text boxes and check boxes.

[1461] Input: User criteria (e.g. "spicy food", "vegetarian", "gluten-free", etc.)

[1462] Output: User condition data

[1463] Step 2:

[1464] The user's condition data is sent from the device to the server. This information is sent to the server using an HTTP request.

[1465] Input: User condition data

[1466] Output: Condition data sent to the server

[1467] Step 3:

[1468] The server then uses natural language processing technology to analyze the received user requirements. This analysis involves vectorizing the text data using the Scikit-learn library, and then calculating cosine similarity to extract dishes from the menu that match the user's requirements.

[1469] Input: Condition data sent to the server

[1470] Output: A list of dishes that match the user's criteria

[1471] Step 4:

[1472] The server sends a list of selected dishes to the device, and returns this information to the device in an HTTP response.

[1473] Input: A list of dishes that match the user's criteria

[1474] Output: Recipe list sent to the device

[1475] Step 5:

[1476] The terminal displays the list of dishes received from the server to the user, who then checks the list and selects the dish they wish to order.

[1477] Input: Recipe list sent to device

[1478] Output: User-selected dishes

[1479] Step 6:

[1480] The user's selected dish data is sent from the device to the server, again using an HTTP request.

[1481] Input: User-selected dish data

[1482] Output: Final order data sent to the server

[1483] Step 7:

[1484] The server receives the final order data and confirms the order. The order details are saved on the server and sent to the restaurant. The order details are then sent to the restaurant's management system, and the staff begins preparing the food.

[1485] Input: Final order data sent to the server

[1486] Output: Order details notified to the store

[1487] Step 8:

[1488] The server manages the order history and makes this information available to users when they place repeat orders in the future. An interface is provided to make it easy to place repeat orders based on past order history.

[1489] Input: User's past order data

[1490] Output: Historical data available for repeat orders

[1491] This allows users to efficiently select dishes and confirm their orders, and by utilizing past order history, they can easily place repeat orders.

[1492] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1493] This invention relates to combining an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which a user inputs order conditions, selects dishes suggested by a server based on the conditions and the user's emotions, and finally confirms the order.

[1494] System Overview:

[1495] 1. User Input

[1496] A user launches a restaurant's application on a device such as a smartphone or tablet, and the menu is displayed. The device then sends a request to the server, which retrieves the menu list and displays it on the screen. The user then inputs preferences, allergies, and other conditions. For example, they can specify conditions such as "spicy food," "vegetarian," or "gluten-free." This input is done through text boxes and options on the device.

[1497] 2. Emotion recognition

[1498] The emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. For example, it uses facial recognition technology and voice analysis technology to identify the user's emotional state, such as whether they are happy, excited, relaxed, etc.

[1499] 3. Sending conditions and emotions

[1500] The device sends the input conditions and recognized emotions to the server, which simultaneously sends the user's conditions and emotion information to the server, providing the data that will be used as the basis for analysis.

[1501] 4. Server Analysis

[1502] The server analyzes the order conditions and emotional data it receives using natural language processing and emotion analysis technology. Specifically, it extracts keywords from the conditions and determines the user's emotional state from the emotional data. For example, if a user requests "spicy food" and is "excited," it will prioritize suggesting spicy food.

[1503] 5. Food selection

[1504] Based on the analysis results, the server searches for the dish from the menu that best suits the conditions and emotions. It searches the menu items in the database and generates a list of dishes according to the conditions and emotions. For example, for a user who likes "spicy food" and is "excited," it selects "super spicy mapo tofu" or "spicy curry."

[1505] 6. Submit your proposal

[1506] The server sends a list of selected dishes to the device. The list is encoded in JSON format or similar and sent to the device.

[1507] 7. User Presentation

[1508] The device displays the suggestions sent from the server to the user. The suggested dishes are displayed in a list format on the screen. For example, dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are presented.

[1509] 8. Order Selection

[1510] The user selects the desired dish from the suggested dishes, taps on the desired dish, and presses the "Order" button.

[1511] 9. Submitting your order

[1512] The device sends the selected order details to the server, which then encodes the selected dishes in JSON format and sends them to the server.

[1513] 10. Notice to Stores

[1514] The server notifies the restaurant of the final order details, which are then transmitted to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[1515] In this way, by combining and analyzing user input and emotional information, more accurate and personalized dish suggestions can be made, thereby increasing user satisfaction and store operational efficiency.

[1516] The processing flow will be explained below.

[1517] Step 1:

[1518] A user launches a restaurant application on their device, which sends a request to the server, downloads a menu list, and displays it on the screen.

[1519] Step 2:

[1520] The user inputs ordering conditions, such as "spicy food" or "vegetarian food preferred," into text boxes and check boxes.

[1521] Step 3:

[1522] The device recognizes the user's facial expressions and voice using an emotion engine. The device's camera captures the user's facial expressions and the microphone records the user's voice. The emotion engine analyzes this data to identify the user's emotional state (e.g., happy, excited, relaxed).

[1523] Step 4:

[1524] The terminal transmits the input conditions and the recognized emotion data to the server, thereby providing the user's order conditions and emotion information to the server.

[1525] Step 5:

[1526] The server analyzes the received conditions and emotional data. It uses natural language processing technology to analyze the order conditions and simultaneously analyzes the emotional data to determine the user's emotional state. For example, it analyzes the emotional state of "spicy food" and "excited."

[1527] Step 6:

[1528] The server selects dishes based on the conditions and emotional data. From the menu in the database, it selects "spicy dishes" that are suitable for "excited" users. For example, it suggests "super spicy mapo tofu" or "spicy curry."

[1529] Step 7:

[1530] The server sends the list of selected dishes to the device, which then encodes the list in an appropriate data format (e.g., JSON) and sends it to the device.

[1531] Step 8:

[1532] The device displays the suggested dishes to the user. Dishes such as "Super Spicy Mapo Tofu" and "Spicy Curry" are displayed in a list format on the screen.

[1533] Step 9:

[1534] The user selects the desired dish from the suggested dishes and presses the "Order" button. For example, the user selects and orders "Super Spicy Mapo Tofu."

[1535] Step 10:

[1536] The terminal sends the selected order details to the server. The dishes selected by the user are encoded in a data format and sent to the server.

[1537] Step 11:

[1538] The server notifies the restaurant of the final order details, which are then sent to the restaurant's management system, where the restaurant confirms the order and begins cooking.

[1539] In this way, the combined analysis of user input conditions and emotional information leads to more personalized suggestions and an improved user experience.

[1540] Example 2

[1541] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1542] Conventional restaurant ordering systems mainly provide suggestions based on the user's preferences, but lack personalized suggestions that take into account the user's emotional state. There is a need for a method to improve satisfaction by providing dish suggestions that adapt to the user's emotions.

[1543] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means by which the user can input conditions, a transmission means for transmitting the conditions input by the input means and the user's emotional information to the server, a selection means by which the server selects dishes to suggest based on the conditions and emotional information, a display means for presenting the dishes selected by the selection means to the user, a selection means by which the user selects an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration.

[1544] "Input means" refers to a device or interface for inputting desired conditions and information by the user.

[1545] The "transmission means" is a device or system that transmits input information or data to a server via a network.

[1546] The "selection means" is a device or algorithm that performs the process of selecting appropriate proposals based on the conditions and information received by the server.

[1547] The "display means" is a device or interface that visually shows the proposal selected by the selection means to the user.

[1548] The "selection means" is a device or system that allows the user to select a desired item from the displayed suggestions.

[1549] The "order sending means" is a device or system that sends the order details selected by the user to the server.

[1550] "Natural language processing" is a technology that enables computers to understand, analyze, and respond to human language.

[1551] "Emotion analysis" is a technology for identifying a user's emotional state from their facial expressions and voice.

[1552] The "notification means" is a device or system that allows the server to transmit the order details to the store.

[1553] This invention combines an emotion engine with a system that allows users to efficiently place orders at restaurants. This system involves a process in which the user inputs their order conditions, selects dishes suggested by the server based on those conditions and the user's emotions, and finally confirms the order.

[1554] Basic system configuration

[1555] This system is composed of a user input means, a transmission means, a selection means, a display means, a selection means, and an order transmission means. Each means will be explained in detail below.

[1556] User input method

[1557] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu. The device then sends an HTTP request to the server to retrieve the menu list. This retrieval process includes retrieving menu information from a MySQL database, encoding it in JSON format, and sending it to the device. The retrieved menu is then displayed to the user on the device using a UI created with React Native or Flutter.

[1558] Users input their preferences, allergies, and other conditions via text boxes and radio buttons on the terminal.

[1559] emotion recognition means

[1560] As the user enters their conditions, the device's camera and microphone capture their facial expressions and voice, which are then used to recognize their emotions using Google's Face API, Amazon's Rekognition, Microsoft's AI services, and more. This emotion recognition identifies the user's emotional state, such as whether they are happy, excited, or relaxed.

[1561] For example, if a user smiles while looking at a menu, the camera captures that smile and each AI service recognizes the emotion as "enjoying."

[1562] Server analysis and selection method

[1563] The entered conditions and recognized emotion information are encoded in JSON format by the device and sent to the server. The server receives this data and analyzes it using natural language processing (NLP) and emotion analysis techniques. Specifically, it uses libraries such as NLTK and spaCy to extract keywords from the conditions and determine the user's emotional state from the emotion data.

[1564] For example, the server receives data such as "spicy food" and "enjoying it," and determines that "spicy food" matches "super spicy mapo tofu" or "spicy curry."

[1565] The server then selects the dish from the menu that best suits the criteria and emotion, using a MySQL query to generate a list of dishes based on the criteria and emotion.

[1566] Proposal and ordering methods

[1567] The selected suggestions are encoded in JSON format and sent to the device as an HTTP response. The device then displays the received suggestions to the user. A list of appropriate dishes is displayed on the screen using a UI created with React Native or Flutter.

[1568] The user selects the desired dish from the presented dishes and presses the "Order" button. The selected order details are also encoded in JSON format and sent to the server.

[1569] Finally, the server notifies the store's order management system (e.g., a POS system) of the order details, which then confirms the order and begins cooking.

[1570] Specific examples and prompts for the generative AI model

[1571] For example, if a user requests "spicy food" and their emotion is recognized as "enjoying it," the system will suggest dishes such as "super spicy mapo tofu" or "spicy curry."

[1572] Prompt sentence for generative AI model

[1573] Example 1: "If the user selects a spicy dish and facial analysis indicates that they are enjoying it, please recommend a dish."

[1574] Example 2: "If a user selects gluten-free as their allergy information and a happy expression is recognized, suggest appropriate dishes."

[1575] As a result, personalized recipe suggestions that take into account the user's emotional state become possible, thereby improving user satisfaction.

[1576] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1577] Step 1:

[1578] User Input

[1579] A user launches a restaurant application on a device such as a smartphone or tablet and displays the menu.

[1580] Specific operation: When a user opens the app, the device sends an HTTP request (e.g., GET / menu) to the server to retrieve a menu list. The server retrieves menu information from the MySQL database and sends it to the device in JSON format. Input is done through text boxes and check boxes. Input contents include conditions such as "spicy food," "vegetarian," and "gluten-free."

[1581] Input: User criteria (e.g. "spicy food" or "vegetarian")

[1582] Output: Menu list displayed on the device

[1583] Step 2:

[1584] emotion recognition

[1585] The device captures the user's facial expressions and voice using a camera and microphone.

[1586] How it works: The device uses Google's Face API, Amazon's Rekognition, and Microsoft's AI services to analyze the user's emotions, recognizing emotions such as "enjoyed" or "excited" based on the user's facial expressions and voice patterns.

[1587] Input: User's facial expressions and voice data

[1588] Output: Parsed emotion data (e.g., "having fun")

[1589] Step 3:

[1590] Sending conditions and emotions

[1591] The device sends the input conditions and recognized emotions to the server in JSON format.

[1592] Specific operation: The device sends a request to the POST / analyze endpoint: {"conditions": ["spicy", "vegetarian"], "emotion": "enjoying"}.

[1593] Input: User condition and emotion data

[1594] Output: Data sent to the server

[1595] Step 4:

[1596] Server Analysis

[1597] The server analyzes the order conditions and emotional data received using natural language processing (NLP) and emotion analysis technology.

[1598] Specific operation: The server uses libraries such as NLTK and spaCy to extract keywords from the conditions and identify the user's emotions from the emotional data. For example, if the conditions are "spicy food" and "enjoying it," the server will extract "super spicy mapo tofu" and "spicy curry" as matching candidates.

[1599] Input: User condition and emotion data

[1600] Output: Parsed list of recommended dishes

[1601] Step 5:

[1602] Food selection

[1603] Based on the analysis results, the server selects the dish from the restaurant's menu that best suits the conditions and emotions.

[1604] What it does: The server runs the query SELECT FROM menu WHERE spicy='true' AND category='vegetarian' against the MySQL database to extract dishes that fit the criteria, generating a list of appropriate dishes.

[1605] Input: Parsed conditions and emotion data

[1606] Output: A list of recommended dishes based on criteria and sentiment

[1607] Step 6:

[1608] Submit your proposal

[1609] The server encodes the list of selected dishes in JSON format and sends it to the terminal as an HTTP response.

[1610] Specific operation: For example, generate JSON data {"dishes": ["Super spicy mapo tofu", "spicy curry"]} and send it to the terminal.

[1611] Input: Recommended dish list

[1612] Output: Suggestion sent to device

[1613] Step 7:

[1614] What the user sees

[1615] The terminal displays the received proposal to the user.

[1616] Specific operation: A list of dishes is displayed on the device screen through a UI created with React Native or Flutter. For example, "Super Spicy Mapo Tofu" and "Spicy Curry" are listed.

[1617] Input: Proposal received from the server

[1618] Output: A list of dishes displayed to the user

[1619] Step 8:

[1620] Order Selection

[1621] The user selects the desired dish from the suggested dishes and presses the "Order" button.

[1622] Specific operation: The user taps on "Super Spicy Mapo Tofu" and presses the "Order" button to confirm the selection.

[1623] Input: User's food selection

[1624] Output: Selected order details

[1625] Step 9:

[1626] Sending order details

[1627] The terminal encodes the selected order details in JSON format and sends them to the server.

[1628] Specific operation: The terminal sends a request to the POST / order endpoint: {"order": "Super spicy mapo tofu"}.

[1629] Input: Selected order details

[1630] Output: Order data sent to the server

[1631] Step 10:

[1632] Notification to stores

[1633] The server notifies the store's order management system of the final order details.

[1634] Specific operation: The server sends a request to the store's order management API: {"order": "Super spicy mapo tofu"}. The store then confirms the order and begins cooking.

[1635] Input: User's order details

[1636] Output: Order details sent to the store

[1637] (Application example 2)

[1638] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1639] Conventional restaurant and food delivery systems only consider user preferences and allergies when suggesting dishes, which means they are unable to suggest dishes that take into account the user's emotional state. This makes it difficult for users to select an appropriate dish that matches their current mood and emotions. This can lead to a decrease in user satisfaction, so there is a need for a suggestion system that incorporates emotions.

[1640] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1641] In this invention, the server includes an input means for allowing a user to input conditions and emotions, a transmission means for transmitting the conditions and emotions input by the input means to the server, a selection means for the server to select dishes to suggest based on the conditions and emotions, a display means for presenting the dishes selected by the selection means to the user, a selection means for the user to select an order from the suggested dishes, and an order transmission means for transmitting the order selected by the selection means to the server. This enables personalized dish suggestions that take the user's emotional state into consideration, thereby increasing user satisfaction.

[1642] "Input means" refers to a device or interface for a user to input conditions and emotions.

[1643] The "transmission means" is a device or function for transmitting the conditions and emotions input by the input means to the server.

[1644] The "selection means" is a device or function that allows the server to select the most suitable dish based on the above conditions and emotions.

[1645] The "display means" is a device or function for presenting the dish selected by the selection means to the user.

[1646] The "selection means" is a device or interface that allows the user to select a desired dish from the dishes presented.

[1647] The "order sending means" is a device or function for sending the order selected by the selection means to the server.

[1648] The "emotion recognition means" is a device or function for analyzing the user's facial expressions and voice to recognize the user's emotional state.

[1649] The "notification means" is a device or function that allows the server to notify the store of the final order details.

[1650] This invention is a system that allows users to efficiently use restaurants and food delivery services, and by combining an emotion recognition engine, suggests dishes based on the user's emotional state. This system includes input means, transmission means, selection means, display means, selection means, order transmission means, emotion recognition means, and notification means.

[1651] 1. Overall system configuration

[1652] The main components of the system are as follows:

[1653] The user inputs conditions and emotions from a device such as a smartphone or tablet.

[1654] The terminal transmits the input data to the server.

[1655] The server analyzes the user's conditions and emotions and selects the most suitable dish.

[1656] The terminal presents the selected dishes to the user, and the user selects an order.

[1657] The terminal transmits the selected order to the server, which then notifies the store.

[1658] 2. Hardware and Software Used

[1659] Input methods: smartphone or tablet touchscreen, text box, selection options.

[1660] Emotion recognition means: smartphone or tablet camera, microphone, and facial recognition and voice analysis software via EmotionEngine (e.g., OpenCV or Google Cloud Speech-to-Text API).

[1661] Transmission method: Send data to the server (in JSON format via HTTP request) over an internet connection.

[1662] Selection method: In-server natural language processing and sentiment analysis algorithms.

[1663] Display medium: Smartphone or tablet display.

[1664] Options: Smartphone or tablet touchscreen interface.

[1665] Order sending method: Notification from the server to the store system (HTTP request or WebSocket communication).

[1666] Notification method: Store order management system.

[1667] 3. Example of the process

[1668] First, the user uses a smartphone application to input their preferred food preferences (e.g., "spicy food" or "vegetarian"). Next, the device's camera and microphone are used to recognize and analyze the user's emotions. This allows the system to identify the user's emotional state, such as "excited" or "relaxed."

[1669] The above data is sent to the server, which then performs natural language processing and sentiment analysis based on the data. For example, if the user requests "spicy food" and is in an "excited" state, spicy dishes (such as "super spicy mapo tofu" or "spicy curry") will be suggested. The suggested dishes are displayed on the smartphone application, and the user can select the desired dish and confirm the order.

[1670] 4. Examples of prompts

[1671] Below are some specific examples of input prompts for generative AI models.

[1672] User input criteria: spicy food, vegetarian

[1673] User sentiment: Excited

[1674] Generate suggested dishes.

[1675] Using this prompt, the AI ​​model can generate optimal recipe suggestions based on the user's emotions and cooking requirements, thereby suggesting dishes that match the user's current emotions and providing a highly satisfying user experience.

[1676] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1677] Step 1:

[1678] The user launches the app on their smartphone or tablet and enters the food preferences (e.g., "spicy food," "vegetarian," etc.).

[1679] Input: User condition input (text box, choice, etc.)

[1680] Output: Data of the entered conditions (JSON format)

[1681] Step 2:

[1682] The device's camera and microphone are used to collect the user's emotions, which are then analyzed by an emotion recognition engine (EmotionEngine).

[1683] Input: Camera video, microphone audio

[1684] Output: Recognized emotion data (e.g., "excited," "relaxed," etc.)

[1685] Step 3:

[1686] The terminal transmits the input conditions and the recognized emotion data to the server.

[1687] Input: Condition data, emotion data

[1688] Output: Request sent to the server (JSON format data)

[1689] Step 4:

[1690] Based on the conditions and emotional data received by the server, natural language processing and emotional analysis are performed to select the most suitable dish.

[1691] Input: Condition data, emotion data

[1692] Output: List of selected dishes (JSON format)

[1693] Step 5:

[1694] The server sends a list of selected dishes to the terminal.

[1695] Input: List of selected dishes

[1696] Output: A list of dishes sent to the device (JSON format)

[1697] Step 6:

[1698] The terminal displays the list of dishes sent from the server to the user, who then selects the dish they want from the list.

[1699] Input: list of dishes

[1700] Output: User's selected dish

[1701] Step 7:

[1702] The terminal transmits the dish selected by the user to the server.

[1703] Input: Selected dish data (JSON format)

[1704] Output: Order data sent to the server

[1705] Step 8:

[1706] The server notifies the restaurant of the final order, and the restaurant begins cooking based on the order.

[1707] Input: Order data

[1708] Output: Order notified to the store

[1709] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1710] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1711] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1712] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1713] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1714] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1715] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1716] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1717] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1718] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1719] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1720] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1721] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1722] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1723] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1724] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1725] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1726] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1727] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1728] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1729] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1730] The following is further disclosed regarding the above embodiment.

[1731] (Claim 1)

[1732] an input means by which a user can input conditions;

[1733] a transmission means for transmitting the conditions input by the input means to a server;

[1734] a selection means for selecting a dish to be proposed by the server based on the conditions;

[1735] a display means for presenting the dishes selected by the selection means to the user;

[1736] A selection means for allowing a user to select an order from the presented dishes;

[1737] The system includes an order sending means for sending the order selected by the selection means to a server.

[1738] (Claim 2)

[1739] 2. The system according to claim 1, wherein the server includes means for analyzing the condition based on natural language processing.

[1740] (Claim 3)

[1741] 2. The system according to claim 1, wherein the server includes a notification means for notifying the store of the order.

[1742] "Example 1"

[1743] (Claim 1)

[1744] an input means by which a user can input conditions;

[1745] a transmission means for transmitting the conditions input by the input means to a server;

[1746] a selection means for selecting a dish to be proposed by the server based on the conditions;

[1747] a display means for presenting the dishes selected by the selection means to the user;

[1748] A selection means for allowing a user to select an order from the presented dishes;

[1749] an order sending means for sending the order selected by the selection means to a server;

[1750] The system includes a notification means for notifying the store management system of the order details.

[1751] (Claim 2)

[1752] 2. The system according to claim 1, wherein the server includes an analysis means for analyzing the condition based on natural language processing.

[1753] (Claim 3)

[1754] 2. The system of claim 1, wherein the terminal includes means for transmitting user input data to the server in JSON format.

[1755] "Application Example 1"

[1756] (Claim 1)

[1757] an input means by which a user can input conditions;

[1758] a transmission means for transmitting the conditions input by the input means to a server;

[1759] a selection means for selecting a dish to be proposed by the server based on the conditions;

[1760] a display means for presenting the dishes selected by the selection means to the user;

[1761] A selection means for allowing a user to select an order from the presented dishes;

[1762] an order sending means for sending the order selected by the selection means to a server;

[1763] An analysis means for analyzing the conditions entered by the user based on natural language processing, calculating cosine similarity, and selecting the dish that most closely matches the user's conditions;

[1764] a suggestion means for generating a list of dishes selected by the analysis means and suggesting the list to a user;

[1765] The system includes an order history management means that allows repeat orders to be easily placed based on past order history.

[1766] (Claim 2)

[1767] 2. The system according to claim 1, wherein the server analyzes the conditions based on natural language processing and suggests dishes using the suggestion means.

[1768] (Claim 3)

[1769] 2. The system according to claim 1, further comprising means for notifying the store of the order and managing the order contents, wherein the server notifies the store of the order.

[1770] "Example 2: Combining Emotion Engines"

[1771] (Claim 1)

[1772] an input means by which a user can input conditions;

[1773] a transmission means for transmitting the conditions input by the input means and the user's emotion information to a server;

[1774] a selection means for selecting a dish to be proposed by the server based on the conditions and emotion information;

[1775] a display means for presenting the dishes selected by the selection means to the user;

[1776] A selection means for allowing a user to select an order from the presented dishes;

[1777] The system includes an order sending means for sending the order selected by the selection means to a server.

[1778] (Claim 2)

[1779] The system of claim 1, wherein the server comprises means for analyzing the condition and emotion information based on natural language processing and emotion analysis.

[1780] (Claim 3)

[1781] 2. The system according to claim 1, wherein the server includes a notification means for notifying the store of the order.

[1782] "Application example 2 when combining emotion engines"

[1783] (Claim 1)

[1784] an input means by which a user can input conditions and emotions;

[1785] a transmission means for transmitting the condition and emotion input by the input means to a server;

[1786] a selection means for selecting a dish to be recommended by the server based on the conditions and emotions;

[1787] a display means for presenting the dishes selected by the selection means to the user;

[1788] A selection means for allowing a user to select an order from the presented dishes;

[1789] an order sending means for sending the order selected by the selection means to a server;

[1790] emotion recognition means for recognizing an emotion of a user;

[1791] A system including:

[1792] (Claim 2)

[1793] 10. The system of claim 1, wherein the server comprises means for analyzing the conditions and emotions based on natural language processing and sentiment analysis.

[1794] (Claim 3)

[1795] 2. The system according to claim 1, wherein the server includes a notification means for notifying the store of the order. [Explanation of symbols]

[1796] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. an input means by which a user can input conditions; a transmission means for transmitting the conditions input by the input means to a server; a selection means for selecting a dish to be proposed by the server based on the conditions; a display means for presenting the dishes selected by the selection means to the user; A selection means for allowing a user to select an order from the presented dishes; and an order sending means for sending the order selected by the selection means to a server.

2. The system according to claim 1 , wherein the server includes means for analyzing the conditions based on natural language processing.

3. 2. The system according to claim 1, wherein the server includes a notification means for notifying the store of the order.

Images