System

Abstract

To allow a user to have a conversation regarding food experiences and foods.SOLUTION: A food experience system includes: means that acquires information on at least any one of the origin, characteristics for every country, ingredients, nutrients, flavors, and recipes of foods; natural language processing means that explains foods and answers to questions for a user on the basis of the information; machine learning means that includes analyzing the user's preference from the user's questions through machine learning, and proposing recommended foods for the user on the basis of the analysis; means that includes virtual reality technique for generating a virtual chef that has a conversation with the user in virtual reality and provides explanation and advice for how to make foods; means that utilizes the virtual reality technique to provide experiences for making foods to the user; and order processing means that receives an order from the user and executes food delivery arrangement.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional methods of providing cooking information allow users to learn about cooking through text information and images, but they have the drawback of not being able to actually experience or interact with the cooking. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system that allows a user to experience cuisines of various countries in virtual reality and obtain information about the cuisines. Specifically, the problem is solved by using the following means.

[0006] 1. Provide a means to obtain information on a food's origin, national characteristics, ingredients, nutritional value, taste, and recipes based on the user's request.

[0007] 2. Utilizing generative AI, we provide natural language processing tools to explain dishes to users and answer their questions.

[0008] 3. Utilizing virtual reality technology, we provide a means for users to experience cuisine from various countries in virtual reality and order food and drinks in real time.

[0009] 4. Use machine learning to analyze user questions and preferences and provide a means to suggest recommended dishes.

[0010] 5. Leverage virtual reality technology to provide a means for users to interact with a virtual chef in virtual reality and learn how to cook a dish.

[0011] 6. Providing an order processing means, capable of receiving user orders and arranging for food delivery.

[0012] These methods allow users to gain a deeper understanding and enjoy cooking through experiences and dialogue that were not possible with conventional methods of providing cooking information.

[0013] A "culinary knowledge database" is a database that contains information on the origins of various dishes, their national characteristics, ingredients, nutritional value, taste, and recipes.

[0014] "Generative AI" is an AI (artificial intelligence) model that uses natural language processing to interact with users and generate descriptions of dishes and answers to questions.

[0015] "Virtual reality technology" refers to technology that uses computer technology to create a virtual reality space in which users can feel a physical presence.

[0016] "Machine learning methods" refer to techniques and algorithms for analyzing large amounts of data, learning patterns and trends, and making predictions and judgments.

[0017] A "virtual chef" is a virtual chef that interacts with a user in a virtual reality environment and provides cooking instructions and advice. A virtual chef is generated using virtual reality technology.

[0018] The "order processing means" refers to a method or system for accepting user orders, arranging for food delivery, and managing order information. [Brief description of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 4 is a sequence diagram showing a process flow of the data processing system according to the first embodiment. FIG. [Figure 12] 11 is a sequence diagram showing a process flow of the data processing system in application example 1. FIG. [Figure 13] FIG. 11 is a sequence diagram showing the flow of processing of the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in application example 2 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0023] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The embodiment for carrying out the present invention comprises the following elements.

[0041] 1. Server: Access to the cooking knowledge database and obtain information The server accesses a cooking knowledge database based on a user's request to obtain information on the origin of the dish, its national characteristics, ingredients, nutritional value, taste, and recipe.

[0042] 2. Generative AI: Generating recipe descriptions and answers to questions The generative AI is activated in response to a user request and generates an explanation of the dish or an answer to the question. The generative AI provides the user with an appropriate explanation and answer based on information obtained from a cooking knowledge database. For example, if a user is interested in Italian cuisine, the user can receive an explanation from the generative AI about the origin and characteristics of Italian cuisine, as well as recommended recipes. In another example, if a user asks, "What kind of wine goes well with this dish?" the generative AI will explain the appropriate types and characteristics of wines and suggest recommended wines.

[0043] 3. Virtual reality technology: Providing culinary experiences and fulfilling orders Using virtual reality technology, users can experience cuisine from various countries in virtual reality. The cooking experience includes cooking the food, learning how to cook the food, and eating the food. Users can also order food and drinks in real time while listening to explanations from generative AI.

[0044] 4. Machine learning tools: Analyzing user questions and preferences and suggesting recommended dishes Using machine learning techniques, the system analyzes the user's questions and preferences to suggest appropriate answers and recommended dishes. The machine learning techniques learn the user's past behavior and preferences and make suggestions tailored to each individual user. For example, if a user says, "I like spicy food," the generative AI will suggest recipes and recommended menu items that specialize in spicy food.

[0045] 5. Virtual Chef: Cooking instructions and practice support The virtual chef in the virtual reality world provides explanations and advice to help users learn how to cook. Users can practice cooking by following the virtual chef's instructions. In addition, users have a full 360-degree view in the virtual reality world, so they can move freely within the image. Users can move to the location of the generative AI virtual chef and receive instructions on how to cook a dish. For example, if a user wants to know how to make pasta, the user can move to the location of the virtual chef and be taught how to make pasta by the generative AI virtual chef.

[0046] 6. Order Processing: Accepting your orders and arranging delivery The server accepts the user's order and arranges for the food to be delivered. The user orders food in virtual reality, and the food is delivered to their home or a specified location.

[0047] By combining the above elements, the user can experience cooking in virtual reality and obtain information about cooking. The user can enjoy cooking experiences from all over the world while staying at home. In addition, the system also provides suggestions for recommended dishes based on the user's questions and preferences, and provides explanations and practice support on how to cook the dishes. The order processing means allows the user to order food in virtual reality, and the food is delivered to the user's home or a specified location.

[0048] The process flow will be explained below.

[0049] Step 1: The server retrieves information from the cooking knowledge database. The server receives the user's request and accesses the cooking knowledge database. The server retrieves information from a database about the food's origin, national characteristics, ingredients, nutritional value, taste, and recipe.

[0050] Step 2: The server starts the generative AI to generate recipe descriptions and answers to questions. The server launches the generative AI in response to a user request. The generative AI generates descriptions of dishes and answers to questions based on information obtained from a cooking knowledge database.

[0051] Step 3: The user experiences and orders food in virtual reality Users can experience cuisine from various countries in virtual reality. Users order food and drinks in real time while listening to descriptions of the dishes provided by generative AI.

[0052] Step 4: The server analyzes the user's questions and preferences and makes recommendations on dishes. The server analyzes the user's questions and preferences. The server uses machine learning methods to suggest appropriate answers and dish recommendations to the user.

[0053] Step 5: User interacts with the virtual chef and learns how to cook the dish The user interacts with the virtual chef in virtual reality. Users can practice cooking while receiving instructions and advice on how to prepare the dish from the virtual chef.

[0054] Step 6: The server receives the user's order and arranges for the food to be delivered. The server accepts the user's order. The server utilizes the order processing means to arrange for the food to be delivered.

[0055] Through the above processing steps, a cooking experience system is realized between the server and the user. The server obtains information from the cooking knowledge database and activates a generative AI to generate explanations of dishes and answers to questions. The user can experience cooking and place an order in virtual reality. The server analyzes the user's questions and preferences and suggests recommended dishes. The user can also interact with the virtual chef and learn how to cook. The server receives the user's order and arranges for the delivery of the food. Example 1

[0056] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0057] In conventional cooking experience systems, it was difficult for users to not only obtain cooking information but also to actually experience cooking and receive cooking suggestions according to their preferences. In addition, there was a lack of technical combinations to quickly respond to user requests and provide appropriate cooking information and recipes. Furthermore, there is a demand for a system that provides cooking experiences using virtual reality technology and provides appropriate feedback to users.

[0058] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0059] In this invention, the server includes a means for acquiring information, a means for performing natural language processing on the user, a machine learning means for analyzing the user's questions and preferences, a means for interacting with a virtual chef using virtual reality technology, an order processing means for receiving the user's order and arranging for delivery, a means for receiving requests via the user's terminal, a means for providing a cooking experience using virtual reality technology, and a means for generating answers to questions using generative AI. This allows the user to actually experience cooking in virtual reality and receive suggestions for dishes according to their preferences. In addition, the server can quickly respond to the user's requests and provide appropriate cooking information and recipes.

[0060] "Means for acquiring information" refers to a function for acquiring necessary information from a database or an external information source based on a user request.

[0061] "Means of natural language processing" refers to technology that analyzes text and voice data entered by the user, understands the meaning, and generates appropriate answers and information.

[0062] "Machine learning methods" are algorithms and technologies that analyze a user's past behavioral data and preferences, and use that data to improve future suggestions and responses.

[0063] The "means for interacting with a virtual chef" using "virtual reality technology" is a means for a user to interactively communicate with a virtual chef within a virtual reality environment and learn how to cook.

[0064] The "order processing means for receiving an order and arranging delivery" is a function for processing an order for food or the like placed by a user, and arranging for the appropriate delivery route and supplier for the order details.

[0065] The "means for receiving a request" is a function whereby the system receives a request input by a user via a terminal and initiates appropriate processing.

[0066] "Means for providing a cooking experience using virtual reality technology" refers to technology that uses a virtual reality device to provide users with a variety of cooking experiences in a virtual environment.

[0067] A "means for generating answers to questions using generative AI" is a function that uses AI technology to automatically generate appropriate answers and explanations to users' questions.

[0068] To implement this invention, the following hardware and software are mainly required: a server, a terminal, a virtual reality device, generative AI technology, natural language processing technology, machine learning technology, and a database.

[0069] 1. Server The server plays a central role in receiving user requests and retrieving the necessary information. Specifically, the server accesses a food knowledge database to retrieve information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipes. The server can use database management systems such as MySQL (registered trademark) and PostgreSQL (registered trademark).

[0070] 2. Terminal A user sends a request to the system via his / her own terminal (e.g., a personal computer or a smartphone). The request from the terminal is sent to the server, where it is processed. A browser or a dedicated application is used on the terminal.

[0071] 3. Virtual reality devices A user wears a virtual reality (VR) device (e.g., Oculus Rift® or HTC Vive®) and accesses a virtual reality environment. Within the virtual reality environment, the user can interact with a virtual chef and enjoy a cooking experience.

[0072] 4. Generative AI technology Generative AI uses natural language processing (NLP) technology to generate appropriate answers and explanations to user questions. For example, advanced generative AI models such as OpenAI (registered trademark) ChatGPT (registered trademark) and BERT (Bidirectional Encoder Representations from Transformers) can be used. Generative AI is responsible for creating meaningful answers to users based on information obtained from a cooking knowledge database.

[0073] 5. Natural Language Processing Technology Natural language processing technology is used to analyze and understand the meaning of user input text and voice. Questions and requests entered by users on their devices are analyzed by natural language processing technology and sent to a server or generative AI.

[0074] 6. Machine Learning Technology Machine learning technology is used to analyze a user's past behavioral data and preferences, and to improve future suggestions and answers based on that data. For example, machine learning libraries such as Scikit-learn (registered trademark) and TENSORFLOW (registered trademark) can be used.

[0075] 7. Database The database stores a wide range of information related to cooking knowledge. MySQL (registered trademark) or PostgreSQL (registered trademark) is a suitable database management system (DBMS). The database quickly provides information in response to queries from the server.

[0076] A specific example will be described below. For example, if a user requests, "I want to know about Italian cuisine," the following prompt sentence is input to the generative AI model.

[0077] Example prompt: User: "I want to know about Italian food." Generative AI: "Italian cuisine has a long history, and each region has its own unique characteristics. For example, in southern Italy, there are many dishes that use tomatoes, while in northern Italy, cream sauces are the norm. Also, typical Italian dishes include spaghetti, pizza, and risotto. This explanation is based on information obtained from a cooking knowledge database."

[0078] Through this system, users can experience cooking in virtual reality and receive suggestions on dishes based on their preferences. It also responds quickly to user requests and provides appropriate cooking information and recipes.

[0079] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0080] Step 1: The user sends a request from the terminal. Specifically, the user enters "I want to know about Italian cuisine" into the input form on the terminal and presses the send button. Input: User request (e.g. "I want to know about Italian cuisine") Output: The user's request is sent to the server.

[0081] Step 2: The server receives the request and queries a cooking knowledge database. The server analyzes the user request and sends an SQL query to the database to retrieve relevant information (the dish's origin, characteristics, recipe, etc.). Input: User request (e.g. "I want to know about Italian cuisine") Output: Information retrieval results from the cooking knowledge database via SQL queries

[0082] Step 3: The server sends the acquired information to the generative AI. The server sends the information retrieved from the database to the generative AI to generate appropriate answers for the user. The generative AI uses this information to generate answers to the user's questions. Input: Information retrieved from databases (e.g. origins, characteristics, ingredients, recipes of Italian cuisine) Output: Answers generated by the generative AI for the user

[0083] Step 4: Generative AI generates answers using natural language processing technology. Generative AI uses natural language processing technology to convert the acquired information into appropriate sentences and generate answers in a form that is easy for the user to understand. Input: Information from a cooking knowledge database Output: A natural language processed answer for the user (e.g. "Italian cuisine has a long history and varies by region. For example...")

[0084] Step 5: The server transmits the generated answer to the user's terminal. The server sends the answers received from the generative AI to the user's device, allowing the user to view them. Input: Answer generated by generative AI Output: The answer displayed on the user's terminal

[0085] Step 6: The user puts on the virtual reality device and begins the cooking experience. Wearing the virtual reality device, a user enters the virtual reality space provided by the system and enjoys a cooking experience. Input: User interests and requests (e.g. Italian food experiences) Output: A user's cooking experience in virtual reality (e.g., interacting with a chef in a virtual Italian restaurant).

[0086] Step 7: Machine learning technology analyzes user preferences and uses them to inform the next request. The server collects user behavior data, analyzes it using machine learning algorithms, and updates a preference model to make future suggestions. Input: User behavior data in the virtual reality (e.g., which dishes they tried, which descriptions they were interested in) Output: Updated user preference model

[0087] Step 8: The user orders food and the server processes the order. Users order the food they experience in virtual reality, and the server receives the order and arranges for cooking and delivery. Input: User's order (e.g., order Italian food) Output: Meal ordered for delivery

[0088] According to the above specific processing steps, the system provides users with an advanced cooking experience. (Application example 1)

[0089] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0090] Although there have been services providing information and recipes for cooking, it was difficult for users to obtain more detailed information or personalized suggestions. In addition, there was a lack of a system that could provide guidance for users when actually cooking, immediate answers to questions, and a consistent system for ordering and delivering food. Furthermore, there were insufficient means to provide real-life experiences in a virtual space. This has created a need for an environment where users can efficiently learn to cook at home and order while experiencing it.

[0091] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0092] In this invention, the server includes: a means for acquiring information on the origin of a dish, characteristics of each country, ingredients, nutrition, taste, and recipes; a natural language processing means for explaining dishes to a user and answering questions; a machine learning means for analyzing the user's questions and preferences and proposing recommended dishes; a means including virtual reality technology in which a virtual cooking instructor interacts with the user in virtual reality and teaches the user how to cook a dish; an order processing means for receiving an order from the user and arranging for delivery of the dish; a means for a user to experience cooking through a smartphone and provide information about the dish and place an order; and a means for a generative AI to generate explanations of the dish and answers to questions. This enables an environment in which a user can obtain detailed cooking information while at home, experience actual cooking in a virtual space, and place an order while receiving personalized suggestions.

[0093] A "server" is a computer device that obtains information from a cooking knowledge database based on a user request and performs various processes.

[0094] "Cuisine" information is detailed data on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipes.

[0095] "Natural language processing means" refers to technology that understands and generates natural language to explain dishes to users and answer their questions.

[0096] "Machine learning tools" refers to algorithms that learn and improve from data to analyze user questions and preferences and provide dish recommendations.

[0097] "Virtual reality technology" refers to technology that allows users to have experiences in virtual reality, in particular by providing interaction and instruction with a virtual cooking instructor.

[0098] A "virtual cooking instructor" is a virtual entity that provides cooking tips and advice to users within a virtual reality space.

[0099] "Order processing means" refers to the function of accepting food orders from users and arranging for delivery.

[0100] "Smartphone" refers to a portable computing device that a user uses to conduct a cooking experience.

[0101] "Generative AI" is artificial intelligence that generates descriptions of dishes and answers to questions based on user requests.

[0102] The system that realizes this invention is composed of a server, a smartphone, a natural language processing means, a machine learning means, virtual reality technology, a virtual cooking instructor, and a generative AI. The details are described below.

[0103] Server Roles The server retrieves information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe from the food knowledge database. The server accesses the database based on the user's request, extracts the information, and processes it. The server also runs a generative AI to generate prompts to answer the user's questions and explain the dish.

[0104] The role of smartphones The smartphone is the interface through which users access the system. Users can input questions about food through the smartphone and receive answers from the generative AI in real time. Users can also order food and arrange for delivery through the smartphone.

[0105] Natural language processing methods and the role of generative AI The natural language processing means analyzes questions and requests entered by the user and sends them to the server in an appropriate format. The generative AI generates a description of the dish and an answer to the question based on the request. For example, if a user asks, "What kind of wine would go well with this dish?", the generative AI generates an answer about the appropriate type and characteristics of the wine.

[0106] Example prompt: User: "What are your recommendations for Italian food?" Prompt for AI: "What are your recommendations for Italian food?"

[0107] The role of machine learning methods The machine learning means learns the user's past behavior and preferences and recommends dishes tailored to the individual user. For example, if the user repeatedly searches for spicy dishes, the machine learning means preferentially recommends dishes based on spiciness.

[0108] Virtual reality technology and virtual cooking instructors By using virtual reality technology, users can enjoy a virtual cooking experience. A virtual cooking instructor can teach the user how to cook in the virtual space, providing specific cooking steps and tips. The user can proceed with cooking in real time while receiving instruction in the virtual space.

[0109] Order Processing Method The server accepts food orders from users and arranges for food delivery based on the details of the orders. The order processing means automatically performs various processes for delivering food to a location specified by the user, and completes the delivery quickly and accurately.

[0110] By combining these elements, users can obtain detailed recipe information, receive answers to their specific questions, and receive cooking instruction in a virtual space without leaving their home. They can also easily order food and have it delivered to their designated location.

[0111] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0112] Step 1: Users use their smartphones to access the application and input cooking questions. Input: "What are your recommendations for Italian food?" How it works: The smartphone receives the entered question and forwards it to a natural language processing means. Output: The entered question is sent to the server in parsed form.

[0113] Step 2: The server uses natural language processing means to analyze the user's question and generate a corresponding prompt. Input: User question: "What are the best Italian dishes?" How it works: A natural language processor analyzes the question and generates a prompt suitable for a generative AI. Output: Prompt text "What are your recommendations for Italian food?"

[0114] Step 3: A generative AI generates dish descriptions and recommendations based on prompts received from the server. Input: Prompt text "What are your recommendations for Italian food?" How it works: The generative AI searches a database based on the prompt text and generates the best answer. Output: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza."

[0115] Step 4: The server sends the answer generated by the generative AI to the smartphone. Input: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza." How it works: The server receives the generated answer, converts it into the appropriate format, and sends it to the smartphone. Output: The smartphone displays the answer to the user.

[0116] Step 5: The user checks the displayed answer on the smartphone, and if he / she wishes to order food, he / she enters the order information. Input: A request to order "spaghetti with tomato sauce" How it works: The user selects the food they want to order, enters the necessary information and submits it. Output: The smartphone sends the order information to the server.

[0117] Step 6: The server executes the order processing means based on the order information received from the user and makes delivery arrangements. Input: A request to order "spaghetti with tomato sauce" How it works: The server verifies the order information and works with the delivery company and restaurant to fulfill the order. Output: The user is notified of the delivery status, expected delivery time, etc.

[0118] Step 7: Using virtual reality technology, users receive real-time cooking instruction from a virtual cooking instructor on their smartphone. Input: Virtual cooking progress How it works: Based on virtual reality technology, a virtual cooking instructor gives instructions and advice. Output: The user receives guidance and advice in a timely manner.

[0119] Step 8: Machine learning techniques analyze a user's past behavior and preferences and incorporate them into the next dish recommendations. Input: User's past behavior history, preference data How it works: Machine learning algorithms analyze data and learn the best recipes for users. Output: The next food recommendation will be more personalized.

[0120] Through the above steps, the system provides a series of processes that allow users to efficiently obtain food information and place an order while experiencing actual cooking.

[0121] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0122] The embodiment for carrying out the present invention comprises the following elements.

[0123] 1. Server: Incorporating emotion engine and processing emotion data The server incorporates an emotion engine and performs processing to recognize the user's emotions. The server collects and analyzes the user's emotional data to understand the user's emotional state.

[0124] 2. Generative AI: Emotion-based dialogue generation and recipe suggestions adjustment The generative AI receives emotional data and generates dialogue based on the user's emotions. The generative AI will adjust the food suggestions and make food suggestions that match the user's emotions.

[0125] 3. Virtual reality technology: Virtual chef's facial expressions and attitude change based on emotions The virtual chef, existing in virtual reality, recognizes the user's emotions and changes his or her facial expressions and attitude. The virtual chef will show expressions such as smiles and surprise in response to the user's emotions, providing a more realistic cooking experience.

[0126] 4. Order processing means: Providing services and gifts according to emotions The server uses the user's emotional data to provide appropriate services and gifts when delivering food. By providing special services and thank-you messages tailored to the user's emotions, user satisfaction can be increased.

[0127] By combining the above elements, a system that recognizes the user's emotions and reflects them in the cooking experience is realized. The server incorporates an emotion engine and processes the user's emotion data. The generative AI generates dialogue and adjusts cooking suggestions according to emotions, and the virtual reality technology changes the virtual chef's facial expression and attitude based on emotions. The order processing means provides services and gifts according to the user's emotions, improving user satisfaction.

[0128] The process flow will be explained below.

[0129] Step 1: The server incorporates an emotion engine and processes emotion data. The server incorporates an emotion engine and starts the process to recognize the user's emotion. The server receives user input and sensor data and extracts emotion data.

[0130] The server analyzes the emotion data to understand the user's emotional state.

[0131] Step 2: The generative AI receives the emotion data and adjusts the dialogue and dish suggestions based on the emotion. The generative AI processes the emotional data received from the server and generates dialogue and adjusts cooking suggestions based on the emotion. The generative AI will adjust the content of the dialogue with the user and the dishes it suggests based on emotional data. Generative AI uses expressions and language that match emotions, making interactions with users more natural.

[0132] Step 3: The virtual reality technology receives the emotion data and processes it to control the changes in the virtual chef's facial expressions and attitude. The virtual chef, existing in virtual reality, receives emotional data and changes its facial expressions and attitude. The virtual chef will show facial expressions such as smiling or surprised based on emotional data, providing realistic reactions that match the user's emotions. The virtual chef's changing facial expressions and attitude allow users to enjoy a more familiar cooking experience.

[0133] Step 4: The order processing means uses the emotion data to provide services or gifts The server uses emotional data to provide appropriate services and gifts when delivering food. The server provides special services, thank-you messages, and other surprises based on the emotion data, and is tailored to the user's emotions. The server analyzes the emotional data and provides special services tailored to the user's tastes and preferences.

[0134] The above processing steps realize a cooking experience system that combines an emotion engine between the server and the user. The server processes the emotion data, and the generative AI and virtual reality technology control the dialogue and the virtual chef's facial expressions and attitudes based on the emotion data. The order processing means provides special services and gifts using the emotion data, providing a satisfying cooking experience that matches the user's emotions. Example 2

[0135] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0136] Conventional cooking experience systems lack the functionality to reflect the user's emotions and preferences in detail, making it difficult to provide an optimal experience that meets individual needs. In addition, the system does not analyze the user's emotional state in real time and generate appropriate dialogue or provide services based on that, so it has not been possible to sufficiently improve user satisfaction.

[0137] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0138] In this invention, the server includes a means for collecting and analyzing emotion data, a means for generating a dialogue based on the emotion data and suggesting a dish, a means for a virtual character in virtual reality to show an expression and attitude according to the emotion of the user, and a means for providing a service or a gift according to the emotion of the user. This makes it possible to analyze the emotion of the user in detail and provide an optimal dialogue and service according to individual needs.

[0139] "Means for collecting and analyzing emotional data" refers to technology for acquiring data such as a user's voice, facial expressions, and text input, analyzing this data using machine learning algorithms, etc., and classifying the user's emotional state.

[0140] The "means for generating dialogue based on emotional data and suggesting dishes" refers to a technology that uses natural language processing technology and machine learning models based on collected emotional data to generate dialogue with the user and suggest dishes that suit the user's emotions.

[0141] "A means for a virtual character in virtual reality to show facial expressions and attitudes that correspond to the user's emotions" refers to a technology that uses virtual reality (VR) technology to change the facial expressions and attitudes of a virtual character (e.g. a virtual chef) in real time based on the user's emotional data.

[0142] "Means for providing services and gifts according to the user's emotions" refers to technology that automatically selects and provides special services (e.g., message cards, desserts) and gifts based on the user's emotional data.

[0143] In an embodiment of the present invention, the system is composed of a server, a terminal, a generative AI model, virtual reality technology, and an order processing means. This system collects and analyzes the user's emotion data, generates dialogue and suggests dishes based on the emotion, and further provides a realistic experience using virtual reality technology.

[0144] Collecting and analyzing emotion data The server communicates with the user's device (smartphone, tablet, PC, etc.) and acquires data such as voice, facial expression, and text input. This allows the user's emotional state to be understood in real time. Specifically, when the user says "I'm tired today" into the device, the voice data is sent to the server. The server collects the emotional data and classifies the user's emotions by analyzing it using a machine learning model such as Google's TensorFlow.

[0145] Dialogue generation and dish suggestions Based on the analysis of the emotion data, the generative AI model generates dialogue and suggests dishes. For example, if the user is recognized as "tired," the generative AI model (e.g., ChatGPT (registered trademark) from OpenAI) generates dialogue such as "You seem tired today. How about a relaxing herbal tea?" In addition, IBM (registered trademark)'s Watson (registered trademark) is used to suggest dishes, and recipes with a relaxing effect can be suggested. The generated dialogue and dish suggestions are sent to the user's device via the server.

[0146] Changes in facial expressions and attitudes of virtual characters When a user puts on a VR device (e.g., Oculus Quest (registered trademark)) and enters the virtual reality, the server uses the Unity engine to control the facial expressions and attitudes of a virtual character (e.g., a virtual chef). For example, if the user's emotion is analyzed as "happy," the virtual character will smile. This allows the user to enjoy a realistic cooking experience.

[0147] Providing services and gifts The server provides special services and gifts according to the user's emotions. Specifically, it uses AWS's Lambda (registered trademark) as a trigger to prepare special message cards and desserts based on the user's emotion data. For example, if the user is recognized as "happy" on their birthday, it automatically provides a special message card and dessert service and notifies the user's device.

[0148] A specific example will be described below.

[0149] As a concrete example, consider a case where a user feels tired. When the user says to the terminal, "I'm tired today," the server collects the voice data and detects "fatigue" using TensorFlow (registered trademark). Next, the generative AI model generates a message to the user saying, "You seem tired today. How about some relaxing herbal tea?" and suggests it to the user. When the user puts on the VR device, the virtual chef shows a gentle smile and plays relaxing music while cooking. In this way, the user can enjoy a realistic cooking experience.

[0150] Examples of prompt statements If the user is tired, generate encouraging dialogue or suggest dishes to help them relax. Analyze user sentiment data and generate warm messages. Please suggest a menu that will help you relax. Keep the tone of the conversation friendly and create a relaxed atmosphere.

[0151] In this way, the present invention is a system that performs detailed analysis of a user's emotions and provides optimal dialogue and services in response to individual needs.

[0152] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0153] The flow of this system's program processing Step 1: Collect emotion data The user provides data such as voice, facial expressions, and text input through the device. The device's camera and microphone capture this data and send it to the server. Input: User's voice, facial expressions, and text data Data processing: The terminal captures data in real time and sends it to the server Output: Emotion data sent to the server

[0154] Step 2: Analyze the sentiment data The server analyzes the data it receives using machine learning models such as Google's TensorFlow (registered trademark) to classify the user's emotions. Input: Sentiment data collected in step 1 Data Computing: Use TensorFlow® for voice, image and text analysis and sentiment classification Output: Classified emotion data (e.g. "Tired", "Angry", "Happy", etc.)

[0155] Step 3: Dialogue generation The server sends the classified emotion data to the generative AI model, which then generates the appropriate dialogue. The generative AI model then processes natural language to generate the appropriate dialogue. Input: Classified emotion data Data processing: A generative AI model (e.g. ChatGPT(R) by OpenAI(R)) generates dialogue Output: Generated dialogue (e.g. "You seem to have had a tiring day today. Would you like some herbal tea to relax?")

[0156] Step 4: Food suggestions The generative AI model suggests dishes based on the user's emotional state. For example, if the user is tired, it suggests dishes that have a relaxing effect. Here, IBM's Watson is used to analyze the recipes. Input: Emotion data and analysis results Data processing: Analyze and select suitable recipes using Watson® Output: Cooking suggestions (e.g. "Relaxing herbal tea")

[0157] Step 5: Interact and send recipe suggestions The server transmits the generated dialogue and recipe suggestions to the user's terminal, and the user receives the dialogue and recipe suggestions through the terminal. Input: Generated dialogue and dish suggestions Data processing: Sending data from the server to the user's device Output: Dialogue and recipe suggestions displayed on the user's device

[0158] Step 6: Changing the virtual character's facial expression and attitude The user puts on the VR device and enters the virtual reality world. The server uses the Unity engine to control the facial expressions and attitudes of the virtual character according to the user's emotional state. Input: Classified emotion data Data processing: Change the facial expressions and attitudes of virtual characters in real time using the Unity engine Output: Changed facial expressions and attitudes of virtual characters displayed on a VR device

[0159] Step 7: Offer a special service or gift The server uses AWS Lambda to trigger special services and gifts based on the user's emotional state, including providing special message cards and desserts. Input: Emotion data and analysis results Data processing: Service preparation and trigger settings using AWS Lambda Output: Notification of special offers and gifts that have been arranged

[0160] Step 8: Notify and serve The server notifies the user's terminal that the service or gift is ready, and the user receives the special service. Input: Special offers and gift information Data transmission: Notification from the server to the user device Output: The special offer or gift the user receives

[0161] In this way, the entire system analyzes the user's emotions and provides optimal interactions and services tailored to individual needs, thereby increasing user satisfaction. (Application example 2)

[0162] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] Current food delivery systems have the problem that they do not provide appropriate food suggestions or special services that correspond to the user's emotions and preferences. In addition, conventional systems have a one-way dialogue with the user, which lacks an interactive experience, resulting in low user satisfaction.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0165] In this invention, the server includes an emotion recognition means for recognizing the emotion of the user, a generative AI means for generating a dialogue based on the emotion of the user, and a special service means for proposing special services and gifts based on the emotion of the user. This allows the server to provide appropriate dish suggestions, dialogue, and special services according to the emotional state of the user, thereby improving the satisfaction of the user.

[0166] "Emotion recognition means" refers to a technology for analyzing a user's facial image and voice data and identifying the user's emotions.

[0167] "Generative AI means" refers to artificial intelligence technology that generates natural language dialogue, cooking suggestions, etc. based on the user's emotional data.

[0168] The "special service means" is a function for providing special services or gifts according to the user's emotional state.

[0169] "Virtual reality technology" is a technology that provides users with a realistic virtual environment in which they can interact.

[0170] The "order processing means" is a system for accepting food orders from users and arranging for food delivery based on those orders.

[0171] "Natural language processing means" refers to technology that understands questions and input data from users and generates appropriate answers to them.

[0172] A "server" is a computer system that executes various processes such as emotion recognition means and generative AI means, and manages interactions with users.

[0173] "Dish suggestion" is the process of selecting and recommending appropriate dishes based on the user's emotions and preferences.

[0174] A "virtual chef" is a virtual character that teaches the user how to cook and interacts with them in a virtual reality environment.

[0175] "User's emotional state" refers to the mental and emotional state of the user that can be inferred from the user's facial expressions, tone of voice, etc.

[0176] The present invention relates to a food delivery system that recognizes the emotions of a user and responds individually based on the emotions. Specific embodiments of the present invention are described below.

[0177] 1. Implementation of emotion recognition measures A camera is installed on a user terminal (such as a smartphone or a head-mounted display) and is used to capture a facial image of the user. The captured facial image is sent to a server, where it is analyzed by an emotion recognition engine (e.g., using OpenCV (registered trademark) or dlib (registered trademark)). This analysis identifies the user's emotional state.

[0178] 2. Implementing generative AI methods The server inputs the emotion data acquired from the emotion recognition means into a generative AI model (e.g., ChatGPT). The generative AI model generates natural language dialogue and recipe suggestions based on the input emotion data. For example, if the user is determined to be "tired," the generative AI will suggest relaxing dishes and generate soothing dialogue.

[0179] Example prompt: The user is feeling tired. Based on that, suggest appropriate dialogue and dishes. The suggested dishes should be relaxing and the dialogue should include soothing words.

[0180] 3. Interact with the Virtual Chef The server uses virtual reality technology (e.g., Unity (registered trademark) 3D) to realize virtual chef interaction in order to provide the user with dialogue and cooking suggestions generated by the generative AI means. When the user puts on the head-mounted display, the virtual chef appears and teaches the user how to cook in real time and engages in dialogue according to the user's emotions.

[0181] 4. Implementation of special service measures The server has a special service means for providing special services or gifts based on the emotion data of the user. For example, when the user is recognized as being "happy," a special dessert is provided free of charge.

[0182] 5. Implementation of order processing procedures The server accepts the user's order and arranges for the delivery of an appropriate meal based on the order. The order processing means cooperates with the special service means to arrange for a special message or gift to be delivered at the same time.

[0183] A specific example will be described below.

[0184] For example, a user uses this system when he or she comes home from work and feels tired. The user puts on the HMD and the emotion recognition means judges that the user is "tired." The generative AI means generates relaxing food suggestions and soothing dialogue, and the virtual chef makes the suggestions. Once the order is confirmed, the server arranges for the delivery of relaxing food and herbal tea as a special service.

[0185] This makes it possible to provide a fulfilling food delivery experience that responds to the user's emotions.

[0186] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0187] Step 1: The user puts on a smartphone or a head-mounted display (HMD) and launches the food delivery system application. Input: User's facial image and voice data Specific operation: The device's camera captures the user's facial image and transmits it to the server in real time.

[0188] Step 2: The server receives the facial image data and analyzes the user's emotions using emotion recognition means (OpenCV (registered trademark) or dlib (registered trademark)). Input: User's facial image data Data processing: Facial landmark detection and facial expression analysis Output: Emotional state data (e.g. tired, happy, etc.) Specific operation: The server detects facial landmarks from the facial image data and analyzes the emotional state using an emotion recognition engine.

[0189] Step 3: The server inputs the emotional state data into a generative AI model (ChatGPT (registered trademark)) to generate dialogue and cooking suggestions. Input: Emotional state data Data Computation: Emotion-Based Prompt Generation and Dialogue Generation Output: Generated dialogue and recipe suggestions Specific operation: Based on the emotional state, the server generates a prompt such as, "The user is feeling tired. Please suggest appropriate conversations and dishes based on that," and inputs it into ChatGPT.

[0190] Step 4: The generated dialogue and cooking suggestions are loaded into a virtual chef using virtual reality technology (Unity (registered trademark) 3D), which then begins interacting with the user. Input: Generated dialogue and recipe suggestions Data calculation: Controlling the facial expressions and movements of the virtual chef Output: The virtual chef's dialogue and cooking suggestions are displayed to the user. Specific actions: Through the HMD, the virtual chef speaks to the user, suggests appropriate dishes, and explains cooking methods.

[0191] Step 5: The user selects the food he / she likes and confirms the order, at which time special service measures are applied. Input: Order data and emotional state data from users Data processing: selection of special services Output: Special message or gift when order is confirmed Specific operation: The server receives the order data, determines special messages and gifts according to the emotional state, and reflects them in the order contents.

[0192] Step 6: The server then transmits the final order data to the food supplier and arranges for delivery. Input: Final order data Data processing: generating delivery instructions Output: Delivery instruction data for the food provider Specific operation: The server sends delivery instruction data to the food provider and notifies them of the order details, including any special services or gifts.

[0193] Step 7: The food is delivered to the user and received along with special offers and gifts, allowing the user to enjoy a relaxing cooking experience. Specific operation: The delivery person delivers the food, special services, and gifts to the user and sends a notification of completion of receipt to the server. This will enable food delivery that is personalized according to the user's emotions.

[0194] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0195] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0196] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0197] [Second embodiment]

[0198] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0199] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0200] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0201] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0202] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0203] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0204] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0205] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0206] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0207] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0208] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0209] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0210] The embodiment for carrying out the present invention comprises the following elements.

[0211] 1. Server: Access to the cooking knowledge database and obtain information The server accesses a cooking knowledge database based on a user's request to obtain information on the origin of the dish, its national characteristics, ingredients, nutritional value, taste, and recipe.

[0212] 2. Generative AI: Generating recipe descriptions and answers to questions The generative AI is activated in response to a user request and generates an explanation of the dish or an answer to the question. The generative AI provides the user with an appropriate explanation and answer based on information obtained from a cooking knowledge database. For example, if a user is interested in Italian cuisine, the user can receive an explanation from the generative AI about the origin and characteristics of Italian cuisine, as well as recommended recipes. In another example, if a user asks, "What kind of wine goes well with this dish?" the generative AI will explain the appropriate types and characteristics of wines and suggest recommended wines.

[0213] 3. Virtual reality technology: Providing culinary experiences and fulfilling orders Using virtual reality technology, users can experience cuisine from various countries in virtual reality. The cooking experience includes cooking the food, learning how to cook the food, and eating the food. Users can also order food and drinks in real time while listening to explanations from generative AI.

[0214] 4. Machine learning tools: Analyzing user questions and preferences and suggesting recommended dishes Using machine learning techniques, the system analyzes the user's questions and preferences to suggest appropriate answers and recommended dishes. The machine learning techniques learn the user's past behavior and preferences and make suggestions tailored to each individual user. For example, if a user says, "I like spicy food," the generative AI will suggest recipes and recommended menu items that specialize in spicy food.

[0215] 5. Virtual Chef: Cooking instructions and practice support The virtual chef in the virtual reality world provides explanations and advice to help users learn how to cook. Users can practice cooking by following the virtual chef's instructions. In addition, users have a full 360-degree view in the virtual reality world, so they can move freely within the image. Users can move to the location of the generative AI virtual chef and receive instructions on how to cook a dish. For example, if a user wants to know how to make pasta, the user can move to the location of the virtual chef and be taught how to make pasta by the generative AI virtual chef.

[0216] 6. Order Processing: Accepting your orders and arranging delivery The server accepts the user's order and arranges for the food to be delivered. The user orders food in virtual reality, and the food is delivered to their home or a specified location.

[0217] By combining the above elements, the user can experience cooking in virtual reality and obtain information about cooking. The user can enjoy cooking experiences from all over the world while staying at home. In addition, the system also provides suggestions for recommended dishes based on the user's questions and preferences, and provides explanations and practice support on how to cook the dishes. The order processing means allows the user to order food in virtual reality, and the food is delivered to the user's home or a specified location.

[0218] The process flow will be explained below.

[0219] Step 1: The server retrieves information from the cooking knowledge database. The server receives the user's request and accesses the cooking knowledge database. The server retrieves information from a database about the food's origin, national characteristics, ingredients, nutritional value, taste, and recipe.

[0220] Step 2: The server starts the generative AI to generate recipe descriptions and answers to questions. The server launches the generative AI in response to a user request. The generative AI generates descriptions of dishes and answers to questions based on information obtained from a cooking knowledge database.

[0221] Step 3: The user experiences and orders food in virtual reality Users can experience cuisine from various countries in virtual reality. Users order food and drinks in real time while listening to descriptions of the dishes provided by generative AI.

[0222] Step 4: The server analyzes the user's questions and preferences and makes recommendations on dishes. The server analyzes the user's questions and preferences. The server uses machine learning methods to suggest appropriate answers and dish recommendations to the user.

[0223] Step 5: User interacts with the virtual chef and learns how to cook the dish The user interacts with the virtual chef in virtual reality. Users can practice cooking while receiving instructions and advice on how to prepare the dish from the virtual chef.

[0224] Step 6: The server receives the user's order and arranges for the food to be delivered. The server accepts the user's order. The server utilizes the order processing means to arrange for the food to be delivered.

[0225] Through the above processing steps, a cooking experience system is realized between the server and the user. The server obtains information from the cooking knowledge database and activates a generative AI to generate explanations of dishes and answers to questions. The user can experience cooking and place an order in virtual reality. The server analyzes the user's questions and preferences and suggests recommended dishes. The user can also interact with the virtual chef and learn how to cook. The server receives the user's order and arranges for the delivery of the food. Example 1

[0226] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0227] In conventional cooking experience systems, it was difficult for users to not only obtain cooking information but also to actually experience cooking and receive cooking suggestions according to their preferences. In addition, there was a lack of technical combinations to quickly respond to user requests and provide appropriate cooking information and recipes. Furthermore, there is a demand for a system that provides cooking experiences using virtual reality technology and provides appropriate feedback to users.

[0228] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0229] In this invention, the server includes a means for acquiring information, a means for performing natural language processing on the user, a machine learning means for analyzing the user's questions and preferences, a means for interacting with a virtual chef using virtual reality technology, an order processing means for receiving the user's order and arranging for delivery, a means for receiving requests via the user's terminal, a means for providing a cooking experience using virtual reality technology, and a means for generating answers to questions using generative AI. This allows the user to actually experience cooking in virtual reality and receive suggestions for dishes according to their preferences. In addition, the server can quickly respond to the user's requests and provide appropriate cooking information and recipes.

[0230] "Means for acquiring information" refers to a function for acquiring necessary information from a database or an external information source based on a user request.

[0231] "Means of natural language processing" refers to technology that analyzes text and voice data entered by the user, understands the meaning, and generates appropriate answers and information.

[0232] "Machine learning methods" are algorithms and technologies that analyze a user's past behavioral data and preferences, and use that data to improve future suggestions and responses.

[0233] The "means for interacting with a virtual chef" using "virtual reality technology" is a means for a user to interactively communicate with a virtual chef within a virtual reality environment and learn how to cook.

[0234] The "order processing means for receiving an order and arranging delivery" is a function for processing an order for food or the like placed by a user, and arranging for the appropriate delivery route and supplier for the order details.

[0235] The "means for receiving a request" is a function whereby the system receives a request input by a user via a terminal and initiates appropriate processing.

[0236] "Means for providing a cooking experience using virtual reality technology" refers to technology that uses a virtual reality device to provide users with a variety of cooking experiences in a virtual environment.

[0237] A "means for generating answers to questions using generative AI" is a function that uses AI technology to automatically generate appropriate answers and explanations to users' questions.

[0238] To implement this invention, the following hardware and software are mainly required: a server, a terminal, a virtual reality device, generative AI technology, natural language processing technology, machine learning technology, and a database.

[0239] 1. Server The server plays a central role in receiving user requests and retrieving the necessary information. Specifically, the server accesses a food knowledge database to retrieve information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe. The server can use a database management system such as MySQL or PostgreSQL.

[0240] 2. Terminal A user sends a request to the system via his / her own terminal (e.g., a personal computer or a smartphone). The request from the terminal is sent to the server, where it is processed. A browser or a dedicated application is used on the terminal.

[0241] 3. Virtual reality devices A user wears a virtual reality (VR) device (e.g., Oculus Rift® or HTC Vive®) and accesses a virtual reality environment. Within the virtual reality environment, the user can interact with a virtual chef and enjoy a cooking experience.

[0242] 4. Generative AI technology Generative AI uses natural language processing (NLP) technology to generate appropriate answers and explanations to user questions. For example, advanced generative AI models such as ChatGPT (registered trademark) from OpenAI (registered trademark) and BERT (Bidirectional Encoder Representations from Transformers) can be used. Generative AI is responsible for creating meaningful answers for users based on information obtained from a cooking knowledge database.

[0243] 5. Natural Language Processing Technology Natural language processing technology is used to analyze and understand the meaning of user input text and voice. Questions and requests entered by users on their devices are analyzed by natural language processing technology and sent to a server or generative AI.

[0244] 6. Machine Learning Technology Machine learning technology is used to analyze a user's past behavioral data and preferences, and to improve future suggestions and answers based on that data. For example, machine learning libraries such as Scikit-learn (registered trademark) and TensorFlow (registered trademark) can be used.

[0245] 7. Database The database stores a wide range of information related to cooking knowledge. MySQL (registered trademark) or PostgreSQL (registered trademark) is a suitable database management system (DBMS). The database quickly provides information in response to queries from the server.

[0246] A specific example will be described below.

[0247] For example, if a user requests, "I want to know about Italian cuisine," the following prompt sentence is input to the generative AI model.

[0248] Example prompt: User: "I want to know about Italian food." Generative AI: "Italian cuisine has a long history, and each region has its own unique characteristics. For example, in southern Italy, there are many dishes that use tomatoes, while in northern Italy, cream sauces are the norm. Also, typical Italian dishes include spaghetti, pizza, and risotto. This explanation is based on information obtained from a cooking knowledge database."

[0249] Through this system, users can experience cooking in virtual reality and receive suggestions on dishes based on their preferences. It also responds quickly to user requests and provides appropriate cooking information and recipes.

[0250] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0251] Step 1: The user sends a request from the terminal. Specifically, the user enters "I want to know about Italian cuisine" into the input form on the terminal and presses the send button. Input: User request (e.g. "I want to know about Italian cuisine") Output: The user's request is sent to the server.

[0252] Step 2: The server receives the request and queries a cooking knowledge database. The server analyzes the user request and sends an SQL query to the database to retrieve relevant information (the dish's origin, characteristics, recipe, etc.). Input: User request (e.g. "I want to know about Italian cuisine") Output: Information retrieval results from the cooking knowledge database via SQL queries

[0253] Step 3: The server sends the acquired information to the generative AI. The server sends the information retrieved from the database to the generative AI to generate appropriate answers for the user. The generative AI uses this information to generate answers to the user's questions. Input: Information retrieved from databases (e.g. origins, characteristics, ingredients, recipes of Italian cuisine) Output: Answers generated by the generative AI for the user

[0254] Step 4: Generative AI generates answers using natural language processing technology. Generative AI uses natural language processing technology to convert the acquired information into appropriate sentences and generate answers in a form that is easy for the user to understand. Input: Information from a cooking knowledge database Output: A natural language processed answer for the user (e.g. "Italian cuisine has a long history and varies by region. For example...")

[0255] Step 5: The server transmits the generated answer to the user's terminal. The server sends the answers received from the generative AI to the user's device, allowing the user to view them. Input: Answer generated by generative AI Output: The answer displayed on the user's terminal

[0256] Step 6: The user puts on the virtual reality device and begins the cooking experience. Wearing the virtual reality device, a user enters the virtual reality space provided by the system and enjoys a cooking experience. Input: User interests and requests (e.g. Italian food experiences) Output: A user's cooking experience in virtual reality (e.g., interacting with a chef in a virtual Italian restaurant).

[0257] Step 7: Machine learning technology analyzes user preferences and uses them to inform the next request. The server collects user behavior data, analyzes it using machine learning algorithms, and updates a preference model to make future suggestions. Input: User behavior data in the virtual reality (e.g., which dishes they tried, which descriptions they were interested in) Output: Updated user preference model

[0258] Step 8: The user orders food and the server processes the order. Users order the food they experience in virtual reality, and the server receives the order and arranges for cooking and delivery. Input: User's order (e.g., order Italian food) Output: Meal ordered for delivery

[0259] According to the above specific processing steps, the system provides users with an advanced cooking experience. (Application example 1)

[0260] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0261] Although there have been services providing information and recipes for cooking, it was difficult for users to obtain more detailed information or personalized suggestions. In addition, there was a lack of a system that could provide guidance for users when actually cooking, immediate answers to questions, and a consistent system for ordering and delivering food. Furthermore, there were insufficient means to provide real-life experiences in a virtual space. This has created a need for an environment where users can efficiently learn to cook at home and order while experiencing it.

[0262] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0263] In this invention, the server includes: a means for acquiring information on the origin of a dish, characteristics of each country, ingredients, nutrition, taste, and recipes; a natural language processing means for explaining dishes to a user and answering questions; a machine learning means for analyzing the user's questions and preferences and proposing recommended dishes; a means including virtual reality technology in which a virtual cooking instructor interacts with the user in virtual reality and teaches the user how to cook a dish; an order processing means for receiving an order from the user and arranging for delivery of the dish; a means for a user to experience cooking through a smartphone and provide information about the dish and place an order; and a means for a generative AI to generate explanations of the dish and answers to questions. This enables an environment in which a user can obtain detailed cooking information while at home, experience actual cooking in a virtual space, and place an order while receiving personalized suggestions.

[0264] A "server" is a computer device that obtains information from a cooking knowledge database based on a user request and performs various processes.

[0265] "Cuisine" information is detailed data on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipes.

[0266] "Natural language processing means" refers to technology that understands and generates natural language to explain dishes to users and answer their questions.

[0267] "Machine learning tools" refers to algorithms that learn and improve from data to analyze user questions and preferences and provide dish recommendations.

[0268] "Virtual reality technology" refers to technology that allows users to have experiences in virtual reality, in particular by providing interaction and instruction with a virtual cooking instructor.

[0269] A "virtual cooking instructor" is a virtual entity that provides cooking tips and advice to users within a virtual reality space.

[0270] "Order processing means" refers to the function of accepting food orders from users and arranging for delivery.

[0271] "Smartphone" refers to a portable computing device that a user uses to conduct a cooking experience.

[0272] "Generative AI" is artificial intelligence that generates descriptions of dishes and answers to questions based on user requests.

[0273] The system that realizes this invention is composed of a server, a smartphone, a natural language processing means, a machine learning means, virtual reality technology, a virtual cooking instructor, and a generative AI. The details are described below.

[0274] Server Roles The server retrieves information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe from the food knowledge database. The server accesses the database based on the user's request, extracts the information, and processes it. The server also runs a generative AI to generate prompts to answer the user's questions and explain the dish.

[0275] The role of smartphones The smartphone is the interface through which users access the system. Users can input questions about food through the smartphone and receive answers from the generative AI in real time. Users can also order food and arrange for delivery through the smartphone.

[0276] Natural language processing methods and the role of generative AI The natural language processing means analyzes questions and requests entered by the user and sends them to the server in an appropriate format. The generative AI generates a description of the dish and an answer to the question based on the request. For example, if a user asks, "What kind of wine would go well with this dish?", the generative AI generates an answer about the appropriate type and characteristics of the wine.

[0277] Example prompt: User: "What are your recommendations for Italian food?" Prompt for AI: "What are your recommendations for Italian food?"

[0278] The role of machine learning methods The machine learning means learns the user's past behavior and preferences and recommends dishes tailored to the individual user. For example, if the user repeatedly searches for spicy dishes, the machine learning means preferentially recommends dishes based on spiciness.

[0279] Virtual reality technology and virtual cooking instructors By using virtual reality technology, users can enjoy a virtual cooking experience. A virtual cooking instructor can teach the user how to cook in the virtual space, providing specific cooking steps and tips. The user can proceed with cooking in real time while receiving instruction in the virtual space.

[0280] Order Processing Method The server accepts food orders from users and arranges for food delivery based on the details of the orders. The order processing means automatically performs various processes for delivering food to a location specified by the user, and completes the delivery quickly and accurately.

[0281] By combining these elements, users can obtain detailed recipe information, receive answers to their specific questions, and receive cooking instruction in a virtual space without leaving their home. They can also easily order food and have it delivered to their designated location.

[0282] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0283] Step 1: Users use their smartphones to access the application and input cooking questions. Input: "What are your recommendations for Italian food?" How it works: The smartphone receives the entered question and forwards it to a natural language processing means. Output: The entered question is sent to the server in parsed form.

[0284] Step 2: The server uses natural language processing means to analyze the user's question and generate a corresponding prompt. Input: User question: "What are the best Italian dishes?" How it works: A natural language processor analyzes the question and generates a prompt suitable for a generative AI. Output: Prompt text "What are your recommendations for Italian food?"

[0285] Step 3: A generative AI generates dish descriptions and recommendations based on prompts received from the server. Input: Prompt text "What are your recommendations for Italian food?" How it works: The generative AI searches a database based on the prompt text and generates the best answer. Output: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza."

[0286] Step 4: The server sends the answer generated by the generative AI to the smartphone. Input: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza." How it works: The server receives the generated answer, converts it into the appropriate format, and sends it to the smartphone. Output: The smartphone displays the answer to the user.

[0287] Step 5: The user checks the displayed answer on the smartphone, and if he / she wishes to order food, he / she enters the order information. Input: A request to order "spaghetti with tomato sauce" How it works: The user selects the food they want to order, enters the necessary information and submits it. Output: The smartphone sends the order information to the server.

[0288] Step 6: The server executes the order processing means based on the order information received from the user and makes delivery arrangements. Input: A request to order "spaghetti with tomato sauce" How it works: The server verifies the order information and works with the delivery company and restaurant to fulfill the order. Output: The user is notified of the delivery status, expected delivery time, etc.

[0289] Step 7: Using virtual reality technology, users receive real-time cooking instruction from a virtual cooking instructor on their smartphone. Input: Virtual cooking progress How it works: Based on virtual reality technology, a virtual cooking instructor gives instructions and advice. Output: The user receives guidance and advice in a timely manner.

[0290] Step 8: Machine learning techniques analyze a user's past behavior and preferences and incorporate them into the next dish recommendations. Input: User's past behavior history, preference data How it works: Machine learning algorithms analyze data and learn the best recipes for users. Output: The next food recommendation will be more personalized.

[0291] Through the above steps, the system provides a series of processes that allow users to efficiently obtain food information and place an order while experiencing actual cooking.

[0292] Furthermore, an emotion engine that estimates the user's emotion may be combined.

[0293] The identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0294] The embodiment for carrying out the present invention comprises the following elements.

[0295] 1. Server: Incorporating emotion engine and processing emotion data The server incorporates an emotion engine and performs processing to recognize the user's emotions. The server collects and analyzes the user's emotional data to understand the user's emotional state.

[0296] 2. Generative AI: Emotion-based dialogue generation and recipe suggestions adjustment The generative AI receives emotional data and generates dialogue based on the user's emotions. The generative AI will adjust the food suggestions and make food suggestions that match the user's emotions.

[0297] 3. Virtual reality technology: Virtual chef's facial expressions and attitude change based on emotions The virtual chef, existing in virtual reality, recognizes the user's emotions and changes his or her facial expressions and attitude. The virtual chef will show expressions such as smiles and surprise in response to the user's emotions, providing a more realistic cooking experience.

[0298] 4. Order processing means: Providing services and gifts according to emotions The server uses the user's emotional data to provide appropriate services and gifts when delivering food. By providing special services and thank-you messages tailored to the user's emotions, user satisfaction can be increased.

[0299] By combining the above elements, a system that recognizes the user's emotions and reflects them in the cooking experience is realized. The server incorporates an emotion engine and processes the user's emotion data. The generative AI generates dialogue and adjusts cooking suggestions according to emotions, and the virtual reality technology changes the virtual chef's facial expression and attitude based on emotions. The order processing means provides services and gifts according to the user's emotions, improving user satisfaction.

[0300] The process flow will be explained below.

[0301] Step 1: The server incorporates an emotion engine and processes emotion data. The server incorporates an emotion engine and starts the process to recognize the user's emotion. The server receives user input and sensor data and extracts emotion data. The server analyzes the emotion data to understand the user's emotional state.

[0302] Step 2: The generative AI receives the emotion data and adjusts the dialogue and dish suggestions based on the emotion. The generative AI processes the emotional data received from the server and generates dialogue and adjusts cooking suggestions based on the emotion. The generative AI will adjust the content of the dialogue with the user and the dishes it suggests based on emotional data. Generative AI uses expressions and language that match emotions, making interactions with users more natural.

[0303] Step 3: The virtual reality technology receives the emotion data and processes it to control the changes in the virtual chef's facial expressions and attitude. The virtual chef, existing in virtual reality, receives emotional data and changes its facial expressions and attitude. The virtual chef will show facial expressions such as smiling or surprised based on emotional data, providing realistic reactions that match the user's emotions. The virtual chef's changing facial expressions and attitude allow users to enjoy a more familiar cooking experience.

[0304] Step 4: The order processing means uses the emotion data to provide services or gifts The server uses emotional data to provide appropriate services and gifts when delivering food. The server provides special services, thank-you messages, and other surprises based on the emotion data, and is tailored to the user's emotions. The server analyzes the emotional data and provides special services tailored to the user's tastes and preferences.

[0305] The above processing steps realize a cooking experience system that combines an emotion engine between the server and the user. The server processes the emotion data, and the generative AI and virtual reality technology control the dialogue and the virtual chef's facial expressions and attitudes based on the emotion data. The order processing means provides special services and gifts using the emotion data, providing a satisfying cooking experience that matches the user's emotions. Example 2

[0306] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0307] Conventional cooking experience systems lack the functionality to reflect the user's emotions and preferences in detail, making it difficult to provide an optimal experience that meets individual needs. In addition, the system does not analyze the user's emotional state in real time and generate appropriate dialogue or provide services based on that, so it has not been possible to sufficiently improve user satisfaction.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0309] In this invention, the server includes a means for collecting and analyzing emotion data, a means for generating a dialogue based on the emotion data and suggesting a dish, a means for a virtual character in virtual reality to show an expression and attitude according to the emotion of the user, and a means for providing a service or a gift according to the emotion of the user. This makes it possible to analyze the emotion of the user in detail and provide an optimal dialogue and service according to individual needs.

[0310] "Means for collecting and analyzing emotional data" refers to technology for acquiring data such as a user's voice, facial expressions, and text input, analyzing this data using machine learning algorithms, etc., and classifying the user's emotional state.

[0311] The "means for generating dialogue based on emotional data and suggesting dishes" refers to a technology that uses natural language processing technology and machine learning models based on collected emotional data to generate dialogue with the user and suggest dishes that suit the user's emotions.

[0312] "A means for a virtual character in virtual reality to show facial expressions and attitudes that correspond to the user's emotions" refers to a technology that uses virtual reality (VR) technology to change the facial expressions and attitudes of a virtual character (e.g. a virtual chef) in real time based on the user's emotional data.

[0313] "Means for providing services and gifts according to the user's emotions" refers to technology that automatically selects and provides special services (e.g., message cards, desserts) and gifts based on the user's emotional data.

[0314] In an embodiment of the present invention, the system is composed of a server, a terminal, a generative AI model, virtual reality technology, and an order processing means. This system collects and analyzes the user's emotion data, generates dialogue and suggests dishes based on the emotion, and further provides a realistic experience using virtual reality technology.

[0315] Collecting and analyzing emotion data The server communicates with the user's device (smartphone, tablet, PC, etc.) and acquires data such as voice, facial expression, and text input. This allows the user's emotional state to be understood in real time. Specifically, when the user says "I'm tired today" into the device, the voice data is sent to the server. The server collects the emotional data and classifies the user's emotions by analyzing it using a machine learning model such as Google's TensorFlow.

[0316] Dialogue generation and dish suggestions Based on the analysis of the emotion data, the generative AI model generates dialogue and suggests dishes. For example, if the user is recognized as "tired," the generative AI model (e.g. ChatGPT (registered trademark) from OpenAI (registered trademark)) generates dialogue such as "You seem tired today. How about a relaxing herbal tea?" In addition, IBM's Watson is used to suggest dishes, and recipes with a relaxing effect can be suggested. The generated dialogue and dish suggestions are sent to the user's device via the server.

[0317] Changes in facial expressions and attitudes of virtual characters When a user puts on a VR device (e.g., Oculus Quest (registered trademark)) and enters the virtual reality, the server uses the Unity engine to control the facial expressions and attitudes of a virtual character (e.g., a virtual chef). For example, if the user's emotion is analyzed as "happy," the virtual character will smile. This allows the user to enjoy a realistic cooking experience.

[0318] Providing services and gifts The server provides special services and gifts according to the user's emotions. Specifically, it uses AWS's Lambda (registered trademark) as a trigger to prepare special message cards and desserts based on the user's emotion data. For example, if the user is recognized as "happy" on their birthday, it automatically provides a special message card and dessert service and notifies the user's device.

[0319] A specific example will be described below.

[0320] As a concrete example, consider a case where a user feels tired. When the user says to the terminal, "I'm tired today," the server collects the voice data and detects "fatigue" using TensorFlow (registered trademark). Next, the generative AI model generates a message to the user saying, "You seem tired today. How about some relaxing herbal tea?" and suggests it to the user. When the user puts on the VR device, the virtual chef shows a gentle smile and plays relaxing music while cooking. In this way, the user can enjoy a realistic cooking experience.

[0321] Examples of prompt statements If the user is tired, generate encouraging dialogue or suggest dishes to help them relax. Analyze user sentiment data and generate warm messages. Please suggest a menu that will help you relax. Keep the tone of the conversation friendly and create a relaxed atmosphere.

[0322] In this way, the present invention is a system that performs detailed analysis of a user's emotions and provides optimal dialogue and services in response to individual needs.

[0323] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0324] The flow of this system's program processing Step 1: Collect emotion data The user provides data such as voice, facial expressions, and text input through the device. The device's camera and microphone capture this data and send it to the server. Input: User's voice, facial expressions, and text data Data processing: The terminal captures data in real time and sends it to the server Output: Emotion data sent to the server

[0325] Step 2: Analyze the sentiment data The server analyzes the data it receives using machine learning models such as Google's TensorFlow (registered trademark) to classify the user's emotions. Input: Sentiment data collected in step 1 Data Computing: Use TensorFlow® for voice, image and text analysis and sentiment classification Output: Classified emotion data (e.g. "Tired", "Angry", "Happy", etc.)

[0326] Step 3: Dialogue generation The server sends the classified emotion data to the generative AI model, which then generates the appropriate dialogue. The generative AI model then processes natural language to generate the appropriate dialogue. Input: Classified emotion data Data processing: A generative AI model (e.g. ChatGPT(R) by OpenAI(R)) generates dialogue Output: Generated dialogue (e.g. "You seem to have had a tiring day today. Would you like some herbal tea to relax?")

[0327] Step 4: Food suggestions The generative AI model suggests dishes based on the user's emotional state. For example, if the user is tired, it suggests dishes that have a relaxing effect. Here, IBM's Watson is used to analyze the recipes. Input: Emotion data and analysis results Data processing: Analyze and select suitable recipes using Watson® Output: Cooking suggestions (e.g. "Relaxing herbal tea")

[0328] Step 5: Interact and send recipe suggestions The server transmits the generated dialogue and recipe suggestions to the user's terminal, and the user receives the dialogue and recipe suggestions through the terminal. Input: Generated dialogue and dish suggestions Data processing: Sending data from the server to the user's device Output: Dialogue and recipe suggestions displayed on the user's device

[0329] Step 6: Changing the virtual character's facial expression and attitude The user puts on the VR device and enters the virtual reality world. The server uses the Unity engine to control the facial expressions and attitudes of the virtual character according to the user's emotional state. Input: Classified emotion data Data processing: Change the facial expressions and attitudes of virtual characters in real time using the Unity engine Output: Changed facial expressions and attitudes of virtual characters displayed on a VR device

[0330] Step 7: Offer a special service or gift The server uses AWS Lambda to trigger special services and gifts based on the user's emotional state, including providing special message cards and desserts. Input: Emotion data and analysis results Data processing: Service preparation and trigger settings using AWS Lambda Output: Notification of special offers and gifts that have been arranged

[0331] Step 8: Notify and serve The server notifies the user's terminal that the service or gift is ready, and the user receives the special service. Input: Special offers and gift information Data transmission: Notification from the server to the user device Output: The special offer or gift the user receives

[0332] In this way, the entire system analyzes the user's emotions and provides optimal interactions and services tailored to individual needs, thereby increasing user satisfaction. (Application example 2)

[0333] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0334] Current food delivery systems have the problem that they do not provide appropriate food suggestions or special services that correspond to the user's emotions and preferences. In addition, conventional systems have a one-way dialogue with the user, which lacks an interactive experience, resulting in low user satisfaction.

[0335] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0336] In this invention, the server includes an emotion recognition means for recognizing the emotion of the user, a generative AI means for generating a dialogue based on the emotion of the user, and a special service means for proposing special services and gifts based on the emotion of the user. This allows the server to provide appropriate dish suggestions, dialogue, and special services according to the emotional state of the user, thereby improving the satisfaction of the user.

[0337] "Emotion recognition means" refers to a technology for analyzing a user's facial image and voice data and identifying the user's emotions.

[0338] "Generative AI means" refers to artificial intelligence technology that generates natural language dialogue, cooking suggestions, etc. based on the user's emotional data.

[0339] The "special service means" is a function for providing special services or gifts according to the user's emotional state.

[0340] "Virtual reality technology" is a technology that provides users with a realistic virtual environment in which they can interact.

[0341] The "order processing means" is a system for accepting food orders from users and arranging for food delivery based on those orders.

[0342] "Natural language processing means" refers to technology that understands questions and input data from users and generates appropriate answers to them.

[0343] A "server" is a computer system that executes various processes such as emotion recognition means and generative AI means, and manages interactions with users.

[0344] "Dish suggestion" is the process of selecting and recommending appropriate dishes based on the user's emotions and preferences.

[0345] A "virtual chef" is a virtual character that teaches the user how to cook and interacts with them in a virtual reality environment.

[0346] "User's emotional state" refers to the mental and emotional state of the user that can be inferred from the user's facial expressions, tone of voice, etc.

[0347] The present invention relates to a food delivery system that recognizes the emotions of a user and responds individually based on the emotions. Specific embodiments of the present invention are described below.

[0348] 1. Implementation of emotion recognition measures A camera is installed on a user terminal (such as a smartphone or a head-mounted display) and is used to capture a facial image of the user. The captured facial image is sent to a server, where it is analyzed by an emotion recognition engine (e.g., using OpenCV (registered trademark) or dlib (registered trademark)). This analysis identifies the user's emotional state.

[0349] 2. Implementing generative AI methods The server inputs the emotion data acquired from the emotion recognition means into a generative AI model (e.g., ChatGPT (registered trademark)). The generative AI model generates natural language dialogue and recipe suggestions based on the input emotion data. For example, if the user is determined to be "tired," the generative AI will suggest relaxing dishes and generate soothing dialogue.

[0350] Example prompt: The user is feeling tired. Based on that, suggest appropriate dialogue and dishes. The suggested dishes should be relaxing and the dialogue should include soothing words.

[0351] 3. Interact with the Virtual Chef The server uses virtual reality technology (e.g., Unity (registered trademark) 3D) to realize virtual chef interaction in order to provide the user with dialogue and cooking suggestions generated by the generative AI means. When the user puts on the head-mounted display, the virtual chef appears and teaches the user how to cook in real time and engages in dialogue according to the user's emotions.

[0352] 4. Implementation of special service measures The server has a special service means for providing special services or gifts based on the emotion data of the user. For example, when the user is recognized as being "happy," a special dessert is provided free of charge.

[0353] 5. Implementation of order processing procedures The server accepts the user's order and arranges for the delivery of an appropriate meal based on the order. The order processing means cooperates with the special service means to arrange for a special message or gift to be delivered at the same time.

[0354] A specific example will be described below.

[0355] For example, a user uses this system when he or she comes home from work and feels tired. The user puts on the HMD and the emotion recognition means judges that the user is "tired." The generative AI means generates relaxing food suggestions and soothing dialogue, and the virtual chef makes the suggestions. Once the order is confirmed, the server arranges for the delivery of relaxing food and herbal tea as a special service.

[0356] This makes it possible to provide a fulfilling food delivery experience that responds to the user's emotions.

[0357] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0358] Step 1: The user puts on a smartphone or a head-mounted display (HMD) and launches the food delivery system application. Input: User's facial image and voice data Specific operation: The device's camera captures the user's facial image and transmits it to the server in real time.

[0359] Step 2: The server receives the facial image data and analyzes the user's emotions using emotion recognition means (OpenCV (registered trademark) or dlib (registered trademark)). Input: User's facial image data Data processing: Facial landmark detection and facial expression analysis Output: Emotional state data (e.g. tired, happy, etc.) Specific operation: The server detects facial landmarks from the facial image data and analyzes the emotional state using an emotion recognition engine.

[0360] Step 3: The server inputs the emotional state data into a generative AI model (ChatGPT (registered trademark)) to generate dialogue and cooking suggestions. Input: Emotional state data Data Computation: Emotion-Based Prompt Generation and Dialogue Generation Output: Generated dialogue and recipe suggestions Specific operation: Based on the emotional state, the server generates a prompt sentence such as "The user is feeling tired. Please suggest appropriate conversations and dishes based on that," and inputs it into ChatGPT (registered trademark).

[0361] Step 4: The generated dialogue and cooking suggestions are then loaded into a virtual chef using virtual reality technology (e.g., Unity (registered trademark) 3D) to begin interacting with the user. Input: Generated dialogue and recipe suggestions Data calculation: Controlling the facial expressions and movements of the virtual chef Output: The virtual chef's dialogue and cooking suggestions are displayed to the user. Specific actions: Through the HMD, the virtual chef speaks to the user, suggests appropriate dishes, and explains cooking methods.

[0362] Step 5: The user selects the food he / she likes and confirms the order, at which time special service measures are applied. Input: Order data and emotional state data from users Data processing: selection of special services Output: Special message or gift when order is confirmed Specific operation: The server receives the order data, determines special messages and gifts according to the emotional state, and reflects them in the order contents.

[0363] Step 6: The server then transmits the final order data to the food supplier and arranges for delivery. Input: Final order data Data processing: generating delivery instructions Output: Delivery instruction data for the food provider Specific operation: The server sends delivery instruction data to the food provider and notifies them of the order details, including any special services or gifts.

[0364] Step 7: The food is delivered to the user and received along with special offers and gifts, allowing the user to enjoy a relaxing cooking experience. Specific operation: The delivery person delivers the food, special services, and gifts to the user and sends a notification of completion of receipt to the server.

[0365] This will enable food delivery that is personalized according to the user's emotions.

[0366] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0367] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0368] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0369] [Third embodiment]

[0370] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0371] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0372] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0373] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0374] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0375] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0376] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0377] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0378] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0379] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0380] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0381] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server", and the headset type terminal 314 will be referred to as the "terminal".

[0382] The embodiment for carrying out the present invention comprises the following elements.

[0383] 1. Server: Access to the cooking knowledge database and obtain information The server accesses a cooking knowledge database based on a user's request to obtain information on the origin of the dish, its national characteristics, ingredients, nutritional value, taste, and recipe.

[0384] 2. Generative AI: Generating recipe descriptions and answers to questions The generative AI is activated in response to a user request and generates an explanation of the dish or an answer to the question. The generative AI provides the user with an appropriate explanation and answer based on information obtained from a cooking knowledge database. For example, if a user is interested in Italian cuisine, the user can receive an explanation from the generative AI about the origin and characteristics of Italian cuisine, as well as recommended recipes. In another example, if a user asks, "What kind of wine goes well with this dish?" the generative AI will explain the appropriate types and characteristics of wines and suggest recommended wines.

[0385] 3. Virtual reality technology: Providing culinary experiences and fulfilling orders Using virtual reality technology, users can experience cuisine from various countries in virtual reality. The cooking experience includes cooking the food, learning how to cook the food, and eating the food. Users can also order food and drinks in real time while listening to explanations from generative AI.

[0386] 4. Machine learning tools: Analyzing user questions and preferences and suggesting recommended dishes Using machine learning techniques, the system analyzes the user's questions and preferences to suggest appropriate answers and recommended dishes. The machine learning techniques learn the user's past behavior and preferences and make suggestions tailored to each individual user. For example, if a user says, "I like spicy food," the generative AI will suggest recipes and recommended menu items that specialize in spicy food.

[0387] 5. Virtual Chef: Cooking instructions and practice support The virtual chef in the virtual reality world provides explanations and advice to help users learn how to cook. Users can practice cooking by following the virtual chef's instructions. In addition, users have a full 360-degree view in the virtual reality world, so they can move freely within the image. Users can move to the location of the generative AI virtual chef and receive instructions on how to cook a dish. For example, if a user wants to know how to make pasta, the user can move to the location of the virtual chef and be taught how to make pasta by the generative AI virtual chef.

[0388] 6. Order Processing: Accepting your orders and arranging delivery The server accepts the user's order and arranges for the food to be delivered. The user orders food in virtual reality, and the food is delivered to their home or a specified location.

[0389] By combining the above elements, the user can experience cooking in virtual reality and obtain information about cooking. The user can enjoy cooking experiences from all over the world while staying at home. In addition, the system also provides suggestions for recommended dishes based on the user's questions and preferences, and provides explanations and practice support on how to cook the dishes. The order processing means allows the user to order food in virtual reality, and the food is delivered to the user's home or a specified location.

[0390] The process flow will be explained below.

[0391] Step 1: The server retrieves information from the cooking knowledge database. The server receives the user's request and accesses the cooking knowledge database. The server retrieves information from a database about the food's origin, national characteristics, ingredients, nutritional value, taste, and recipe.

[0392] Step 2: The server starts the generative AI to generate recipe descriptions and answers to questions. The server launches the generative AI in response to a user request. The generative AI generates descriptions of dishes and answers to questions based on information obtained from a cooking knowledge database.

[0393] Step 3: The user experiences and orders food in virtual reality Users can experience cuisine from various countries in virtual reality. Users order food and drinks in real time while listening to descriptions of the dishes provided by generative AI.

[0394] Step 4: The server analyzes the user's questions and preferences and makes recommendations on dishes. The server analyzes the user's questions and preferences. The server uses machine learning methods to suggest appropriate answers and dish recommendations to the user.

[0395] Step 5: User interacts with the virtual chef and learns how to cook the dish The user interacts with the virtual chef in virtual reality. Users can practice cooking while receiving instructions and advice on how to prepare the dish from the virtual chef.

[0396] Step 6: The server receives the user's order and arranges for the food to be delivered. The server accepts the user's order. The server utilizes the order processing means to arrange for the food to be delivered.

[0397] Through the above processing steps, a cooking experience system is realized between the server and the user. The server obtains information from the cooking knowledge database and activates a generative AI to generate explanations of dishes and answers to questions. The user can experience cooking and place an order in virtual reality. The server analyzes the user's questions and preferences and suggests recommended dishes. The user can also interact with the virtual chef and learn how to cook. The server receives the user's order and arranges for the delivery of the food. Example 1

[0398] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0399] In conventional cooking experience systems, it was difficult for users to not only obtain cooking information but also to actually experience cooking and receive cooking suggestions according to their preferences. In addition, there was a lack of technical combinations to quickly respond to user requests and provide appropriate cooking information and recipes. Furthermore, there is a demand for a system that provides cooking experiences using virtual reality technology and provides appropriate feedback to users.

[0400] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0401] In this invention, the server includes a means for acquiring information, a means for performing natural language processing on the user, a machine learning means for analyzing the user's questions and preferences, a means for interacting with a virtual chef using virtual reality technology, an order processing means for receiving the user's order and arranging for delivery, a means for receiving requests via the user's terminal, a means for providing a cooking experience using virtual reality technology, and a means for generating answers to questions using generative AI. This allows the user to actually experience cooking in virtual reality and receive suggestions for dishes according to their preferences. In addition, the server can quickly respond to the user's requests and provide appropriate cooking information and recipes.

[0402] "Means for acquiring information" refers to a function for acquiring necessary information from a database or an external information source based on a user request.

[0403] "Means of natural language processing" refers to technology that analyzes text and voice data entered by the user, understands the meaning, and generates appropriate answers and information.

[0404] "Machine learning methods" are algorithms and technologies that analyze a user's past behavioral data and preferences, and use that data to improve future suggestions and responses.

[0405] The "means for interacting with a virtual chef" using "virtual reality technology" is a means for a user to interactively communicate with a virtual chef within a virtual reality environment and learn how to cook.

[0406] The "order processing means for receiving an order and arranging delivery" is a function for processing an order for food or the like placed by a user, and arranging for the appropriate delivery route and supplier for the order details.

[0407] The "means for receiving a request" is a function whereby the system receives a request input by a user via a terminal and initiates appropriate processing.

[0408] "Means for providing a cooking experience using virtual reality technology" refers to technology that uses a virtual reality device to provide users with a variety of cooking experiences in a virtual environment.

[0409] A "means for generating answers to questions using generative AI" is a function that uses AI technology to automatically generate appropriate answers and explanations to users' questions.

[0410] To implement this invention, the following hardware and software are mainly required: a server, a terminal, a virtual reality device, generative AI technology, natural language processing technology, machine learning technology, and a database.

[0411] 1. Server The server plays a central role in receiving user requests and retrieving the necessary information. Specifically, the server accesses a food knowledge database to retrieve information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe. The server can use a database management system such as MySQL or PostgreSQL.

[0412] 2. Terminal A user sends a request to the system via his / her own terminal (e.g., a personal computer or a smartphone). The request from the terminal is sent to the server, where it is processed. A browser or a dedicated application is used on the terminal.

[0413] 3. Virtual reality devices A user wears a virtual reality (VR) device (e.g., Oculus Rift® or HTC Vive®) and accesses a virtual reality environment. Within the virtual reality environment, the user can interact with a virtual chef and enjoy a cooking experience.

[0414] 4. Generative AI technology Generative AI uses natural language processing (NLP) technology to generate appropriate answers and explanations to user questions. For example, advanced generative AI models such as ChatGPT (registered trademark) from OpenAI (registered trademark) and BERT (Bidirectional Encoder Representations from Transformers) can be used. Generative AI is responsible for creating meaningful answers for users based on information obtained from a cooking knowledge database.

[0415] 5. Natural Language Processing Technology Natural language processing technology is used to analyze and understand the meaning of user input text and voice. Questions and requests entered by users on their devices are analyzed by natural language processing technology and sent to a server or generative AI.

[0416] 6. Machine Learning Technology Machine learning technology is used to analyze a user's past behavioral data and preferences, and to improve future suggestions and answers based on that data. For example, machine learning libraries such as Scikit-learn (registered trademark) and TensorFlow (registered trademark) can be used.

[0417] 7. Database The database stores a wide range of information related to cooking knowledge. MySQL (registered trademark) or PostgreSQL (registered trademark) is a suitable database management system (DBMS). The database quickly provides information in response to queries from the server.

[0418] A specific example will be described below.

[0419] For example, if a user requests, "I want to know about Italian cuisine," the following prompt sentence is input to the generative AI model.

[0420] Example prompt: User: "I want to know about Italian food." Generative AI: "Italian cuisine has a long history, and each region has its own unique characteristics. For example, in southern Italy, there are many dishes that use tomatoes, while in northern Italy, cream sauces are the norm. Also, typical Italian dishes include spaghetti, pizza, and risotto. This explanation is based on information obtained from a cooking knowledge database."

[0421] Through this system, users can experience cooking in virtual reality and receive suggestions on dishes based on their preferences. It also responds quickly to user requests and provides appropriate cooking information and recipes.

[0422] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0423] Step 1: The user sends a request from the terminal. Specifically, the user enters "I want to know about Italian cuisine" into the input form on the terminal and presses the send button. Input: User request (e.g. "I want to know about Italian cuisine") Output: The user's request is sent to the server.

[0424] Step 2: The server receives the request and queries a cooking knowledge database. The server analyzes the user request and sends an SQL query to the database to retrieve relevant information (the dish's origin, characteristics, recipe, etc.). Input: User request (e.g. "I want to know about Italian cuisine") Output: Information retrieval results from the cooking knowledge database via SQL queries

[0425] Step 3: The server sends the acquired information to the generative AI. The server sends the information retrieved from the database to the generative AI to generate appropriate answers for the user. The generative AI uses this information to generate answers to the user's questions. Input: Information retrieved from databases (e.g. origins, characteristics, ingredients, recipes of Italian cuisine) Output: Answers generated by the generative AI for the user

[0426] Step 4: Generative AI generates answers using natural language processing technology. Generative AI uses natural language processing technology to convert the acquired information into appropriate sentences and generate answers in a form that is easy for the user to understand. Input: Information from a cooking knowledge database Output: A natural language processed answer for the user (e.g. "Italian cuisine has a long history and varies by region. For example...")

[0427] Step 5: The server transmits the generated answer to the user's terminal. The server sends the answers received from the generative AI to the user's device, allowing the user to view them. Input: Answer generated by generative AI Output: The answer displayed on the user's terminal

[0428] Step 6: The user puts on the virtual reality device and begins the cooking experience. Wearing the virtual reality device, a user enters the virtual reality space provided by the system and enjoys a cooking experience. Input: User interests and requests (e.g. Italian food experiences) Output: A user's cooking experience in virtual reality (e.g., interacting with a chef in a virtual Italian restaurant).

[0429] Step 7: Machine learning technology analyzes user preferences and uses them to inform the next request. The server collects user behavior data, analyzes it using machine learning algorithms, and updates a preference model to make future suggestions. Input: User behavior data in the virtual reality (e.g., which dishes they tried, which descriptions they were interested in) Output: Updated user preference model

[0430] Step 8: The user orders food and the server processes the order. Users order the food they experience in virtual reality, and the server receives the order and arranges for cooking and delivery. Input: User's order (e.g., order Italian food) Output: Meal ordered for delivery

[0431] According to the above specific processing steps, the system provides users with an advanced cooking experience. (Application example 1)

[0432] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0433] Although there have been services providing information and recipes for cooking, it was difficult for users to obtain more detailed information or personalized suggestions. In addition, there was a lack of a system that could provide guidance for users when actually cooking, immediate answers to questions, and a consistent system for ordering and delivering food. Furthermore, there were insufficient means to provide real-life experiences in a virtual space. This has created a need for an environment where users can efficiently learn to cook at home and order while experiencing it.

[0434] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0435] In this invention, the server includes: a means for acquiring information on the origin of a dish, characteristics of each country, ingredients, nutrition, taste, and recipes; a natural language processing means for explaining dishes to a user and answering questions; a machine learning means for analyzing the user's questions and preferences and proposing recommended dishes; a means including virtual reality technology in which a virtual cooking instructor interacts with the user in virtual reality and teaches the user how to cook a dish; an order processing means for receiving an order from the user and arranging for delivery of the dish; a means for a user to experience cooking through a smartphone and provide information about the dish and place an order; and a means for a generative AI to generate explanations of the dish and answers to questions. This enables an environment in which a user can obtain detailed cooking information while at home, experience actual cooking in a virtual space, and place an order while receiving personalized suggestions.

[0436] A "server" is a computer device that obtains information from a cooking knowledge database based on a user request and performs various processes.

[0437] "Cuisine" information is detailed data on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipes.

[0438] "Natural language processing means" refers to technology that understands and generates natural language to explain dishes to users and answer their questions.

[0439] "Machine learning tools" refers to algorithms that learn and improve from data to analyze user questions and preferences and provide dish recommendations.

[0440] "Virtual reality technology" refers to technology that allows users to have experiences in virtual reality, in particular by providing interaction and instruction with a virtual cooking instructor.

[0441] A "virtual cooking instructor" is a virtual entity that provides cooking tips and advice to users within a virtual reality space.

[0442] "Order processing means" refers to the function of accepting food orders from users and arranging for delivery.

[0443] "Smartphone" refers to a portable computing device that a user uses to conduct a cooking experience.

[0444] "Generative AI" is artificial intelligence that generates descriptions of dishes and answers to questions based on user requests.

[0445] The system that realizes this invention is composed of a server, a smartphone, a natural language processing means, a machine learning means, virtual reality technology, a virtual cooking instructor, and a generative AI. The details are described below.

[0446] Server Roles The server retrieves information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe from the food knowledge database. The server accesses the database based on the user's request, extracts the information, and processes it. The server also runs a generative AI to generate prompts to answer the user's questions and explain the dish.

[0447] The role of smartphones The smartphone is the interface through which users access the system. Users can input questions about food through the smartphone and receive answers from the generative AI in real time. Users can also order food and arrange for delivery through the smartphone.

[0448] Natural language processing methods and the role of generative AI The natural language processing means analyzes questions and requests entered by the user and sends them to the server in an appropriate format. The generative AI generates a description of the dish and an answer to the question based on the request. For example, if a user asks, "What kind of wine would go well with this dish?", the generative AI generates an answer about the appropriate type and characteristics of the wine.

[0449] Example prompt: User: "What are your recommendations for Italian food?" Prompt for AI: "What are your recommendations for Italian food?"

[0450] The role of machine learning methods The machine learning means learns the user's past behavior and preferences and recommends dishes tailored to the individual user. For example, if the user repeatedly searches for spicy dishes, the machine learning means preferentially recommends dishes based on spiciness.

[0451] Virtual reality technology and virtual cooking instructors By using virtual reality technology, users can enjoy a virtual cooking experience. A virtual cooking instructor can teach the user how to cook in the virtual space, providing specific cooking steps and tips. The user can proceed with cooking in real time while receiving instruction in the virtual space.

[0452] Order Processing Method The server accepts food orders from users and arranges for food delivery based on the details of the orders. The order processing means automatically performs various processes for delivering food to a location specified by the user, and completes the delivery quickly and accurately.

[0453] By combining these elements, users can obtain detailed recipe information, receive answers to their specific questions, and receive cooking instruction in a virtual space without leaving their home. They can also easily order food and have it delivered to their designated location.

[0454] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0455] Step 1: Users use their smartphones to access the application and input cooking questions. Input: "What are your recommendations for Italian food?" How it works: The smartphone receives the entered question and forwards it to a natural language processing means. Output: The entered question is sent to the server in parsed form.

[0456] Step 2: The server uses natural language processing means to analyze the user's question and generate a corresponding prompt. Input: User question: "What are the best Italian dishes?" How it works: A natural language processor analyzes the question and generates a prompt suitable for a generative AI. Output: Prompt text "What are your recommendations for Italian food?"

[0457] Step 3: A generative AI generates dish descriptions and recommendations based on prompts received from the server. Input: Prompt text "What are your recommendations for Italian food?" How it works: The generative AI searches a database based on the prompt text and generates the best answer. Output: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza."

[0458] Step 4: The server sends the answer generated by the generative AI to the smartphone. Input: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza." How it works: The server receives the generated answer, converts it into the appropriate format, and sends it to the smartphone. Output: The smartphone displays the answer to the user.

[0459] Step 5: The user checks the displayed answer on the smartphone, and if he / she wishes to order food, he / she enters the order information. Input: A request to order "spaghetti with tomato sauce" How it works: The user selects the food they want to order, enters the necessary information and submits it. Output: The smartphone sends the order information to the server.

[0460] Step 6: The server executes the order processing means based on the order information received from the user and makes delivery arrangements. Input: A request to order "spaghetti with tomato sauce" How it works: The server verifies the order information and works with the delivery company and restaurant to fulfill the order. Output: The user is notified of the delivery status, expected delivery time, etc.

[0461] Step 7: Using virtual reality technology, users receive real-time cooking instruction from a virtual cooking instructor on their smartphone. Input: Virtual cooking progress How it works: Based on virtual reality technology, a virtual cooking instructor gives instructions and advice. Output: The user receives guidance and advice in a timely manner.

[0462] Step 8: Machine learning techniques analyze a user's past behavior and preferences and incorporate them into the next dish recommendations. Input: User's past behavior history, preference data How it works: Machine learning algorithms analyze data and learn the best recipes for users. Output: The next food recommendation will be more personalized.

[0463] Through the above steps, the system provides a series of processes that allow users to efficiently obtain food information and place an order while experiencing actual cooking.

[0464] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0465] The embodiment for carrying out the present invention comprises the following elements.

[0466] 1. Server: Incorporating emotion engine and processing emotion data The server incorporates an emotion engine and performs processing to recognize the user's emotions. The server collects and analyzes the user's emotional data to understand the user's emotional state.

[0467] 2. Generative AI: Emotion-based dialogue generation and recipe suggestions adjustment The generative AI receives emotional data and generates dialogue based on the user's emotions. The generative AI will adjust the food suggestions and make food suggestions that match the user's emotions.

[0468] 3. Virtual reality technology: Virtual chef's facial expressions and attitude change based on emotions The virtual chef, existing in virtual reality, recognizes the user's emotions and changes his or her facial expressions and attitude. The virtual chef will show expressions such as smiles and surprise in response to the user's emotions, providing a more realistic cooking experience.

[0469] 4. Order processing means: Providing services and gifts according to emotions The server uses the user's emotional data to provide appropriate services and gifts when delivering food. By providing special services and thank-you messages tailored to the user's emotions, user satisfaction can be increased.

[0470] By combining the above elements, a system that recognizes the user's emotions and reflects them in the cooking experience is realized. The server incorporates an emotion engine and processes the user's emotion data. The generative AI generates dialogue and adjusts cooking suggestions according to emotions, and the virtual reality technology changes the virtual chef's facial expression and attitude based on emotions. The order processing means provides services and gifts according to the user's emotions, improving user satisfaction.

[0471] The process flow will be explained below.

[0472] Step 1: The server incorporates an emotion engine and processes emotion data. The server incorporates an emotion engine and starts the process to recognize the user's emotion. The server receives user input and sensor data and extracts emotion data. The server analyzes the emotion data to understand the user's emotional state.

[0473] Step 2: The generative AI receives the emotion data and adjusts the dialogue and dish suggestions based on the emotion. The generative AI processes the emotional data received from the server and generates dialogue and adjusts cooking suggestions based on the emotion. The generative AI will adjust the content of the dialogue with the user and the dishes it suggests based on emotional data. Generative AI uses expressions and language that match emotions, making interactions with users more natural.

[0474] Step 3: The virtual reality technology receives the emotion data and processes it to control the changes in the virtual chef's facial expressions and attitude. The virtual chef, existing in virtual reality, receives emotional data and changes its facial expressions and attitude. The virtual chef will show facial expressions such as smiling or surprised based on emotional data, providing realistic reactions that match the user's emotions. The virtual chef's changing facial expressions and attitude allow users to enjoy a more familiar cooking experience.

[0475] Step 4: The order processing means uses the emotion data to provide services or gifts The server uses emotional data to provide appropriate services and gifts when delivering food. The server provides special services, thank-you messages, and other surprises based on the emotion data, and is tailored to the user's emotions. The server analyzes the emotional data and provides special services tailored to the user's tastes and preferences.

[0476] The above processing steps realize a cooking experience system that combines an emotion engine between the server and the user. The server processes the emotion data, and the generative AI and virtual reality technology control the dialogue and the virtual chef's facial expressions and attitudes based on the emotion data. The order processing means provides special services and gifts using the emotion data, providing a satisfying cooking experience that matches the user's emotions. Example 2

[0477] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0478] Conventional cooking experience systems lack the functionality to reflect the user's emotions and preferences in detail, making it difficult to provide an optimal experience that meets individual needs. In addition, the system does not analyze the user's emotional state in real time and generate appropriate dialogue or provide services based on that, so it has not been possible to sufficiently improve user satisfaction.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0480] In this invention, the server includes a means for collecting and analyzing emotion data, a means for generating a dialogue based on the emotion data and suggesting a dish, a means for a virtual character in virtual reality to show an expression and attitude according to the emotion of the user, and a means for providing a service or a gift according to the emotion of the user. This makes it possible to analyze the emotion of the user in detail and provide an optimal dialogue and service according to individual needs.

[0481] "Means for collecting and analyzing emotional data" refers to technology for acquiring data such as a user's voice, facial expressions, and text input, analyzing this data using machine learning algorithms, etc., and classifying the user's emotional state.

[0482] The "means for generating dialogue based on emotional data and suggesting dishes" refers to a technology that uses natural language processing technology and machine learning models based on collected emotional data to generate dialogue with the user and suggest dishes that suit the user's emotions.

[0483] "A means for a virtual character in virtual reality to show facial expressions and attitudes that correspond to the user's emotions" refers to a technology that uses virtual reality (VR) technology to change the facial expressions and attitudes of a virtual character (e.g. a virtual chef) in real time based on the user's emotional data.

[0484] "Means for providing services and gifts according to the user's emotions" refers to technology that automatically selects and provides special services (e.g., message cards, desserts) and gifts based on the user's emotional data.

[0485] In an embodiment of the present invention, the system is composed of a server, a terminal, a generative AI model, virtual reality technology, and an order processing means. This system collects and analyzes the user's emotion data, generates dialogue and suggests dishes based on the emotion, and further provides a realistic experience using virtual reality technology.

[0486] Collecting and analyzing emotion data The server communicates with the user's device (smartphone, tablet, PC, etc.) and acquires data such as voice, facial expression, and text input. This allows the user's emotional state to be understood in real time. Specifically, when the user says "I'm tired today" into the device, the voice data is sent to the server. The server collects the emotional data and classifies the user's emotions by analyzing it using a machine learning model such as Google's TensorFlow.

[0487] Dialogue generation and dish suggestions Based on the analysis of the emotion data, the generative AI model generates dialogue and suggests dishes. For example, if the user is recognized as "tired," the generative AI model (e.g. ChatGPT (registered trademark) from OpenAI (registered trademark)) generates dialogue such as "You seem tired today. How about a relaxing herbal tea?" In addition, IBM's Watson is used to suggest dishes, and recipes with a relaxing effect can be suggested. The generated dialogue and dish suggestions are sent to the user's device via the server.

[0488] Changes in facial expressions and attitudes of virtual characters When a user puts on a VR device (e.g., Oculus Quest (registered trademark)) and enters the virtual reality, the server uses the Unity engine to control the facial expressions and attitudes of a virtual character (e.g., a virtual chef). For example, if the user's emotion is analyzed as "happy," the virtual character will smile. This allows the user to enjoy a realistic cooking experience.

[0489] Providing services and gifts The server provides special services and gifts according to the user's emotions. Specifically, it uses AWS's Lambda (registered trademark) as a trigger to prepare special message cards and desserts based on the user's emotion data. For example, if the user is recognized as "happy" on their birthday, it automatically provides a special message card and dessert service and notifies the user's device.

[0490] A specific example will be described below.

[0491] As a concrete example, consider a case where a user feels tired. When the user says to the terminal, "I'm tired today," the server collects the voice data and detects "fatigue" using TensorFlow (registered trademark). Next, the generative AI model generates a message to the user saying, "You seem tired today. How about some relaxing herbal tea?" and suggests it to the user. When the user puts on the VR device, the virtual chef shows a gentle smile and plays relaxing music while cooking. In this way, the user can enjoy a realistic cooking experience.

[0492] Examples of prompt statements If the user is tired, generate encouraging dialogue or suggest dishes to help them relax. Analyze user sentiment data and generate warm messages. Please suggest a menu that will help you relax. Keep the tone of the conversation friendly and create a relaxed atmosphere.

[0493] In this way, the present invention is a system that performs detailed analysis of a user's emotions and provides optimal dialogue and services in response to individual needs.

[0494] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0495] The flow of this system's program processing Step 1: Collect emotion data The user provides data such as voice, facial expressions, and text input through the device. The device's camera and microphone capture this data and send it to the server. Input: User's voice, facial expressions, and text data Data processing: The terminal captures data in real time and sends it to the server Output: Emotion data sent to the server

[0496] Step 2: Analyze the sentiment data The server analyzes the data it receives using machine learning models such as Google's TensorFlow (registered trademark) to classify the user's emotions. Input: Sentiment data collected in step 1 Data Computing: Use TensorFlow® for voice, image and text analysis and sentiment classification Output: Classified emotion data (e.g. "Tired", "Angry", "Happy", etc.)

[0497] Step 3: Dialogue generation The server sends the classified emotion data to the generative AI model, which then generates the appropriate dialogue. The generative AI model then processes natural language to generate the appropriate dialogue. Input: Classified emotion data Data processing: A generative AI model (e.g. ChatGPT(R) by OpenAI(R)) generates dialogue Output: Generated dialogue (e.g. "You seem to have had a tiring day today. Would you like some herbal tea to relax?")

[0498] Step 4: Food suggestions The generative AI model suggests dishes based on the user's emotional state. For example, if the user is tired, it suggests dishes that have a relaxing effect. Here, IBM's Watson is used to analyze the recipes. Input: Emotion data and analysis results Data processing: Analyze and select suitable recipes using Watson® Output: Cooking suggestions (e.g. "Relaxing herbal tea")

[0499] Step 5: Interact and send recipe suggestions The server transmits the generated dialogue and recipe suggestions to the user's terminal, and the user receives the dialogue and recipe suggestions through the terminal. Input: Generated dialogue and dish suggestions Data processing: Sending data from the server to the user's device Output: Dialogue and recipe suggestions displayed on the user's device

[0500] Step 6: Changing the virtual character's facial expression and attitude The user puts on the VR device and enters the virtual reality world. The server uses the Unity engine to control the facial expressions and attitudes of the virtual character according to the user's emotional state. Input: Classified emotion data Data processing: Change the facial expressions and attitudes of virtual characters in real time using the Unity engine Output: Changed facial expressions and attitudes of virtual characters displayed on a VR device

[0501] Step 7: Offer a special service or gift The server uses AWS Lambda to trigger special services and gifts based on the user's emotional state, including providing special message cards and desserts. Input: Emotion data and analysis results Data processing: Service preparation and trigger settings using AWS Lambda Output: Notification of special offers and gifts that have been arranged

[0502] Step 8: Notify and serve The server notifies the user's terminal that the service or gift is ready, and the user receives the special service. Input: Special offers and gift information Data transmission: Notification from the server to the user device Output: The special offer or gift the user receives

[0503] In this way, the entire system analyzes the user's emotions and provides optimal interactions and services tailored to individual needs, thereby increasing user satisfaction. (Application example 2)

[0504] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[0505] Current food delivery systems have the problem that they do not provide appropriate food suggestions or special services that correspond to the user's emotions and preferences. In addition, conventional systems have a one-way dialogue with the user, which lacks an interactive experience, resulting in low user satisfaction.

[0506] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0507] In this invention, the server includes an emotion recognition means for recognizing the emotion of the user, a generative AI means for generating a dialogue based on the emotion of the user, and a special service means for proposing special services and gifts based on the emotion of the user. This allows the server to provide appropriate dish suggestions, dialogue, and special services according to the emotional state of the user, thereby improving the satisfaction of the user.

[0508] "Emotion recognition means" refers to a technology for analyzing a user's facial image and voice data and identifying the user's emotions.

[0509] "Generative AI means" refers to artificial intelligence technology that generates natural language dialogue, cooking suggestions, etc. based on the user's emotional data.

[0510] The "special service means" is a function for providing special services or gifts according to the user's emotional state.

[0511] "Virtual reality technology" is a technology that provides users with a realistic virtual environment in which they can interact.

[0512] The "order processing means" is a system for accepting food orders from users and arranging for food delivery based on those orders.

[0513] "Natural language processing means" refers to technology that understands questions and input data from users and generates appropriate answers to them.

[0514] A "server" is a computer system that executes various processes such as emotion recognition means and generative AI means, and manages interactions with users.

[0515] "Dish suggestion" is the process of selecting and recommending appropriate dishes based on the user's emotions and preferences.

[0516] A "virtual chef" is a virtual character that teaches the user how to cook and interacts with them in a virtual reality environment.

[0517] "User's emotional state" refers to the mental and emotional state of the user that can be inferred from the user's facial expressions, tone of voice, etc.

[0518] The present invention relates to a food delivery system that recognizes the emotions of a user and responds individually based on the emotions. Specific embodiments of the present invention are described below.

[0519] 1. Implementation of emotion recognition measures A camera is installed on a user terminal (such as a smartphone or a head-mounted display) and is used to capture a facial image of the user. The captured facial image is sent to a server, where it is analyzed by an emotion recognition engine (e.g., using OpenCV (registered trademark) or dlib (registered trademark)). This analysis identifies the user's emotional state.

[0520] 2. Implementing generative AI methods The server inputs the emotion data acquired from the emotion recognition means into a generative AI model (e.g., ChatGPT). The generative AI model generates natural language dialogue and recipe suggestions based on the input emotion data. For example, if the user is determined to be "tired," the generative AI will suggest relaxing dishes and generate soothing dialogue.

[0521] Example prompt: The user is feeling tired. Based on that, suggest appropriate dialogue and dishes. The suggested dishes should be relaxing and the dialogue should include soothing words.

[0522] 3. Interact with the Virtual Chef The server uses virtual reality technology (e.g., Unity (registered trademark) 3D) to realize virtual chef interaction in order to provide the user with dialogue and cooking suggestions generated by the generative AI means. When the user puts on the head-mounted display, the virtual chef appears and teaches the user how to cook in real time and engages in dialogue according to the user's emotions.

[0523] 4. Implementation of special service measures The server has a special service means for providing special services or gifts based on the emotion data of the user. For example, when the user is recognized as being "happy," a special dessert is provided free of charge.

[0524] 5. Implementation of order processing procedures The server accepts the user's order and arranges for the delivery of an appropriate meal based on the order. The order processing means cooperates with the special service means to arrange for a special message or gift to be delivered at the same time.

[0525] A specific example will be described below.

[0526] For example, a user uses this system when he or she comes home from work and feels tired. The user puts on the HMD and the emotion recognition means judges that the user is "tired." The generative AI means generates relaxing food suggestions and soothing dialogue, and the virtual chef makes the suggestions. Once the order is confirmed, the server arranges for the delivery of relaxing food and herbal tea as a special service.

[0527] This makes it possible to provide a fulfilling food delivery experience that responds to the user's emotions.

[0528] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0529] Step 1: The user puts on a smartphone or a head-mounted display (HMD) and launches the food delivery system application. Input: User's facial image and voice data Specific operation: The device's camera captures the user's facial image and transmits it to the server in real time.

[0530] Step 2: The server receives the facial image data and analyzes the user's emotions using emotion recognition means (OpenCV (registered trademark) or dlib (registered trademark)). Input: User's facial image data Data processing: Facial landmark detection and facial expression analysis Output: Emotional state data (e.g. tired, happy, etc.) Specific operation: The server detects facial landmarks from the facial image data and analyzes the emotional state using an emotion recognition engine.

[0531] Step 3: The server inputs the emotional state data into a generative AI model (ChatGPT (registered trademark)) to generate dialogue and cooking suggestions. Input: Emotional state data Data Computation: Emotion-Based Prompt Generation and Dialogue Generation Output: Generated dialogue and recipe suggestions Specific operation: Based on the emotional state, the server generates a prompt such as, "The user is feeling tired. Please suggest appropriate conversations and dishes based on that," and inputs it into ChatGPT.

[0532] Step 4: The generated dialogue and cooking suggestions are then loaded into a virtual chef using virtual reality technology (e.g., Unity (registered trademark) 3D) to begin interacting with the user. Input: Generated dialogue and recipe suggestions Data calculation: Controlling the facial expressions and movements of the virtual chef Output: The virtual chef's dialogue and cooking suggestions are displayed to the user. Specific actions: Through the HMD, the virtual chef speaks to the user, suggests appropriate dishes, and explains cooking methods.

[0533] Step 5: The user selects the food he / she likes and confirms the order, at which time special service measures are applied. Input: Order data and emotional state data from users Data processing: selection of special services Output: Special message or gift when order is confirmed Specific operation: The server receives the order data, determines special messages and gifts according to the emotional state, and reflects them in the order contents.

[0534] Step 6: The server then transmits the final order data to the food supplier and arranges for delivery. Input: Final order data Data processing: generating delivery instructions Output: Delivery instruction data for the food provider Specific operation: The server sends delivery instruction data to the food provider and notifies them of the order details, including any special services or gifts.

[0535] Step 7: The food is delivered to the user and received along with special offers and gifts, allowing the user to enjoy a relaxing cooking experience. Specific operation: The delivery person delivers the food, special services, and gifts to the user and sends a notification of completion of receipt to the server.

[0536] This will enable food delivery that is personalized according to the user's emotions.

[0537] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0538] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0539] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0540] [Fourth embodiment]

[0541] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0542] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0543] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0544] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0545] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0546] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0547] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0548] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0549] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0550] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0551] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0552] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0553] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0554] The embodiment for carrying out the present invention comprises the following elements.

[0555] 1. Server: Access to the cooking knowledge database and obtain information The server accesses a cooking knowledge database based on a user's request to obtain information on the origin of the dish, its national characteristics, ingredients, nutritional value, taste, and recipe.

[0556] 2. Generative AI: Generating recipe descriptions and answers to questions The generative AI is activated in response to a user request and generates an explanation of the dish or an answer to the question. The generative AI provides the user with an appropriate explanation and answer based on information obtained from a cooking knowledge database. For example, if a user is interested in Italian cuisine, the user can receive an explanation from the generative AI about the origin and characteristics of Italian cuisine, as well as recommended recipes. In another example, if a user asks, "What kind of wine goes well with this dish?" the generative AI will explain the appropriate types and characteristics of wines and suggest recommended wines.

[0557] 3. Virtual reality technology: Providing culinary experiences and fulfilling orders Using virtual reality technology, users can experience cuisine from various countries in virtual reality. The cooking experience includes cooking the food, learning how to cook the food, and eating the food. Users can also order food and drinks in real time while listening to explanations from generative AI.

[0558] 4. Machine learning tools: Analyzing user questions and preferences and suggesting recommended dishes Using machine learning techniques, the system analyzes the user's questions and preferences to suggest appropriate answers and recommended dishes. The machine learning techniques learn the user's past behavior and preferences and make suggestions tailored to each individual user. For example, if a user says, "I like spicy food," the generative AI will suggest recipes and recommended menu items that specialize in spicy food.

[0559] 5. Virtual Chef: Cooking instructions and practice support The virtual chef in the virtual reality world provides explanations and advice to help users learn how to cook. Users can practice cooking by following the virtual chef's instructions. In addition, users have a full 360-degree view in the virtual reality world, so they can move freely within the image. Users can move to the location of the generative AI virtual chef and receive instructions on how to cook a dish. For example, if a user wants to know how to make pasta, the user can move to the location of the virtual chef and be taught how to make pasta by the generative AI virtual chef.

[0560] 6. Order Processing: Accepting your orders and arranging delivery The server accepts the user's order and arranges for the food to be delivered. The user orders food in virtual reality, and the food is delivered to their home or a specified location.

[0561] By combining the above elements, the user can experience cooking in virtual reality and obtain information about cooking. The user can enjoy cooking experiences from all over the world while staying at home. In addition, the system also provides suggestions for recommended dishes based on the user's questions and preferences, and provides explanations and practice support on how to cook the dishes. The order processing means allows the user to order food in virtual reality, and the food is delivered to the user's home or a specified location.

[0562] The process flow will be explained below.

[0563] Step 1: The server retrieves information from the cooking knowledge database. The server receives the user's request and accesses the cooking knowledge database. The server retrieves information from a database about the food's origin, national characteristics, ingredients, nutritional value, taste, and recipe.

[0564] Step 2: The server starts the generative AI to generate recipe descriptions and answers to questions. The server launches the generative AI in response to a user request. The generative AI generates descriptions of dishes and answers to questions based on information obtained from a cooking knowledge database.

[0565] Step 3: The user experiences and orders food in virtual reality Users can experience cuisine from various countries in virtual reality. Users order food and drinks in real time while listening to descriptions of the dishes provided by generative AI.

[0566] Step 4: The server analyzes the user's questions and preferences and makes recommendations on dishes. The server analyzes the user's questions and preferences. The server uses machine learning methods to suggest appropriate answers and dish recommendations to the user.

[0567] Step 5: User interacts with the virtual chef and learns how to cook the dish The user interacts with the virtual chef in virtual reality. Users can practice cooking while receiving instructions and advice on how to prepare the dish from the virtual chef.

[0568] Step 6: The server receives the user's order and arranges for the food to be delivered. The server accepts the user's order. The server utilizes the order processing means to arrange for the food to be delivered.

[0569] Through the above processing steps, a cooking experience system is realized between the server and the user. The server obtains information from the cooking knowledge database and activates a generative AI to generate explanations of dishes and answers to questions. The user can experience cooking and place an order in virtual reality. The server analyzes the user's questions and preferences and suggests recommended dishes. The user can also interact with the virtual chef and learn how to cook. The server receives the user's order and arranges for the delivery of the food. Example 1

[0570] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0571] In conventional cooking experience systems, it was difficult for users to not only obtain cooking information but also to actually experience cooking and receive cooking suggestions according to their preferences. In addition, there was a lack of technical combinations to quickly respond to user requests and provide appropriate cooking information and recipes. Furthermore, there is a demand for a system that provides cooking experiences using virtual reality technology and provides appropriate feedback to users.

[0572] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0573] In this invention, the server includes a means for acquiring information, a means for performing natural language processing on the user, a machine learning means for analyzing the user's questions and preferences, a means for interacting with a virtual chef using virtual reality technology, an order processing means for receiving the user's order and arranging for delivery, a means for receiving requests via the user's terminal, a means for providing a cooking experience using virtual reality technology, and a means for generating answers to questions using generative AI. This allows the user to actually experience cooking in virtual reality and receive suggestions for dishes according to their preferences. In addition, the server can quickly respond to the user's requests and provide appropriate cooking information and recipes.

[0574] "Means for acquiring information" refers to a function for acquiring necessary information from a database or an external information source based on a user request.

[0575] "Means of natural language processing" refers to technology that analyzes text and voice data entered by the user, understands the meaning, and generates appropriate answers and information.

[0576] "Machine learning methods" are algorithms and technologies that analyze a user's past behavioral data and preferences, and use that data to improve future suggestions and responses.

[0577] The "means for interacting with a virtual chef" using "virtual reality technology" is a means for a user to interactively communicate with a virtual chef within a virtual reality environment and learn how to cook.

[0578] The "order processing means for receiving an order and arranging delivery" is a function for processing an order for food or the like placed by a user, and arranging for the appropriate delivery route and supplier for the order details.

[0579] The "means for receiving a request" is a function whereby the system receives a request input by a user via a terminal and initiates appropriate processing.

[0580] "Means for providing a cooking experience using virtual reality technology" refers to technology that uses a virtual reality device to provide users with a variety of cooking experiences in a virtual environment.

[0581] A "means for generating answers to questions using generative AI" is a function that uses AI technology to automatically generate appropriate answers and explanations to users' questions.

[0582] To implement this invention, the following hardware and software are mainly required: a server, a terminal, a virtual reality device, generative AI technology, natural language processing technology, machine learning technology, and a database.

[0583] 1. Server The server plays a central role in receiving user requests and retrieving the necessary information. Specifically, the server accesses a food knowledge database to retrieve information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe. The server can use a database management system such as MySQL or PostgreSQL.

[0584] 2. Terminal A user sends a request to the system via his / her own terminal (e.g., a personal computer or a smartphone). The request from the terminal is sent to the server, where it is processed. A browser or a dedicated application is used on the terminal.

[0585] 3. Virtual reality devices A user wears a virtual reality (VR) device (e.g., Oculus Rift® or HTC Vive®) and accesses a virtual reality environment. Within the virtual reality environment, the user can interact with a virtual chef and enjoy a cooking experience.

[0586] 4. Generative AI technology Generative AI uses natural language processing (NLP) technology to generate appropriate answers and explanations to user questions. For example, advanced generative AI models such as ChatGPT (registered trademark) from OpenAI (registered trademark) and BERT (Bidirectional Encoder Representations from Transformers) can be used. Generative AI is responsible for creating meaningful answers for users based on information obtained from a cooking knowledge database.

[0587] 5. Natural Language Processing Technology Natural language processing technology is used to analyze and understand the meaning of user input text and voice. Questions and requests entered by users on their devices are analyzed by natural language processing technology and sent to a server or generative AI.

[0588] 6. Machine Learning Technology Machine learning technology is used to analyze a user's past behavioral data and preferences, and to improve future suggestions and answers based on that data. For example, machine learning libraries such as Scikit-learn (registered trademark) and TensorFlow (registered trademark) can be used.

[0589] 7. Database The database stores a wide range of information related to cooking knowledge. MySQL (registered trademark) or PostgreSQL (registered trademark) is a suitable database management system (DBMS). The database quickly provides information in response to queries from the server.

[0590] A specific example will be described below.

[0591] For example, if a user requests, "I want to know about Italian cuisine," the following prompt sentence is input to the generative AI model.

[0592] Example prompt: User: "I want to know about Italian food." Generative AI: "Italian cuisine has a long history, and each region has its own unique characteristics. For example, in southern Italy, there are many dishes that use tomatoes, while in northern Italy, cream sauces are the norm. Also, typical Italian dishes include spaghetti, pizza, and risotto. This explanation is based on information obtained from a cooking knowledge database."

[0593] Through this system, users can experience cooking in virtual reality and receive suggestions on dishes based on their preferences. It also responds quickly to user requests and provides appropriate cooking information and recipes.

[0594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0595] Step 1: The user sends a request from the terminal. Specifically, the user enters "I want to know about Italian cuisine" into the input form on the terminal and presses the send button. Input: User request (e.g. "I want to know about Italian cuisine") Output: The user's request is sent to the server.

[0596] Step 2: The server receives the request and queries a cooking knowledge database. The server analyzes the user request and sends an SQL query to the database to retrieve relevant information (the dish's origin, characteristics, recipe, etc.). Input: User request (e.g. "I want to know about Italian cuisine") Output: Information retrieval results from the cooking knowledge database via SQL queries

[0597] Step 3: The server sends the acquired information to the generative AI. The server sends the information retrieved from the database to the generative AI to generate appropriate answers for the user. The generative AI uses this information to generate answers to the user's questions. Input: Information retrieved from databases (e.g. origins, characteristics, ingredients, recipes of Italian cuisine) Output: Answers generated by the generative AI for the user

[0598] Step 4: Generative AI generates answers using natural language processing technology. Generative AI uses natural language processing technology to convert the acquired information into appropriate sentences and generate answers in a form that is easy for the user to understand. Input: Information from a cooking knowledge database Output: A natural language processed answer for the user (e.g. "Italian cuisine has a long history and varies by region. For example...")

[0599] Step 5: The server transmits the generated answer to the user's terminal. The server sends the answers received from the generative AI to the user's device, allowing the user to view them. Input: Answer generated by generative AI Output: The answer displayed on the user's terminal

[0600] Step 6: The user puts on the virtual reality device and begins the cooking experience. Wearing the virtual reality device, a user enters the virtual reality space provided by the system and enjoys a cooking experience. Input: User interests and requests (e.g. Italian food experiences) Output: A user's cooking experience in virtual reality (e.g., interacting with a chef in a virtual Italian restaurant).

[0601] Step 7: Machine learning technology analyzes user preferences and uses them to inform the next request. The server collects user behavior data, analyzes it using machine learning algorithms, and updates a preference model to make future suggestions. Input: User behavior data in the virtual reality (e.g., which dishes they tried, which descriptions they were interested in) Output: Updated user preference model

[0602] Step 8: The user orders food and the server processes the order. Users order the food they experience in virtual reality, and the server receives the order and arranges for cooking and delivery. Input: User's order (e.g., order Italian food) Output: Meal ordered for delivery

[0603] According to the above specific processing steps, the system provides users with an advanced cooking experience. (Application example 1)

[0604] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0605] Although there have been services providing information and recipes for cooking, it was difficult for users to obtain more detailed information or personalized suggestions. In addition, there was a lack of a system that could provide guidance for users when actually cooking, immediate answers to questions, and a consistent system for ordering and delivering food. Furthermore, there were insufficient means to provide real-life experiences in a virtual space. This has created a need for an environment where users can efficiently learn to cook at home and order while experiencing it.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0607] In this invention, the server includes: a means for acquiring information on the origin of a dish, characteristics of each country, ingredients, nutrition, taste, and recipes; a natural language processing means for explaining dishes to a user and answering questions; a machine learning means for analyzing the user's questions and preferences and proposing recommended dishes; a means including virtual reality technology in which a virtual cooking instructor interacts with the user in virtual reality and teaches the user how to cook a dish; an order processing means for receiving an order from the user and arranging for delivery of the dish; a means for a user to experience cooking through a smartphone and provide information about the dish and place an order; and a means for a generative AI to generate explanations of the dish and answers to questions. This enables an environment in which a user can obtain detailed cooking information while at home, experience actual cooking in a virtual space, and place an order while receiving personalized suggestions.

[0608] A "server" is a computer device that obtains information from a cooking knowledge database based on a user request and performs various processes.

[0609] "Cuisine" information is detailed data on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipes.

[0610] "Natural language processing means" refers to technology that understands and generates natural language to explain dishes to users and answer their questions.

[0611] "Machine learning tools" refers to algorithms that learn and improve from data to analyze user questions and preferences and provide dish recommendations.

[0612] "Virtual reality technology" refers to technology that allows users to have experiences in virtual reality, in particular by providing interaction and instruction with a virtual cooking instructor.

[0613] A "virtual cooking instructor" is a virtual entity that provides cooking tips and advice to users within a virtual reality space.

[0614] "Order processing means" refers to the function of accepting food orders from users and arranging for delivery.

[0615] "Smartphone" refers to a portable computing device that a user uses to conduct a cooking experience.

[0616] "Generative AI" is artificial intelligence that generates descriptions of dishes and answers to questions based on user requests.

[0617] The system that realizes this invention is composed of a server, a smartphone, a natural language processing means, a machine learning means, virtual reality technology, a virtual cooking instructor, and a generative AI. The details are described below.

[0618] Server Roles The server retrieves information on the origin of a dish, its national characteristics, ingredients, nutrition, taste, and recipe from the food knowledge database. The server accesses the database based on the user's request, extracts the information, and processes it. The server also runs a generative AI to generate prompts to answer the user's questions and explain the dish.

[0619] The role of smartphones The smartphone is the interface through which users access the system. Users can input questions about food through the smartphone and receive answers from the generative AI in real time. Users can also order food and arrange for delivery through the smartphone.

[0620] Natural language processing methods and the role of generative AI The natural language processing means analyzes questions and requests entered by the user and sends them to the server in an appropriate format. The generative AI generates a description of the dish and an answer to the question based on the request. For example, if a user asks, "What kind of wine would go well with this dish?", the generative AI generates an answer about the appropriate type and characteristics of the wine.

[0621] Example prompt: User: "What are your recommendations for Italian food?" Prompt for AI: "What are your recommendations for Italian food?"

[0622] The role of machine learning methods The machine learning means learns the user's past behavior and preferences and recommends dishes tailored to the individual user. For example, if the user repeatedly searches for spicy dishes, the machine learning means preferentially recommends dishes based on spiciness.

[0623] Virtual reality technology and virtual cooking instructors By using virtual reality technology, users can enjoy a virtual cooking experience. A virtual cooking instructor can teach the user how to cook in the virtual space, providing specific cooking steps and tips. The user can proceed with cooking in real time while receiving instruction in the virtual space.

[0624] Order Processing Method The server accepts food orders from users and arranges for food delivery based on the details of the orders. The order processing means automatically performs various processes for delivering food to a location specified by the user, and completes the delivery quickly and accurately.

[0625] By combining these elements, users can obtain detailed recipe information, receive answers to their specific questions, and receive cooking instruction in a virtual space without leaving their home. They can also easily order food and have it delivered to their designated location.

[0626] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0627] Step 1: Users use their smartphones to access the application and input cooking questions. Input: "What are your recommendations for Italian food?" How it works: The smartphone receives the entered question and forwards it to a natural language processing means. Output: The entered question is sent to the server in parsed form.

[0628] Step 2: The server uses natural language processing means to analyze the user's question and generate a corresponding prompt. Input: User question: "What are the best Italian dishes?" How it works: A natural language processor analyzes the question and generates a prompt suitable for a generative AI. Output: Prompt text "What are your recommendations for Italian food?"

[0629] Step 3: A generative AI generates dish descriptions and recommendations based on prompts received from the server. Input: Prompt text "What are your recommendations for Italian food?" How it works: The generative AI searches a database based on the prompt text and generates the best answer. Output: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza."

[0630] Step 4: The server sends the answer generated by the generative AI to the smartphone.

[0631] Input: "Recommended Italian dishes include spaghetti with tomato sauce and Margherita pizza." How it works: The server receives the generated answer, converts it into the appropriate format, and sends it to the smartphone. Output: The smartphone displays the answer to the user.

[0632] Step 5: The user checks the displayed answer on the smartphone, and if he / she wishes to order food, he / she enters the order information. Input: A request to order "spaghetti with tomato sauce" How it works: The user selects the food they want to order, enters the necessary information and submits it. Output: The smartphone sends the order information to the server.

[0633] Step 6: The server executes the order processing means based on the order information received from the user and makes delivery arrangements. Input: A request to order "spaghetti with tomato sauce" How it works: The server verifies the order information and works with the delivery company and restaurant to fulfill the order. Output: The user is notified of the delivery status, expected delivery time, etc.

[0634] Step 7: Using virtual reality technology, users receive real-time cooking instruction from a virtual cooking instructor on their smartphone. Input: Virtual cooking progress How it works: Based on virtual reality technology, a virtual cooking instructor gives instructions and advice. Output: The user receives guidance and advice in a timely manner.

[0635] Step 8: Machine learning techniques analyze a user's past behavior and preferences and incorporate them into the next dish recommendations. Input: User's past behavior history, preference data How it works: Machine learning algorithms analyze data and learn the best recipes for users. Output: The next food recommendation will be more personalized.

[0636] Through the above steps, the system provides a series of processes that allow users to efficiently obtain food information and place an order while experiencing actual cooking.

[0637] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0638] The embodiment for carrying out the present invention comprises the following elements.

[0639] 1. Server: Incorporating emotion engine and processing emotion data The server incorporates an emotion engine and performs processing to recognize the user's emotions. The server collects and analyzes the user's emotional data to understand the user's emotional state.

[0640] 2. Generative AI: Emotion-based dialogue generation and recipe suggestions adjustment The generative AI receives emotional data and generates dialogue based on the user's emotions. The generative AI will adjust the food suggestions and make food suggestions that match the user's emotions.

[0641] 3. Virtual reality technology: Virtual chef's facial expressions and attitude change based on emotions The virtual chef, existing in virtual reality, recognizes the user's emotions and changes his or her facial expressions and attitude. The virtual chef will show expressions such as smiles and surprise in response to the user's emotions, providing a more realistic cooking experience.

[0642] 4. Order processing means: Providing services and gifts according to emotions The server uses the user's emotional data to provide appropriate services and gifts when delivering food. By providing special services and thank-you messages tailored to the user's emotions, user satisfaction can be increased.

[0643] By combining the above elements, a system that recognizes the user's emotions and reflects them in the cooking experience is realized. The server incorporates an emotion engine and processes the user's emotion data. The generative AI generates dialogue and adjusts cooking suggestions according to emotions, and the virtual reality technology changes the virtual chef's facial expression and attitude based on emotions. The order processing means provides services and gifts according to the user's emotions, improving user satisfaction.

[0644] The process flow will be explained below.

[0645] Step 1: The server incorporates an emotion engine and processes emotion data. The server incorporates an emotion engine and starts the process to recognize the user's emotion. The server receives user input and sensor data and extracts emotion data. The server analyzes the emotion data to understand the user's emotional state.

[0646] Step 2: The generative AI receives the emotion data and adjusts the dialogue and dish suggestions based on the emotion. The generative AI processes the emotional data received from the server and generates dialogue and adjusts cooking suggestions based on the emotion. The generative AI will adjust the content of the dialogue with the user and the dishes it suggests based on emotional data. Generative AI uses expressions and language that match emotions, making interactions with users more natural.

[0647] Step 3: The virtual reality technology receives the emotion data and processes it to control the changes in the virtual chef's facial expressions and attitude. The virtual chef, existing in virtual reality, receives emotional data and changes its facial expressions and attitude. The virtual chef will show facial expressions such as smiling or surprised based on emotional data, providing realistic reactions that match the user's emotions. The virtual chef's changing facial expressions and attitude allow users to enjoy a more familiar cooking experience.

[0648] Step 4: The order processing means uses the emotion data to provide services or gifts The server uses emotional data to provide appropriate services and gifts when delivering food. The server provides special services, thank-you messages, and other surprises based on the emotion data, and is tailored to the user's emotions. The server analyzes the emotional data and provides special services tailored to the user's tastes and preferences.

[0649] The above processing steps realize a cooking experience system that combines an emotion engine between the server and the user. The server processes the emotion data, and the generative AI and virtual reality technology control the dialogue and the virtual chef's facial expressions and attitudes based on the emotion data. The order processing means provides special services and gifts using the emotion data, providing a satisfying cooking experience that matches the user's emotions. Example 2

[0650] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0651] Conventional cooking experience systems lack the functionality to reflect the user's emotions and preferences in detail, making it difficult to provide an optimal experience that meets individual needs. In addition, the system does not analyze the user's emotional state in real time and generate appropriate dialogue or provide services based on that, so it has not been possible to sufficiently improve user satisfaction.

[0652] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0653] In this invention, the server includes a means for collecting and analyzing emotion data, a means for generating a dialogue based on the emotion data and suggesting a dish, a means for a virtual character in virtual reality to show an expression and attitude according to the emotion of the user, and a means for providing a service or a gift according to the emotion of the user. This makes it possible to analyze the emotion of the user in detail and provide an optimal dialogue and service according to individual needs.

[0654] "Means for collecting and analyzing emotional data" refers to technology for acquiring data such as a user's voice, facial expressions, and text input, analyzing this data using machine learning algorithms, etc., and classifying the user's emotional state.

[0655] The "means for generating dialogue based on emotional data and suggesting dishes" refers to a technology that uses natural language processing technology and machine learning models based on collected emotional data to generate dialogue with the user and suggest dishes that suit the user's emotions.

[0656] "A means for a virtual character in virtual reality to show facial expressions and attitudes that correspond to the user's emotions" refers to a technology that uses virtual reality (VR) technology to change the facial expressions and attitudes of a virtual character (e.g. a virtual chef) in real time based on the user's emotional data.

[0657] "Means for providing services and gifts according to the user's emotions" refers to technology that automatically selects and provides special services (e.g., message cards, desserts) and gifts based on the user's emotional data.

[0658] In an embodiment of the present invention, the system is composed of a server, a terminal, a generative AI model, virtual reality technology, and an order processing means. This system collects and analyzes the user's emotion data, generates dialogue and suggests dishes based on the emotion, and further provides a realistic experience using virtual reality technology.

[0659] Collecting and analyzing emotion data The server communicates with the user's device (smartphone, tablet, PC, etc.) and acquires data such as voice, facial expression, and text input. This allows the user's emotional state to be understood in real time. Specifically, when the user says "I'm tired today" into the device, the voice data is sent to the server. The server collects the emotional data and classifies the user's emotions by analyzing it using a machine learning model such as Google's TensorFlow.

[0660] Dialogue generation and dish suggestions Based on the analysis of the emotion data, the generative AI model generates dialogue and suggests dishes. For example, if the user is recognized as "tired," the generative AI model (e.g. ChatGPT (registered trademark) from OpenAI (registered trademark)) generates dialogue such as "You seem tired today. How about a relaxing herbal tea?" In addition, IBM's Watson is used to suggest dishes, and recipes with a relaxing effect can be suggested. The generated dialogue and dish suggestions are sent to the user's device via the server.

[0661] Changes in facial expressions and attitudes of virtual characters When a user puts on a VR device (e.g., Oculus Quest (registered trademark)) and enters the virtual reality, the server uses the Unity engine to control the facial expressions and attitudes of a virtual character (e.g., a virtual chef). For example, if the user's emotion is analyzed as "happy," the virtual character will smile. This allows the user to enjoy a realistic cooking experience.

[0662] Providing services and gifts The server provides special services and gifts according to the user's emotions. Specifically, AWS Lambda is used as a trigger to prepare special message cards and desserts based on the user's emotion data. For example, if a user is recognized as "happy" on their birthday, a special message card and dessert service is automatically provided and a notification is sent to the user's device.

[0663] A specific example will be described below.

[0664] As a concrete example, consider a case where a user feels tired. When the user says to the terminal, "I'm tired today," the server collects the voice data and detects "fatigue" using TensorFlow (registered trademark). Next, the generative AI model generates a message to the user saying, "You seem tired today. How about some relaxing herbal tea?" and suggests it to the user. When the user puts on the VR device, the virtual chef shows a gentle smile and plays relaxing music while cooking. In this way, the user can enjoy a realistic cooking experience.

[0665] Examples of prompt statements If the user is tired, generate encouraging dialogue or suggest dishes to help them relax. Analyze user sentiment data and generate warm messages. Please suggest a menu that will help you relax. Keep the tone of the conversation friendly and create a relaxed atmosphere.

[0666] In this way, the present invention is a system that performs detailed analysis of a user's emotions and provides optimal dialogue and services in response to individual needs.

[0667] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0668] The flow of this system's program processing Step 1: Collect emotion data The user provides data such as voice, facial expressions, and text input through the device. The device's camera and microphone capture this data and send it to the server. Input: User's voice, facial expressions, and text data Data processing: The terminal captures data in real time and sends it to the server Output: Emotion data sent to the server

[0669] Step 2: Analyze the sentiment data The server analyzes the data it receives using machine learning models such as Google's TensorFlow (registered trademark) to classify the user's emotions. Input: Sentiment data collected in step 1 Data Computing: Use TensorFlow® for voice, image and text analysis and sentiment classification Output: Classified emotion data (e.g. "Tired", "Angry", "Happy", etc.)

[0670] Step 3: Dialogue generation The server sends the classified emotion data to the generative AI model, which then generates the appropriate dialogue. The generative AI model then processes natural language to generate the appropriate dialogue. Input: Classified emotion data Data processing: A generative AI model (e.g. ChatGPT(R) by OpenAI(R)) generates dialogue Output: Generated dialogue (e.g. "You seem to have had a tiring day today. Would you like some herbal tea to relax?")

[0671] Step 4: Food suggestions The generative AI model suggests dishes based on the user's emotional state. For example, if the user is tired, it suggests dishes that have a relaxing effect. Here, IBM's Watson is used to analyze the recipes. Input: Emotion data and analysis results Data processing: Analyze and select suitable recipes using Watson® Output: Cooking suggestions (e.g. "Relaxing herbal tea")

[0672] Step 5: Interact and send recipe suggestions The server transmits the generated dialogue and recipe suggestions to the user's terminal, and the user receives the dialogue and recipe suggestions through the terminal. Input: Generated dialogue and dish suggestions Data processing: Sending data from the server to the user's device Output: Dialogue and recipe suggestions displayed on the user's device

[0673] Step 6: Changing the virtual character's facial expression and attitude The user puts on the VR device and enters the virtual reality world. The server uses the Unity engine to control the facial expressions and attitudes of the virtual character according to the user's emotional state. Input: Classified emotion data Data processing: Change the facial expressions and attitudes of virtual characters in real time using the Unity engine Output: Changed facial expressions and attitudes of virtual characters displayed on a VR device

[0674] Step 7: Offer a special service or gift The server uses AWS Lambda to trigger special services and gifts based on the user's emotional state, including providing special message cards and desserts. Input: Emotion data and analysis results Data processing: Service preparation and trigger settings using AWS Lambda Output: Notification of special offers and gifts that have been arranged

[0675] Step 8: Notify and serve The server notifies the user's terminal that the service or gift is ready, and the user receives the special service. Input: Special offers and gift information Data transmission: Notification from the server to the user device Output: The special offer or gift the user receives

[0676] In this way, the entire system analyzes the user's emotions and provides optimal interactions and services tailored to individual needs, thereby increasing user satisfaction. (Application example 2)

[0677] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0678] Current food delivery systems have the problem that they do not provide appropriate food suggestions or special services that correspond to the user's emotions and preferences. In addition, conventional systems have a one-way dialogue with the user, which lacks an interactive experience, resulting in low user satisfaction.

[0679] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0680] In this invention, the server includes an emotion recognition means for recognizing the emotion of the user, a generative AI means for generating a dialogue based on the emotion of the user, and a special service means for proposing special services and gifts based on the emotion of the user. This allows the server to provide appropriate dish suggestions, dialogue, and special services according to the emotional state of the user, thereby improving the satisfaction of the user.

[0681] "Emotion recognition means" refers to a technology for analyzing a user's facial image and voice data and identifying the user's emotions.

[0682] "Generative AI means" refers to artificial intelligence technology that generates natural language dialogue, cooking suggestions, etc. based on the user's emotional data.

[0683] The "special service means" is a function for providing special services or gifts according to the user's emotional state.

[0684] "Virtual reality technology" is a technology that provides users with a realistic virtual environment in which they can interact.

[0685] The "order processing means" is a system for accepting food orders from users and arranging for food delivery based on those orders.

[0686] "Natural language processing means" refers to technology that understands questions and input data from users and generates appropriate answers to them.

[0687] A "server" is a computer system that executes various processes such as emotion recognition means and generative AI means, and manages interactions with users.

[0688] "Dish suggestion" is the process of selecting and recommending appropriate dishes based on the user's emotions and preferences.

[0689] A "virtual chef" is a virtual character that teaches the user how to cook and interacts with them in a virtual reality environment.

[0690] "User's emotional state" refers to the mental and emotional state of the user that can be inferred from the user's facial expressions, tone of voice, etc.

[0691] The present invention relates to a food delivery system that recognizes the emotions of a user and responds individually based on the emotions. Specific embodiments of the present invention are described below.

[0692] 1. Implementation of emotion recognition measures A camera is installed on a user terminal (such as a smartphone or a head-mounted display) and is used to capture a facial image of the user. The captured facial image is sent to a server, where it is analyzed by an emotion recognition engine (e.g., using OpenCV (registered trademark) or dlib (registered trademark)). This analysis identifies the user's emotional state.

[0693] 2. Implementing generative AI methods The server inputs the emotion data acquired from the emotion recognition means into a generative AI model (e.g., ChatGPT (registered trademark)). The generative AI model generates natural language dialogue and recipe suggestions based on the input emotion data. For example, if the user is determined to be "tired," the generative AI will suggest relaxing dishes and generate soothing dialogue.

[0694] Example prompt: The user is feeling tired. Based on that, suggest appropriate dialogue and dishes. The suggested dishes should be relaxing and the dialogue should include soothing words.

[0695] 3. Interact with the Virtual Chef The server uses virtual reality technology (e.g., Unity3D and VR SDK) to realize virtual chef interaction in order to provide the user with dialogue and cooking suggestions generated by generative AI means. When the user wears a head-mounted display, a virtual chef appears and teaches the user how to cook in real time and engages in dialogue according to the user's emotions.

[0696] 4. Implementation of special service measures The server has a special service means for providing special services or gifts based on the emotion data of the user. For example, when the user is recognized as being "happy," a special dessert is provided free of charge.

[0697] 5. Implementation of order processing procedures The server accepts the user's order and arranges for the delivery of an appropriate meal based on the order. The order processing means cooperates with the special service means to arrange for a special message or gift to be delivered at the same time.

[0698] A specific example will be described below.

[0699] For example, a user uses this system when he or she comes home from work and feels tired. The user puts on the HMD and the emotion recognition means judges that the user is "tired." The generative AI means generates relaxing food suggestions and soothing dialogue, and the virtual chef makes the suggestions. Once the order is confirmed, the server arranges for the delivery of relaxing food and herbal tea as a special service.

[0700] This makes it possible to provide a fulfilling food delivery experience that responds to the user's emotions.

[0701] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0702] Step 1: The user puts on a smartphone or a head-mounted display (HMD) and launches the food delivery system application. Input: User's facial image and voice data Specific operation: The device's camera captures the user's facial image and transmits it to the server in real time.

[0703] Step 2: The server receives the facial image data and analyzes the user's emotions using emotion recognition means (OpenCV (registered trademark) or dlib (registered trademark)). Input: User's facial image data Data processing: Facial landmark detection and facial expression analysis Output: Emotional state data (e.g. tired, happy, etc.) Specific operation: The server detects facial landmarks from the facial image data and analyzes the emotional state using an emotion recognition engine.

[0704] Step 3: The server inputs the emotional state data into a generative AI model (ChatGPT) and generates dialogue and cooking suggestions. Input: Emotional state data Data Computation: Emotion-Based Prompt Generation and Dialogue Generation Output: Generated dialogue and recipe suggestions Specific operation: Based on the emotional state, the server generates a prompt such as, "The user is feeling tired. Please suggest appropriate conversations and dishes based on that," and inputs it into ChatGPT.

[0705] Step 4: The generated dialogue and cooking suggestions are then loaded into a virtual chef using virtual reality technology (e.g., Unity (registered trademark) 3D) to begin interacting with the user. Input: Generated dialogue and recipe suggestions Data calculation: Controlling the facial expressions and movements of the virtual chef Output: The virtual chef's dialogue and cooking suggestions are displayed to the user. Specific actions: Through the HMD, the virtual chef speaks to the user, suggests appropriate dishes, and explains cooking methods.

[0706] Step 5: The user selects the food he / she likes and confirms the order, at which time special service measures are applied. Input: Order data and emotional state data from users Data processing: selection of special services Output: Special message or gift when order is confirmed Specific operation: The server receives the order data, determines special messages and gifts according to the emotional state, and reflects them in the order contents.

[0707] Step 6: The server then transmits the final order data to the food supplier and arranges for delivery. Input: Final order data Data processing: generating delivery instructions Output: Delivery instruction data for the food provider Specific operation: The server sends delivery instruction data to the food provider and notifies them of the order details, including any special services or gifts.

[0708] Step 7: The food is delivered to the user and received along with special offers and gifts, allowing the user to enjoy a relaxing cooking experience. Specific operation: The delivery person delivers the food, special services, and gifts to the user and sends a notification of completion of receipt to the server.

[0709] This will enable food delivery that is personalized according to the user's emotions.

[0710] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A outputs the result of the specific processing to the speaker 240 and the control target 443.

[0711] The microphone 238 acquires a voice indicating a user input for a result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0712] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0713] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[0714] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0715] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0716] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0717] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0718] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0719] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0720] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0721] Although the system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, the system according to the present disclosure is not necessarily implemented in a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[0722] In the above embodiment, an example is given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to input data.

[0723] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0724] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0725] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0726] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0727] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0728] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0729] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0730] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0731] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0732] The following is further disclosed regarding the above embodiment.

[0733] (Claim 1) A means for obtaining information on the origin, national characteristics, ingredients, nutrition, taste, and recipes of a dish; A natural language processing means for explaining dishes to a user and answering questions; A machine learning means for analyzing the user's questions and preferences and suggesting recommended dishes; A means including virtual reality technology for allowing a virtual chef to interact with the user in virtual reality and teach the user how to cook a dish; an order processing means for receiving an order from the user and arranging for delivery of food; A cooking experience system including:

[0734] (Claim 2) A server acquires necessary information from a cooking knowledge database based on a request from the user; The generative AI uses natural language processing to generate descriptions of dishes and answers to questions. The cooking experience system according to claim 1,

[0735] (Claim 3) A means for the server to start the generative AI in response to a request from the user, and for the generative AI to provide an explanation of the dish and answer questions; A means for the generative AI to explain to the user information regarding the origin, characteristics, ingredients, nutrition, taste, and recipe of a dish; The machine learning means generates answers in response to questions posed by the user and analyzes the preferences of the user; The cooking experience system according to claim 2,

[0736] (Claim 4) The cooking experience system according to claim 1, characterized in that an emotion engine analyzes the emotions of the user and changes the attitude and facial expression of the virtual chef in the virtual reality, thereby providing a realistic cooking experience.

[0737] "Example 1" (Claim 1) A means of obtaining information; a means for performing natural language processing on a user; Machine learning means to analyze user questions and preferences; A means of interacting with the virtual chef using virtual reality technology; an order processing means for receiving an order from a user and arranging delivery; means for receiving a request via a user terminal; A means to provide a cooking experience using virtual reality technology; A means for generating answers to questions using generative AI; A system including:

[0738] (Claim 2) A means for the server to obtain information based on a user request; The generative AI uses natural language processing to generate descriptions of dishes and answers to questions. 2. The system of claim 1, comprising:

[0739] (Claim 3) A means for the server to start the generative AI in response to a user request; A means for generative AI to explain information to users, A machine learning means for analyzing user preferences; 2. The system of claim 1, comprising:

[0740] "Application example 1" (Claim 1) A means for obtaining information on the origin, national characteristics, ingredients, nutrition, taste, and recipes of a dish; A natural language processing means for explaining dishes to a user and answering questions; A machine learning means for analyzing the user's questions and preferences and suggesting recommended dishes; A means including a virtual reality technology for allowing a virtual cooking instructor to interact with the user in virtual reality and teach the user how to cook; an order processing means for receiving an order from the user and arranging for delivery of food; A means for users to experience cooking through their smartphones, obtain information about dishes, and place orders; A means for generative AI to generate cooking explanations and answers to questions, A system including:

[0741] (Claim 2) A server acquires necessary information from a cooking knowledge database based on a request from the user; The system of claim 1, wherein the generative AI includes a natural language processing means for generating descriptions of dishes and answers to questions.

[0742] (Claim 3) A means for the server to start the generative AI in response to a request from the user, and for the generative AI to provide an explanation of the dish and answer questions; A means for the generative AI to explain to the user information regarding the origin, characteristics, ingredients, nutrition, taste, and recipe of a dish; 2. The system of claim 1, wherein the machine learning means includes means for generating answers in response to questions of the user and analyzing preferences of the user.

[0743] "Example 2 of combining emotion engines" (Claim 1) A means for collecting and analyzing emotion data; A means for generating dialogue and suggesting dishes based on emotion data; A means for a virtual character to show facial expressions and attitudes according to the user's emotions in virtual reality; A means for providing services and gifts according to the user's emotions; A system including:

[0744] (Claim 2) A means for acquiring information based on a user's request and responding with the generated information in natural language; 2. The system of claim 1, comprising:

[0745] (Claim 3) A means for generating an appropriate dialogue based on emotion data and providing a service or a gift according to the emotion; 2. The system of claim 1, comprising:

[0746] "Application example 2 when combining emotion engines" (Claim 1) A means for obtaining information on the origin, national characteristics, ingredients, nutrition, taste, and recipes of a dish; A natural language processing means for explaining dishes to a user and answering questions; A machine learning means for analyzing the user's questions and preferences and suggesting recommended dishes; A means including virtual reality technology for allowing a virtual chef to interact with the user in virtual reality and teach the user how to cook a dish; An emotion recognition means for recognizing an emotion of the user; A generative AI means for generating a dialogue based on the user's emotions; A special service means for proposing a special service or a gift based on the user's emotion; an order processing means for receiving an order from the user and arranging for delivery of food; A system including:

[0747] (Claim 2) A server acquires necessary information from a cooking knowledge database based on a request from the user; The generative AI uses natural language processing to generate descriptions of dishes and answers to questions. 2. The system of claim 1, comprising:

[0748] (Claim 3) A means for the server to start the generative AI in response to a request from the user, and for the generative AI to provide an explanation of the dish and answer questions; A means for the generative AI to explain to the user information regarding the origin, characteristics, ingredients, nutrition, taste, and recipe of a dish; The machine learning means generates answers in response to questions posed by the user and analyzes the preferences of the user; 2. The system of claim 1, comprising: [Explanation of symbols]

[0749] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring information on at least one of the origin, national characteristics, ingredients, nutrition, taste, and recipe of a dish; a natural language processing means for explaining dishes to a user and answering questions based on the information; a machine learning means for analyzing the user's preferences from the question of the user by machine learning and suggesting recommended dishes to the user based on the analysis; A means including virtual reality technology for generating a virtual chef, which is a virtual chef that interacts with the user in virtual reality and provides instructions and advice on how to cook a dish; A means for providing the user with a cooking experience using the virtual reality technology; an order processing means for receiving an order from the user and arranging for food delivery; A cooking experience system including:

2. The cooking experience system includes an emotion recognition means for recognizing an emotion of the user; A means for generating a dialogue by a generative AI based on the user's emotion; means for providing a service and / or a gift in conjunction with the generated interaction; The cooking experience system according to claim 1 , further comprising:

3. The means including the virtual reality technology changes the attitude and facial expression of the virtual chef in the virtual reality based on the emotion of the user analyzed by the emotion recognition means, thereby providing the user with a cooking experience. The cooking experience system according to claim 2.

Images