system

Abstract

Provide a system. A means for inputting user information before a user starts eating; A means of regularly recording your dietary progress; A means for analyzing eating speed and detecting abnormal eating speed; A means for generating appropriate questions based on past conversations and user profiles; means for presenting the generated questions to a user and receiving answers; means for analyzing the received answers and generating further questions; A means for adjusting timing to balance the eating speed of all users; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When eating, the stress and discomfort caused by different eating speeds among participants can hinder the enjoyment of the meal. In particular, when some participants eat extremely fast or slow, other participants may feel impatient and have to wait, which can cause stress. This makes it difficult to ensure a fair meal and provide a dining experience that all participants can enjoy. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system with the following features. First, it provides a means for the user to input user information before starting a meal. Second, it includes a means for periodically recording the progress of the meal. It also includes a means for analyzing the eating speed and detecting abnormal eating speeds. It also includes a means for generating appropriate questions based on past conversation content and the user profile. It provides a means for presenting the generated questions to the user and receiving answers. It also includes a means for analyzing the received answers and generating further questions. Finally, it provides a means for adjusting the timing to keep the eating speeds of all users balanced. This makes it possible to provide a dining experience in which all users can progress fairly.

[0006] "User information" refers to basic information such as user ID and meal details that a user enters when starting a meal.

[0007] "Meal progress" is data that indicates how much a user has eaten during a meal, and is recorded periodically.

[0008] "Eating speed" is an index that indicates how much food a user has eaten within a certain period of time.

[0009] "Abnormal eating speed" refers to an eating pace that is significantly faster or slower than other users.

[0010] A "user profile" refers to a collection of information that records a user's individual characteristics, such as past conversations and interests.

[0011] "Generating questions" means creating questions that are appropriate for the user based on the collected user profile and conversation content.

[0012] "Presenting a question" means displaying the generated question on the user's terminal and showing it to the user.

[0013] "Receiving an answer" means obtaining an answer entered by the user in response to a presented question.

[0014] "Analyzing responses" refers to the process of evaluating the responses received from users and determining appropriate questions or comments to lead to next.

[0015] "Adjusting the timing" means managing time to keep the eating speed of all users balanced and ensure smooth progress. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] System Overview

[0038] This invention is a system for balancing the eating speeds of all users when multiple users are eating together. The system operates by having users record and transmit their meal progress using their own devices. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[0039] Initial Setup

[0040] 1. Before starting a meal, the user inputs the "start eating" information into their device. The device then sends this information to the server.

[0041] 2. Based on the information received, the server records each user's session in the database and starts the session.

[0042] Monitoring and sending meal progress

[0043] 3. While eating, the user periodically inputs the amount eaten and progress into the device. The device then transmits this data to the server at regular intervals.

[0044] 4. The server analyzes the received data and calculates each user's eating speed.

[0045] Data Analysis and Anomaly Detection

[0046] 5. The server compares the eating speeds of all users and detects abnormal eating speeds of a particular user if the eating speed of that user is significantly faster or slower.

[0047] Question generation and presentation

[0048] 6. The server generates questions appropriate for a user with an abnormal eating speed based on past conversations and the user profile. For example, for a user who likes traveling, it generates a question such as "Where have you traveled recently?"

[0049] 7. The server sends the generated question to the user's device.

[0050] Answering questions and analyzing

[0051] 8. The device presents the received question to the user, who answers it. The answer is then sent back to the server via the device.

[0052] 9. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0053] Adjusting meal timing

[0054] 10. The server monitors the eating speed of all users in real time and continues to ask questions as appropriate to align everyone's eating timing.

[0055] 11. When all users have finished eating, they input the "Dinner finished" information into their terminals. The terminals then send this information to the server.

[0056] 12. The server confirms that all users have finished and ends the session.

[0057] Specific examples

[0058] Initial Setup

[0059] When user A and user B simultaneously input "start meal" into the terminal, the terminal sends the start information to the server.

[0060] The server initiates the session and records information for each user.

[0061] Monitoring meal progress

[0062] User A enters "I ate three bites of rice" into the device. The device sends this information to the server.

[0063] The server analyzes the received data and calculates User A's eating speed.

[0064] Data analysis

[0065] The server compares the data of all users and determines that User A eats quickly.

[0066] Question generation and presentation

[0067] The server determines that user A likes to travel and generates the question, "Where have you traveled recently?"

[0068] The server sends the question to User A's terminal.

[0069] Answering questions and analyzing

[0070] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[0071] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[0072] The server sends the following question to User A's terminal:

[0073] Adjusting meal timing

[0074] The server continues asking questions until User A's eating speed is in balance with the other users.

[0075] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[0076] The server confirms that all users have finished and ends the session.

[0077] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversations, thereby providing a satisfying dining experience for everyone.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] Before starting a meal, the user inputs "Start meal" into the terminal, which then sends basic information such as the start time, user ID, and meal contents to the server.

[0081] Step 2:

[0082] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[0083] Step 3:

[0084] As the user continues eating, they periodically input the amount eaten and their progress into the terminal.

[0085] Step 4:

[0086] The terminal transmits the user's input data to the server at regular intervals.

[0087] Step 5:

[0088] The server analyzes the received data and calculates each user's eating speed.

[0089] Step 6:

[0090] The server compares the eating speeds of all users, and if a particular user's eating speed is significantly faster or slower than other users, the abnormal eating speed is detected.

[0091] Step 7:

[0092] The server refers to past conversations and user profiles to generate questions suited to users with abnormal eating speeds.

[0093] Step 8:

[0094] The server generates a query and sends it to the user's terminal.

[0095] Step 9:

[0096] The terminal displays the received question to the user.

[0097] Step 10:

[0098] The user answers the displayed questions and enters them into the terminal.

[0099] Step 11:

[0100] The terminal sends the user's answer to the server.

[0101] Step 12:

[0102] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0103] Step 13:

[0104] The server generates additional questions and comments and sends them to the user's device.

[0105] Step 14:

[0106] The device will then prompt the user again with any additional questions or comments.

[0107] Step 15:

[0108] The server monitors the eating speed of all users in real time and adjusts questions and comments to align the eating timing of all users.

[0109] Step 16:

[0110] When all users input "Done eating," the terminal sends the information to the server.

[0111] Step 17:

[0112] The server confirms that all users have finished and saves the session data to the database.

[0113] Step 18:

[0114] The server sends a "meal end" notification to all terminals and ends the session.

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] When multiple users eat together, variations in the eating speeds of each user can disrupt the rhythm of conversation, or some users may eat too fast or too slow. This makes it difficult to provide a dining experience that is satisfying for everyone. To solve this problem, the present invention aims to balance the eating speeds of users and provide a dining experience that is satisfying for everyone through appropriate conversation.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating questions using a generative AI model and sending the questions to the user's terminal, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, and means for adjusting the timing to keep the eating speeds of all users balanced. This makes it possible to monitor each user's eating speed in real time, generate appropriate questions to promote conversation, and keep everyone's eating speeds balanced.

[0120] "User Information" refers to personal attributes and profile information, such as a user's name, interests, and concerns.

[0121] "Meal progress" refers to information indicating the amount of food the user is currently eating and the progress of the meal.

[0122] "Meal speed" refers to the speed at which a meal progresses, calculated based on the amount of food eaten by a user within a certain period of time.

[0123] "Abnormal eating speed" refers to an eating speed that is significantly faster or slower than the average eating speed of other users.

[0124] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate questions that are appropriate for the user.

[0125] "Question" refers to an inquiry or query presented to a user to facilitate relevant conversation.

[0126] An "answer" refers to a reply or response that a user enters to a question.

[0127] "Adjusting the timing" refers to generating questions and continuing the conversation at the right moment to balance the eating speed of all users.

[0128] The present invention relates to a system for balancing the eating speeds of all users when a group of users is eating together. How to implement this system will be described in detail below.

[0129] System Configuration

[0130] This system consists of a device used by the user, a server that processes the data, and an algorithm that uses a generative AI model to generate appropriate questions. The device is a computing device such as a smartphone or tablet, and the server is often installed on the cloud.

[0131] Hardware and software used

[0132] Terminal: A device that a user operates, such as a smartphone, tablet, or PC.

[0133] Server: Hardware for processing and storing data, such as a cloud server or on-premise server.

[0134] Generative AI models: Questions are generated using artificial intelligence models such as the GPT series from OpenAI®.

[0135] Data Processing and Data Calculation

[0136] 1. Data entered by the user:

[0137] Users input information about the start, progress, and end of their meal into the terminal, which allows the server to analyze each user's eating speed.

[0138] 2. Data transmission and analysis:

[0139] The device periodically sends the user's input data to the server, which analyzes the received data and calculates each user's eating speed.

[0140] 3. Detecting abnormal eating speed:

[0141] The server compares the eating speeds of all users and detects an abnormality when a particular user's eating speed is significantly faster or slower.

[0142] 4. Question generation:

[0143] If an anomaly is detected, the server generates appropriate questions using a generative AI model based on past conversations and the user profile, such as OpenAI's GPT series.

[0144] 5. Presenting and answering questions to the user:

[0145] The server generates questions and sends them to the user's device, which then presents them to the user. The user answers the questions, and the answers are sent back to the server, allowing the server to generate further questions and continue the conversation.

[0146] Specific examples

[0147] Example of question generation

[0148] By inputting the following prompt sentence into the generative AI model, a question appropriate for the user can be generated.

[0149] Based on the following user information and conversation content, please generate the following questions for User A.

[0150] User Profile:

[0151] Name: User A

[0152] Interests: Travel, music, cooking

[0153] Recent conversations:

[0154] User A: "I went to Hokkaido last month."

[0155] Based on this prompt, the generative AI model generates a question such as, "What kind of food was delicious in Hokkaido?" This question is sent from the server to the user's device, prompting the user to respond.

[0156] The system monitors the eating speed of users and encourages conversation at appropriate times to provide a satisfying dining experience for everyone. Furthermore, by using a generative AI model, it is possible to dynamically generate questions suited to each user, ensuring smooth conversation.

[0157] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0158] Step 1:

[0159] Before starting a meal, the user enters the "start meal" information into the device. Specifically, the user taps the "start" button on the smartphone or tablet app. This input data includes the user ID and start time. The device then sends this data to the server.

[0160] Step 2:

[0161] The server starts each user's session based on the "start meal" information it receives. At this point, the server records data such as the user ID, start time, and session ID in a database. This makes it possible to track which user started eating and when.

[0162] Step 3:

[0163] While eating, the user periodically inputs their progress into the device. A specific action would be to enter something like "I ate three bites of rice" into an input form within the app. This input data includes the user ID, progress, and time of input. The device then sends this data to the server at regular intervals.

[0164] Step 4:

[0165] The server analyzes the received data and calculates each user's eating speed. Here, the server calculates the eating speed for each user and records the results in the database. For example, if user A eats 3 bites in 10 minutes, their eating speed is "10 minutes / 3 bites."

[0166] Step 5:

[0167] The server compares the eating speeds of all users and detects abnormal eating speeds. In this detection process, the server calculates the average eating speed of all users and determines that speeds that are significantly faster or slower than that are abnormal. Specifically, if user B's eating speed is significantly faster than the average, it is detected as abnormal.

[0168] Step 6:

[0169] The server uses the generative AI model to generate questions appropriate for users with abnormal eating speeds. The input data for the prompt sentences used for generation includes the user profile (e.g., interests and past conversations) and user ID. For example, for a user who likes to travel, the question generated is "Where have you traveled recently?"

[0170] Step 7:

[0171] The server sends the generated question to the user's device. The sent data includes the question content, user ID, and time of sending. The device then presents the received question to the user.

[0172] Step 8:

[0173] The user answers the questions through the device. Specifically, the user enters the answer into the input form on the device and taps the "Send" button. This input data includes the answer, user ID, and time of input. The device then sends the answer to the server.

[0174] Step 9:

[0175] The server analyzes the user's answers and generates additional questions as necessary. Here, the server analyzes the answers and generates questions to encourage further conversation. For example, the next question generated might be, "What kind of food was delicious in Hokkaido?" The generated question is then returned to step 7 and sent again.

[0176] Step 10:

[0177] All users enter their end of meal time into their device, which then sends this information to the server. Specifically, users tap the "End" button in the app. The data sent includes the user ID and end time.

[0178] Step 11:

[0179] The server confirms the end information of all users and ends the session. At this time, all users' meal data and conversation content are saved in a database and recorded for future analysis and improvement.

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] In conventional online dining experiences, users often eat at different speeds, resulting in a lack of unity between them. Communication can also be disrupted, leading to uneven mealtime progress. Furthermore, sharing and coordinating real-time meal plans in a virtual environment is difficult.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and user profiles, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for generating conversations based on user interests and concerns using an AI model, and means for monitoring and sharing users' eating experiences in real time in a virtual environment, thereby enabling adjustment of different eating speeds and smooth communication.

[0185] The "means for inputting user information" refers to an interface that allows a user to input their own information into the system, and can include basic information such as name, age, and hobbies.

[0186] The "means for periodically recording the progress of a meal" is a means for recording the current progress of a meal at time intervals as the user eats, and for collecting data such as the amount and speed of eating.

[0187] The "means for analyzing eating speed and detecting abnormal eating speed" analyzes the progress data entered by the user and detects eating speeds that are too fast or too slow compared to other users based on preset criteria.

[0188] The "means for generating appropriate questions based on past conversation content and user profile" is a means for generating appropriate questions to interest the user based on previous conversations and the user's registration information.

[0189] The "means for presenting the generated question to the user and receiving an answer" is a means for displaying the generated question on the user's device and receiving an answer from the user.

[0190] The "means for analyzing the received answers and generating further questions" is a means for analyzing the answers from the user and generating new questions based on them.

[0191] The "means for adjusting timing to balance the eating speeds of all users" is a means for monitoring the eating progress of all users in real time and adjusting each user's eating speed to keep it balanced.

[0192] "Means for generating conversations based on user interests and concerns using an AI model" refers to using a generative AI model to automatically generate relevant conversations based on the interests and concerns derived from a user's profile and past comments.

[0193] "Means for monitoring and sharing users' meal experiences in real time in a virtual environment" refers to a method for monitoring a user's meal progress in real time through a virtual environment and sharing that information with other users, thereby creating a sense of unity.

[0194] The present invention provides a system for promoting enjoyable communication while balancing the eating speeds of all users when multiple users share a meal in a virtual environment. Specific embodiments of the system are described below.

[0195] Hardware and software used

[0196] Hardware: Smartphone, Head-Mounted Display (HMD)

[0197] Software: Servers for real-time data analysis, AI models for chatbots (e.g., OpenAI GPT-4 (registered trademark)), database management systems (e.g., MySQL (registered trademark))

[0198] System Overview

[0199] The system includes the following means:

[0200] 1. Means for inputting user information: Before starting a meal, each user inputs their information (name, age, hobbies, etc.) into the system via their smartphone or HMD. This information is sent to the server, and an individual session begins.

[0201] 2. A means to periodically record the progress of the meal: During the meal, the user inputs their progress (e.g., the amount eaten) via their smartphone or HMD. This information is sent to the server at regular intervals.

[0202] 3. Means for analyzing eating speed and detecting abnormal eating speed: The server analyzes the received data in real time, calculates each user's eating speed, and detects abnormally fast or slow eating speeds based on set criteria.

[0203] 4. Generating appropriate questions based on past conversations and user profiles: Using an AI model (e.g., GPT-4), appropriate questions are generated based on the user's profile and past conversations. For example, for a user who likes traveling, the system generates the question, "Where have you traveled recently?"

[0204] 5. A means for presenting the generated questions to the user and receiving answers: The generated questions are presented in real time to the user's smartphone or HMD, and the user answers the questions. The answers are then sent back to the server.

[0205] 6. Means for analyzing the received answers and generating further questions: The server analyzes the user's answers using an AI model and generates further questions to continue the conversation, thereby allowing the conversation to continue uninterrupted.

[0206] 7. A means for adjusting timing to balance the eating speed of all users: The server monitors the eating speed of all users and generates questions and comments as appropriate to keep the balance.

[0207] 8. Using an AI model to generate conversations based on the user's interests: Using a generative AI model, a conversation can be conducted based on the user's interests. For example, if a user enjoys reading, the conversation can be asked questions such as, "What books have you read recently?"

[0208] 9. A means of monitoring and sharing users' meal experiences in a virtual environment in real time: The system monitors and shares users' meal progress in real time, allowing everyone to enjoy the same experience.

[0209] Specific examples

[0210] Example 1:

[0211] Prompt: "Tell me about your most recent travel destination."

[0212] User A: "I went to Hokkaido last month."

[0213] Post-analysis question: "What kind of food did you like in Hokkaido?"

[0214] In this way, the AI ​​model is used to generate conversations based on the user's interests, allowing for adjustment of eating speed and smooth communication.

[0215] Example 2:

[0216] Prompt: "Tell me about the last book you read."

[0217] User B: "I was reading a mystery novel."

[0218] Post-analysis questions: "What was the title of the mystery novel? What part did you like?"

[0219] Specific prompts for the AI ​​model:

[0220] The user has started their meal progress. To encourage conversation with other users and adjust the eating speed, generate appropriate questions from the following input data. The user's recent interests are "travel" and "reading." Example: "Tell me about the places you've traveled recently and the books you've read recently."

[0221] In this way, the dining experience in a virtual environment can be tailored to be more enjoyable.

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] Before a user starts eating, they use their smartphone or HMD to enter user information (such as name, age, and hobbies) into the system. This information is sent from the device to the server. The server records the received user information in a database and starts the session. Input data: User information (such as name, age, and hobbies). Output data: User information recorded in the database and a session start notification.

[0225] Step 2:

[0226] While eating, the user periodically inputs their progress (e.g., amount eaten) into the device. The input data is sent from the device to the server at regular intervals. The server records the received progress data in a database and prepares it for analysis in real time. Input data: Progress (e.g., amount eaten). Output data: Progress data recorded in the database in preparation for analysis.

[0227] Step 3:

[0228] The server analyzes the received progress data in real time and calculates each user's eating speed. As a result of the analysis, it detects users whose eating speed is abnormally fast or slow compared to the set criteria. Input data: progress data. Output data: detection results of each user's eating speed and abnormal speed.

[0229] Step 4:

[0230] The server uses a generative AI model (e.g., GPT-4) to generate appropriate questions for users with abnormal eating speeds based on past conversations and user profiles. Input data: past conversations, user profiles. Output data: generated questions.

[0231] Step 5:

[0232] The generated questions are presented from the server to the user's smartphone or HMD. The user answers the presented questions. Input data: Generated questions. Output data: User answers.

[0233] Step 6:

[0234] The server analyzes the answers received from the user using an AI model and generates further questions, allowing the conversation to continue uninterrupted. Input data: User answers. Output data: Additional questions.

[0235] Step 7:

[0236] The server monitors the eating speed of all users in real time and generates questions and comments as needed to maintain balance. Input data: Progress and eating speed data of all users. Output data: Additional questions and comments for adjustment.

[0237] Step 8:

[0238] The server uses a generative AI model to engage in conversations based on the user's interests. For example, if a user has a hobby of reading, it will ask questions such as, "What books have you read recently?" Input data: User profile, past conversations. Output data: Conversation content based on interests.

[0239] Step 9:

[0240] The system monitors and shares the meal progress of users in real time, allowing everyone to enjoy the same experience and fostering a sense of unity. Input data: Progress data of all users. Output data: Shared progress data and real-time sense of unity.

[0241] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0242] System Overview

[0243] This invention is a system for balancing the eating speeds of all users when dining together, and provides a richer conversation experience by combining it with an emotion engine that recognizes users' emotions. This system works by having users record their meal progress and emotion data using their own devices and send it to a server. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[0244] Initial Setup

[0245] 1. Before starting a meal, the user inputs the "start eating" information into the device. The device then sends the start time, user ID, meal contents, past conversation log, and emotion data to the server.

[0246] 2. Based on the information received, the server records each user's session in the database and starts the session.

[0247] Monitoring and sending meal progress and emotion data

[0248] 3. While eating, the user periodically inputs the amount eaten, progress, and emotional data into the device. Emotional data can be automatically acquired using emotion engines such as voice analysis and facial expression analysis.

[0249] 4. The device sends the user's input data and emotion data to the server at regular intervals.

[0250] Data Analysis and Anomaly Detection

[0251] 5. The server analyzes the received data and calculates each user's eating speed and emotional state.

[0252] 6. The server compares the eating speeds of all users and detects abnormal eating speeds of specific users if they are significantly faster or slower. In addition, it analyzes the user's emotional state and obtains emotional indicators such as stress and relaxation.

[0253] Question generation and presentation

[0254] 7. The server generates questions appropriate for users with abnormal eating speed based on past conversations, user profiles, and emotional data. For example, for a user who loves traveling and is feeling stressed, the server generates a question such as, "Where was your most recent relaxing trip?"

[0255] 8. The server sends the generated question to the user's device.

[0256] Answering questions and analyzing

[0257] 9. The device presents the received question to the user, who answers it. The answer is sent to the server via the device.

[0258] 10. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0259] Adjusting meal timing

[0260] 11. The server monitors the eating speed and emotional data of all users in real time and continues to ask questions as appropriate to adjust everyone's eating timing and emotional state.

[0261] 12. When all users have finished eating, they enter the "Drinking is over" information into their terminals. The terminals then send this information to the server.

[0262] 13. The server confirms that all users have exited and saves the session data to the database.

[0263] Specific examples

[0264] Initial Setup

[0265] When user A and user B simultaneously input "start eating" into the terminal, the terminal sends the start information and emotion data to the server.

[0266] The server initiates the session and records information for each user.

[0267] Monitoring meal progress and emotional data

[0268] User A inputs "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[0269] The server analyzes the received data and calculates User A's eating speed and emotional state.

[0270] Data analysis

[0271] The server compares the data of all users and determines that User A eats quickly and is feeling stressed.

[0272] Question generation and presentation

[0273] The server determines that User A likes to travel and generates the question, "Where is the most relaxing travel destination you have recently visited?"

[0274] The server sends the question to User A's terminal.

[0275] Answering questions and analyzing

[0276] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[0277] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[0278] The server sends the following question to User A's terminal:

[0279] Meal timing and emotion regulation

[0280] The server continues asking questions until User A's eating speed and emotions are in balance with other users.

[0281] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[0282] The server confirms that all users have finished and ends the session.

[0283] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[0284] The processing flow will be explained below.

[0285] Step 1:

[0286] Before starting a meal, the user enters "Start Meal" into the terminal. The user then enters basic information such as ID and meal contents. The terminal then sends this user information along with the start time to the server.

[0287] Step 2:

[0288] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[0289] Step 3:

[0290] As the user eats, they periodically input information about the amount eaten and their progress into the device. Emotional data is also collected at the same time, using an emotion engine that analyzes voice and facial expressions.

[0291] Step 4:

[0292] The terminal transmits the user's input data and emotion data to the server at regular intervals.

[0293] Step 5:

[0294] The server analyzes the received data and calculates each user's eating speed and emotional state.

[0295] Step 6:

[0296] The server compares the eating speeds of all users and detects abnormal eating speeds. If an abnormal eating speed is detected, the server also checks the emotional state of the user.

[0297] Step 7:

[0298] The server generates appropriate questions based on past conversations, user profiles, and emotional data, such as "What are some of the most relaxing travel destinations you've visited recently?" for a stressed travel-loving user.

[0299] Step 8:

[0300] The server generates a query and sends it to the user's terminal.

[0301] Step 9:

[0302] The terminal displays the received question to the user.

[0303] Step 10:

[0304] The user answers the displayed questions and enters them into the terminal.

[0305] Step 11:

[0306] The terminal sends the user's answer to the server.

[0307] Step 12:

[0308] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0309] Step 13:

[0310] The server generates additional questions and comments and sends them to the user's device.

[0311] Step 14:

[0312] The device will then prompt the user again with any additional questions or comments.

[0313] Step 15:

[0314] The server monitors all users' eating speed and emotional data in real time, and adjusts questions and comments to align the eating timing and emotional state of all users.

[0315] Step 16:

[0316] When all users input "Done eating," the terminal sends the information to the server.

[0317] Step 17:

[0318] The server confirms that all users have finished and saves the session data to the database.

[0319] Step 18:

[0320] The server sends a "meal end" notification to all terminals and ends the session.

[0321] Example 2

[0322] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] In today's world, when multiple users eat together, differences in eating speed can create imbalances, making it difficult for everyone to have a satisfying dining experience. Furthermore, systems lack the ability to generate questions at the right time to facilitate smooth conversation, or to communicate in a way that takes users' emotions into account. There is a need to solve these problems and provide a richer shared dining experience.

[0324] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and a user profile, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for acquiring emotional data using voice analysis or facial expression analysis, means for adjusting questions based on the acquired emotional data, and means for transmitting, receiving, and analyzing data in real time. This makes it possible to generate questions that take emotions into consideration while maintaining a balance in eating speeds among users, thereby enriching the conversation experience.

[0325] "User information" refers to information including the user ID, start time, meal contents, past conversation log, and emotion data.

[0326] "Meal progress" refers to data about the amount eaten and progress recorded by the user during the meal.

[0327] "Eating speed" is an index that measures and represents the speed at which a user eats.

[0328] An "abnormal eating speed" is an eating speed that is judged to be significantly fast or slow when compared with the eating speeds of all users.

[0329] "Past conversation content" refers to the history of interactions between users recorded in previous conversation sessions.

[0330] A "user profile" refers to personal attribute data such as a user's interests, concerns, and past behavioral history.

[0331] "Emotion data" refers to data that indicates the emotional state of the user, obtained based on voice analysis and facial expression analysis.

[0332] A "question" refers to a dialogue prompt that is generated based on the user's interests and current emotional state.

[0333] An "answer" refers to a response that a user enters in response to a question.

[0334] "Means for transmitting, receiving, and analyzing data in real time" refers to technical means for instantly transmitting and receiving data using communication technology and for quickly analyzing it.

[0335] This system balances the eating speeds of all users when dining together, and provides a rich conversational experience by using an emotion engine. This system works by having users record their meal progress and emotion data using their own devices and then sending this data to a server.

[0336] Initial Setup

[0337] 1. User: Before starting a meal, the user inputs the "start meal" information into their device. The input information includes the start time, user ID, meal contents, past conversation log, and emotion data.

[0338] 2. Terminal: The terminal collects the input information and sends it to the server, using an Internet communication protocol such as HTTP.

[0339] 3. Server: Based on the received information, the server records each user's session in a database (e.g., MySQL) and starts the session.

[0340] Monitoring meal progress and emotional data

[0341] 1. User: While eating, the user inputs their meal progress and emotional data into the device. Emotional data is automatically acquired using the Google® Cloud Speech-to-Text API and the Microsoft® Azure® Emotion API.

[0342] 2. Terminal: The terminal sends the user's input data and emotion data to the server at regular intervals (e.g., every 5 minutes).

[0343] Data Analysis and Anomaly Detection

[0344] 1. Server: The server analyzes the received data using Python's pandas library and Scikit-learn to quantify eating speed and emotional state.

[0345] 2. Server: Compares the data of all users, identifies users with abnormal eating speeds (e.g., significantly faster or slower), and analyzes their emotional state.

[0346] Question generation and presentation

[0347] 1. Server: Generates questions appropriate for users with abnormal eating speeds based on past conversations, user profiles, and emotional data. For example, if the generative AI model determines that a user loves traveling and is feeling stressed, it generates a question such as, "Where was the most relaxing trip you've taken recently?"

[0348] 2. Server: Sends the generated question to the user's device.

[0349] Answering questions and analyzing

[0350] 1. Terminal: The terminal presents the received question to the user. The user answers the question and enters it into the terminal. For example, the answer might be, "I went to Hokkaido last month."

[0351] 2. Terminal: The terminal sends the answer to the server.

[0352] 3. Server: The server analyzes the user's answers and generates additional questions or comments to continue the conversation, such as "What kind of food did you like in Hokkaido?"

[0353] Adjusting meal timing

[0354] 1. Server: The server monitors the eating speed and emotional data of all users in real time and adjusts everyone's eating timing and emotional state by asking questions appropriately.

[0355] 2. All users: When finishing a meal, they input the "Done" information into their terminal. The terminal sends this information to the server.

[0356] 3. Server: The server checks the completion of all users and stores the session data in a database, for example, recording the completion time and final emotion data.

[0357] Specific examples

[0358] 1. User A and User B: Simultaneously input "start eating" into their devices, and the devices send the start information and emotion data to the server.

[0359] 2. Server: Initiates sessions and records information about each user.

[0360] 3. User A: Enters "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[0361] 4. Server: Analyzes the received data and calculates User A's eating speed and emotional state.

[0362] 5. Server: Compares the data of all users and determines that User A eats quickly and is stressed.

[0363] 6. Server: Determines that User A loves to travel and generates the question, "Where was your last relaxing trip?"

[0364] 7. Server: Sends the question to User A's device.

[0365] 8. User A: "I went to Hokkaido last month." The device sends the answer to the server.

[0366] 9. Server: Analyzes the answer and generates the next question: "What kind of food was delicious in Hokkaido?". Sends the next question to User A's device.

[0367] 10. Server: Continue asking questions until User A's eating speed and emotions are balanced with the other users.

[0368] 11. All users: When you enter "Done eating" into the terminal, the terminal will send that information to the server.

[0369] 12. Server: Confirm that all users have finished and end the session.

[0370] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[0371] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0372] Step 1:

[0373] Before starting a meal, the user enters the "start meal" information into the device. At that time, the user enters the user ID, start time, meal contents, past conversation log, and emotion data. Specifically, the data is entered into an input form using a smartphone app or tablet. The entered data is temporarily stored in the device's memory.

[0374] Input: User ID, start time, meal contents, past conversation log, emotion data

[0375] Output: User data in device memory

[0376] Step 2:

[0377] The device collects the entered user information and sends it to the server. The HTTP protocol is used for transmission, and the data is packetized in JSON format and sent. The transmitted data is received by the server-side API and stored in a database.

[0378] Input: User data in the terminal memory

[0379] Output: User information stored in a database on the server

[0380] Step 3:

[0381] The server starts a session for each user based on the received information. The received data (user ID, start time, meal contents, etc.) is recorded in a database, and the session start time is saved as a timestamp.

[0382] Input: User information on the server

[0383] Output: Session data recorded in a database

[0384] Step 4:

[0385] The user inputs the amount of food eaten and their progress during the meal into the device. Emotion data is also automatically acquired using an emotion engine. For example, the user's voice is analyzed using the Google Cloud Speech-to-Text API to evaluate their stress level.

[0386] Input: User's food intake, progress, and voice data

[0387] Output: New dataset in device (progress and emotion ratings)

[0388] Step 5:

[0389] The device sends the user's progress and emotion data to the server at regular intervals (for example, every 5 minutes). This communication also uses the HTTP protocol, and the data is sent to the server in JSON format. The server then formats the received data appropriately and adds it to the database.

[0390] Input: New progress and emotion data

[0391] Output: Updated user database on the server.

[0392] Step 6:

[0393] The server analyzes the received data using Python's pandas library and Scikit-learn. Specifically, it calculates eating speed based on the amount of food eaten and progress, and runs algorithms to evaluate stress levels and relaxation state based on emotional data.

[0394] Input: Updated user database

[0395] Output: Eating speed and emotional state of each user

[0396] Step 7:

[0397] The server compares the data of all users and detects abnormal eating speeds. For example, it identifies users whose eating speeds deviate significantly from the average speed and also checks their emotional state. If an abnormality is detected, it flags the data as abnormal.

[0398] Input: Eating speed and emotional state of each user

[0399] Output: A list of users for whom anomalies were detected

[0400] Step 8:

[0401] The server generates appropriate questions for users with abnormal eating speeds. Using past conversations, user profiles, and emotional data, the generative AI model inputs prompt sentences to generate specific questions. For example, it generates a question like, "Where was your most recent relaxing trip?"

[0402] Input: A list of users for whom anomalies were detected, and a prompt for the AI ​​model

[0403] Output: Generated questions

[0404] Step 9:

[0405] The server sends the generated question to the user's device. The server uses the HTTP protocol to send the generated question in JSON format to the device. The device receives it and displays it on the user interface.

[0406] Input: Generated Question

[0407] Output: The question displayed on the user's terminal

[0408] Step 10:

[0409] The user inputs an answer to the question displayed on the device. For example, the answer might be "I went to Hokkaido last month." The answer is input using the keyboard on the smartphone or tablet. The answer data is saved in the device's memory.

[0410] Input: User's answer

[0411] Output: Answer data stored in the device's memory

[0412] Step 11:

[0413] The device sends the response data to the server using the HTTP protocol, packetizing the response data in JSON format and sending it to the server. The server stores the received response data in a database and analyzes it.

[0414] Input: Answer data stored in the device's memory

[0415] Output: Answer data saved on the server

[0416] Step 12:

[0417] The server analyzes the received response data, for example by applying a text analysis algorithm to generate the next question from what the user has said about their travel destination, and then uses a generative AI model to generate the next question.

[0418] Input: Answer data saved on the server

[0419] Output: The following questions are generated:

[0420] Step 13:

[0421] The server generates the next question and sends it to the user's device. The server then packets the question in JSON format and sends it using the HTTP protocol. The device receives it and displays it on the user interface.

[0422] Input: The next generated question

[0423] Output: The following question displayed on the user's screen:

[0424] Step 14:

[0425] The server monitors all users' eating speed and emotional data in real time and continues to ask questions accordingly, forming a feedback loop to adjust the eating timing and emotional state of all users.

[0426] Input: Eating speed and emotion data of all users

[0427] Output: Feedback loop of adjusted questions

[0428] Step 15:

[0429] When all users have finished eating, they input "meal finished" information into their terminal. The terminal then sends the end information to the server. The server updates the session data based on the received end information and confirms that all users have finished.

[0430] Input: User exit information

[0431] Output: Finished data saved on the server

[0432] Step 16:

[0433] The server confirms the end of each user session and stores the session data in a database, including the end time and final emotion data.

[0434] Input: Finished data sent to the server

[0435] Output: Session termination data stored in the database

[0436] (Application example 2)

[0437] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0438] In today's world, when multiple users eat together, imbalances in eating speed can cause some users to feel isolated or conversations to be interrupted. This problem significantly reduces the enjoyment and satisfaction of the meal and requires special consideration. Furthermore, because emotional fluctuations often affect the pace of eating, a system that takes this into account is needed. However, current systems do not provide a system that can reflect users' emotions in real time and generate appropriate conversations.

[0439] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0440] In this invention, the server includes a means for periodically recording the progress of the meal using a smart device, a means for acquiring the user's emotion data using an emotion engine, and a means for generating questions using a generative AI model, which enables the server to monitor the user's eating speed and emotions in real time and balance the eating speed and conversation by asking appropriate questions.

[0441] A "smart device" refers to a device that has internet connectivity and is equipped with various sensors and displays.

[0442] An "emotion engine" refers to a software or hardware system that analyzes a user's facial expressions and voice and recognizes their psychological state and emotions.

[0443] A "generative AI model" refers to a system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[0444] A "server" refers to a computer system that centrally manages, analyzes, and processes data sent from multiple users.

[0445] "Means for generating questions" refers to algorithms or programs for automatically generating appropriate questions based on past conversation content and user profiles.

[0446] "Eating speed" refers to the progress of a user's eating within a specific period of time.

[0447] "Means for analyzing eating speed" refers to a program or algorithm that evaluates and analyzes a user's eating speed based on collected data.

[0448] This invention is a system that uses smart devices to record and transmit users' meal progress and emotional data, and a server analyzes this data and generates appropriate questions to adjust the balance between eating speed and conversation for all users. This system enriches the dining experience among users and is expected to be used particularly in brick-and-mortar establishments such as restaurants.

[0449] The system configuration includes the following hardware and software:

[0450] Hardware

[0451] Smart device: A device with internet connectivity and equipped with various sensors and displays. Here, we use smart glasses as an example.

[0452] Sensors: To keep track of your eating progress, sensors built into the smart glasses measure the amount of food you eat.

[0453] Server: A computer system that centrally manages, analyzes, and processes data sent from multiple users.

[0454] software

[0455] Emotion engine: An engine for analyzing the user's facial expressions and voice to obtain emotional data.

[0456] Generative AI model: A system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[0457] ServerCommunicator: A module that is responsible for communicating with the server and sending and receiving data.

[0458] QuestionGenerator: A module that generates appropriate questions based on emotional data, user profiles, and conversation history.

[0459] SmartGlass® esDisplay: Software capable of displaying questions on the screen of smart glasses.

[0460] System Operation

[0461] In this system, the user puts on the smart glasses before starting a meal and inputs the "start eating" information. The smart glasses then send the start time, user ID, meal content, past conversation log, and emotional data to the server. The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. This data is then sent to the server at regular intervals, where it is analyzed and used to monitor eating speed and emotional state.

[0462] The server then compares each user's eating speed, and if an abnormal eating speed is detected, the AI ​​model generates appropriate questions based on the user's profile and emotional data. The generated questions are displayed on the smart glasses' display, and the user answers them. The answers are then sent back to the server, which generates additional questions to further encourage conversation.

[0463] Specific examples

[0464] For example, if the user is recognized as feeling stressed, the following example prompt sentences are fed into the generative AI model:

[0465] A user is feeling stressed. Generate a question for User A, who loves to travel: "Where was the most relaxing trip you've taken recently?"

[0466] The generated question looks like this:

[0467] "Where's the last relaxing trip you went to?"

[0468] This question is displayed on the smart glasses display, and the user responds, "I went to Hokkaido last month." This response is sent to the server, which then generates the next question, such as, "What kind of food did you like in Hokkaido?" Through this process, the eating speed and emotional state of all users are balanced.

[0469] This will enrich the dining experience between users and allow for mealtimes and conversations that are satisfying for everyone.

[0470] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0471] Step 1:

[0472] Before starting a meal, the user puts on the smart glasses and inputs the "start eating" information. The user operates the interface of the smart glasses to input the start time, user ID, and meal contents. This information is sent to the server along with the start trigger.

[0473] Input: Start time, User ID, Meal details

[0474] Output: Start trigger and user information sent to the server

[0475] Step 2:

[0476] The server starts a session based on the received start information, user ID, and meal details. A session ID is generated and recorded in the database. Each user's progress is managed based on this session ID.

[0477] Input: Start trigger and user information

[0478] Output: Generate a session ID and record it in the database

[0479] Step 3:

[0480] The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. The camera and microphone built into the smart glasses capture facial expressions and voice, which are then sent to the emotion engine for analysis.

[0481] Input: User facial expression images, voice data

[0482] Output: Obtaining emotion data

[0483] Step 4:

[0484] Sensors in the smart glasses periodically measure the amount of food consumed and send the data to a server. For example, a weight sensor measures the amount of food consumed and sends it as progress data.

[0485] Input: Food quantity data

[0486] Output: Getting and sending progress data

[0487] Step 5:

[0488] The server analyzes the received emotion data and progress data to calculate the user's eating speed. It also compares the eating speeds of multiple users to check for abnormal speeds.

[0489] Input: Emotion data, progress data

[0490] Output: Detection results of eating speed and abnormal speed

[0491] Step 6:

[0492] If the server detects abnormal eating speed, it uses a generative AI model to generate appropriate questions based on the user's profile and emotional data. The generative AI model generates questions based on the prompt text, and the server sends the questions to the smart glasses.

[0493] Input: User profile, emotion data, prompt sentence

[0494] Output: Generated questions

[0495] Step 7:

[0496] A question is displayed on the smart glasses display, and the user responds to the question by voice or text, which is then sent back to the server.

[0497] Input: Generated Question

[0498] Output: User's answer

[0499] Step 8:

[0500] The server analyzes the answers it receives and generates further questions as needed, repeating this process until all users' eating speeds and conversations are balanced.

[0501] Input: User's answer

[0502] Output: Further generated questions

[0503] Step 9:

[0504] All users input the "meal end" information into their smart glasses. This information is sent to the server, which ends the session and saves all data in a database.

[0505] Input: Meal end information

[0506] Output: Save session data and end session

[0507] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0508] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0509] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0510] [Second embodiment]

[0511] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0512] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0513] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0514] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0515] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0516] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0517] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0518] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0519] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0520] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0521] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0522] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0523] System Overview

[0524] This invention is a system for balancing the eating speeds of all users when multiple users are eating together. The system operates by having users record and transmit their meal progress using their own devices. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[0525] Initial Setup

[0526] 1. Before starting a meal, the user inputs the "start eating" information into their device. The device then sends this information to the server.

[0527] 2. Based on the information received, the server records each user's session in the database and starts the session.

[0528] Monitoring and sending meal progress

[0529] 3. While eating, the user periodically inputs the amount eaten and progress into the device. The device then transmits this data to the server at regular intervals.

[0530] 4. The server analyzes the received data and calculates each user's eating speed.

[0531] Data Analysis and Anomaly Detection

[0532] 5. The server compares the eating speeds of all users and detects abnormal eating speeds of a particular user if the eating speed of that user is significantly faster or slower.

[0533] Question generation and presentation

[0534] 6. The server generates questions appropriate for a user with an abnormal eating speed based on past conversations and the user profile. For example, for a user who likes traveling, it generates a question such as "Where have you traveled recently?"

[0535] 7. The server sends the generated question to the user's device.

[0536] Answering questions and analyzing

[0537] 8. The device presents the received question to the user, who answers it. The answer is then sent back to the server via the device.

[0538] 9. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0539] Adjusting meal timing

[0540] 10. The server monitors the eating speed of all users in real time and continues to ask questions as appropriate to align everyone's eating timing.

[0541] 11. When all users have finished eating, they input the "Dinner finished" information into their terminals. The terminals then send this information to the server.

[0542] 12. The server confirms that all users have finished and ends the session.

[0543] Specific examples

[0544] Initial Setup

[0545] When user A and user B simultaneously input "start meal" into the terminal, the terminal sends the start information to the server.

[0546] The server initiates the session and records information for each user.

[0547] Monitoring meal progress

[0548] User A enters "I ate three bites of rice" into the device. The device sends this information to the server.

[0549] The server analyzes the received data and calculates User A's eating speed.

[0550] Data analysis

[0551] The server compares the data of all users and determines that User A eats quickly.

[0552] Question generation and presentation

[0553] The server determines that user A likes to travel and generates the question, "Where have you traveled recently?"

[0554] The server sends the question to User A's terminal.

[0555] Answering questions and analyzing

[0556] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[0557] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[0558] The server sends the following question to User A's terminal:

[0559] Adjusting meal timing

[0560] The server continues asking questions until User A's eating speed is in balance with the other users.

[0561] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[0562] The server confirms that all users have finished and ends the session.

[0563] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversations, thereby providing a satisfying dining experience for everyone.

[0564] The processing flow will be explained below.

[0565] Step 1:

[0566] Before starting a meal, the user inputs "Start meal" into the terminal, which then sends basic information such as the start time, user ID, and meal contents to the server.

[0567] Step 2:

[0568] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[0569] Step 3:

[0570] As the user continues eating, they periodically input the amount eaten and their progress into the terminal.

[0571] Step 4:

[0572] The terminal transmits the user's input data to the server at regular intervals.

[0573] Step 5:

[0574] The server analyzes the received data and calculates each user's eating speed.

[0575] Step 6:

[0576] The server compares the eating speeds of all users, and if a particular user's eating speed is significantly faster or slower than other users, the abnormal eating speed is detected.

[0577] Step 7:

[0578] The server refers to past conversations and user profiles to generate questions suited to users with abnormal eating speeds.

[0579] Step 8:

[0580] The server generates a query and sends it to the user's terminal.

[0581] Step 9:

[0582] The terminal displays the received question to the user.

[0583] Step 10:

[0584] The user answers the displayed questions and enters them into the terminal.

[0585] Step 11:

[0586] The terminal sends the user's answer to the server.

[0587] Step 12:

[0588] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0589] Step 13:

[0590] The server generates additional questions and comments and sends them to the user's device.

[0591] Step 14:

[0592] The device will then prompt the user again with any additional questions or comments.

[0593] Step 15:

[0594] The server monitors the eating speed of all users in real time and adjusts questions and comments to align the eating timing of all users.

[0595] Step 16:

[0596] When all users input "Done eating," the terminal sends the information to the server.

[0597] Step 17:

[0598] The server confirms that all users have finished and saves the session data to the database.

[0599] Step 18:

[0600] The server sends a "meal end" notification to all terminals and ends the session.

[0601] Example 1

[0602] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0603] When multiple users eat together, variations in the eating speeds of each user can disrupt the rhythm of conversation, or some users may eat too fast or too slow. This makes it difficult to provide a dining experience that is satisfying for everyone. To solve this problem, the present invention aims to balance the eating speeds of users and provide a dining experience that is satisfying for everyone through appropriate conversation.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0605] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating questions using a generative AI model and sending the questions to the user's terminal, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, and means for adjusting the timing to keep the eating speeds of all users balanced. This makes it possible to monitor each user's eating speed in real time, generate appropriate questions to promote conversation, and keep everyone's eating speeds balanced.

[0606] "User Information" refers to personal attributes and profile information, such as a user's name, interests, and concerns.

[0607] "Meal progress" refers to information indicating the amount of food the user is currently eating and the progress of the meal.

[0608] "Meal speed" refers to the speed at which a meal progresses, calculated based on the amount of food eaten by a user within a certain period of time.

[0609] "Abnormal eating speed" refers to an eating speed that is significantly faster or slower than the average eating speed of other users.

[0610] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate questions that are appropriate for the user.

[0611] "Question" refers to an inquiry or query presented to a user to facilitate relevant conversation.

[0612] An "answer" refers to a reply or response that a user enters to a question.

[0613] "Adjusting the timing" refers to generating questions and continuing the conversation at the right moment to balance the eating speed of all users.

[0614] The present invention relates to a system for balancing the eating speeds of all users when a group of users is eating together. How to implement this system will be described in detail below.

[0615] System Configuration

[0616] This system consists of a device used by the user, a server that processes the data, and an algorithm that uses a generative AI model to generate appropriate questions. The device is a computing device such as a smartphone or tablet, and the server is often installed on the cloud.

[0617] Hardware and software used

[0618] Terminal: A device that a user operates, such as a smartphone, tablet, or PC.

[0619] Server: Hardware for processing and storing data, such as a cloud server or on-premise server.

[0620] Generative AI models: Generate questions using artificial intelligence models such as OpenAI's GPT series.

[0621] Data Processing and Data Arithmetic

[0622] 1. Data entered by the user:

[0623] Users input information about the start, progress, and end of their meal into the terminal, which allows the server to analyze each user's eating speed.

[0624] 2. Data transmission and analysis:

[0625] The device periodically sends the user's input data to the server, which analyzes the received data and calculates each user's eating speed.

[0626] 3. Detecting abnormal eating speed:

[0627] The server compares the eating speeds of all users and detects an abnormality when a particular user's eating speed is significantly faster or slower.

[0628] 4. Question generation:

[0629] If an anomaly is detected, the server generates appropriate questions using a generative AI model based on past conversations and the user profile, such as OpenAI's GPT series.

[0630] 5. Presenting and answering questions to the user:

[0631] The server generates questions and sends them to the user's device, which then presents them to the user. The user answers the questions, and the answers are sent back to the server, allowing the server to generate further questions and continue the conversation.

[0632] Specific examples

[0633] Example of question generation

[0634] By inputting the following prompt sentence into the generative AI model, a question appropriate for the user can be generated.

[0635] Based on the following user information and conversation content, please generate the following questions for User A.

[0636] User Profile:

[0637] Name: User A

[0638] Interests: Travel, music, cooking

[0639] Recent conversations:

[0640] User A: "I went to Hokkaido last month."

[0641] Based on this prompt, the generative AI model generates a question such as, "What kind of food was delicious in Hokkaido?" This question is sent from the server to the user's device, prompting the user to respond.

[0642] The system monitors the eating speed of users and encourages conversation at appropriate times to provide a satisfying dining experience for everyone. Furthermore, by using a generative AI model, it is possible to dynamically generate questions suited to each user, ensuring smooth conversation.

[0643] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0644] Step 1:

[0645] Before starting a meal, the user enters the "start meal" information into the device. Specifically, the user taps the "start" button on the smartphone or tablet app. This input data includes the user ID and start time. The device then sends this data to the server.

[0646] Step 2:

[0647] The server starts each user's session based on the "start meal" information it receives. At this point, the server records data such as the user ID, start time, and session ID in a database. This makes it possible to track which user started eating and when.

[0648] Step 3:

[0649] While eating, the user periodically inputs their progress into the device. A specific action would be to enter something like "I ate three bites of rice" into an input form within the app. This input data includes the user ID, progress, and time of input. The device then sends this data to the server at regular intervals.

[0650] Step 4:

[0651] The server analyzes the received data and calculates each user's eating speed. Here, the server calculates the eating speed for each user and records the results in the database. For example, if user A eats 3 bites in 10 minutes, their eating speed is "10 minutes / 3 bites."

[0652] Step 5:

[0653] The server compares the eating speeds of all users and detects abnormal eating speeds. In this detection process, the server calculates the average eating speed of all users and determines that speeds that are significantly faster or slower than that are abnormal. Specifically, if user B's eating speed is significantly faster than the average, it is detected as abnormal.

[0654] Step 6:

[0655] The server uses the generative AI model to generate questions appropriate for users with abnormal eating speeds. The input data for the prompt sentences used for generation includes the user profile (e.g., interests and past conversations) and user ID. For example, for a user who likes to travel, the question generated is "Where have you traveled recently?"

[0656] Step 7:

[0657] The server sends the generated question to the user's device. The sent data includes the question content, user ID, and time of sending. The device then presents the received question to the user.

[0658] Step 8:

[0659] The user answers the questions through the device. Specifically, the user enters the answer into the input form on the device and taps the "Send" button. This input data includes the answer, user ID, and time of input. The device then sends the answer to the server.

[0660] Step 9:

[0661] The server analyzes the user's answers and generates additional questions as necessary. Here, the server analyzes the answers and generates questions to encourage further conversation. For example, the next question generated might be, "What kind of food was delicious in Hokkaido?" The generated question is then returned to step 7 and sent again.

[0662] Step 10:

[0663] All users enter their end of meal time into their device, which then sends this information to the server. Specifically, users tap the "End" button in the app. The data sent includes the user ID and end time.

[0664] Step 11:

[0665] The server confirms the end information of all users and ends the session. At this time, all users' meal data and conversation content are saved in a database and recorded for future analysis and improvement.

[0666] (Application example 1)

[0667] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] In conventional online dining experiences, users often eat at different speeds, resulting in a lack of unity between them. Communication can also be disrupted, leading to uneven mealtime progress. Furthermore, sharing and coordinating real-time meal plans in a virtual environment is difficult.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0670] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and user profiles, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for generating conversations based on user interests and concerns using an AI model, and means for monitoring and sharing users' eating experiences in real time in a virtual environment, thereby enabling adjustment of different eating speeds and smooth communication.

[0671] The "means for inputting user information" refers to an interface that allows a user to input their own information into the system, and can include basic information such as name, age, and hobbies.

[0672] The "means for periodically recording the progress of a meal" is a means for recording the current progress of a meal at time intervals as the user eats, and for collecting data such as the amount and speed of eating.

[0673] The "means for analyzing eating speed and detecting abnormal eating speed" analyzes the progress data entered by the user and detects eating speeds that are too fast or too slow compared to other users based on preset criteria.

[0674] The "means for generating appropriate questions based on past conversation content and user profile" is a means for generating appropriate questions to interest the user based on previous conversations and the user's registration information.

[0675] The "means for presenting the generated question to the user and receiving an answer" is a means for displaying the generated question on the user's device and receiving an answer from the user.

[0676] The "means for analyzing the received answers and generating further questions" is a means for analyzing the answers from the user and generating new questions based on them.

[0677] The "means for adjusting timing to balance the eating speeds of all users" is a means for monitoring the eating progress of all users in real time and adjusting each user's eating speed to keep it balanced.

[0678] "Means for generating conversations based on user interests and concerns using an AI model" refers to using a generative AI model to automatically generate relevant conversations based on the interests and concerns derived from a user's profile and past comments.

[0679] "Means for monitoring and sharing users' meal experiences in real time in a virtual environment" refers to a method for monitoring a user's meal progress in real time through a virtual environment and sharing that information with other users, thereby creating a sense of unity.

[0680] The present invention provides a system for promoting enjoyable communication while balancing the eating speeds of all users when multiple users share a meal in a virtual environment. Specific embodiments of the system are described below.

[0681] Hardware and software used

[0682] Hardware: Smartphone, Head-Mounted Display (HMD)

[0683] Software: Servers for real-time data analysis, AI models for chatbots (e.g., OpenAI GPT-4), database management systems (e.g., MySQL)

[0684] System Overview

[0685] The system includes the following means:

[0686] 1. Means for inputting user information: Before starting a meal, each user inputs their information (name, age, hobbies, etc.) into the system via their smartphone or HMD. This information is sent to the server, and an individual session begins.

[0687] 2. A means to periodically record the progress of the meal: During the meal, the user inputs their progress (e.g., the amount eaten) via their smartphone or HMD. This information is sent to the server at regular intervals.

[0688] 3. Means for analyzing eating speed and detecting abnormal eating speed: The server analyzes the received data in real time, calculates each user's eating speed, and detects abnormally fast or slow eating speeds based on set criteria.

[0689] 4. Generating appropriate questions based on past conversations and user profiles: Using an AI model (e.g., GPT-4), appropriate questions are generated based on the user's profile and past conversations. For example, for a user who likes traveling, the system generates the question, "Where have you traveled recently?"

[0690] 5. A means for presenting the generated questions to the user and receiving answers: The generated questions are presented in real time to the user's smartphone or HMD, and the user answers the questions. The answers are then sent back to the server.

[0691] 6. Means for analyzing the received answers and generating further questions: The server analyzes the user's answers using an AI model and generates further questions to continue the conversation, thereby allowing the conversation to continue uninterrupted.

[0692] 7. A means for adjusting timing to balance the eating speed of all users: The server monitors the eating speed of all users and generates questions and comments as appropriate to keep the balance.

[0693] 8. Using an AI model to generate conversations based on the user's interests: Using a generative AI model, a conversation can be conducted based on the user's interests. For example, if a user enjoys reading, the conversation can be asked questions such as, "What books have you read recently?"

[0694] 9. A means of monitoring and sharing users' meal experiences in a virtual environment in real time: The system monitors and shares users' meal progress in real time, allowing everyone to enjoy the same experience.

[0695] Specific examples

[0696] Example 1:

[0697] Prompt: "Tell me about your most recent travel destination."

[0698] User A: "I went to Hokkaido last month."

[0699] Post-analysis question: "What kind of food did you like in Hokkaido?"

[0700] In this way, the AI ​​model is used to generate conversations based on the user's interests, allowing for adjustment of eating speed and smooth communication.

[0701] Example 2:

[0702] Prompt: "Tell me about the last book you read."

[0703] User B: "I was reading a mystery novel."

[0704] Post-analysis questions: "What was the title of the mystery novel? What part did you like?"

[0705] Specific prompts for the AI ​​model:

[0706] The user has started their meal progress. To encourage conversation with other users and adjust the eating speed, generate appropriate questions from the following input data. The user's recent interests are "travel" and "reading." Example: "Tell me about the places you've traveled recently and the books you've read recently."

[0707] In this way, the dining experience in a virtual environment can be tailored to be more enjoyable.

[0708] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0709] Step 1:

[0710] Before a user starts eating, they use their smartphone or HMD to enter user information (such as name, age, and hobbies) into the system. This information is sent from the device to the server. The server records the received user information in a database and starts the session. Input data: User information (such as name, age, and hobbies). Output data: User information recorded in the database and a session start notification.

[0711] Step 2:

[0712] While eating, the user periodically inputs their progress (e.g., amount eaten) into the device. The input data is sent from the device to the server at regular intervals. The server records the received progress data in a database and prepares it for analysis in real time. Input data: Progress (e.g., amount eaten). Output data: Progress data recorded in the database in preparation for analysis.

[0713] Step 3:

[0714] The server analyzes the received progress data in real time and calculates each user's eating speed. As a result of the analysis, it detects users whose eating speed is abnormally fast or slow compared to the set criteria. Input data: progress data. Output data: detection results of each user's eating speed and abnormal speed.

[0715] Step 4:

[0716] The server uses a generative AI model (e.g., GPT-4) to generate appropriate questions for users with abnormal eating speeds based on past conversations and user profiles. Input data: past conversations, user profiles. Output data: generated questions.

[0717] Step 5:

[0718] The generated questions are presented from the server to the user's smartphone or HMD. The user answers the presented questions. Input data: Generated questions. Output data: User answers.

[0719] Step 6:

[0720] The server analyzes the answers received from the user using an AI model and generates further questions, allowing the conversation to continue uninterrupted. Input data: User answers. Output data: Additional questions.

[0721] Step 7:

[0722] The server monitors the eating speed of all users in real time and generates questions and comments as needed to maintain balance. Input data: Progress and eating speed data of all users. Output data: Additional questions and comments for adjustment.

[0723] Step 8:

[0724] The server uses a generative AI model to engage in conversations based on the user's interests. For example, if a user has a hobby of reading, it will ask questions such as, "What books have you read recently?" Input data: User profile, past conversations. Output data: Conversation content based on interests.

[0725] Step 9:

[0726] The system monitors and shares the meal progress of users in real time, allowing everyone to enjoy the same experience and fostering a sense of unity. Input data: Progress data of all users. Output data: Shared progress data and real-time sense of unity.

[0727] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0728] System Overview

[0729] This invention is a system for balancing the eating speeds of all users when dining together, and provides a richer conversation experience by combining it with an emotion engine that recognizes users' emotions. This system works by having users record their meal progress and emotion data using their own devices and send it to a server. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[0730] Initial Setup

[0731] 1. Before starting a meal, the user inputs the "start eating" information into the device. The device then sends the start time, user ID, meal contents, past conversation log, and emotion data to the server.

[0732] 2. Based on the information received, the server records each user's session in the database and starts the session.

[0733] Monitoring and sending meal progress and emotion data

[0734] 3. While eating, the user periodically inputs the amount eaten, progress, and emotional data into the device. Emotional data can be automatically acquired using emotion engines such as voice analysis and facial expression analysis.

[0735] 4. The device sends the user's input data and emotion data to the server at regular intervals.

[0736] Data Analysis and Anomaly Detection

[0737] 5. The server analyzes the received data and calculates each user's eating speed and emotional state.

[0738] 6. The server compares the eating speeds of all users and detects abnormal eating speeds of specific users if they are significantly faster or slower. In addition, it analyzes the user's emotional state and obtains emotional indicators such as stress and relaxation.

[0739] Question generation and presentation

[0740] 7. The server generates questions appropriate for users with abnormal eating speed based on past conversations, user profiles, and emotional data. For example, for a user who loves traveling and is feeling stressed, the server generates a question such as, "Where was your most recent relaxing trip?"

[0741] 8. The server sends the generated question to the user's device.

[0742] Answering questions and analyzing

[0743] 9. The device presents the received question to the user, who answers it. The answer is sent to the server via the device.

[0744] 10. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0745] Adjusting meal timing

[0746] 11. The server monitors the eating speed and emotional data of all users in real time and continues to ask questions as appropriate to adjust everyone's eating timing and emotional state.

[0747] 12. When all users have finished eating, they enter the "Drinking is over" information into their terminals. The terminals then send this information to the server.

[0748] 13. The server confirms that all users have exited and saves the session data to the database.

[0749] Specific examples

[0750] Initial Setup

[0751] When user A and user B simultaneously input "start eating" into the terminal, the terminal sends the start information and emotion data to the server.

[0752] The server initiates the session and records information for each user.

[0753] Monitoring meal progress and emotional data

[0754] User A inputs "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[0755] The server analyzes the received data and calculates User A's eating speed and emotional state.

[0756] Data analysis

[0757] The server compares the data of all users and determines that User A eats quickly and is feeling stressed.

[0758] Question generation and presentation

[0759] The server determines that User A likes to travel and generates the question, "Where is the most relaxing travel destination you have recently visited?"

[0760] The server sends the question to User A's terminal.

[0761] Answering questions and analyzing

[0762] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[0763] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[0764] The server sends the following question to User A's terminal:

[0765] Meal timing and emotion regulation

[0766] The server continues asking questions until User A's eating speed and emotions are in balance with other users.

[0767] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[0768] The server confirms that all users have finished and ends the session.

[0769] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[0770] The processing flow will be explained below.

[0771] Step 1:

[0772] Before starting a meal, the user enters "Start Meal" into the terminal. The user then enters basic information such as ID and meal contents. The terminal then sends this user information along with the start time to the server.

[0773] Step 2:

[0774] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[0775] Step 3:

[0776] As the user eats, they periodically input information about the amount eaten and their progress into the device. Emotional data is also collected at the same time, using an emotion engine that analyzes voice and facial expressions.

[0777] Step 4:

[0778] The terminal transmits the user's input data and emotion data to the server at regular intervals.

[0779] Step 5:

[0780] The server analyzes the received data and calculates each user's eating speed and emotional state.

[0781] Step 6:

[0782] The server compares the eating speeds of all users and detects abnormal eating speeds. If an abnormal eating speed is detected, the server also checks the emotional state of the user.

[0783] Step 7:

[0784] The server generates appropriate questions based on past conversations, user profiles, and emotional data, such as "What are some of the most relaxing travel destinations you've visited recently?" for a stressed travel-loving user.

[0785] Step 8:

[0786] The server generates a query and sends it to the user's terminal.

[0787] Step 9:

[0788] The terminal displays the received question to the user.

[0789] Step 10:

[0790] The user answers the displayed questions and enters them into the terminal.

[0791] Step 11:

[0792] The terminal sends the user's answer to the server.

[0793] Step 12:

[0794] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[0795] Step 13:

[0796] The server generates additional questions and comments and sends them to the user's device.

[0797] Step 14:

[0798] The device will then prompt the user again with any additional questions or comments.

[0799] Step 15:

[0800] The server monitors all users' eating speed and emotional data in real time, and adjusts questions and comments to align the eating timing and emotional state of all users.

[0801] Step 16:

[0802] When all users input "Done eating," the terminal sends the information to the server.

[0803] Step 17:

[0804] The server confirms that all users have finished and saves the session data to the database.

[0805] Step 18:

[0806] The server sends a "meal end" notification to all terminals and ends the session.

[0807] Example 2

[0808] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0809] In today's world, when multiple users eat together, differences in eating speed can create imbalances, making it difficult for everyone to have a satisfying dining experience. Furthermore, systems lack the ability to generate questions at the right time to facilitate smooth conversation, or to communicate in a way that takes users' emotions into account. There is a need to solve these problems and provide a richer shared dining experience.

[0810] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and a user profile, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for acquiring emotional data using voice analysis or facial expression analysis, means for adjusting questions based on the acquired emotional data, and means for transmitting, receiving, and analyzing data in real time. This makes it possible to generate questions that take emotions into consideration while maintaining a balance in eating speeds among users, thereby enriching the conversation experience.

[0811] "User information" refers to information including the user ID, start time, meal contents, past conversation log, and emotion data.

[0812] "Meal progress" refers to data about the amount eaten and progress recorded by the user during the meal.

[0813] "Eating speed" is an index that measures and represents the speed at which a user eats.

[0814] An "abnormal eating speed" is an eating speed that is judged to be significantly fast or slow when compared with the eating speeds of all users.

[0815] "Past conversation content" refers to the history of interactions between users recorded in previous conversation sessions.

[0816] A "user profile" refers to personal attribute data such as a user's interests, concerns, and past behavioral history.

[0817] "Emotion data" refers to data that indicates the emotional state of the user, obtained based on voice analysis and facial expression analysis.

[0818] A "question" refers to a dialogue prompt that is generated based on the user's interests and current emotional state.

[0819] An "answer" refers to a response that a user enters in response to a question.

[0820] "Means for transmitting, receiving, and analyzing data in real time" refers to technical means for instantly transmitting and receiving data using communication technology and for quickly analyzing it.

[0821] This system balances the eating speeds of all users when dining together, and provides a rich conversational experience by using an emotion engine. This system works by having users record their meal progress and emotion data using their own devices and then sending this data to a server.

[0822] Initial Setup

[0823] 1. User: Before starting a meal, the user inputs the "start meal" information into their device. The input information includes the start time, user ID, meal contents, past conversation log, and emotion data.

[0824] 2. Terminal: The terminal collects the input information and sends it to the server, using an Internet communication protocol such as HTTP.

[0825] 3. Server: Based on the received information, the server records each user's session in a database (e.g., MySQL) and starts the session.

[0826] Monitoring meal progress and emotional data

[0827] 1. User: While eating, the user inputs their eating progress and emotional data into the device. Emotional data is automatically acquired using the Google Cloud Speech-to-Text API and the Microsoft Azure Emotion API.

[0828] 2. Terminal: The terminal sends the user's input data and emotion data to the server at regular intervals (e.g., every 5 minutes).

[0829] Data Analysis and Anomaly Detection

[0830] 1. Server: The server analyzes the received data using Python's pandas library and Scikit-learn to quantify eating speed and emotional state.

[0831] 2. Server: Compares the data of all users, identifies users with abnormal eating speeds (e.g., significantly faster or slower), and analyzes their emotional state.

[0832] Question generation and presentation

[0833] 1. Server: Generates questions appropriate for users with abnormal eating speeds based on past conversations, user profiles, and emotional data. For example, if the generative AI model determines that a user loves traveling and is feeling stressed, it generates a question such as, "Where was the most relaxing trip you've taken recently?"

[0834] 2. Server: Sends the generated question to the user's device.

[0835] Answering questions and analyzing

[0836] 1. Terminal: The terminal presents the received question to the user. The user answers the question and enters it into the terminal. For example, the answer might be, "I went to Hokkaido last month."

[0837] 2. Terminal: The terminal sends the answer to the server.

[0838] 3. Server: The server analyzes the user's answers and generates additional questions or comments to continue the conversation, such as "What kind of food did you like in Hokkaido?"

[0839] Adjusting meal timing

[0840] 1. Server: The server monitors the eating speed and emotional data of all users in real time and adjusts everyone's eating timing and emotional state by asking questions appropriately.

[0841] 2. All users: When finishing a meal, they input the "Done" information into their terminal. The terminal sends this information to the server.

[0842] 3. Server: The server checks the completion of all users and stores the session data in a database, for example, recording the completion time and final emotion data.

[0843] Specific examples

[0844] 1. User A and User B: Simultaneously input "start eating" into their devices, and the devices send the start information and emotion data to the server.

[0845] 2. Server: Initiates sessions and records information about each user.

[0846] 3. User A: Enters "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[0847] 4. Server: Analyzes the received data and calculates User A's eating speed and emotional state.

[0848] 5. Server: Compares the data of all users and determines that User A eats quickly and is stressed.

[0849] 6. Server: Determines that User A loves to travel and generates the question, "Where was your last relaxing trip?"

[0850] 7. Server: Sends the question to User A's device.

[0851] 8. User A: "I went to Hokkaido last month." The device sends the answer to the server.

[0852] 9. Server: Analyzes the answer and generates the next question: "What kind of food was delicious in Hokkaido?". Sends the next question to User A's device.

[0853] 10. Server: Continue asking questions until User A's eating speed and emotions are balanced with the other users.

[0854] 11. All users: When you enter "Done eating" into the terminal, the terminal will send that information to the server.

[0855] 12. Server: Confirm that all users have finished and end the session.

[0856] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[0857] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0858] Step 1:

[0859] Before starting a meal, the user enters the "start meal" information into the device. At that time, the user enters the user ID, start time, meal contents, past conversation log, and emotion data. Specifically, the data is entered into an input form using a smartphone app or tablet. The entered data is temporarily stored in the device's memory.

[0860] Input: User ID, start time, meal contents, past conversation log, emotion data

[0861] Output: User data in device memory

[0862] Step 2:

[0863] The device collects the entered user information and sends it to the server. The HTTP protocol is used for transmission, and the data is packetized in JSON format and sent. The transmitted data is received by the server-side API and stored in a database.

[0864] Input: User data in the terminal memory

[0865] Output: User information stored in a database on the server

[0866] Step 3:

[0867] The server starts a session for each user based on the received information. The received data (user ID, start time, meal contents, etc.) is recorded in a database, and the session start time is saved as a timestamp.

[0868] Input: User information on the server

[0869] Output: Session data recorded in a database

[0870] Step 4:

[0871] The user inputs the amount of food eaten and their progress during the meal into the device. Emotion data is also automatically acquired using an emotion engine. For example, the user's voice is analyzed using the Google Cloud Speech-to-Text API to evaluate their stress level.

[0872] Input: User's food intake, progress, and voice data

[0873] Output: New dataset in device (progress and emotion ratings)

[0874] Step 5:

[0875] The device sends the user's progress and emotion data to the server at regular intervals (for example, every 5 minutes). This communication also uses the HTTP protocol, and the data is sent to the server in JSON format. The server then formats the received data appropriately and adds it to the database.

[0876] Input: New progress and emotion data

[0877] Output: Updated user database on the server.

[0878] Step 6:

[0879] The server analyzes the received data using Python's pandas library and Scikit-learn. Specifically, it calculates eating speed based on the amount of food eaten and progress, and runs algorithms to evaluate stress levels and relaxation state based on emotional data.

[0880] Input: Updated user database

[0881] Output: Eating speed and emotional state of each user

[0882] Step 7:

[0883] The server compares the data of all users and detects abnormal eating speeds. For example, it identifies users whose eating speeds deviate significantly from the average speed and also checks their emotional state. If an abnormality is detected, it flags the data as abnormal.

[0884] Input: Eating speed and emotional state of each user

[0885] Output: A list of users for whom anomalies were detected

[0886] Step 8:

[0887] The server generates appropriate questions for users with abnormal eating speeds. Using past conversations, user profiles, and emotional data, the generative AI model inputs prompt sentences to generate specific questions. For example, it generates a question like, "Where was your most recent relaxing trip?"

[0888] Input: A list of users for whom anomalies were detected, and a prompt for the AI ​​model

[0889] Output: Generated questions

[0890] Step 9:

[0891] The server sends the generated question to the user's device. The server uses the HTTP protocol to send the generated question in JSON format to the device. The device receives it and displays it on the user interface.

[0892] Input: Generated Question

[0893] Output: The question displayed on the user's terminal

[0894] Step 10:

[0895] The user inputs an answer to the question displayed on the device. For example, the answer might be "I went to Hokkaido last month." The answer is input using the keyboard on the smartphone or tablet. The answer data is saved in the device's memory.

[0896] Input: User's answer

[0897] Output: Answer data stored in the device's memory

[0898] Step 11:

[0899] The device sends the response data to the server using the HTTP protocol, packetizing the response data in JSON format and sending it to the server. The server stores the received response data in a database and analyzes it.

[0900] Input: Answer data stored in the device's memory

[0901] Output: Answer data saved on the server

[0902] Step 12:

[0903] The server analyzes the received response data, for example by applying a text analysis algorithm to generate the next question from what the user has said about their travel destination, and then uses a generative AI model to generate the next question.

[0904] Input: Answer data saved on the server

[0905] Output: The following questions are generated:

[0906] Step 13:

[0907] The server generates the next question and sends it to the user's device. The server then packets the question in JSON format and sends it using the HTTP protocol. The device receives it and displays it on the user interface.

[0908] Input: The next generated question

[0909] Output: The following question displayed on the user's screen:

[0910] Step 14:

[0911] The server monitors all users' eating speed and emotional data in real time and continues to ask questions accordingly, forming a feedback loop to adjust the eating timing and emotional state of all users.

[0912] Input: Eating speed and emotion data of all users

[0913] Output: Feedback loop of adjusted questions

[0914] Step 15:

[0915] When all users have finished eating, they input "meal finished" information into their terminal. The terminal then sends the end information to the server. The server updates the session data based on the received end information and confirms that all users have finished.

[0916] Input: User exit information

[0917] Output: Finished data saved on the server

[0918] Step 16:

[0919] The server confirms the end of each user session and stores the session data in a database, including the end time and final emotion data.

[0920] Input: Finished data sent to the server

[0921] Output: Session termination data stored in the database

[0922] (Application example 2)

[0923] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0924] In today's world, when multiple users eat together, imbalances in eating speed can cause some users to feel isolated or conversations to be interrupted. This problem significantly reduces the enjoyment and satisfaction of the meal and requires special consideration. Furthermore, because emotional fluctuations often affect the pace of eating, a system that takes this into account is needed. However, current systems do not provide a system that can reflect users' emotions in real time and generate appropriate conversations.

[0925] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0926] In this invention, the server includes a means for periodically recording the progress of the meal using a smart device, a means for acquiring the user's emotion data using an emotion engine, and a means for generating questions using a generative AI model, which enables the server to monitor the user's eating speed and emotions in real time and balance the eating speed and conversation by asking appropriate questions.

[0927] A "smart device" refers to a device that has internet connectivity and is equipped with various sensors and displays.

[0928] An "emotion engine" refers to a software or hardware system that analyzes a user's facial expressions and voice and recognizes their psychological state and emotions.

[0929] A "generative AI model" refers to a system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[0930] A "server" refers to a computer system that centrally manages, analyzes, and processes data sent from multiple users.

[0931] "Means for generating questions" refers to algorithms or programs for automatically generating appropriate questions based on past conversation content and user profiles.

[0932] "Eating speed" refers to the progress of a user's eating within a specific period of time.

[0933] "Means for analyzing eating speed" refers to a program or algorithm that evaluates and analyzes a user's eating speed based on collected data.

[0934] This invention is a system that uses smart devices to record and transmit users' meal progress and emotional data, and a server analyzes this data and generates appropriate questions to adjust the balance between eating speed and conversation for all users. This system enriches the dining experience among users and is expected to be used particularly in brick-and-mortar establishments such as restaurants.

[0935] The system configuration includes the following hardware and software:

[0936] Hardware

[0937] Smart device: A device with internet connectivity and equipped with various sensors and displays. Here, we use smart glasses as an example.

[0938] Sensors: To keep track of your eating progress, sensors built into the smart glasses measure the amount of food you eat.

[0939] Server: A computer system that centrally manages, analyzes, and processes data sent from multiple users.

[0940] software

[0941] Emotion engine: An engine for analyzing the user's facial expressions and voice to obtain emotional data.

[0942] Generative AI model: A system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[0943] ServerCommunicator: A module that is responsible for communicating with the server and sending and receiving data.

[0944] QuestionGenerator: A module that generates appropriate questions based on emotional data, user profiles, and conversation history.

[0945] SmartGlassesDisplay: Software with the ability to display questions on the screen of smart glasses.

[0946] System Operation

[0947] In this system, the user puts on the smart glasses before starting a meal and inputs the "start eating" information. The smart glasses then send the start time, user ID, meal content, past conversation log, and emotional data to the server. The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. This data is then sent to the server at regular intervals, where it is analyzed and used to monitor eating speed and emotional state.

[0948] The server then compares each user's eating speed, and if an abnormal eating speed is detected, the AI ​​model generates appropriate questions based on the user's profile and emotional data. The generated questions are displayed on the smart glasses' display, and the user answers them. The answers are then sent back to the server, which generates additional questions to further encourage conversation.

[0949] Specific examples

[0950] For example, if the user is recognized as feeling stressed, the following example prompt sentences are fed into the generative AI model:

[0951] A user is feeling stressed. Generate a question for User A, who loves to travel: "Where was the most relaxing trip you've taken recently?"

[0952] The generated question looks like this:

[0953] "Where's the last relaxing trip you went to?"

[0954] This question is displayed on the smart glasses display, and the user responds, "I went to Hokkaido last month." This response is sent to the server, which then generates the next question, such as, "What kind of food did you like in Hokkaido?" Through this process, the eating speed and emotional state of all users are balanced.

[0955] This will enrich the dining experience between users and allow for mealtimes and conversations that are satisfying for everyone.

[0956] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0957] Step 1:

[0958] Before starting a meal, the user puts on the smart glasses and inputs the "start eating" information. The user operates the interface of the smart glasses to input the start time, user ID, and meal contents. This information is sent to the server along with the start trigger.

[0959] Input: Start time, User ID, Meal details

[0960] Output: Start trigger and user information sent to the server

[0961] Step 2:

[0962] The server starts a session based on the received start information, user ID, and meal details. A session ID is generated and recorded in the database. Each user's progress is managed based on this session ID.

[0963] Input: Start trigger and user information

[0964] Output: Generate a session ID and record it in the database

[0965] Step 3:

[0966] The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. The camera and microphone built into the smart glasses capture facial expressions and voice, which are then sent to the emotion engine for analysis.

[0967] Input: User facial expression images, voice data

[0968] Output: Obtaining emotion data

[0969] Step 4:

[0970] Sensors in the smart glasses periodically measure the amount of food consumed and send the data to a server. For example, a weight sensor measures the amount of food consumed and sends it as progress data.

[0971] Input: Food quantity data

[0972] Output: Getting and sending progress data

[0973] Step 5:

[0974] The server analyzes the received emotion data and progress data to calculate the user's eating speed. It also compares the eating speeds of multiple users to check for abnormal speeds.

[0975] Input: Emotion data, progress data

[0976] Output: Detection results of eating speed and abnormal speed

[0977] Step 6:

[0978] If the server detects abnormal eating speed, it uses a generative AI model to generate appropriate questions based on the user's profile and emotional data. The generative AI model generates questions based on the prompt text, and the server sends the questions to the smart glasses.

[0979] Input: User profile, emotion data, prompt sentence

[0980] Output: Generated questions

[0981] Step 7:

[0982] A question is displayed on the smart glasses display, and the user responds to the question by voice or text, which is then sent back to the server.

[0983] Input: Generated Question

[0984] Output: User's answer

[0985] Step 8:

[0986] The server analyzes the answers it receives and generates further questions as needed, repeating this process until all users' eating speeds and conversations are balanced.

[0987] Input: User's answer

[0988] Output: Further generated questions

[0989] Step 9:

[0990] All users input the "meal end" information into their smart glasses. This information is sent to the server, which ends the session and saves all data in a database.

[0991] Input: Meal end information

[0992] Output: Save session data and end session

[0993] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0994] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0995] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0996] [Third embodiment]

[0997] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0998] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0999] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1000] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1001] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1002] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1003] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1004] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1005] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1006] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1007] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1008] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1009] System Overview

[1010] This invention is a system for balancing the eating speeds of all users when multiple users are eating together. The system operates by having users record and transmit their meal progress using their own devices. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[1011] Initial Setup

[1012] 1. Before starting a meal, the user inputs the "start eating" information into their device. The device then sends this information to the server.

[1013] 2. Based on the information received, the server records each user's session in the database and starts the session.

[1014] Monitoring and sending meal progress

[1015] 3. While eating, the user periodically inputs the amount eaten and progress into the device. The device then transmits this data to the server at regular intervals.

[1016] 4. The server analyzes the received data and calculates each user's eating speed.

[1017] Data Analysis and Anomaly Detection

[1018] 5. The server compares the eating speeds of all users and detects abnormal eating speeds of a particular user if the eating speed of that user is significantly faster or slower.

[1019] Question generation and presentation

[1020] 6. The server generates questions appropriate for a user with an abnormal eating speed based on past conversations and the user profile. For example, for a user who likes traveling, it generates a question such as "Where have you traveled recently?"

[1021] 7. The server sends the generated question to the user's device.

[1022] Answering questions and analyzing

[1023] 8. The device presents the received question to the user, who answers it. The answer is then sent back to the server via the device.

[1024] 9. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1025] Adjusting meal timing

[1026] 10. The server monitors the eating speed of all users in real time and continues to ask questions as appropriate to align everyone's eating timing.

[1027] 11. When all users have finished eating, they input the "Dinner finished" information into their terminals. The terminals then send this information to the server.

[1028] 12. The server confirms that all users have finished and ends the session.

[1029] Specific examples

[1030] Initial Setup

[1031] When user A and user B simultaneously input "start meal" into the terminal, the terminal sends the start information to the server.

[1032] The server initiates the session and records information for each user.

[1033] Monitoring meal progress

[1034] User A enters "I ate three bites of rice" into the device. The device sends this information to the server.

[1035] The server analyzes the received data and calculates User A's eating speed.

[1036] Data analysis

[1037] The server compares the data of all users and determines that User A eats quickly.

[1038] Question generation and presentation

[1039] The server determines that user A likes to travel and generates the question, "Where have you traveled recently?"

[1040] The server sends the question to User A's terminal.

[1041] Answering questions and analyzing

[1042] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[1043] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[1044] The server sends the following question to User A's terminal:

[1045] Adjusting meal timing

[1046] The server continues asking questions until User A's eating speed is in balance with the other users.

[1047] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[1048] The server confirms that all users have finished and ends the session.

[1049] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversations, thereby providing a satisfying dining experience for everyone.

[1050] The processing flow will be explained below.

[1051] Step 1:

[1052] Before starting a meal, the user inputs "Start meal" into the terminal, which then sends basic information such as the start time, user ID, and meal contents to the server.

[1053] Step 2:

[1054] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[1055] Step 3:

[1056] As the user continues eating, they periodically input the amount eaten and their progress into the terminal.

[1057] Step 4:

[1058] The terminal transmits the user's input data to the server at regular intervals.

[1059] Step 5:

[1060] The server analyzes the received data and calculates each user's eating speed.

[1061] Step 6:

[1062] The server compares the eating speeds of all users, and if a particular user's eating speed is significantly faster or slower than other users, the abnormal eating speed is detected.

[1063] Step 7:

[1064] The server refers to past conversations and user profiles to generate questions suited to users with abnormal eating speeds.

[1065] Step 8:

[1066] The server generates a query and sends it to the user's terminal.

[1067] Step 9:

[1068] The terminal displays the received question to the user.

[1069] Step 10:

[1070] The user answers the displayed questions and enters them into the terminal.

[1071] Step 11:

[1072] The terminal sends the user's answer to the server.

[1073] Step 12:

[1074] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1075] Step 13:

[1076] The server generates additional questions and comments and sends them to the user's device.

[1077] Step 14:

[1078] The device will then prompt the user again with any additional questions or comments.

[1079] Step 15:

[1080] The server monitors the eating speed of all users in real time and adjusts questions and comments to align the eating timing of all users.

[1081] Step 16:

[1082] When all users input "Done eating," the terminal sends the information to the server.

[1083] Step 17:

[1084] The server confirms that all users have finished and saves the session data to the database.

[1085] Step 18:

[1086] The server sends a "meal end" notification to all terminals and ends the session.

[1087] Example 1

[1088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1089] When multiple users eat together, variations in the eating speeds of each user can disrupt the rhythm of conversation, or some users may eat too fast or too slow. This makes it difficult to provide a dining experience that is satisfying for everyone. To solve this problem, the present invention aims to balance the eating speeds of users and provide a dining experience that is satisfying for everyone through appropriate conversation.

[1090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1091] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating questions using a generative AI model and sending the questions to the user's terminal, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, and means for adjusting the timing to keep the eating speeds of all users balanced. This makes it possible to monitor each user's eating speed in real time, generate appropriate questions to promote conversation, and keep everyone's eating speeds balanced.

[1092] "User Information" refers to personal attributes and profile information, such as a user's name, interests, and concerns.

[1093] "Meal progress" refers to information indicating the amount of food the user is currently eating and the progress of the meal.

[1094] "Meal speed" refers to the speed at which a meal progresses, calculated based on the amount of food eaten by a user within a certain period of time.

[1095] "Abnormal eating speed" refers to an eating speed that is significantly faster or slower than the average eating speed of other users.

[1096] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate questions that are appropriate for the user.

[1097] "Question" refers to an inquiry or query presented to a user to facilitate relevant conversation.

[1098] An "answer" refers to a reply or response that a user enters to a question.

[1099] "Adjusting the timing" refers to generating questions and continuing the conversation at the right moment to balance the eating speed of all users.

[1100] The present invention relates to a system for balancing the eating speeds of all users when a group of users is eating together. How to implement this system will be described in detail below.

[1101] System Configuration

[1102] This system consists of a device used by the user, a server that processes the data, and an algorithm that uses a generative AI model to generate appropriate questions. The device is a computing device such as a smartphone or tablet, and the server is often installed on the cloud.

[1103] Hardware and software used

[1104] Terminal: A device that a user operates, such as a smartphone, tablet, or PC.

[1105] Server: Hardware for processing and storing data, such as a cloud server or on-premise server.

[1106] Generative AI models: Generate questions using artificial intelligence models such as OpenAI's GPT series.

[1107] Data Processing and Data Calculation

[1108] 1. Data entered by the user:

[1109] Users input information about the start, progress, and end of their meal into the terminal, which allows the server to analyze each user's eating speed.

[1110] 2. Data transmission and analysis:

[1111] The device periodically sends the user's input data to the server, which analyzes the received data and calculates each user's eating speed.

[1112] 3. Detecting abnormal eating speed:

[1113] The server compares the eating speeds of all users and detects an abnormality when a particular user's eating speed is significantly faster or slower.

[1114] 4. Question generation:

[1115] If an anomaly is detected, the server generates appropriate questions using a generative AI model based on past conversations and the user profile, such as OpenAI's GPT series.

[1116] 5. Presenting and answering questions to the user:

[1117] The server generates questions and sends them to the user's device, which then presents them to the user. The user answers the questions, and the answers are sent back to the server, allowing the server to generate further questions and continue the conversation.

[1118] Specific examples

[1119] Example of question generation

[1120] By inputting the following prompt sentence into the generative AI model, a question appropriate for the user can be generated.

[1121] Based on the following user information and conversation content, please generate the following questions for User A.

[1122] User Profile:

[1123] Name: User A

[1124] Interests: Travel, music, cooking

[1125] Recent conversations:

[1126] User A: "I went to Hokkaido last month."

[1127] Based on this prompt, the generative AI model generates a question such as, "What kind of food was delicious in Hokkaido?" This question is sent from the server to the user's device, prompting the user to respond.

[1128] The system monitors the eating speed of users and encourages conversation at appropriate times to provide a satisfying dining experience for everyone. Furthermore, by using a generative AI model, it is possible to dynamically generate questions suited to each user, ensuring smooth conversation.

[1129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1130] Step 1:

[1131] Before starting a meal, the user enters the "start meal" information into the device. Specifically, the user taps the "start" button on the smartphone or tablet app. This input data includes the user ID and start time. The device then sends this data to the server.

[1132] Step 2:

[1133] The server starts each user's session based on the "start meal" information it receives. At this point, the server records data such as the user ID, start time, and session ID in a database. This makes it possible to track which user started eating and when.

[1134] Step 3:

[1135] While eating, the user periodically inputs their progress into the device. A specific action would be to enter something like "I ate three bites of rice" into an input form within the app. This input data includes the user ID, progress, and time of input. The device then sends this data to the server at regular intervals.

[1136] Step 4:

[1137] The server analyzes the received data and calculates each user's eating speed. Here, the server calculates the eating speed for each user and records the results in the database. For example, if user A eats 3 bites in 10 minutes, their eating speed is "10 minutes / 3 bites."

[1138] Step 5:

[1139] The server compares the eating speeds of all users and detects abnormal eating speeds. In this detection process, the server calculates the average eating speed of all users and determines that speeds that are significantly faster or slower than that are abnormal. Specifically, if user B's eating speed is significantly faster than the average, it is detected as abnormal.

[1140] Step 6:

[1141] The server uses the generative AI model to generate questions appropriate for users with abnormal eating speeds. The input data for the prompt sentences used for generation includes the user profile (e.g., interests and past conversations) and user ID. For example, for a user who likes to travel, the question generated is "Where have you traveled recently?"

[1142] Step 7:

[1143] The server sends the generated question to the user's device. The sent data includes the question content, user ID, and time of sending. The device then presents the received question to the user.

[1144] Step 8:

[1145] The user answers the questions through the device. Specifically, the user enters the answer into the input form on the device and taps the "Send" button. This input data includes the answer, user ID, and time of input. The device then sends the answer to the server.

[1146] Step 9:

[1147] The server analyzes the user's answers and generates additional questions as necessary. Here, the server analyzes the answers and generates questions to encourage further conversation. For example, the next question generated might be, "What kind of food was delicious in Hokkaido?" The generated question is then returned to step 7 and sent again.

[1148] Step 10:

[1149] All users enter their end of meal time into their device, which then sends this information to the server. Specifically, users tap the "End" button in the app. The data sent includes the user ID and end time.

[1150] Step 11:

[1151] The server confirms the end information of all users and ends the session. At this time, all users' meal data and conversation content are saved in a database and recorded for future analysis and improvement.

[1152] (Application example 1)

[1153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1154] In conventional online dining experiences, users often eat at different speeds, resulting in a lack of unity between them. Communication can also be disrupted, leading to uneven mealtime progress. Furthermore, sharing and coordinating real-time meal plans in a virtual environment is difficult.

[1155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1156] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and user profiles, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for generating conversations based on user interests and concerns using an AI model, and means for monitoring and sharing users' eating experiences in real time in a virtual environment, thereby enabling adjustment of different eating speeds and smooth communication.

[1157] The "means for inputting user information" refers to an interface that allows a user to input their own information into the system, and can include basic information such as name, age, and hobbies.

[1158] The "means for periodically recording the progress of a meal" is a means for recording the current progress of a meal at time intervals as the user eats, and for collecting data such as the amount and speed of eating.

[1159] The "means for analyzing eating speed and detecting abnormal eating speed" analyzes the progress data entered by the user and detects eating speeds that are too fast or too slow compared to other users based on preset criteria.

[1160] The "means for generating appropriate questions based on past conversation content and user profile" is a means for generating appropriate questions to interest the user based on previous conversations and the user's registration information.

[1161] The "means for presenting the generated question to the user and receiving an answer" is a means for displaying the generated question on the user's device and receiving an answer from the user.

[1162] The "means for analyzing the received answers and generating further questions" is a means for analyzing the answers from the user and generating new questions based on them.

[1163] The "means for adjusting timing to balance the eating speeds of all users" is a means for monitoring the eating progress of all users in real time and adjusting each user's eating speed to keep it balanced.

[1164] "Means for generating conversations based on user interests and concerns using an AI model" refers to using a generative AI model to automatically generate relevant conversations based on the interests and concerns derived from a user's profile and past comments.

[1165] "Means for monitoring and sharing users' meal experiences in real time in a virtual environment" refers to a method for monitoring a user's meal progress in real time through a virtual environment and sharing that information with other users, thereby creating a sense of unity.

[1166] The present invention provides a system for promoting enjoyable communication while balancing the eating speeds of all users when multiple users share a meal in a virtual environment. Specific embodiments of the system are described below.

[1167] Hardware and software used

[1168] Hardware: Smartphone, Head-Mounted Display (HMD)

[1169] Software: Servers for real-time data analysis, AI models for chatbots (e.g., OpenAI GPT-4), database management systems (e.g., MySQL)

[1170] System Overview

[1171] The system includes the following means:

[1172] 1. Means for inputting user information: Before starting a meal, each user inputs their information (name, age, hobbies, etc.) into the system via their smartphone or HMD. This information is sent to the server, and an individual session begins.

[1173] 2. A means to periodically record the progress of the meal: During the meal, the user inputs their progress (e.g., the amount eaten) via their smartphone or HMD. This information is sent to the server at regular intervals.

[1174] 3. Means for analyzing eating speed and detecting abnormal eating speed: The server analyzes the received data in real time, calculates each user's eating speed, and detects abnormally fast or slow eating speeds based on set criteria.

[1175] 4. Generating appropriate questions based on past conversations and user profiles: Using an AI model (e.g., GPT-4), appropriate questions are generated based on the user's profile and past conversations. For example, for a user who likes traveling, the system generates the question, "Where have you traveled recently?"

[1176] 5. A means for presenting the generated questions to the user and receiving answers: The generated questions are presented in real time to the user's smartphone or HMD, and the user answers the questions. The answers are then sent back to the server.

[1177] 6. Means for analyzing the received answers and generating further questions: The server analyzes the user's answers using an AI model and generates further questions to continue the conversation, thereby allowing the conversation to continue uninterrupted.

[1178] 7. A means for adjusting timing to balance the eating speed of all users: The server monitors the eating speed of all users and generates questions and comments as appropriate to keep the balance.

[1179] 8. Using an AI model to generate conversations based on the user's interests: Using a generative AI model, a conversation can be conducted based on the user's interests. For example, if a user enjoys reading, the conversation can be asked questions such as, "What books have you read recently?"

[1180] 9. A means of monitoring and sharing users' meal experiences in a virtual environment in real time: The system monitors and shares users' meal progress in real time, allowing everyone to enjoy the same experience.

[1181] Specific examples

[1182] Example 1:

[1183] Prompt: "Tell me about your most recent travel destination."

[1184] User A: "I went to Hokkaido last month."

[1185] Post-analysis question: "What kind of food did you like in Hokkaido?"

[1186] In this way, the AI ​​model is used to generate conversations based on the user's interests, allowing for adjustment of eating speed and smooth communication.

[1187] Example 2:

[1188] Prompt: "Tell me about the last book you read."

[1189] User B: "I was reading a mystery novel."

[1190] Post-analysis questions: "What was the title of the mystery novel? What part did you like?"

[1191] Specific prompts for the AI ​​model:

[1192] The user has started their meal progress. To encourage conversation with other users and adjust the eating speed, generate appropriate questions from the following input data. The user's recent interests are "travel" and "reading." Example: "Tell me about the places you've traveled recently and the books you've read recently."

[1193] In this way, the dining experience in a virtual environment can be tailored to be more enjoyable.

[1194] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1195] Step 1:

[1196] Before a user starts eating, they use their smartphone or HMD to enter user information (such as name, age, and hobbies) into the system. This information is sent from the device to the server. The server records the received user information in a database and starts the session. Input data: User information (such as name, age, and hobbies). Output data: User information recorded in the database and a session start notification.

[1197] Step 2:

[1198] While eating, the user periodically inputs their progress (e.g., amount eaten) into the device. The input data is sent from the device to the server at regular intervals. The server records the received progress data in a database and prepares it for analysis in real time. Input data: Progress (e.g., amount eaten). Output data: Progress data recorded in the database in preparation for analysis.

[1199] Step 3:

[1200] The server analyzes the received progress data in real time and calculates each user's eating speed. As a result of the analysis, it detects users whose eating speed is abnormally fast or slow compared to the set criteria. Input data: progress data. Output data: detection results of each user's eating speed and abnormal speed.

[1201] Step 4:

[1202] The server uses a generative AI model (e.g., GPT-4) to generate appropriate questions for users with abnormal eating speeds based on past conversations and user profiles. Input data: past conversations, user profiles. Output data: generated questions.

[1203] Step 5:

[1204] The generated questions are presented from the server to the user's smartphone or HMD. The user answers the presented questions. Input data: Generated questions. Output data: User answers.

[1205] Step 6:

[1206] The server analyzes the answers received from the user using an AI model and generates further questions, allowing the conversation to continue uninterrupted. Input data: User answers. Output data: Additional questions.

[1207] Step 7:

[1208] The server monitors the eating speed of all users in real time and generates questions and comments as needed to maintain balance. Input data: Progress and eating speed data of all users. Output data: Additional questions and comments for adjustment.

[1209] Step 8:

[1210] The server uses a generative AI model to engage in conversations based on the user's interests. For example, if a user has a hobby of reading, it will ask questions such as, "What books have you read recently?" Input data: User profile, past conversations. Output data: Conversation content based on interests.

[1211] Step 9:

[1212] The system monitors and shares the meal progress of users in real time, allowing everyone to enjoy the same experience and fostering a sense of unity. Input data: Progress data of all users. Output data: Shared progress data and real-time sense of unity.

[1213] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1214] System Overview

[1215] This invention is a system for balancing the eating speeds of all users when dining together, and provides a richer conversation experience by combining it with an emotion engine that recognizes users' emotions. This system works by having users record their meal progress and emotion data using their own devices and send it to a server. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[1216] Initial Setup

[1217] 1. Before starting a meal, the user inputs the "start eating" information into the device. The device then sends the start time, user ID, meal contents, past conversation log, and emotion data to the server.

[1218] 2. Based on the information received, the server records each user's session in the database and starts the session.

[1219] Monitoring and sending meal progress and emotion data

[1220] 3. While eating, the user periodically inputs the amount eaten, progress, and emotional data into the device. Emotional data can be automatically acquired using emotion engines such as voice analysis and facial expression analysis.

[1221] 4. The device sends the user's input data and emotion data to the server at regular intervals.

[1222] Data Analysis and Anomaly Detection

[1223] 5. The server analyzes the received data and calculates each user's eating speed and emotional state.

[1224] 6. The server compares the eating speeds of all users and detects abnormal eating speeds of specific users if they are significantly faster or slower. In addition, it analyzes the user's emotional state and obtains emotional indicators such as stress and relaxation.

[1225] Question generation and presentation

[1226] 7. The server generates questions appropriate for users with abnormal eating speed based on past conversations, user profiles, and emotional data. For example, for a user who loves traveling and is feeling stressed, the server generates a question such as, "Where was your most recent relaxing trip?"

[1227] 8. The server sends the generated question to the user's device.

[1228] Answering questions and analyzing

[1229] 9. The device presents the received question to the user, who answers it. The answer is sent to the server via the device.

[1230] 10. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1231] Adjusting meal timing

[1232] 11. The server monitors the eating speed and emotional data of all users in real time and continues to ask questions as appropriate to adjust everyone's eating timing and emotional state.

[1233] 12. When all users have finished eating, they enter the "Drinking is over" information into their terminals. The terminals then send this information to the server.

[1234] 13. The server confirms that all users have exited and saves the session data to the database.

[1235] Specific examples

[1236] Initial Setup

[1237] When user A and user B simultaneously input "start eating" into the terminal, the terminal sends the start information and emotion data to the server.

[1238] The server initiates the session and records information for each user.

[1239] Monitoring meal progress and emotional data

[1240] User A inputs "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[1241] The server analyzes the received data and calculates User A's eating speed and emotional state.

[1242] Data analysis

[1243] The server compares the data of all users and determines that User A eats quickly and is feeling stressed.

[1244] Question generation and presentation

[1245] The server determines that User A likes to travel and generates the question, "Where is the most relaxing travel destination you have recently visited?"

[1246] The server sends the question to User A's terminal.

[1247] Answering questions and analyzing

[1248] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[1249] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[1250] The server sends the following question to User A's terminal:

[1251] Meal timing and emotion regulation

[1252] The server continues asking questions until User A's eating speed and emotions are in balance with other users.

[1253] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[1254] The server confirms that all users have finished and ends the session.

[1255] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[1256] The processing flow will be explained below.

[1257] Step 1:

[1258] Before starting a meal, the user enters "Start Meal" into the terminal. The user then enters basic information such as ID and meal contents. The terminal then sends this user information along with the start time to the server.

[1259] Step 2:

[1260] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[1261] Step 3:

[1262] As the user eats, they periodically input information about the amount eaten and their progress into the device. Emotional data is also collected at the same time, using an emotion engine that analyzes voice and facial expressions.

[1263] Step 4:

[1264] The terminal transmits the user's input data and emotion data to the server at regular intervals.

[1265] Step 5:

[1266] The server analyzes the received data and calculates each user's eating speed and emotional state.

[1267] Step 6:

[1268] The server compares the eating speeds of all users and detects abnormal eating speeds. If an abnormal eating speed is detected, the server also checks the emotional state of the user.

[1269] Step 7:

[1270] The server generates appropriate questions based on past conversations, user profiles, and emotional data, such as "What are some of the most relaxing travel destinations you've visited recently?" for a stressed travel-loving user.

[1271] Step 8:

[1272] The server generates a query and sends it to the user's terminal.

[1273] Step 9:

[1274] The terminal displays the received question to the user.

[1275] Step 10:

[1276] The user answers the displayed questions and enters them into the terminal.

[1277] Step 11:

[1278] The terminal sends the user's answer to the server.

[1279] Step 12:

[1280] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1281] Step 13:

[1282] The server generates additional questions and comments and sends them to the user's device.

[1283] Step 14:

[1284] The device will then prompt the user again with any additional questions or comments.

[1285] Step 15:

[1286] The server monitors all users' eating speed and emotional data in real time, and adjusts questions and comments to align the eating timing and emotional state of all users.

[1287] Step 16:

[1288] When all users input "Done eating," the terminal sends the information to the server.

[1289] Step 17:

[1290] The server confirms that all users have finished and saves the session data to the database.

[1291] Step 18:

[1292] The server sends a "meal end" notification to all terminals and ends the session.

[1293] Example 2

[1294] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1295] In today's world, when multiple users eat together, differences in eating speed can create imbalances, making it difficult for everyone to have a satisfying dining experience. Furthermore, systems lack the ability to generate questions at the right time to facilitate smooth conversation, or to communicate in a way that takes users' emotions into account. There is a need to solve these problems and provide a richer shared dining experience.

[1296] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and a user profile, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for acquiring emotional data using voice analysis or facial expression analysis, means for adjusting questions based on the acquired emotional data, and means for transmitting, receiving, and analyzing data in real time. This makes it possible to generate questions that take emotions into consideration while maintaining a balance in eating speeds among users, thereby enriching the conversation experience.

[1297] "User information" refers to information including the user ID, start time, meal contents, past conversation log, and emotion data.

[1298] "Meal progress" refers to data about the amount eaten and progress recorded by the user during the meal.

[1299] "Eating speed" is an index that measures and represents the speed at which a user eats.

[1300] An "abnormal eating speed" is an eating speed that is judged to be significantly fast or slow when compared with the eating speeds of all users.

[1301] "Past conversation content" refers to the history of interactions between users recorded in previous conversation sessions.

[1302] A "user profile" refers to personal attribute data such as a user's interests, concerns, and past behavioral history.

[1303] "Emotion data" refers to data that indicates the emotional state of the user, obtained based on voice analysis and facial expression analysis.

[1304] A "question" refers to a dialogue prompt that is generated based on the user's interests and current emotional state.

[1305] An "answer" refers to a response that a user enters in response to a question.

[1306] "Means for transmitting, receiving, and analyzing data in real time" refers to technical means for instantly transmitting and receiving data using communication technology and for quickly analyzing it.

[1307] This system balances the eating speeds of all users when dining together, and provides a rich conversational experience by using an emotion engine. This system works by having users record their meal progress and emotion data using their own devices and then sending this data to a server.

[1308] Initial Setup

[1309] 1. User: Before starting a meal, the user inputs the "start meal" information into their device. The input information includes the start time, user ID, meal contents, past conversation log, and emotion data.

[1310] 2. Terminal: The terminal collects the input information and sends it to the server, using an Internet communication protocol such as HTTP.

[1311] 3. Server: Based on the received information, the server records each user's session in a database (e.g., MySQL) and starts the session.

[1312] Monitoring meal progress and emotional data

[1313] 1. User: While eating, the user inputs their eating progress and emotional data into the device. Emotional data is automatically acquired using the Google Cloud Speech-to-Text API and the Microsoft Azure Emotion API.

[1314] 2. Terminal: The terminal sends the user's input data and emotion data to the server at regular intervals (e.g., every 5 minutes).

[1315] Data Analysis and Anomaly Detection

[1316] 1. Server: The server analyzes the received data using Python's pandas library and Scikit-learn to quantify eating speed and emotional state.

[1317] 2. Server: Compares the data of all users, identifies users with abnormal eating speeds (e.g., significantly faster or slower), and analyzes their emotional state.

[1318] Question generation and presentation

[1319] 1. Server: Generates questions appropriate for users with abnormal eating speeds based on past conversations, user profiles, and emotional data. For example, if the generative AI model determines that a user loves traveling and is feeling stressed, it generates a question such as, "Where was the most relaxing trip you've taken recently?"

[1320] 2. Server: Sends the generated question to the user's device.

[1321] Answering questions and analyzing

[1322] 1. Terminal: The terminal presents the received question to the user. The user answers the question and enters it into the terminal. For example, the answer might be, "I went to Hokkaido last month."

[1323] 2. Terminal: The terminal sends the answer to the server.

[1324] 3. Server: The server analyzes the user's answers and generates additional questions or comments to continue the conversation, such as "What kind of food did you like in Hokkaido?"

[1325] Adjusting meal timing

[1326] 1. Server: The server monitors the eating speed and emotional data of all users in real time and adjusts everyone's eating timing and emotional state by asking questions appropriately.

[1327] 2. All users: When finishing a meal, they input the "Done" information into their terminal. The terminal sends this information to the server.

[1328] 3. Server: The server checks the completion of all users and stores the session data in a database, for example, recording the completion time and final emotion data.

[1329] Specific examples

[1330] 1. User A and User B: Simultaneously input "start eating" into their devices, and the devices send the start information and emotion data to the server.

[1331] 2. Server: Initiates sessions and records information about each user.

[1332] 3. User A: Enters "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[1333] 4. Server: Analyzes the received data and calculates User A's eating speed and emotional state.

[1334] 5. Server: Compares the data of all users and determines that User A eats quickly and is stressed.

[1335] 6. Server: Determines that User A loves to travel and generates the question, "Where was your last relaxing trip?"

[1336] 7. Server: Sends the question to User A's device.

[1337] 8. User A: "I went to Hokkaido last month." The device sends the answer to the server.

[1338] 9. Server: Analyzes the answer and generates the next question: "What kind of food was delicious in Hokkaido?". Sends the next question to User A's device.

[1339] 10. Server: Continue asking questions until User A's eating speed and emotions are balanced with the other users.

[1340] 11. All users: When you enter "Done eating" into the terminal, the terminal will send that information to the server.

[1341] 12. Server: Confirm that all users have finished and end the session.

[1342] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[1343] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1344] Step 1:

[1345] Before starting a meal, the user enters the "start meal" information into the device. At that time, the user enters the user ID, start time, meal contents, past conversation log, and emotion data. Specifically, the data is entered into an input form using a smartphone app or tablet. The entered data is temporarily stored in the device's memory.

[1346] Input: User ID, start time, meal contents, past conversation log, emotion data

[1347] Output: User data in device memory

[1348] Step 2:

[1349] The device collects the entered user information and sends it to the server. The HTTP protocol is used for transmission, and the data is packetized in JSON format and sent. The transmitted data is received by the server-side API and stored in a database.

[1350] Input: User data in the terminal memory

[1351] Output: User information stored in a database on the server

[1352] Step 3:

[1353] The server starts a session for each user based on the received information. The received data (user ID, start time, meal contents, etc.) is recorded in a database, and the session start time is saved as a timestamp.

[1354] Input: User information on the server

[1355] Output: Session data recorded in a database

[1356] Step 4:

[1357] The user inputs the amount of food eaten and their progress during the meal into the device. Emotion data is also automatically acquired using an emotion engine. For example, the user's voice is analyzed using the Google Cloud Speech-to-Text API to evaluate their stress level.

[1358] Input: User's food intake, progress, and voice data

[1359] Output: New dataset in device (progress and emotion ratings)

[1360] Step 5:

[1361] The device sends the user's progress and emotion data to the server at regular intervals (for example, every 5 minutes). This communication also uses the HTTP protocol, and the data is sent to the server in JSON format. The server then formats the received data appropriately and adds it to the database.

[1362] Input: New progress and emotion data

[1363] Output: Updated user database on the server.

[1364] Step 6:

[1365] The server analyzes the received data using Python's pandas library and Scikit-learn. Specifically, it calculates eating speed based on the amount of food eaten and progress, and runs algorithms to evaluate stress levels and relaxation state based on emotional data.

[1366] Input: Updated user database

[1367] Output: Eating speed and emotional state of each user

[1368] Step 7:

[1369] The server compares the data of all users and detects abnormal eating speeds. For example, it identifies users whose eating speeds deviate significantly from the average speed and also checks their emotional state. If an abnormality is detected, it flags the data as abnormal.

[1370] Input: Eating speed and emotional state of each user

[1371] Output: A list of users for whom anomalies were detected

[1372] Step 8:

[1373] The server generates appropriate questions for users with abnormal eating speeds. Using past conversations, user profiles, and emotional data, the generative AI model inputs prompt sentences to generate specific questions. For example, it generates a question like, "Where was your most recent relaxing trip?"

[1374] Input: A list of users for whom anomalies were detected, and a prompt for the AI ​​model

[1375] Output: Generated questions

[1376] Step 9:

[1377] The server sends the generated question to the user's device. The server uses the HTTP protocol to send the generated question in JSON format to the device. The device receives it and displays it on the user interface.

[1378] Input: Generated Question

[1379] Output: The question displayed on the user's terminal

[1380] Step 10:

[1381] The user inputs an answer to the question displayed on the device. For example, the answer might be "I went to Hokkaido last month." The answer is input using the keyboard on the smartphone or tablet. The answer data is saved in the device's memory.

[1382] Input: User's answer

[1383] Output: Answer data stored in the device's memory

[1384] Step 11:

[1385] The device sends the response data to the server using the HTTP protocol, packetizing the response data in JSON format and sending it to the server. The server stores the received response data in a database and analyzes it.

[1386] Input: Answer data stored in the device's memory

[1387] Output: Answer data saved on the server

[1388] Step 12:

[1389] The server analyzes the received response data, for example by applying a text analysis algorithm to generate the next question from what the user has said about their travel destination, and then uses a generative AI model to generate the next question.

[1390] Input: Answer data saved on the server

[1391] Output: The following questions are generated:

[1392] Step 13:

[1393] The server generates the next question and sends it to the user's device. The server then packets the question in JSON format and sends it using the HTTP protocol. The device receives it and displays it on the user interface.

[1394] Input: The next generated question

[1395] Output: The following question displayed on the user's screen:

[1396] Step 14:

[1397] The server monitors all users' eating speed and emotional data in real time and continues to ask questions accordingly, forming a feedback loop to adjust the eating timing and emotional state of all users.

[1398] Input: Eating speed and emotion data of all users

[1399] Output: Feedback loop of adjusted questions

[1400] Step 15:

[1401] When all users have finished eating, they input "meal finished" information into their terminal. The terminal then sends the end information to the server. The server updates the session data based on the received end information and confirms that all users have finished.

[1402] Input: User exit information

[1403] Output: Finished data saved on the server

[1404] Step 16:

[1405] The server confirms the end of each user session and stores the session data in a database, including the end time and final emotion data.

[1406] Input: Finished data sent to the server

[1407] Output: Session termination data stored in the database

[1408] (Application example 2)

[1409] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1410] In today's world, when multiple users eat together, imbalances in eating speed can cause some users to feel isolated or conversations to be interrupted. This problem significantly reduces the enjoyment and satisfaction of the meal and requires special consideration. Furthermore, because emotional fluctuations often affect the pace of eating, a system that takes this into account is needed. However, current systems do not provide a system that can reflect users' emotions in real time and generate appropriate conversations.

[1411] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1412] In this invention, the server includes a means for periodically recording the progress of the meal using a smart device, a means for acquiring the user's emotion data using an emotion engine, and a means for generating questions using a generative AI model, which enables the server to monitor the user's eating speed and emotions in real time and balance the eating speed and conversation by asking appropriate questions.

[1413] A "smart device" refers to a device that has internet connectivity and is equipped with various sensors and displays.

[1414] An "emotion engine" refers to a software or hardware system that analyzes a user's facial expressions and voice and recognizes their psychological state and emotions.

[1415] A "generative AI model" refers to a system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[1416] A "server" refers to a computer system that centrally manages, analyzes, and processes data sent from multiple users.

[1417] "Means for generating questions" refers to algorithms or programs for automatically generating appropriate questions based on past conversation content and user profiles.

[1418] "Eating speed" refers to the progress of a user's eating within a specific period of time.

[1419] "Means for analyzing eating speed" refers to a program or algorithm that evaluates and analyzes a user's eating speed based on collected data.

[1420] This invention is a system that uses smart devices to record and transmit users' meal progress and emotional data, and a server analyzes this data and generates appropriate questions to adjust the balance between eating speed and conversation for all users. This system enriches the dining experience among users and is expected to be used particularly in brick-and-mortar establishments such as restaurants.

[1421] The system configuration includes the following hardware and software:

[1422] Hardware

[1423] Smart device: A device with internet connectivity and equipped with various sensors and displays. Here, we use smart glasses as an example.

[1424] Sensors: To keep track of your eating progress, sensors built into the smart glasses measure the amount of food you eat.

[1425] Server: A computer system that centrally manages, analyzes, and processes data sent from multiple users.

[1426] software

[1427] Emotion engine: An engine for analyzing the user's facial expressions and voice to obtain emotional data.

[1428] Generative AI model: A system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[1429] ServerCommunicator: A module that is responsible for communicating with the server and sending and receiving data.

[1430] QuestionGenerator: A module that generates appropriate questions based on emotional data, user profiles, and conversation history.

[1431] SmartGlassesDisplay: Software with the ability to display questions on the screen of smart glasses.

[1432] System Operation

[1433] In this system, the user puts on the smart glasses before starting a meal and inputs the "start eating" information. The smart glasses then send the start time, user ID, meal content, past conversation log, and emotional data to the server. The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. This data is then sent to the server at regular intervals, where it is analyzed and used to monitor eating speed and emotional state.

[1434] The server then compares each user's eating speed, and if an abnormal eating speed is detected, the AI ​​model generates appropriate questions based on the user's profile and emotional data. The generated questions are displayed on the smart glasses' display, and the user answers them. The answers are then sent back to the server, which generates additional questions to further encourage conversation.

[1435] Specific examples

[1436] For example, if the user is recognized as feeling stressed, the following example prompt sentences are fed into the generative AI model:

[1437] A user is feeling stressed. Generate a question for User A, who loves to travel: "Where was the most relaxing trip you've taken recently?"

[1438] The generated question looks like this:

[1439] "Where's the last relaxing trip you went to?"

[1440] This question is displayed on the smart glasses display, and the user responds, "I went to Hokkaido last month." This response is sent to the server, which then generates the next question, such as, "What kind of food did you like in Hokkaido?" Through this process, the eating speed and emotional state of all users are balanced.

[1441] This will enrich the dining experience between users and allow for mealtimes and conversations that are satisfying for everyone.

[1442] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1443] Step 1:

[1444] Before starting a meal, the user puts on the smart glasses and inputs the "start eating" information. The user operates the interface of the smart glasses to input the start time, user ID, and meal contents. This information is sent to the server along with the start trigger.

[1445] Input: Start time, User ID, Meal details

[1446] Output: Start trigger and user information sent to the server

[1447] Step 2:

[1448] The server starts a session based on the received start information, user ID, and meal details. A session ID is generated and recorded in the database. Each user's progress is managed based on this session ID.

[1449] Input: Start trigger and user information

[1450] Output: Generate a session ID and record it in the database

[1451] Step 3:

[1452] The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. The camera and microphone built into the smart glasses capture facial expressions and voice, which are then sent to the emotion engine for analysis.

[1453] Input: User facial expression images, voice data

[1454] Output: Obtaining emotion data

[1455] Step 4:

[1456] Sensors in the smart glasses periodically measure the amount of food consumed and send the data to a server. For example, a weight sensor measures the amount of food consumed and sends it as progress data.

[1457] Input: Food quantity data

[1458] Output: Getting and sending progress data

[1459] Step 5:

[1460] The server analyzes the received emotion data and progress data to calculate the user's eating speed. It also compares the eating speeds of multiple users to check for abnormal speeds.

[1461] Input: Emotion data, progress data

[1462] Output: Detection results of eating speed and abnormal speed

[1463] Step 6:

[1464] If the server detects abnormal eating speed, it uses a generative AI model to generate appropriate questions based on the user's profile and emotional data. The generative AI model generates questions based on the prompt text, and the server sends the questions to the smart glasses.

[1465] Input: User profile, emotion data, prompt sentence

[1466] Output: Generated questions

[1467] Step 7:

[1468] A question is displayed on the smart glasses display, and the user responds to the question by voice or text, which is then sent back to the server.

[1469] Input: Generated Question

[1470] Output: User's answer

[1471] Step 8:

[1472] The server analyzes the answers it receives and generates further questions as needed, repeating this process until all users' eating speeds and conversations are balanced.

[1473] Input: User's answer

[1474] Output: Further generated questions

[1475] Step 9:

[1476] All users input the "finished meal" information into their smart glasses. This information is sent to the server, which ends the session and saves all data in a database.

[1477] Input: Meal end information

[1478] Output: Save session data and end session

[1479] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1480] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1481] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1482] [Fourth embodiment]

[1483] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1484] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1485] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1486] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1487] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1488] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1489] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1490] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1491] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1492] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1493] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1494] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1495] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1496] System Overview

[1497] This invention is a system for balancing the eating speeds of all users when multiple users are eating together. The system operates by having users record and transmit their meal progress using their own devices. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[1498] Initial Setup

[1499] 1. Before starting a meal, the user inputs the "start eating" information into their device. The device then sends this information to the server.

[1500] 2. Based on the information received, the server records each user's session in the database and starts the session.

[1501] Monitoring and sending meal progress

[1502] 3. While eating, the user periodically inputs the amount eaten and progress into the device. The device then transmits this data to the server at regular intervals.

[1503] 4. The server analyzes the received data and calculates each user's eating speed.

[1504] Data Analysis and Anomaly Detection

[1505] 5. The server compares the eating speeds of all users and detects abnormal eating speeds of a particular user if the eating speed of that user is significantly faster or slower.

[1506] Question generation and presentation

[1507] 6. The server generates questions appropriate for a user with an abnormal eating speed based on past conversations and the user profile. For example, for a user who likes traveling, it generates a question such as "Where have you traveled recently?"

[1508] 7. The server sends the generated question to the user's device.

[1509] Answering questions and analyzing

[1510] 8. The device presents the received question to the user, who answers it. The answer is then sent back to the server via the device.

[1511] 9. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1512] Adjusting meal timing

[1513] 10. The server monitors the eating speed of all users in real time and continues to ask questions as appropriate to align everyone's eating timing.

[1514] 11. When all users have finished eating, they input the "Dinner finished" information into their terminals. The terminals then send this information to the server.

[1515] 12. The server confirms that all users have finished and ends the session.

[1516] Specific examples

[1517] Initial Setup

[1518] When user A and user B simultaneously input "start meal" into the terminal, the terminal sends the start information to the server.

[1519] The server initiates the session and records information for each user.

[1520] Monitoring meal progress

[1521] User A enters "I ate three bites of rice" into the device. The device sends this information to the server.

[1522] The server analyzes the received data and calculates User A's eating speed.

[1523] Data analysis

[1524] The server compares the data of all users and determines that User A eats quickly.

[1525] Question generation and presentation

[1526] The server determines that user A likes to travel and generates the question, "Where have you traveled recently?"

[1527] The server sends the question to User A's terminal.

[1528] Answering questions and analyzing

[1529] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[1530] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[1531] The server sends the following question to User A's terminal:

[1532] Adjusting meal timing

[1533] The server continues asking questions until User A's eating speed is in balance with the other users.

[1534] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[1535] The server confirms that all users have finished and ends the session.

[1536] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversations, thereby providing a satisfying dining experience for everyone.

[1537] The processing flow will be explained below.

[1538] Step 1:

[1539] Before starting a meal, the user inputs "Start meal" into the terminal, which then sends basic information such as the start time, user ID, and meal contents to the server.

[1540] Step 2:

[1541] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[1542] Step 3:

[1543] As the user continues eating, they periodically input the amount eaten and their progress into the terminal.

[1544] Step 4:

[1545] The terminal transmits the user's input data to the server at regular intervals.

[1546] Step 5:

[1547] The server analyzes the received data and calculates each user's eating speed.

[1548] Step 6:

[1549] The server compares the eating speeds of all users, and if a particular user's eating speed is significantly faster or slower than other users, the abnormal eating speed is detected.

[1550] Step 7:

[1551] The server refers to past conversations and user profiles to generate questions suited to users with abnormal eating speeds.

[1552] Step 8:

[1553] The server generates a query and sends it to the user's terminal.

[1554] Step 9:

[1555] The terminal displays the received question to the user.

[1556] Step 10:

[1557] The user answers the displayed questions and enters them into the terminal.

[1558] Step 11:

[1559] The terminal sends the user's answer to the server.

[1560] Step 12:

[1561] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1562] Step 13:

[1563] The server generates additional questions and comments and sends them to the user's device.

[1564] Step 14:

[1565] The device will then prompt the user again with any additional questions or comments.

[1566] Step 15:

[1567] The server monitors the eating speed of all users in real time and adjusts questions and comments to align the eating timing of all users.

[1568] Step 16:

[1569] When all users input "Done eating," the terminal sends the information to the server.

[1570] Step 17:

[1571] The server confirms that all users have finished and saves the session data to the database.

[1572] Step 18:

[1573] The server sends a "meal end" notification to all terminals and ends the session.

[1574] Example 1

[1575] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1576] When multiple users eat together, variations in the eating speeds of each user can disrupt the rhythm of conversation, or some users may eat too fast or too slow. This makes it difficult to provide a dining experience that is satisfying for everyone. To solve this problem, the present invention aims to balance the eating speeds of users and provide a dining experience that is satisfying for everyone through appropriate conversation.

[1577] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1578] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating questions using a generative AI model and sending the questions to the user's terminal, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, and means for adjusting the timing to keep the eating speeds of all users balanced. This makes it possible to monitor each user's eating speed in real time, generate appropriate questions to promote conversation, and keep everyone's eating speeds balanced.

[1579] "User Information" refers to personal attributes and profile information, such as a user's name, interests, and concerns.

[1580] "Meal progress" refers to information indicating the amount of food the user is currently eating and the progress of the meal.

[1581] "Meal speed" refers to the speed at which a meal progresses, calculated based on the amount of food eaten by a user within a certain period of time.

[1582] "Abnormal eating speed" refers to an eating speed that is significantly faster or slower than the average eating speed of other users.

[1583] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate questions that are appropriate for the user.

[1584] "Question" refers to an inquiry or query presented to a user to facilitate relevant conversation.

[1585] An "answer" refers to a reply or response that a user enters to a question.

[1586] "Adjusting the timing" refers to generating questions and continuing the conversation at the right moment to balance the eating speed of all users.

[1587] The present invention relates to a system for balancing the eating speeds of all users when a group of users is eating together. How to implement this system will be described in detail below.

[1588] System Configuration

[1589] This system consists of a device used by the user, a server that processes the data, and an algorithm that uses a generative AI model to generate appropriate questions. The device is a computing device such as a smartphone or tablet, and the server is often installed on the cloud.

[1590] Hardware and software used

[1591] Terminal: A device that a user operates, such as a smartphone, tablet, or PC.

[1592] Server: Hardware for processing and storing data, such as a cloud server or on-premise server.

[1593] Generative AI models: Generate questions using artificial intelligence models such as OpenAI's GPT series.

[1594] Data Processing and Data Calculation

[1595] 1. Data entered by the user:

[1596] Users input information about the start, progress, and end of their meal into the terminal, which allows the server to analyze each user's eating speed.

[1597] 2. Data transmission and analysis:

[1598] The device periodically sends the user's input data to the server, which analyzes the received data and calculates each user's eating speed.

[1599] 3. Detecting abnormal eating speed:

[1600] The server compares the eating speeds of all users and detects an abnormality when a particular user's eating speed is significantly faster or slower.

[1601] 4. Question generation:

[1602] If an anomaly is detected, the server generates appropriate questions using a generative AI model based on past conversations and the user profile, such as OpenAI's GPT series.

[1603] 5. Presenting and answering questions to the user:

[1604] The server generates questions and sends them to the user's device, which then presents them to the user. The user answers the questions, and the answers are sent back to the server, allowing the server to generate further questions and continue the conversation.

[1605] Specific examples

[1606] Example of question generation

[1607] By inputting the following prompt sentence into the generative AI model, a question appropriate for the user can be generated.

[1608] Based on the following user information and conversation content, please generate the following questions for User A.

[1609] User Profile:

[1610] Name: User A

[1611] Interests: Travel, music, cooking

[1612] Recent conversations:

[1613] User A: "I went to Hokkaido last month."

[1614] Based on this prompt, the generative AI model generates a question such as, "What kind of food was delicious in Hokkaido?" This question is sent from the server to the user's device, prompting the user to respond.

[1615] The system monitors the eating speed of users and encourages conversation at appropriate times to provide a satisfying dining experience for everyone. Furthermore, by using a generative AI model, it is possible to dynamically generate questions suited to each user, ensuring smooth conversation.

[1616] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1617] Step 1:

[1618] Before starting a meal, the user enters the "start meal" information into the device. Specifically, the user taps the "start" button on the smartphone or tablet app. This input data includes the user ID and start time. The device then sends this data to the server.

[1619] Step 2:

[1620] The server starts each user's session based on the "start meal" information it receives. At this point, the server records data such as the user ID, start time, and session ID in a database. This makes it possible to track which user started eating and when.

[1621] Step 3:

[1622] While eating, the user periodically inputs their progress into the device. A specific action would be to enter something like "I ate three bites of rice" into an input form within the app. This input data includes the user ID, progress, and time of input. The device then sends this data to the server at regular intervals.

[1623] Step 4:

[1624] The server analyzes the received data and calculates each user's eating speed. Here, the server calculates the eating speed for each user and records the results in the database. For example, if user A eats 3 bites in 10 minutes, their eating speed is "10 minutes / 3 bites."

[1625] Step 5:

[1626] The server compares the eating speeds of all users and detects abnormal eating speeds. In this detection process, the server calculates the average eating speed of all users and determines that speeds that are significantly faster or slower than that are abnormal. Specifically, if user B's eating speed is significantly faster than the average, it is detected as abnormal.

[1627] Step 6:

[1628] The server uses the generative AI model to generate questions appropriate for users with abnormal eating speeds. The input data for the prompt sentences used for generation includes the user profile (e.g., interests and past conversations) and user ID. For example, for a user who likes to travel, the question generated is "Where have you traveled recently?"

[1629] Step 7:

[1630] The server sends the generated question to the user's device. The sent data includes the question content, user ID, and time of sending. The device then presents the received question to the user.

[1631] Step 8:

[1632] The user answers the questions through the device. Specifically, the user enters the answer into the input form on the device and taps the "Send" button. This input data includes the answer, user ID, and time of input. The device then sends the answer to the server.

[1633] Step 9:

[1634] The server analyzes the user's answers and generates additional questions as necessary. Here, the server analyzes the answers and generates questions to encourage further conversation. For example, the next question generated might be, "What kind of food was delicious in Hokkaido?" The generated question is then returned to step 7 and sent again.

[1635] Step 10:

[1636] All users enter their end of meal time into their device, which then sends this information to the server. Specifically, users tap the "End" button in the app. The data sent includes the user ID and end time.

[1637] Step 11:

[1638] The server confirms the end information of all users and ends the session. At this time, all users' meal data and conversation content are saved in a database and recorded for future analysis and improvement.

[1639] (Application example 1)

[1640] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1641] In conventional online dining experiences, users often eat at different speeds, resulting in a lack of unity between them. Communication can also be disrupted, leading to uneven mealtime progress. Furthermore, sharing and coordinating real-time meal plans in a virtual environment is difficult.

[1642] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1643] In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and user profiles, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for generating conversations based on user interests and concerns using an AI model, and means for monitoring and sharing users' eating experiences in real time in a virtual environment, thereby enabling adjustment of different eating speeds and smooth communication.

[1644] The "means for inputting user information" refers to an interface that allows a user to input their own information into the system, and can include basic information such as name, age, and hobbies.

[1645] The "means for periodically recording the progress of a meal" is a means for recording the current progress of a meal at time intervals as the user eats, and for collecting data such as the amount and speed of eating.

[1646] The "means for analyzing eating speed and detecting abnormal eating speed" analyzes the progress data entered by the user and detects eating speeds that are too fast or too slow compared to other users based on preset criteria.

[1647] The "means for generating appropriate questions based on past conversation content and user profile" is a means for generating appropriate questions to interest the user based on previous conversations and the user's registration information.

[1648] The "means for presenting the generated question to the user and receiving an answer" is a means for displaying the generated question on the user's device and receiving an answer from the user.

[1649] The "means for analyzing the received answers and generating further questions" is a means for analyzing the answers from the user and generating new questions based on them.

[1650] The "means for adjusting timing to balance the eating speeds of all users" is a means for monitoring the eating progress of all users in real time and adjusting each user's eating speed to keep it balanced.

[1651] "Means for generating conversations based on user interests and concerns using an AI model" refers to using a generative AI model to automatically generate relevant conversations based on the interests and concerns derived from a user's profile and past comments.

[1652] "Means for monitoring and sharing users' meal experiences in real time in a virtual environment" refers to a method for monitoring a user's meal progress in real time through a virtual environment and sharing that information with other users, thereby creating a sense of unity.

[1653] The present invention provides a system for promoting enjoyable communication while balancing the eating speeds of all users when multiple users share a meal in a virtual environment. Specific embodiments of the system are described below.

[1654] Hardware and software used

[1655] Hardware: Smartphone, Head-Mounted Display (HMD)

[1656] Software: Servers for real-time data analysis, AI models for chatbots (e.g., OpenAI GPT-4), database management systems (e.g., MySQL)

[1657] System Overview

[1658] The system includes the following means:

[1659] 1. Means for inputting user information: Before starting a meal, each user inputs their information (name, age, hobbies, etc.) into the system via their smartphone or HMD. This information is sent to the server, and an individual session begins.

[1660] 2. A means to periodically record the progress of the meal: During the meal, the user inputs their progress (e.g., the amount eaten) via their smartphone or HMD. This information is sent to the server at regular intervals.

[1661] 3. Means for analyzing eating speed and detecting abnormal eating speed: The server analyzes the received data in real time, calculates each user's eating speed, and detects abnormally fast or slow eating speeds based on set criteria.

[1662] 4. Generating appropriate questions based on past conversations and user profiles: Using an AI model (e.g., GPT-4), appropriate questions are generated based on the user's profile and past conversations. For example, for a user who likes traveling, the system generates the question, "Where have you traveled recently?"

[1663] 5. A means for presenting the generated questions to the user and receiving answers: The generated questions are presented in real time to the user's smartphone or HMD, and the user answers the questions. The answers are then sent back to the server.

[1664] 6. Means for analyzing the received answers and generating further questions: The server analyzes the user's answers using an AI model and generates further questions to continue the conversation, thereby allowing the conversation to continue uninterrupted.

[1665] 7. A means for adjusting timing to balance the eating speed of all users: The server monitors the eating speed of all users and generates questions and comments as appropriate to keep the balance.

[1666] 8. Using an AI model to generate conversations based on the user's interests: Using a generative AI model, a conversation can be conducted based on the user's interests. For example, if a user enjoys reading, the conversation can be asked questions such as, "What books have you read recently?"

[1667] 9. A means of monitoring and sharing users' meal experiences in a virtual environment in real time: The system monitors and shares users' meal progress in real time, allowing everyone to enjoy the same experience.

[1668] Specific examples

[1669] Example 1:

[1670] Prompt: "Tell me about your most recent travel destination."

[1671] User A: "I went to Hokkaido last month."

[1672] Post-analysis question: "What kind of food did you like in Hokkaido?"

[1673] In this way, the AI ​​model is used to generate conversations based on the user's interests, allowing for adjustment of eating speed and smooth communication.

[1674] Example 2:

[1675] Prompt: "Tell me about the last book you read."

[1676] User B: "I was reading a mystery novel."

[1677] Post-analysis questions: "What was the title of the mystery novel? What part did you like?"

[1678] Specific prompts for the AI ​​model:

[1679] The user has started their meal progress. To encourage conversation with other users and adjust the eating speed, generate appropriate questions from the following input data. The user's recent interests are "travel" and "reading." Example: "Tell me about the places you've traveled recently and the books you've read recently."

[1680] In this way, the dining experience in a virtual environment can be tailored to be more enjoyable.

[1681] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1682] Step 1:

[1683] Before a user starts eating, they use their smartphone or HMD to enter user information (such as name, age, and hobbies) into the system. This information is sent from the device to the server. The server records the received user information in a database and starts the session. Input data: User information (such as name, age, and hobbies). Output data: User information recorded in the database and a session start notification.

[1684] Step 2:

[1685] While eating, the user periodically inputs their progress (e.g., amount eaten) into the device. The input data is sent from the device to the server at regular intervals. The server records the received progress data in a database and prepares it for analysis in real time. Input data: Progress (e.g., amount eaten). Output data: Progress data recorded in the database in preparation for analysis.

[1686] Step 3:

[1687] The server analyzes the received progress data in real time and calculates each user's eating speed. As a result of the analysis, it detects users whose eating speed is abnormally fast or slow compared to the set criteria. Input data: progress data. Output data: detection results of each user's eating speed and abnormal speed.

[1688] Step 4:

[1689] The server uses a generative AI model (e.g., GPT-4) to generate appropriate questions for users with abnormal eating speeds based on past conversations and user profiles. Input data: past conversations, user profiles. Output data: generated questions.

[1690] Step 5:

[1691] The generated questions are presented from the server to the user's smartphone or HMD. The user answers the presented questions. Input data: Generated questions. Output data: User answers.

[1692] Step 6:

[1693] The server analyzes the answers received from the user using an AI model and generates further questions, allowing the conversation to continue uninterrupted. Input data: User answers. Output data: Additional questions.

[1694] Step 7:

[1695] The server monitors the eating speed of all users in real time and generates questions and comments as needed to maintain balance. Input data: Progress and eating speed data of all users. Output data: Additional questions and comments for adjustment.

[1696] Step 8:

[1697] The server uses a generative AI model to engage in conversations based on the user's interests. For example, if a user has a hobby of reading, it will ask questions such as, "What books have you read recently?" Input data: User profile, past conversations. Output data: Conversation content based on interests.

[1698] Step 9:

[1699] The system monitors and shares the meal progress of users in real time, allowing everyone to enjoy the same experience and fostering a sense of unity. Input data: Progress data of all users. Output data: Shared progress data and real-time sense of unity.

[1700] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1701] System Overview

[1702] This invention is a system for balancing the eating speeds of all users when dining together, and provides a richer conversation experience by combining it with an emotion engine that recognizes users' emotions. This system works by having users record their meal progress and emotion data using their own devices and send it to a server. The server analyzes this data, generates appropriate questions, and presents them to users to promote conversation and adjust the eating speeds of all users.

[1703] Initial Setup

[1704] 1. Before starting a meal, the user inputs the "start eating" information into the device. The device then sends the start time, user ID, meal contents, past conversation log, and emotion data to the server.

[1705] 2. Based on the information received, the server records each user's session in the database and starts the session.

[1706] Monitoring and sending meal progress and emotion data

[1707] 3. While eating, the user periodically inputs the amount eaten, progress, and emotional data into the device. Emotional data can be automatically acquired using emotion engines such as voice analysis and facial expression analysis.

[1708] 4. The device sends the user's input data and emotion data to the server at regular intervals.

[1709] Data Analysis and Anomaly Detection

[1710] 5. The server analyzes the received data and calculates each user's eating speed and emotional state.

[1711] 6. The server compares the eating speeds of all users and detects abnormal eating speeds of specific users if they are significantly faster or slower. In addition, it analyzes the user's emotional state and obtains emotional indicators such as stress and relaxation.

[1712] Question generation and presentation

[1713] 7. The server generates questions appropriate for users with abnormal eating speed based on past conversations, user profiles, and emotional data. For example, for a user who loves traveling and is feeling stressed, the server generates a question such as, "Where was your most recent relaxing trip?"

[1714] 8. The server sends the generated question to the user's device.

[1715] Answering questions and analyzing

[1716] 9. The device presents the received question to the user, who answers it. The answer is sent to the server via the device.

[1717] 10. The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1718] Adjusting meal timing

[1719] 11. The server monitors the eating speed and emotional data of all users in real time and continues to ask questions as appropriate to adjust everyone's eating timing and emotional state.

[1720] 12. When all users have finished eating, they enter the "Drinking is over" information into their terminals. The terminals then send this information to the server.

[1721] 13. The server confirms that all users have exited and saves the session data to the database.

[1722] Specific examples

[1723] Initial Setup

[1724] When user A and user B simultaneously input "start eating" into the terminal, the terminal sends the start information and emotion data to the server.

[1725] The server initiates the session and records information for each user.

[1726] Monitoring meal progress and emotional data

[1727] User A inputs "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[1728] The server analyzes the received data and calculates User A's eating speed and emotional state.

[1729] Data analysis

[1730] The server compares the data of all users and determines that User A eats quickly and is feeling stressed.

[1731] Question generation and presentation

[1732] The server determines that User A likes to travel and generates the question, "Where is the most relaxing travel destination you have recently visited?"

[1733] The server sends the question to User A's terminal.

[1734] Answering questions and analyzing

[1735] User A enters "I went to Hokkaido last month" into the terminal. The terminal sends the answer to the server.

[1736] The server analyzes the answer and generates the next question: "What kind of food did you like in Hokkaido?"

[1737] The server sends the following question to User A's terminal:

[1738] Meal timing and emotion regulation

[1739] The server continues asking questions until User A's eating speed and emotions are in balance with other users.

[1740] Finally, when all users input "meal finished" into their terminals, the terminals send this information to the server.

[1741] The server confirms that all users have finished and ends the session.

[1742] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[1743] The processing flow will be explained below.

[1744] Step 1:

[1745] Before starting a meal, the user enters "Start Meal" into the terminal. The user then enters basic information such as ID and meal contents. The terminal then sends this user information along with the start time to the server.

[1746] Step 2:

[1747] Based on the data received by the server, the session of each user is recorded in the database and the session is started.

[1748] Step 3:

[1749] As the user eats, they periodically input information about the amount eaten and their progress into the device. Emotional data is also collected at the same time, using an emotion engine that analyzes voice and facial expressions.

[1750] Step 4:

[1751] The terminal transmits the user's input data and emotion data to the server at regular intervals.

[1752] Step 5:

[1753] The server analyzes the received data and calculates each user's eating speed and emotional state.

[1754] Step 6:

[1755] The server compares the eating speeds of all users and detects abnormal eating speeds. If an abnormal eating speed is detected, the server also checks the emotional state of the user.

[1756] Step 7:

[1757] The server generates appropriate questions based on past conversations, user profiles, and emotional data, such as "What are some of the most relaxing travel destinations you've visited recently?" for a stressed travel-loving user.

[1758] Step 8:

[1759] The server generates a query and sends it to the user's terminal.

[1760] Step 9:

[1761] The terminal displays the received question to the user.

[1762] Step 10:

[1763] The user answers the displayed questions and enters them into the terminal.

[1764] Step 11:

[1765] The terminal sends the user's answer to the server.

[1766] Step 12:

[1767] The server analyzes the user's answers and generates follow-up questions or comments to continue the conversation.

[1768] Step 13:

[1769] The server generates additional questions and comments and sends them to the user's device.

[1770] Step 14:

[1771] The device will then prompt the user again with any additional questions or comments.

[1772] Step 15:

[1773] The server monitors all users' eating speed and emotional data in real time, and adjusts questions and comments to align the eating timing and emotional state of all users.

[1774] Step 16:

[1775] When all users input "Done eating," the terminal sends the information to the server.

[1776] Step 17:

[1777] The server confirms that all users have finished and saves the session data to the database.

[1778] Step 18:

[1779] The server sends a "meal end" notification to all terminals and ends the session.

[1780] Example 2

[1781] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1782] In today's world, when multiple users eat together, differences in eating speed can create imbalances, making it difficult for everyone to have a satisfying dining experience. Furthermore, systems lack the ability to generate questions at the right time to facilitate smooth conversation, or to communicate in a way that takes users' emotions into account. There is a need to solve these problems and provide a richer shared dining experience.

[1783] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user information before the user starts eating, means for periodically recording the progress of the meal, means for analyzing eating speed and detecting abnormal eating speed, means for generating appropriate questions based on past conversation content and a user profile, means for presenting the generated questions to the user and receiving answers, means for analyzing the received answers and generating further questions, means for adjusting the timing to balance the eating speeds of all users, means for acquiring emotional data using voice analysis or facial expression analysis, means for adjusting questions based on the acquired emotional data, and means for transmitting, receiving, and analyzing data in real time. This makes it possible to generate questions that take emotions into consideration while maintaining a balance in eating speeds among users, thereby enriching the conversation experience.

[1784] "User information" refers to information including the user ID, start time, meal contents, past conversation log, and emotion data.

[1785] "Meal progress" refers to data about the amount eaten and progress recorded by the user during the meal.

[1786] "Eating speed" is an index that measures and represents the speed at which a user eats.

[1787] An "abnormal eating speed" is an eating speed that is judged to be significantly fast or slow when compared with the eating speeds of all users.

[1788] "Past conversation content" refers to the history of interactions between users recorded in previous conversation sessions.

[1789] A "user profile" refers to personal attribute data such as a user's interests, concerns, and past behavioral history.

[1790] "Emotion data" refers to data that indicates the emotional state of the user, obtained based on voice analysis and facial expression analysis.

[1791] A "question" refers to a dialogue prompt that is generated based on the user's interests and current emotional state.

[1792] An "answer" refers to a response that a user enters in response to a question.

[1793] "Means for transmitting, receiving, and analyzing data in real time" refers to technical means for instantly transmitting and receiving data using communication technology and for quickly analyzing it.

[1794] This system balances the eating speeds of all users when dining together, and provides a rich conversational experience by using an emotion engine. This system works by having users record their meal progress and emotion data using their own devices and then sending this data to a server.

[1795] Initial Setup

[1796] 1. User: Before starting a meal, the user inputs the "start meal" information into their device. The input information includes the start time, user ID, meal contents, past conversation log, and emotion data.

[1797] 2. Terminal: The terminal collects the input information and sends it to the server, using an Internet communication protocol such as HTTP.

[1798] 3. Server: Based on the received information, the server records each user's session in a database (e.g., MySQL) and starts the session.

[1799] Monitoring meal progress and emotional data

[1800] 1. User: While eating, the user inputs their eating progress and emotional data into the device. Emotional data is automatically acquired using the Google Cloud Speech-to-Text API and the Microsoft Azure Emotion API.

[1801] 2. Terminal: The terminal sends the user's input data and emotion data to the server at regular intervals (e.g., every 5 minutes).

[1802] Data Analysis and Anomaly Detection

[1803] 1. Server: The server analyzes the received data using Python's pandas library and Scikit-learn to quantify eating speed and emotional state.

[1804] 2. Server: Compares the data of all users, identifies users with abnormal eating speeds (e.g., significantly faster or slower), and analyzes their emotional state.

[1805] Question generation and presentation

[1806] 1. Server: Generates questions appropriate for users with abnormal eating speeds based on past conversations, user profiles, and emotional data. For example, if the generative AI model determines that a user loves traveling and is feeling stressed, it generates a question such as, "Where was the most relaxing trip you've taken recently?"

[1807] 2. Server: Sends the generated question to the user's device.

[1808] Answering questions and analyzing

[1809] 1. Terminal: The terminal presents the received question to the user. The user answers the question and enters it into the terminal. For example, the answer might be, "I went to Hokkaido last month."

[1810] 2. Terminal: The terminal sends the answer to the server.

[1811] 3. Server: The server analyzes the user's answers and generates additional questions or comments to continue the conversation, such as "What kind of food did you like in Hokkaido?"

[1812] Adjusting meal timing

[1813] 1. Server: The server monitors the eating speed and emotional data of all users in real time and adjusts everyone's eating timing and emotional state by asking questions appropriately.

[1814] 2. All users: When finishing a meal, they input the "Done" information into their terminal. The terminal sends this information to the server.

[1815] 3. Server: The server checks the completion of all users and stores the session data in a database, for example, recording the completion time and final emotion data.

[1816] Specific examples

[1817] 1. User A and User B: Simultaneously input "start eating" into their devices, and the devices send the start information and emotion data to the server.

[1818] 2. Server: Initiates sessions and records information about each user.

[1819] 3. User A: Enters "I ate three bites of rice" into the device. The emotion engine analyzes User A's voice and determines that he or she is feeling stressed. The device then sends this information to the server.

[1820] 4. Server: Analyzes the received data and calculates User A's eating speed and emotional state.

[1821] 5. Server: Compares the data of all users and determines that User A eats quickly and is stressed.

[1822] 6. Server: Determines that User A loves to travel and generates the question, "Where was your last relaxing trip?"

[1823] 7. Server: Sends the question to User A's device.

[1824] 8. User A: "I went to Hokkaido last month." The device sends the answer to the server.

[1825] 9. Server: Analyzes the answer and generates the next question: "What kind of food was delicious in Hokkaido?". Sends the next question to User A's device.

[1826] 10. Server: Continue asking questions until User A's eating speed and emotions are balanced with the other users.

[1827] 11. All users: When you enter "Done eating" into the terminal, the terminal will send that information to the server.

[1828] 12. Server: Confirm that all users have finished and end the session.

[1829] In this way, the present invention monitors the eating speed of users and adjusts the progress through appropriate conversation and emotionally reflective questions, thereby providing a satisfying dining experience for everyone.

[1830] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1831] Step 1:

[1832] Before starting a meal, the user enters the "start meal" information into the device. At that time, the user enters the user ID, start time, meal contents, past conversation log, and emotion data. Specifically, the data is entered into an input form using a smartphone app or tablet. The entered data is temporarily stored in the device's memory.

[1833] Input: User ID, start time, meal contents, past conversation log, emotion data

[1834] Output: User data in device memory

[1835] Step 2:

[1836] The device collects the entered user information and sends it to the server. The HTTP protocol is used for transmission, and the data is packetized in JSON format and sent. The transmitted data is received by the server-side API and stored in a database.

[1837] Input: User data in the terminal memory

[1838] Output: User information stored in a database on the server

[1839] Step 3:

[1840] The server starts a session for each user based on the received information. The received data (user ID, start time, meal contents, etc.) is recorded in a database, and the session start time is saved as a timestamp.

[1841] Input: User information on the server

[1842] Output: Session data recorded in a database

[1843] Step 4:

[1844] The user inputs the amount of food eaten and their progress during the meal into the device. Emotion data is also automatically acquired using an emotion engine. For example, the user's voice is analyzed using the Google Cloud Speech-to-Text API to evaluate their stress level.

[1845] Input: User's food intake, progress, and voice data

[1846] Output: New dataset in device (progress and emotion ratings)

[1847] Step 5:

[1848] The device sends the user's progress and emotion data to the server at regular intervals (for example, every 5 minutes). This communication also uses the HTTP protocol, and the data is sent to the server in JSON format. The server then formats the received data appropriately and adds it to the database.

[1849] Input: New progress and emotion data

[1850] Output: Updated user database on the server.

[1851] Step 6:

[1852] The server analyzes the received data using Python's pandas library and Scikit-learn. Specifically, it calculates eating speed based on the amount of food eaten and progress, and runs algorithms to evaluate stress levels and relaxation state based on emotional data.

[1853] Input: Updated user database

[1854] Output: Eating speed and emotional state of each user

[1855] Step 7:

[1856] The server compares the data of all users and detects abnormal eating speeds. For example, it identifies users whose eating speeds deviate significantly from the average speed and also checks their emotional state. If an abnormality is detected, it flags the data as abnormal.

[1857] Input: Eating speed and emotional state of each user

[1858] Output: A list of users for whom anomalies were detected

[1859] Step 8:

[1860] The server generates appropriate questions for users with abnormal eating speeds. Using past conversations, user profiles, and emotional data, the generative AI model inputs prompt sentences to generate specific questions. For example, it generates a question like, "Where was your most recent relaxing trip?"

[1861] Input: A list of users for whom anomalies were detected, and a prompt for the AI ​​model

[1862] Output: Generated questions

[1863] Step 9:

[1864] The server sends the generated question to the user's device. The server uses the HTTP protocol to send the generated question in JSON format to the device. The device receives it and displays it on the user interface.

[1865] Input: Generated Question

[1866] Output: The question displayed on the user's terminal

[1867] Step 10:

[1868] The user inputs an answer to the question displayed on the device. For example, the answer might be "I went to Hokkaido last month." The answer is input using the keyboard on the smartphone or tablet. The answer data is saved in the device's memory.

[1869] Input: User's answer

[1870] Output: Answer data stored in the device's memory

[1871] Step 11:

[1872] The device sends the response data to the server using the HTTP protocol, packetizing the response data in JSON format and sending it to the server. The server stores the received response data in a database and analyzes it.

[1873] Input: Answer data stored in the device's memory

[1874] Output: Answer data saved on the server

[1875] Step 12:

[1876] The server analyzes the received response data, for example by applying a text analysis algorithm to generate the next question from what the user has said about their travel destination, and then uses a generative AI model to generate the next question.

[1877] Input: Answer data saved on the server

[1878] Output: The following questions are generated:

[1879] Step 13:

[1880] The server generates the next question and sends it to the user's device. The server then packets the question in JSON format and sends it using the HTTP protocol. The device receives it and displays it on the user interface.

[1881] Input: The next generated question

[1882] Output: The following question displayed on the user's screen:

[1883] Step 14:

[1884] The server monitors all users' eating speed and emotional data in real time and continues to ask questions accordingly, forming a feedback loop to adjust the eating timing and emotional state of all users.

[1885] Input: Eating speed and emotion data of all users

[1886] Output: Feedback loop of adjusted questions

[1887] Step 15:

[1888] When all users have finished eating, they input "meal finished" information into their terminal. The terminal then sends the end information to the server. The server updates the session data based on the received end information and confirms that all users have finished.

[1889] Input: User exit information

[1890] Output: Finished data saved on the server

[1891] Step 16:

[1892] The server confirms the end of each user session and stores the session data in a database, including the end time and final emotion data.

[1893] Input: Finished data sent to the server

[1894] Output: Session termination data stored in the database

[1895] (Application example 2)

[1896] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1897] In today's world, when multiple users eat together, imbalances in eating speed can cause some users to feel isolated or conversations to be interrupted. This problem significantly reduces the enjoyment and satisfaction of the meal and requires special consideration. Furthermore, because emotional fluctuations often affect the pace of eating, a system that takes this into account is needed. However, current systems do not provide a system that can reflect users' emotions in real time and generate appropriate conversations.

[1898] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1899] In this invention, the server includes a means for periodically recording the progress of the meal using a smart device, a means for acquiring the user's emotion data using an emotion engine, and a means for generating questions using a generative AI model, which enables the server to monitor the user's eating speed and emotions in real time and balance the eating speed and conversation by asking appropriate questions.

[1900] A "smart device" refers to a device that has internet connectivity and is equipped with various sensors and displays.

[1901] An "emotion engine" refers to a software or hardware system that analyzes a user's facial expressions and voice and recognizes their psychological state and emotions.

[1902] A "generative AI model" refers to a system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[1903] A "server" refers to a computer system that centrally manages, analyzes, and processes data sent from multiple users.

[1904] "Means for generating questions" refers to algorithms or programs for automatically generating appropriate questions based on past conversation content and user profiles.

[1905] "Eating speed" refers to the progress of a user's eating within a specific period of time.

[1906] "Means for analyzing eating speed" refers to a program or algorithm that evaluates and analyzes a user's eating speed based on collected data.

[1907] This invention is a system that uses smart devices to record and transmit users' meal progress and emotional data, and a server analyzes this data and generates appropriate questions to adjust the balance between eating speed and conversation for all users. This system enriches the dining experience among users and is expected to be used particularly in brick-and-mortar establishments such as restaurants.

[1908] The system configuration includes the following hardware and software:

[1909] Hardware

[1910] Smart device: A device with internet connectivity and equipped with various sensors and displays. Here, we use smart glasses as an example.

[1911] Sensors: To keep track of your eating progress, sensors built into the smart glasses measure the amount of food you eat.

[1912] Server: A computer system that centrally manages, analyzes, and processes data sent from multiple users.

[1913] software

[1914] Emotion engine: An engine for analyzing the user's facial expressions and voice to obtain emotional data.

[1915] Generative AI model: A system that uses artificial intelligence technology to automatically generate appropriate questions and comments based on a user's profile and conversation content.

[1916] ServerCommunicator: A module that is responsible for communicating with the server and sending and receiving data.

[1917] QuestionGenerator: A module that generates appropriate questions based on emotional data, user profiles, and conversation history.

[1918] SmartGlassesDisplay: Software with the ability to display questions on the screen of smart glasses.

[1919] System Operation

[1920] In this system, the user puts on the smart glasses before starting a meal and inputs the "start eating" information. The smart glasses then send the start time, user ID, meal content, past conversation log, and emotional data to the server. The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. This data is then sent to the server at regular intervals, where it is analyzed and used to monitor eating speed and emotional state.

[1921] The server then compares each user's eating speed, and if an abnormal eating speed is detected, the AI ​​model generates appropriate questions based on the user's profile and emotional data. The generated questions are displayed on the smart glasses' display, and the user answers them. The answers are then sent back to the server, which generates additional questions to further encourage conversation.

[1922] Specific examples

[1923] For example, if the user is recognized as feeling stressed, the following example prompt sentences are fed into the generative AI model:

[1924] A user is feeling stressed. Generate a question for User A, who loves to travel: "Where was the most relaxing trip you've taken recently?"

[1925] The generated question looks like this:

[1926] "Where's the last relaxing trip you went to?"

[1927] This question is displayed on the smart glasses display, and the user responds, "I went to Hokkaido last month." This response is sent to the server, which then generates the next question, such as, "What kind of food did you like in Hokkaido?" Through this process, the eating speed and emotional state of all users are balanced.

[1928] This will enrich the dining experience between users and allow for mealtimes and conversations that are satisfying for everyone.

[1929] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1930] Step 1:

[1931] Before starting a meal, the user puts on the smart glasses and inputs the "start eating" information. The user operates the interface of the smart glasses to input the start time, user ID, and meal contents. This information is sent to the server along with the start trigger.

[1932] Input: Start time, User ID, Meal details

[1933] Output: Start trigger and user information sent to the server

[1934] Step 2:

[1935] The server starts a session based on the received start information, user ID, and meal details. A session ID is generated and recorded in the database. Each user's progress is managed based on this session ID.

[1936] Input: Start trigger and user information

[1937] Output: Generate a session ID and record it in the database

[1938] Step 3:

[1939] The emotion engine analyzes the user's facial expressions and voice in real time to obtain emotional data. The camera and microphone built into the smart glasses capture facial expressions and voice, which are then sent to the emotion engine for analysis.

[1940] Input: User facial expression images, voice data

[1941] Output: Obtaining emotion data

[1942] Step 4:

[1943] Sensors in the smart glasses periodically measure the amount of food consumed and send the data to a server. For example, a weight sensor measures the amount of food consumed and sends it as progress data.

[1944] Input: Food quantity data

[1945] Output: Getting and sending progress data

[1946] Step 5:

[1947] The server analyzes the received emotion data and progress data to calculate the user's eating speed. It also compares the eating speeds of multiple users to check for abnormal speeds.

[1948] Input: Emotion data, progress data

[1949] Output: Detection results of eating speed and abnormal speed

[1950] Step 6:

[1951] If the server detects abnormal eating speed, it uses a generative AI model to generate appropriate questions based on the user's profile and emotional data. The generative AI model generates questions based on the prompt text, and the server sends the questions to the smart glasses.

[1952] Input: User profile, emotion data, prompt sentence

[1953] Output: Generated questions

[1954] Step 7:

[1955] A question is displayed on the smart glasses display, and the user responds to the question by voice or text, which is then sent back to the server.

[1956] Input: Generated Question

[1957] Output: User's answer

[1958] Step 8:

[1959] The server analyzes the answers it receives and generates further questions as needed, repeating this process until all users' eating speeds and conversations are balanced.

[1960] Input: User's answer

[1961] Output: Further generated questions

[1962] Step 9:

[1963] All users input the "meal end" information into their smart glasses. This information is sent to the server, which ends the session and saves all data in a database.

[1964] Input: Meal end information

[1965] Output: Save session data and end session

[1966] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1967] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1968] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1969] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1970] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1971] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1972] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1973] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1974] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1975] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1976] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1977] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1978] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1979] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1980] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1981] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1982] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1983] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1984] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1985] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1986] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1987] The following is further disclosed regarding the above embodiment.

[1988] (Claim 1)

[1989] a means for inputting user information before the user starts eating;

[1990] A means of regularly recording your dietary progress;

[1991] A means for analyzing eating speed and detecting abnormal eating speed;

[1992] A means for generating appropriate questions based on past conversations and user profiles;

[1993] means for presenting the generated questions to a user and receiving answers;

[1994] means for analyzing the received answers and generating further questions;

[1995] A means for adjusting timing to balance the eating speed of all users;

[1996] A system including:

[1997] (Claim 2)

[1998] 2. The system according to claim 1, wherein when generating questions for a user who eats extremely quickly, the system generates questions based on the user's interests and concerns.

[1999] (Claim 3)

[2000] The system according to claim 1, which monitors in real time whether the eating speeds of all users are being kept in balance, and generates and adjusts appropriate conversations when the balance is disrupted.

[2001] "Example 1"

[2002] (Claim 1)

[2003] a means for inputting user information before the user starts eating;

[2004] A means of regularly recording your dietary progress;

[2005] A means for analyzing eating speed and detecting abnormal eating speed;

[2006] A means for generating appropriate questions based on past conversations and user profiles;

[2007] A means for generating questions using a generative AI model and sending the questions to a user's device;

[2008] means for presenting the generated questions to a user and receiving answers;

[2009] means for analyzing the received answers and generating further questions;

[2010] A means for adjusting timing to balance the eating speed of all users;

[2011] A system including:

[2012] (Claim 2)

[2013] The system according to claim 1, wherein when generating questions for a user who eats extremely quickly, the system generates questions using a generation AI model based on the user's interests and concerns.

[2014] (Claim 3)

[2015] The system of claim 1 monitors in real time whether the eating speeds of all users are balanced, and when the balance is lost, generates and adjusts appropriate conversations using a generative AI model.

[2016] "Application Example 1"

[2017] (Claim 1)

[2018] a means for inputting user information before the user starts eating;

[2019] A means of regularly recording your dietary progress;

[2020] A means for analyzing eating speed and detecting abnormal eating speed;

[2021] A means for generating appropriate questions based on past conversations and user profiles;

[2022] means for presenting the generated questions to a user and receiving answers;

[2023] means for analyzing the received answers and generating further questions;

[2024] A means for adjusting timing to balance the eating speed of all users;

[2025] A means of generating conversations based on user interests using an AI model;

[2026] A means for monitoring and sharing the dining experiences of users in a virtual environment in real time;

[2027] A system including:

[2028] (Claim 2)

[2029] 2. The system according to claim 1, wherein when generating questions for a user who eats extremely quickly, the system generates questions based on the user's interests and concerns.

[2030] (Claim 3)

[2031] The system according to claim 1, characterized in that it monitors in real time whether the eating speeds of all users are being kept in balance, and generates and adjusts appropriate conversations when the balance is disrupted.

[2032] "Example 2: Combining Emotion Engines"

[2033] (Claim 1)

[2034] a means for inputting user information before the user starts eating;

[2035] A means of regularly recording your dietary progress;

[2036] A means for analyzing eating speed and detecting abnormal eating speed;

[2037] A means for generating appropriate questions based on past conversations and user profiles;

[2038] means for presenting the generated questions to a user and receiving answers;

[2039] means for analyzing the received answers and generating further questions;

[2040] A means for adjusting timing to balance the eating speed of all users;

[2041] A means for acquiring emotion data using voice analysis or facial expression analysis;

[2042] a means for adjusting questions based on the acquired sentiment data;

[2043] a means of sending, receiving, and analyzing data in real time;

[2044] A system including:

[2045] (Claim 2)

[2046] 2. The system according to claim 1, wherein when generating questions for a user who eats extremely quickly, the system generates questions based on the user's interests and concerns.

[2047] (Claim 3)

[2048] The system according to claim 1, which monitors in real time whether the eating speeds of all users are being kept in balance, and generates and adjusts appropriate conversations when the balance is disrupted.

[2049] "Application example 2 when combining emotion engines"

[2050] (Claim 1)

[2051] a means for inputting user information before the user starts eating;

[2052] A means to regularly record dietary progress using a smart device;

[2053] means for obtaining user emotion data using an emotion engine;

[2054] means for transmitting emotion data and eating speed to a server;

[2055] A means for the server to analyze the eating speed and detect abnormal eating speed;

[2056] A means for generating appropriate questions based on past conversations and user profiles;

[2057] a means for generating questions using a generative AI model and displaying them on a smart device;

[2058] means for presenting the generated questions to a user and receiving answers;

[2059] means for analyzing the received answers and generating further questions;

[2060] A means for adjusting timing to balance the eating speed of all users;

[2061] A system including:

[2062] (Claim 2)

[2063] 2. The system according to claim 1, wherein when generating questions for a user who eats extremely quickly, the system generates questions based on the user's interests and concerns.

[2064] (Claim 3)

[2065] The system of claim 1 monitors in real time whether the eating speeds of all users are being kept in balance, and when the balance is lost, generates and adjusts appropriate conversations using a generative AI model. [Explanation of symbols]

[2066] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for inputting user information before the user starts eating; A means of regularly recording your dietary progress; A means for analyzing eating speed and detecting abnormal eating speed; A means for generating appropriate questions based on past conversations and user profiles; means for presenting the generated questions to a user and receiving answers; means for analyzing the received answers and generating further questions; A means for adjusting timing to balance the eating speed of all users; A system including:

2. The system according to claim 1, wherein when generating questions for a user who eats very quickly, the system generates questions based on the user's interests and concerns.

3. 2. The system according to claim 1, wherein the system monitors in real time whether the eating speeds of all users are being kept in balance, and generates and adjusts appropriate conversations when the balance is lost.

Images