System

Abstract

A system is provided.SOLUTION: A system comprising: means for scanning a barcode of a food product to obtain nutrition information of the food product; means for performing an analysis based on the obtained nutrition information to predict an effect of ingestion of the food product on a user; image generation means for generating an appearance of the user after several months based on the prediction; and means for displaying the generated appearance of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, as health consciousness grows, information on food calories and ingredients is becoming increasingly important. However, for people who tend to exercise less due to their busy schedules, it is not easy to efficiently collect and understand this information and reflect it in their diet. Furthermore, to prevent overeating and maintain a healthy diet, people need to intuitively understand the long-term effects of food on the body. Therefore, there is a need for a method to efficiently obtain nutritional information on foods and visually understand the effects of food choices on the body. [Means for solving the problem]

[0005] This invention provides a system that obtains nutritional information for food by scanning its barcode. This system uses AI to perform detailed analysis of the obtained nutritional information and predicts the impact that food intake will have on the user. It also has the function of using image generation technology to realistically generate and display what the user will look like several months from now based on this prediction. In this way, users can intuitively understand the impact of food selection and intake, and receive support in leading a healthy diet.

[0006] "Food" is a general term for food and drink that people consume on a daily basis.

[0007] A "barcode" is a code made up of horizontal black and white stripes that contains identifying information for an item.

[0008] "Nutritional information" refers to information about the calories, proteins, fats, carbohydrates, and other components contained in food, as well as their amounts.

[0009] "Analysis" is the act of processing data and extracting useful information from it.

[0010] "Prediction" is the act of predicting future possibilities based on known data.

[0011] "User" refers to a person who uses this system.

[0012] "Image creation" is the process of creating new images using digital technology.

[0013] "Display" is the act of visualizing information or images on the screen of a digital device, etc.

[0014] A "system" is an organized whole in which multiple elements work together to achieve a specific function. [Brief explanation of the drawings]

[0015] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that uses image generation technology to display what the user will look like in a few months' time based on the nutritional information of a food item that the user scans by scanning its barcode. This system mainly consists of the following components:

[0037] System Components

[0038] 1. User Device

[0039] 2. Cloud Server

[0040] 3. Database

[0041] 4. Image generation AI

[0042] User terminal processing

[0043] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When a barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[0044] Cloud server processing

[0045] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[0046] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0047] Image generation AI processing

[0048] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI generates a realistic image of what the user will look like in a few months' time based on the predicted weight gain and changes in body fat percentage. The generated image is displayed on the device screen in a format that is easy for the user to view.

[0049] Specific examples

[0050] As a concrete example, the sequence of events when a user scans a chocolate bar is shown below.

[0051] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0052] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0053] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0054] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0055] 5. The server sends the analysis results and prediction data to the device.

[0056] 6. The device uses image generation AI to generate and display a simulated image of the user's appearance based on the predicted results three months from now, allowing the user to visualize what they will look like in a few months. Through this visualization, users can intuitively understand the impact of their dietary choices in the future and receive support to lead a healthier diet.

[0057] In this way, this system is a powerful tool that allows users to easily obtain nutritional information about foods and use that information to help manage their current and future health.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scanning function. The device's camera then starts up, ready to scan the barcode.

[0061] Step 2:

[0062] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[0063] Step 3:

[0064] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[0065] Step 4:

[0066] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[0067] Step 5:

[0068] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[0069] Step 6:

[0070] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[0071] Step 7:

[0072] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[0073] Step 8:

[0074] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[0075] Step 9:

[0076] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[0077] Step 10:

[0078] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[0079] Step 11:

[0080] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[0081] Step 12:

[0082] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[0083] Step 13:

[0084] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[0085] Step 14:

[0086] Users can view simulated images displayed on their device, along with nutritional information and predictions, to visually understand how their dietary choices affect their body, helping them make healthier eating choices going forward.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] In recent years, health management based on food nutritional information has become increasingly important. However, conventional systems do not simply acquire food nutritional information, but also lack the means to predict and visually confirm future health status based on that information. This makes health management difficult, and users have difficulty intuitively understanding future changes in their body.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information for that food product, means for transmitting the obtained nutritional information to a cloud server, means for searching a database based on the nutritional information received by the cloud server and obtaining the nutritional information for the corresponding food product, means for predicting the impact of consuming the food product on the user based on the obtained nutritional information, means for generating a prompt message based on the prediction result and using image generation technology to generate an image of the user several months from now, and means for displaying the generated image of the user. This allows users to easily obtain nutritional information for foods and visually check their future health status based on that information.

[0092] A "means for scanning food barcodes" is a process or device that reads food barcodes using a user terminal such as a smartphone or tablet.

[0093] The "means for obtaining nutritional information" refers to a process or device that scans a barcode and then obtains the nutritional information of the food item corresponding to the barcode from a database.

[0094] A "cloud server" is a remote server accessible via the Internet, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[0095] A "means for searching a database" is a process or device that queries a database in a cloud server and retrieves nutrition information corresponding to a particular barcode.

[0096] A "means for predicting the impact of food intake on a user" is a process or device that calculates and predicts the impact of food intake on a user's health status, such as weight and body fat percentage, based on acquired nutritional information.

[0097] A "means for generating a prompt sentence" is a process or device that generates textual instructions to be input to the image generation AI based on the prediction results.

[0098] "Means for generating what a user will look like in a few months' time using image generation technology" refers to technology or a device that generates an image that realistically simulates what the user will look like in a few months' time based on the generated prompt text.

[0099] The "means for displaying the generated image of the user" is a process or device for displaying the generated image on the screen of the user terminal.

[0100] This invention is a system that allows a user to scan a food's barcode to obtain nutritional information for that food, and then uses image generation technology to display what the user will look like several months from now. This system mainly consists of the following components: a user terminal, a cloud server, a database, and an image generation AI. How the system is specifically implemented using these components is described below.

[0101] User terminal processing

[0102] The user launches the "Food Scan Assistant" app using a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When the barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server. The user's device can be a standard smartphone (for example, an iPhone or Android device).

[0103] Cloud server processing

[0104] The cloud server receives the barcode data sent from the device. It searches its internal database based on the received barcode data to obtain the nutritional information for the corresponding food. This nutritional information includes calories, protein, fat, carbohydrates, etc. The cloud server uses a remote server such as an EC2 instance on Amazon Web Services (AWS). The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0105] Image generation AI processing

[0106] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI uses a generative model such as Stable Diffusion or DALL-E to generate a realistic image of what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage. The generated image is then displayed on the device screen in a format that is easy for the user to view.

[0107] Specific examples

[0108] As a concrete example, the following shows the sequence of events when a user scans a chocolate bar. The user scans the barcode on the chocolate bar with their smartphone. The device reads the barcode "1234567890123" and sends the data to a cloud server. The server receives the barcode "1234567890123" and retrieves the corresponding chocolate bar's nutritional information (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g) from a database. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months. The server then sends the analysis results and predicted data to the device. The device then uses image generation AI to generate and display an image simulating what the user will look like three months from now based on the predicted results.

[0109] Prompt Sentence Examples

[0110] Here are some examples of specific prompts to input to a generative AI model:

[0111] "Generate an image of the user based on the following data:

[0112] Current weight: 70kg

[0113] Predicted weight after 3 months: 75kg

[0114] Increase in body fat percentage: 3%

[0115] Display the user's appearance after 3 months based on these predictions."

[0116] According to this invention, users can easily obtain nutritional information about foods and use that information to help manage their current and future health.

[0117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0118] Step 1:

[0119] The user launches the "Food Scan Assistant" app.

[0120] Input: Launching an app on a smartphone or tablet.

[0121] What happens: The user selects barcode scan mode within the app and launches the camera.

[0122] Output: The camera is activated and ready to scan a barcode.

[0123] Step 2:

[0124] The user scans the food barcode with the camera.

[0125] Input: Barcode displayed on your smartphone or tablet camera.

[0126] How it works: The camera reads barcode data in real time and converts it into text.

[0127] Output: Stringified barcode data.

[0128] Step 3:

[0129] The terminal transmits the barcode data converted into a string to the cloud server.

[0130] Input: The string data of the scanned barcode.

[0131] Operation: The device generates an HTTP request and sends it to the cloud server.

[0132] Output: The HTTP request sent to the cloud server.

[0133] Step 4:

[0134] The cloud server receives the barcode data and searches the database.

[0135] Input: Barcode data in the HTTP request.

[0136] How it works: The cloud server queries the AWS RDS database and retrieves the corresponding nutrition information.

[0137] Output: Nutritional information (calories, protein, fat, carbohydrates) for the food.

[0138] Step 5:

[0139] The cloud server uses an AI module to perform analysis based on nutritional information.

[0140] Input: Nutritional information of the retrieved food.

[0141] How it works: The AI ​​module predicts the effects of eating a food once a day for three months, calculating total calorie intake and predicted weight gain.

[0142] Output: Analysis results and forecast data.

[0143] Step 6:

[0144] The cloud server sends the analysis results and prediction data to the user's device.

[0145] Input: Analysis results and forecast data.

[0146] Operation: The cloud server generates an HTTP response and sends it to the user device.

[0147] Output: The HTTP response sent to the user's device.

[0148] Step 7:

[0149] The image generation AI is activated based on the analysis results and prediction data received by the user device.

[0150] Input: Analysis results and prediction data received from the cloud server.

[0151] How it works: The device generates a prompt and inputs it into the image generation AI.

[0152] Output: The prompt and the generated image.

[0153] Step 8:

[0154] The user terminal displays the generated image.

[0155] Input: An image generated based on the prompt sentence.

[0156] Action: The device visually displays the image to the user.

[0157] Output: A simulated image after 3 months displayed on the user's screen.

[0158] Through the above steps, the user can easily obtain nutritional information about foods and visually check their future health status based on that information.

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] Previously, when shopping in a physical store, there was no way to know in real time the impact that the food being purchased would have on the user's health. This made it difficult for users to accurately understand the nutritional value of foods and make choices that took future health effects into account. Furthermore, existing systems only provided abstract nutritional information, and were insufficient in providing information in a visually easy-to-understand format. To solve this problem, a system is needed that obtains the nutritional information of foods when purchasing food in a physical store and visualizes the user's future state.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes means for identifying a food identifier and acquiring nutritional information for the food, means for analyzing the acquired nutritional information and predicting the impact that food intake will have on the user, means for generating an image of the user several months from now based on the prediction, means for displaying the generated image of the user, and means for scanning the identifier with the user's mobile device while shopping in a physical store. This allows users to obtain detailed nutritional information for the food they purchase while shopping in a physical store and visualize their appearance several months from now, enabling them to make healthier food choices.

[0164] A "food identifier" is information such as a barcode or QR code that identifies a specific food item.

[0165] "Nutritional information" refers to data on nutrients such as energy, protein, fat, and carbohydrates contained in food.

[0166] "Performing an analysis" refers to the process of conducting mathematical or statistical processing on the nutritional information obtained to assess the impact of food intake.

[0167] "Predictive methods" refer to algorithms or software that use nutritional information to predict future changes in a user's health and body shape.

[0168] "Image generation means" refers to computer graphics or generative AI models that visually represent the future user's appearance based on the prediction results.

[0169] "Means for displaying" refers to a display or application for outputting the generated image to a terminal screen in a format that can be viewed by the user.

[0170] "Brick and mortar store" refers to a physical retail store where customers can physically visit and purchase products.

[0171] A "mobile terminal" is a device with communication capabilities that can be carried by a user, and in this context it mainly refers to a smartphone.

[0172] System Overview

[0173] This system uses a smartphone to obtain nutritional information about food when purchasing it in a physical store, and visualizes future health effects based on that information. The system consists of the following main components: a mobile device, a cloud server, a database, and an image-generating AI.

[0174] Processing steps

[0175] 1. Mobile device operation:

[0176] Users can launch the Smart Nutritional Shopping Assistant application at a physical store and scan the barcode of the food they plan to purchase using the camera function on their mobile device.

[0177] 2. Data transmission:

[0178] The mobile device sends the scanned barcode data to the cloud server, generating an HTTP request to send the barcode data to the server.

[0179] 3. Cloud Server Processing:

[0180] The cloud server searches the database based on the received barcode data and retrieves the nutritional information for the food, including energy, protein, fat, carbohydrates, etc.

[0181] 4. Prediction of health effects:

[0182] The cloud server uses the acquired nutritional information to instruct the AI ​​module to analyze and predict the health effects of consuming that food over a certain period of time. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain.

[0183] 5. Send prediction results:

[0184] The cloud server sends the predicted health effects to a mobile device, which provides the results as predicted data on calorie intake and weight gain.

[0185] 6. Image generation:

[0186] The mobile device launches an image generation AI based on the predicted data received from the cloud server, which then simulates what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage.

[0187] 7. Displaying images:

[0188] The mobile device displays the generated images in an easy-to-view format, allowing users to intuitively understand the impact of the foods they purchase on their future health.

[0189] Hardware and software used

[0190] Mobile device: smartphone.

[0191] Cloud server: Data processing server (e.g., Amazon Web Services, Google Cloud Platform).

[0192] Database: A database system (e.g., MySQL, MongoDB) that stores nutrition information.

[0193] Image generation AI: AI modules for image generation (e.g., TensorFlow, PyTorch).

[0194] Specific examples

[0195] Let's take the example of a user purchasing potato chips at a physical store. The user uses their smartphone to scan the barcode on the potato chips. The mobile device sends the barcode "0123456789012" to a cloud server, which retrieves nutritional information (energy: 150 kcal, fat: 10 g, carbohydrates: 15 g) from a database. The server uses an AI module to predict weight gain and other factors if the user consumes this daily for three months, and sends the prediction results to the mobile device. The mobile device then activates an image generation AI to simulate and display what the user will look like in a few months' time.

[0196] Prompt Sentence Examples

[0197] "Scan food barcodes to get nutritional information and simulate what you'll look like in three months if you eat that food every day."

[0198] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0199] Step 1:

[0200] A user uses a smartphone to scan the barcode of a food item they plan to purchase in a physical store. The input is an image of the barcode captured by the smartphone's camera. The smartphone uses the pyzbar library to decode the barcode from the image and outputs the barcode data as a string.

[0201] Step 2:

[0202] The device sends the acquired barcode data to the cloud server. The input is the barcode data from step 1, and the output is the result of sending an HTTP request to the cloud server. Using the requests library, an HTTP POST request is generated to send the barcode data to the server.

[0203] Step 3:

[0204] The server searches a database based on the received barcode data to obtain the nutritional information of the corresponding food. The input is the received barcode data, and the output is the nutritional information of the corresponding food. The server uses SQL queries to search for the nutritional information of the food from a database (e.g., MySQL, MongoDB) and obtains data such as calories, protein, fat, and carbohydrates.

[0205] Step 4:

[0206] The cloud server then instructs the AI ​​module to analyze the nutritional information and predict the health effects of consuming the food over a certain period of time. The input is the nutritional information obtained from Step 3, and the output is predicted data on health effects. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain. For example, TensorFlow or PyTorch can be used for the analysis.

[0207] Step 5:

[0208] The cloud server sends the predicted health impact results to the mobile device. The input is the predicted data from step 4, and the output is the analysis results sent as an HTTP response to the mobile device. The server generates an HTTP response and sends the predicted data to the device in JSON format.

[0209] Step 6:

[0210] The mobile device launches an image generation AI based on the predicted data received from the cloud server. The input is the received predicted data, and the output is an image that simulates the generated future appearance of the user. The image generation AI generates the user's future appearance taking into account predicted weight gain and changes in body fat percentage. This process uses models such as GAN (generative adversarial networks).

[0211] Step 7:

[0212] The mobile device displays the generated image to the user. The input is the image data generated in step 6, and the output is the image of the future user displayed on the smartphone display. Through this visual information, the user can intuitively understand the impact of the food they plan to purchase on their future health.

[0213] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0214] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[0215] System Components

[0216] 1. User Device

[0217] 2. Cloud Server

[0218] 3. Database

[0219] 4. Image generation AI

[0220] 5. Emotion Engine

[0221] User terminal processing

[0222] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When a barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[0223] Cloud server processing

[0224] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[0225] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0226] Emotion engine processing

[0227] The user device uses the analysis results and prediction data received from the server to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, input, etc., and recognizes the user's current emotional state. The recognized emotional state is classified as, for example, "happiness," "sadness," or "stress."

[0228] Image generation AI processing

[0229] The image generation AI adjusts the parameters used to simulate the user's appearance depending on the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gradually.

[0230] Displaying Information

[0231] The user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[0232] Specific examples

[0233] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[0234] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0235] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0236] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0237] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0238] 5. The server sends the analysis results and prediction data to the device.

[0239] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[0240] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[0241] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[0242] The system allows users to receive specific advice based on their dietary habits and emotional state, providing support to lead a healthier life.

[0243] The processing flow will be explained below.

[0244] Step 1:

[0245] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scanning function. The device's camera then starts up, ready to scan the barcode.

[0246] Step 2:

[0247] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[0248] Step 3:

[0249] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[0250] Step 4:

[0251] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[0252] Step 5:

[0253] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[0254] Step 6:

[0255] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[0256] Step 7:

[0257] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[0258] Step 8:

[0259] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[0260] Step 9:

[0261] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[0262] Step 10:

[0263] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[0264] Step 11:

[0265] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[0266] Step 12:

[0267] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[0268] Step 13:

[0269] The device activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize their current emotional state, such as "happiness," "sadness," or "stress."

[0270] Step 14:

[0271] The parameters of the image generation AI are adjusted based on the emotional state recognized by the device's emotion engine. For example, if "stress" is detected, the settings are changed to emphasize the simulation of weight gain.

[0272] Step 15:

[0273] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[0274] Step 16:

[0275] Along with the generated image, the device will display advice and food choice recommendations based on the user's emotional state, such as "You're under a lot of stress, so avoid sweets."

[0276] Step 17:

[0277] Users can view simulated images displayed on their device, along with nutritional information, predictions, and emotional advice, providing specific guidance on diet and health management. This understanding helps users make healthier eating choices going forward.

[0278] Example 2

[0279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] While conventional food nutrition information acquisition systems allow users to obtain nutritional information about foods, it is difficult for them to obtain long-term predictions based on that information or advice tailored to their emotional state. Furthermore, since these systems do not simply display nutritional information but also provide insufficient support for health management that takes into account the user's emotions, it is difficult to maintain the user's motivation.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0282] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information about the food product; means for analyzing the obtained nutritional information and predicting the impact of consuming the food product on the user; image generation means for generating an image of the user several months from now based on the prediction; means for displaying the generated image of the user; means for recognizing the user's emotional state; and image generation means for adjusting the prediction result based on the user's emotional state. This allows the user to not only obtain nutritional information about the food product but also receive health management advice that predicts the long-term impact of the food product. Furthermore, the simulation and advice provided reflect the user's emotions, thereby maintaining the user's motivation and improving the effectiveness of health management.

[0283] "Scanning food barcodes" means using an image capture device such as a camera to read the barcode attached to the food and obtain the information as digital data.

[0284] "Nutritional information" refers to data about the calories, protein, fat, carbohydrates, and other components contained in food.

[0285] "Performing analysis" means using specific algorithms and AI modules to process the data based on the acquired nutritional information and calculate the impact on the user.

[0286] "Predicting" means estimating future changes in the user's weight and health condition based on past data and current nutritional information.

[0287] An "image generation means" is a device or software module that uses a specific algorithm or AI technology to visually represent the user's future appearance based on a prediction.

[0288] "Display" means visually presenting the generated image or predicted data on the screen of the user terminal.

[0289] A "means for recognizing emotional state" is a device or software module that uses a camera or sensor to analyze a user's facial expressions, voice, input, etc., to identify their current emotional state.

[0290] "Adjusting prediction results based on emotional state" means changing or adjusting prediction results or generated images by taking into account the recognized emotional state of the user.

[0291] A "cloud server" is a remote computing resource accessible via a network, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[0292] MODE FOR CARRYING OUT THE INVENTION

[0293] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[0294] 1. User device: smartphone, tablet, etc.

[0295] 2. Cloud server: A remote server that performs analysis and stores data.

[0296] 3. Database: Data storage for storing nutrition information.

[0297] 4. Image generation AI: An algorithm for generating the user's future appearance.

[0298] 5. Emotion Engine: A module for analyzing user emotions.

[0299] System Operation Overview

[0300] First, the user launches the "Food Scan Assistant" app on their smartphone and uses the function to scan the food barcode. The device's camera reads the barcode, converts it into a string, and creates and sends an HTTP request to the cloud server.

[0301] The cloud server searches its internal database based on the received barcode data and retrieves the nutritional information for the food in question. This information includes calories, protein, fat, carbohydrates, etc. This nutritional information is then passed to an AI module for detailed analysis, which predicts, for example, the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0302] The user device uses the received analysis results and prediction data to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize the user's current emotional state. For example, emotional states are classified as "joy," "sadness," "stress," etc.

[0303] The image generation AI adjusts simulation parameters depending on the user's perceived emotional state. For example, if the user is stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gently.

[0304] Finally, the user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[0305] Specific examples

[0306] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[0307] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0308] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0309] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0310] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0311] 5. The server sends the analysis results and prediction data to the device.

[0312] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[0313] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[0314] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[0315] Examples of prompt statements

[0316] "Explain how the system works when a user scans the barcode '1234567890123' on a chocolate bar and wants to know the nutritional information for that food and its effects after three months."

[0317] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0318] Step 1:

[0319] The user launches the "Food Scan Assistant" app on their smartphone, which displays the app's home screen.

[0320] Input: Touch operation on smartphone

[0321] Output: App home screen

[0322] Specifically, the user selects the barcode scanning function within the app and taps the option to launch the camera, which activates the smartphone's camera module.

[0323] Step 2:

[0324] The device reads the food barcode through the camera. The camera captures the barcode and obtains its image data. Then, it uses a barcode parsing library to decode the barcode image and convert it into string data.

[0325] Input: Camera image data

[0326] Output: Stringified barcode data (e.g. "1234567890123")

[0327] Specifically, the device extracts the barcode data and generates an HTTP request to send it to the cloud server, which includes the barcode data.

[0328] Step 3:

[0329] The device generates an HTTP request and sends it to the cloud server. Use the HTTP client library to send the request, including the barcode data.

[0330] Input: HTTP request (barcode data)

[0331] Output: Sending status (e.g. 200 OK)

[0332] Specifically, the device sends a request containing barcode data to a specific API endpoint, and the process is asynchronous, waiting for a response from the server.

[0333] Step 4:

[0334] The server receives the HTTP request, extracts the barcode data, and then searches a database to retrieve the corresponding food's nutritional information, including calories, protein, fat, carbohydrates, etc.

[0335] Input: HTTP request (barcode data)

[0336] Output: Nutritional information (e.g., Calories: 200 kcal, Protein: 2 g, Fat: 12 g, Carbohydrates: 24 g)

[0337] Specifically, the server executes a database query to obtain the nutritional information of the relevant food, and prepares to pass the obtained nutritional information to the AI ​​module.

[0338] Step 5:

[0339] The server's AI module performs a detailed analysis based on the nutritional information, for example, calculating the cumulative calorie intake and predicted weight gain if the food is eaten once a day for three months.

[0340] Input: Nutritional Information

[0341] Output: Predicted data (e.g., weight gain after 3 months)

[0342] Specifically, the server's AI module uses machine learning models to simulate the long-term effects of nutritional information and returns the predictions to the server.

[0343] Step 6:

[0344] The server receives the analysis results and prediction data and generates an HTTP response to send to the user's device, which includes nutritional information and prediction results.

[0345] Input: Forecast data

[0346] Output: HTTP response (nutritional information and prediction results)

[0347] Specifically, the server generates an HTTP response and sends it to the user's device. This response includes nutritional information and prediction data.

[0348] Step 7:

[0349] The user device receives the HTTP response from the server and activates the emotion engine, which uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize their current emotional state.

[0350] Input: HTTP response (nutritional information and prediction results), user input, facial expression, and voice data

[0351] Output: Perceived emotional state (e.g., stress)

[0352] Specifically, the user device runs an emotion engine to analyze various data about the user, and classifies the user's emotional state based on the analysis results.

[0353] Step 8:

[0354] The image generation AI on the user's device adjusts its predictions based on the perceived emotional state, and uses the generative AI model to generate a simulated image of what the user will look like in, say, three months.

[0355] Input: Emotional state, prediction data

[0356] Output: Adjusted simulation image

[0357] Specifically, the image generation AI on the user device adjusts the parameters of the image to be generated based on the output of the emotion engine, and generates a visual simulation image.

[0358] Step 9:

[0359] The user device visually displays the generated image and predicted data to the user, and also displays food selection recommendations and advice according to the emotional state recognized by the emotion engine.

[0360] Input: simulated image, predicted data, recognized emotional state

[0361] Output: Images and advice visually presented to the user

[0362] Specifically, the user terminal displays data on a user interface and provides information in a format that the user can intuitively understand.

[0363] (Application example 2)

[0364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0365] Conventional nutrition information acquisition systems can analyze the nutritional information of foods, but they do not provide comprehensive health advice that takes into account the user's emotional state. As a result, users find it difficult to make food choices that take into account their emotional state and stress level, resulting in insufficient long-term health management. To solve this problem, a system is needed that recognizes the user's emotional state and provides accurate advice on food selection based on that state.

[0366] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for scanning the barcode of a food product and acquiring nutritional information about the food product, means for performing analysis based on the acquired nutritional information and predicting the impact that food intake will have on the user, means for recognizing the user's emotional state, image generation means for generating an image of the user several months from now based on the prediction, and means for displaying advice based on the generated image of the user and their emotional state. This enables the user to receive comprehensive and personalized support regarding food selection and health management.

[0367] "Nutritional information" refers to data on the ingredients contained in food, such as calories, protein, fat, and carbohydrates.

[0368] "Emotional state" is information that represents the psychological state of the user, and includes specific emotions such as joy, sadness, and stress.

[0369] "Means for scanning barcodes" refers to the ability to use the camera of a device such as a smartphone or smart glasses to read the barcode of a food item and obtain its data.

[0370] The "means for performing analysis" is a function that performs calculations and data processing to predict the user's health condition and future weight changes based on the acquired nutritional information.

[0371] A "predictor" is an algorithm that estimates the impact of food intake on a user based on nutritional information and multiple future scenarios.

[0372] The "image generation means" is a combination of software and hardware that uses the acquired data and analysis results to visually reproduce the user's future appearance.

[0373] "Display means" is a function for informing the user of the generated images, nutritional information, analysis results, and advice using a display such as a user terminal or smart glasses.

[0374] A "cloud server" is a remote server that stores, processes, and transfers data over the Internet, and supports data processing for the entire system.

[0375] The "means of acquisition" is a function that scans the barcode, sends the data obtained to a cloud server, and retrieves the corresponding nutritional information.

[0376] "Means of recognition" refers to a function that analyzes the user's facial expressions and voice through a camera or microphone to determine their current emotional state.

[0377] This invention is a system that supports health management by allowing users to scan food barcodes to obtain nutritional information, conducting analysis based on that information, and taking into account the user's emotional state. The system includes the following components: a user terminal, a cloud server, a database, an image generation AI, and an emotion engine. This allows users to make more effective food choices and manage their health.

[0378] System Components

[0379] 1. User Device

[0380] The user uses a smartphone or smart glasses as the device, launches the "Food Scan Assistant" application, and scans the food barcode.

[0381] 2. Cloud Server

[0382] It receives data and searches for nutritional information in a database, analyzes the nutritional content of multiple foods in detail, and works with an emotion engine to make predictions based on the user's emotions.

[0383] 3. Database

[0384] It stores nutritional information for foods and responds to searches from a cloud server.

[0385] 4. Image generation AI

[0386] The system simulates the user's future appearance based on the user's emotional state and nutritional information.

[0387] 5. Emotion Engine

[0388] It analyzes the user's facial expressions and voice to recognize their current emotional state.

[0389] Implementation details

[0390] User terminal processing

[0391] When a user scans a food barcode in a physical store, the camera in the smart glasses or smartphone reads it and sends the barcode data to a cloud server, which also uses the camera and microphone in the smart glasses to analyze the user's emotional state.

[0392] Cloud server processing

[0393] The cloud server receives the barcode data and retrieves nutritional information from the database. Based on this information, the AI ​​module then makes predictions, analyzing the impact of calorie accumulation and weight gain over the next three months, for example. Furthermore, the emotion engine determines the user's emotional state and generates corresponding data.

[0394] Emotion engine processing

[0395] The emotion engine analyzes the user's emotional state and adjusts the nutritional information analysis accordingly—for example, if the user is stressed, the weight gain prediction will be emphasized.

[0396] Image generation AI processing

[0397] The image generation AI simulates the user's future appearance based on the adjusted nutritional information analysis results, and the generated image is displayed on the smart glasses or smartphone screen for easy visual understanding.

[0398] Specific examples

[0399] As a concrete use case, consider the scenario where a user scans a chocolate bar in a supermarket.

[0400] 1. A user scans the barcode on a chocolate bar using smart glasses.

[0401] 2. The barcode data is sent to a cloud server and nutritional information is retrieved.

[0402] 3. The server analyzes calories and ingredients based on nutritional information and generates a prediction result.

[0403] 4. The emotion engine recognizes the user's emotional state and adjusts predictions accordingly.

[0404] 5. Image generation AI generates a future image of the user based on the adjusted predictive data.

[0405] 6. The generated image is displayed on the smart glasses display, and advice on food choices is provided.

[0406] Prompt Sentence Examples

[0407] A user picks up a chocolate bar at the supermarket. The camera in the eyewear scans the barcode and retrieves nutritional information. At the same time, the emotion engine recognizes that the user is feeling stressed. Visually display this information to simulate what the user would look like if they continued to eat a chocolate bar every day for three months.

[0408] This allows users to receive comprehensive and personalized support regarding food choices and health management.

[0409] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0410] Step 1:

[0411] A user scans a food barcode with smart glasses in a brick-and-mortar store.

[0412] Input: Barcode data read using the smart glasses camera.

[0413] Output: HTTP request for barcode data to be sent to the cloud server.

[0414] The smart glasses activate their camera to scan the barcode of the food item selected by the user and send the data to a cloud server.

[0415] Step 2:

[0416] The cloud server receives the barcode data and searches the database to obtain the corresponding nutritional information.

[0417] Input: Barcode data received by the cloud server.

[0418] Output: Nutritional information (calories, protein, fat, carbohydrates, etc.).

[0419] The cloud server searches a database based on the barcode data and obtains nutritional information for related foods.

[0420] Step 3:

[0421] The acquired nutritional information is passed to an AI module to predict the long-term effects of food intake.

[0422] Input: Nutritional information.

[0423] Output: Data predicting the long-term effects of food intake (total calorie intake, predicted weight gain, etc.).

[0424] The AI ​​module in the server analyzes the data based on nutritional information and predicts, for example, accumulated calories and weight gain three months from now.

[0425] Step 4:

[0426] The server sends the analysis results to the user's device.

[0427] Input: Data projecting long-term effects.

[0428] Output: The HTTP response sent to the user device.

[0429] The cloud server compiles the prediction results and generates an HTTP response to send to the user's device.

[0430] Step 5:

[0431] The user terminal activates an emotion engine to recognize the user's emotional state.

[0432] Input: User's facial and voice data.

[0433] Output: Perceived emotional state (e.g., happy, sad, stressed).

[0434] The smart glasses' cameras and microphones are used to analyze the user's facial expressions and voice to recognize their emotional state.

[0435] Step 6:

[0436] Based on the recognized emotional state, image generation AI simulates the user's future appearance.

[0437] Input: Emotional state and prediction data.

[0438] Output: A representation of what the future user will look like.

[0439] The image generation AI simulates the user's future appearance based on their emotional state and predictive data, and generates an image to visually display.

[0440] Step 7:

[0441] The user's device displays the generated image and predicted data and provides health advice according to the user's emotional state.

[0442] Input: Generated image, predicted data, emotional state.

[0443] Output: Images and advice displayed to the user.

[0444] The generated images and predicted data are displayed on the smart glasses' display, and health advice is provided based on the emotional state recognized by the emotion engine.

[0445] Step 8:

[0446] The user makes food choices based on the information provided.

[0447] Input: The displayed image and health advice.

[0448] Output: The food choices selected by the user.

[0449] Users are provided with visual information and advice to help them make healthier food choices.

[0450] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0451] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0452] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0453] [Second embodiment]

[0454] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0455] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0456] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0457] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0458] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0459] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0460] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0461] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0462] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0463] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0464] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0465] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0466] The present invention is a system that uses image generation technology to display what the user will look like in a few months' time based on the nutritional information of a food item that the user scans by scanning its barcode. This system mainly consists of the following components:

[0467] System Components

[0468] 1. User Device

[0469] 2. Cloud Server

[0470] 3. Database

[0471] 4. Image generation AI

[0472] User terminal processing

[0473] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When a barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[0474] Cloud server processing

[0475] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[0476] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0477] Image generation AI processing

[0478] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI generates a realistic image of what the user will look like in a few months' time based on the predicted weight gain and changes in body fat percentage. The generated image is displayed on the device screen in a format that is easy for the user to view.

[0479] Specific examples

[0480] As a concrete example, the sequence of events when a user scans a chocolate bar is shown below.

[0481] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0482] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0483] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0484] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0485] 5. The server sends the analysis results and prediction data to the device.

[0486] 6. The device uses image generation AI to generate and display a simulated image of the user's appearance based on the predicted results three months from now, allowing the user to visualize what they will look like in a few months. Through this visualization, users can intuitively understand the impact of their dietary choices in the future and receive support to lead a healthier diet.

[0487] In this way, this system is a powerful tool that allows users to easily obtain nutritional information about foods and use that information to help manage their current and future health.

[0488] The processing flow will be explained below.

[0489] Step 1:

[0490] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scanning function. The device's camera then starts up, ready to scan the barcode.

[0491] Step 2:

[0492] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[0493] Step 3:

[0494] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[0495] Step 4:

[0496] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[0497] Step 5:

[0498] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[0499] Step 6:

[0500] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[0501] Step 7:

[0502] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[0503] Step 8:

[0504] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[0505] Step 9:

[0506] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[0507] Step 10:

[0508] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[0509] Step 11:

[0510] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[0511] Step 12:

[0512] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[0513] Step 13:

[0514] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[0515] Step 14:

[0516] Users can view simulated images displayed on their device, along with nutritional information and predictions, to visually understand how their dietary choices affect their body, helping them make healthier eating choices going forward.

[0517] Example 1

[0518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0519] In recent years, health management based on food nutritional information has become increasingly important. However, conventional systems do not simply acquire food nutritional information, but also lack the means to predict and visually confirm future health status based on that information. This makes health management difficult, and users have difficulty intuitively understanding future changes in their body.

[0520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0521] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information for that food product, means for transmitting the obtained nutritional information to a cloud server, means for searching a database based on the nutritional information received by the cloud server and obtaining the nutritional information for the corresponding food product, means for predicting the impact of consuming the food product on the user based on the obtained nutritional information, means for generating a prompt message based on the prediction result and using image generation technology to generate an image of the user several months from now, and means for displaying the generated image of the user. This allows users to easily obtain nutritional information for foods and visually check their future health status based on that information.

[0522] A "means for scanning food barcodes" is a process or device that reads food barcodes using a user terminal such as a smartphone or tablet.

[0523] The "means for obtaining nutritional information" refers to a process or device that scans a barcode and then obtains the nutritional information of the food item corresponding to the barcode from a database.

[0524] A "cloud server" is a remote server accessible via the Internet, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[0525] A "means for searching a database" is a process or device that queries a database in a cloud server and retrieves nutrition information corresponding to a particular barcode.

[0526] A "means for predicting the impact of food intake on a user" is a process or device that calculates and predicts the impact of food intake on a user's health status, such as weight and body fat percentage, based on acquired nutritional information.

[0527] A "means for generating a prompt sentence" is a process or device that generates textual instructions to be input to the image generation AI based on the prediction results.

[0528] "Means for generating what a user will look like in a few months' time using image generation technology" refers to technology or a device that generates an image that realistically simulates what the user will look like in a few months' time based on the generated prompt text.

[0529] The "means for displaying the generated image of the user" is a process or device for displaying the generated image on the screen of the user terminal.

[0530] This invention is a system that allows a user to scan a food's barcode to obtain nutritional information for that food, and then uses image generation technology to display what the user will look like several months from now. This system mainly consists of the following components: a user terminal, a cloud server, a database, and an image generation AI. How the system is specifically implemented using these components is described below.

[0531] User terminal processing

[0532] The user launches the "Food Scan Assistant" app using a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When the barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server. The user's device can be a standard smartphone (for example, an iPhone or Android device).

[0533] Cloud server processing

[0534] The cloud server receives the barcode data sent from the device. It searches its internal database based on the received barcode data to obtain the nutritional information for the corresponding food. This nutritional information includes calories, protein, fat, carbohydrates, etc. The cloud server uses a remote server such as an EC2 instance on Amazon Web Services (AWS). The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0535] Image generation AI processing

[0536] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI uses a generative model such as Stable Diffusion or DALL-E to generate a realistic image of what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage. The generated image is then displayed on the device screen in a format that is easy for the user to view.

[0537] Specific examples

[0538] As a concrete example, the following shows the sequence of events when a user scans a chocolate bar. The user scans the barcode on the chocolate bar with their smartphone. The device reads the barcode "1234567890123" and sends the data to a cloud server. The server receives the barcode "1234567890123" and retrieves the corresponding chocolate bar's nutritional information (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g) from a database. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months. The server then sends the analysis results and predicted data to the device. The device then uses image generation AI to generate and display an image simulating what the user will look like three months from now based on the predicted results.

[0539] Prompt Sentence Examples

[0540] Here are some examples of specific prompts to input to a generative AI model:

[0541] "Generate an image of the user based on the following data:

[0542] Current weight: 70kg

[0543] Predicted weight after 3 months: 75kg

[0544] Increase in body fat percentage: 3%

[0545] Display the user's appearance after 3 months based on these predictions."

[0546] According to this invention, users can easily obtain nutritional information about foods and use that information to help manage their current and future health.

[0547] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0548] Step 1:

[0549] The user launches the "Food Scan Assistant" app.

[0550] Input: Launching an app on a smartphone or tablet.

[0551] What happens: The user selects barcode scan mode within the app and launches the camera.

[0552] Output: The camera is activated and ready to scan a barcode.

[0553] Step 2:

[0554] The user scans the food barcode with the camera.

[0555] Input: Barcode displayed on your smartphone or tablet camera.

[0556] How it works: The camera reads barcode data in real time and converts it into text.

[0557] Output: Stringified barcode data.

[0558] Step 3:

[0559] The terminal transmits the barcode data converted into a string to the cloud server.

[0560] Input: The string data of the scanned barcode.

[0561] Operation: The device generates an HTTP request and sends it to the cloud server.

[0562] Output: The HTTP request sent to the cloud server.

[0563] Step 4:

[0564] The cloud server receives the barcode data and searches the database.

[0565] Input: Barcode data in the HTTP request.

[0566] How it works: The cloud server queries the AWS RDS database and retrieves the corresponding nutrition information.

[0567] Output: Nutritional information (calories, protein, fat, carbohydrates) for the food.

[0568] Step 5:

[0569] The cloud server uses an AI module to perform analysis based on nutritional information.

[0570] Input: Nutritional information of the retrieved food.

[0571] How it works: The AI ​​module predicts the effects of eating a food once a day for three months, calculating total calorie intake and predicted weight gain.

[0572] Output: Analysis results and forecast data.

[0573] Step 6:

[0574] The cloud server sends the analysis results and prediction data to the user's device.

[0575] Input: Analysis results and forecast data.

[0576] Operation: The cloud server generates an HTTP response and sends it to the user device.

[0577] Output: The HTTP response sent to the user's device.

[0578] Step 7:

[0579] The image generation AI is activated based on the analysis results and prediction data received by the user device.

[0580] Input: Analysis results and prediction data received from the cloud server.

[0581] How it works: The device generates a prompt and inputs it into the image generation AI.

[0582] Output: The prompt and the generated image.

[0583] Step 8:

[0584] The user terminal displays the generated image.

[0585] Input: An image generated based on the prompt sentence.

[0586] Action: The device visually displays the image to the user.

[0587] Output: A simulated image after 3 months displayed on the user's screen.

[0588] Through the above steps, the user can easily obtain nutritional information about foods and visually check their future health status based on that information.

[0589] (Application example 1)

[0590] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0591] Previously, when shopping in a physical store, there was no way to know in real time the impact that the food being purchased would have on the user's health. This made it difficult for users to accurately understand the nutritional value of foods and make choices that took future health effects into account. Furthermore, existing systems only provided abstract nutritional information, and were insufficient in providing information in a visually easy-to-understand format. To solve this problem, a system is needed that obtains the nutritional information of foods when purchasing food in a physical store and visualizes the user's future state.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0593] In this invention, the server includes means for identifying a food identifier and acquiring nutritional information for the food, means for analyzing the acquired nutritional information and predicting the impact that food intake will have on the user, means for generating an image of the user several months from now based on the prediction, means for displaying the generated image of the user, and means for scanning the identifier with the user's mobile device while shopping in a physical store. This allows users to obtain detailed nutritional information for the food they purchase while shopping in a physical store and visualize their appearance several months from now, enabling them to make healthier food choices.

[0594] A "food identifier" is information such as a barcode or QR code that identifies a specific food item.

[0595] "Nutritional information" refers to data on nutrients such as energy, protein, fat, and carbohydrates contained in food.

[0596] "Performing an analysis" refers to the process of conducting mathematical or statistical processing on the nutritional information obtained to assess the impact of food intake.

[0597] "Predictive methods" refer to algorithms or software that use nutritional information to predict future changes in a user's health and body shape.

[0598] "Image generation means" refers to computer graphics or generative AI models that visually represent the future user's appearance based on the prediction results.

[0599] "Means for displaying" refers to a display or application for outputting the generated image to a terminal screen in a format that can be viewed by the user.

[0600] "Brick and mortar store" refers to a physical retail store where customers can physically visit and purchase products.

[0601] A "mobile terminal" is a device with communication capabilities that can be carried by a user, and in this context it mainly refers to a smartphone.

[0602] System Overview

[0603] This system uses a smartphone to obtain nutritional information about food when purchasing it in a physical store, and visualizes future health effects based on that information. The system consists of the following main components: a mobile device, a cloud server, a database, and an image-generating AI.

[0604] Processing steps

[0605] 1. Mobile device operation:

[0606] Users can launch the Smart Nutritional Shopping Assistant application at a physical store and scan the barcode of the food they plan to purchase using the camera function on their mobile device.

[0607] 2. Data transmission:

[0608] The mobile device sends the scanned barcode data to the cloud server, generating an HTTP request to send the barcode data to the server.

[0609] 3. Cloud Server Processing:

[0610] The cloud server searches the database based on the received barcode data and retrieves the nutritional information for the food, including energy, protein, fat, carbohydrates, etc.

[0611] 4. Prediction of health effects:

[0612] The cloud server uses the acquired nutritional information to instruct the AI ​​module to analyze and predict the health effects of consuming that food over a certain period of time. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain.

[0613] 5. Send prediction results:

[0614] The cloud server sends the predicted health effects to a mobile device, which provides the results as predicted data on calorie intake and weight gain.

[0615] 6. Image generation:

[0616] The mobile device launches an image generation AI based on the predicted data received from the cloud server, which then simulates what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage.

[0617] 7. Displaying images:

[0618] The mobile device displays the generated images in an easy-to-view format, allowing users to intuitively understand the impact of the foods they purchase on their future health.

[0619] Hardware and software used

[0620] Mobile device: smartphone.

[0621] Cloud server: Data processing server (e.g., Amazon Web Services, Google Cloud Platform).

[0622] Database: A database system (e.g., MySQL, MongoDB) that stores nutrition information.

[0623] Image generation AI: AI modules for image generation (e.g., TensorFlow, PyTorch).

[0624] Specific examples

[0625] Let's take the example of a user purchasing potato chips at a physical store. The user uses their smartphone to scan the barcode on the potato chips. The mobile device sends the barcode "0123456789012" to a cloud server, which retrieves nutritional information (energy: 150 kcal, fat: 10 g, carbohydrates: 15 g) from a database. The server uses an AI module to predict weight gain and other factors if the user consumes this daily for three months, and sends the prediction results to the mobile device. The mobile device then activates an image generation AI to simulate and display what the user will look like in a few months' time.

[0626] Prompt Sentence Examples

[0627] "Scan food barcodes to get nutritional information and simulate what you'll look like in three months if you eat that food every day."

[0628] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0629] Step 1:

[0630] A user uses a smartphone to scan the barcode of a food item they plan to purchase in a physical store. The input is an image of the barcode captured by the smartphone's camera. The smartphone uses the pyzbar library to decode the barcode from the image and outputs the barcode data as a string.

[0631] Step 2:

[0632] The device sends the acquired barcode data to the cloud server. The input is the barcode data from step 1, and the output is the result of sending an HTTP request to the cloud server. Using the requests library, an HTTP POST request is generated to send the barcode data to the server.

[0633] Step 3:

[0634] The server searches a database based on the received barcode data to obtain the nutritional information of the corresponding food. The input is the received barcode data, and the output is the nutritional information of the corresponding food. The server uses SQL queries to search for the nutritional information of the food from a database (e.g., MySQL, MongoDB) and obtains data such as calories, protein, fat, and carbohydrates.

[0635] Step 4:

[0636] The cloud server then instructs the AI ​​module to analyze the nutritional information and predict the health effects of consuming the food over a certain period of time. The input is the nutritional information obtained from Step 3, and the output is predicted data on health effects. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain. For example, TensorFlow or PyTorch can be used for the analysis.

[0637] Step 5:

[0638] The cloud server sends the predicted health impact results to the mobile device. The input is the predicted data from step 4, and the output is the analysis results sent as an HTTP response to the mobile device. The server generates an HTTP response and sends the predicted data to the device in JSON format.

[0639] Step 6:

[0640] The mobile device launches an image generation AI based on the predicted data received from the cloud server. The input is the received predicted data, and the output is an image that simulates the generated future appearance of the user. The image generation AI generates the user's future appearance taking into account predicted weight gain and changes in body fat percentage. This process uses models such as GAN (generative adversarial networks).

[0641] Step 7:

[0642] The mobile device displays the generated image to the user. The input is the image data generated in step 6, and the output is the image of the future user displayed on the smartphone display. Through this visual information, the user can intuitively understand the impact of the food they plan to purchase on their future health.

[0643] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0644] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[0645] System Components

[0646] 1. User Device

[0647] 2. Cloud Server

[0648] 3. Database

[0649] 4. Image generation AI

[0650] 5. Emotion Engine

[0651] User terminal processing

[0652] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When a barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[0653] Cloud server processing

[0654] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[0655] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0656] Emotion engine processing

[0657] The user device uses the analysis results and prediction data received from the server to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, input, etc., and recognizes the user's current emotional state. The recognized emotional state is classified as, for example, "happiness," "sadness," or "stress."

[0658] Image generation AI processing

[0659] The image generation AI adjusts the parameters used to simulate the user's appearance depending on the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gradually.

[0660] Displaying Information

[0661] The user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[0662] Specific examples

[0663] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[0664] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0665] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0666] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0667] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0668] 5. The server sends the analysis results and prediction data to the device.

[0669] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[0670] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[0671] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[0672] The system allows users to receive specific advice based on their dietary habits and emotional state, providing support to lead a healthier life.

[0673] The processing flow will be explained below.

[0674] Step 1:

[0675] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scanning function. The device's camera then starts up, ready to scan the barcode.

[0676] Step 2:

[0677] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[0678] Step 3:

[0679] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[0680] Step 4:

[0681] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[0682] Step 5:

[0683] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[0684] Step 6:

[0685] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[0686] Step 7:

[0687] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[0688] Step 8:

[0689] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[0690] Step 9:

[0691] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[0692] Step 10:

[0693] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[0694] Step 11:

[0695] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[0696] Step 12:

[0697] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[0698] Step 13:

[0699] The device activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize their current emotional state, such as "happiness," "sadness," or "stress."

[0700] Step 14:

[0701] The parameters of the image generation AI are adjusted based on the emotional state recognized by the device's emotion engine. For example, if "stress" is detected, the settings are changed to emphasize the simulation of weight gain.

[0702] Step 15:

[0703] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[0704] Step 16:

[0705] Along with the generated image, the device will display advice and food choice recommendations based on the user's emotional state, such as "You're under a lot of stress, so avoid sweets."

[0706] Step 17:

[0707] Users can view simulated images displayed on their device, along with nutritional information, predictions, and emotional advice, providing specific guidance on diet and health management. This understanding helps users make healthier eating choices going forward.

[0708] Example 2

[0709] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0710] While conventional food nutrition information acquisition systems allow users to obtain nutritional information about foods, it is difficult for them to obtain long-term predictions based on that information or advice tailored to their emotional state. Furthermore, since these systems do not simply display nutritional information but also provide insufficient support for health management that takes into account the user's emotions, it is difficult to maintain the user's motivation.

[0711] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0712] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information about the food product; means for analyzing the obtained nutritional information and predicting the impact of consuming the food product on the user; image generation means for generating an image of the user several months from now based on the prediction; means for displaying the generated image of the user; means for recognizing the user's emotional state; and image generation means for adjusting the prediction result based on the user's emotional state. This allows the user to not only obtain nutritional information about the food product but also receive health management advice that predicts the long-term impact of the food product. Furthermore, the simulation and advice provided reflect the user's emotions, thereby maintaining the user's motivation and improving the effectiveness of health management.

[0713] "Scanning food barcodes" means using an image capture device such as a camera to read the barcode attached to the food and obtain the information as digital data.

[0714] "Nutritional information" refers to data about the calories, protein, fat, carbohydrates, and other components contained in food.

[0715] "Performing analysis" means using specific algorithms and AI modules to process the data based on the acquired nutritional information and calculate the impact on the user.

[0716] "Predicting" means estimating future changes in the user's weight and health condition based on past data and current nutritional information.

[0717] An "image generation means" is a device or software module that uses a specific algorithm or AI technology to visually represent the user's future appearance based on a prediction.

[0718] "Display" means visually presenting the generated image or predicted data on the screen of the user terminal.

[0719] A "means for recognizing emotional state" is a device or software module that uses a camera or sensor to analyze a user's facial expressions, voice, input, etc., to identify their current emotional state.

[0720] "Adjusting prediction results based on emotional state" means changing or adjusting prediction results or generated images by taking into account the recognized emotional state of the user.

[0721] A "cloud server" is a remote computing resource accessible via a network, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[0722] MODE FOR CARRYING OUT THE INVENTION

[0723] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[0724] 1. User device: smartphone, tablet, etc.

[0725] 2. Cloud server: A remote server that performs analysis and stores data.

[0726] 3. Database: Data storage for storing nutrition information.

[0727] 4. Image generation AI: An algorithm for generating the user's future appearance.

[0728] 5. Emotion Engine: A module for analyzing user emotions.

[0729] System Operation Overview

[0730] First, the user launches the "Food Scan Assistant" app on their smartphone and uses the function to scan the food barcode. The device's camera reads the barcode, converts it into a string, and creates and sends an HTTP request to the cloud server.

[0731] The cloud server searches its internal database based on the received barcode data and retrieves the nutritional information for the food in question. This information includes calories, protein, fat, carbohydrates, etc. This nutritional information is then passed to an AI module for detailed analysis, which predicts, for example, the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0732] The user device uses the received analysis results and prediction data to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize the user's current emotional state. For example, emotional states are classified as "joy," "sadness," "stress," etc.

[0733] The image generation AI adjusts simulation parameters depending on the user's perceived emotional state. For example, if the user is stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gently.

[0734] Finally, the user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[0735] Specific examples

[0736] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[0737] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0738] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0739] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0740] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0741] 5. The server sends the analysis results and prediction data to the device.

[0742] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[0743] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[0744] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[0745] Examples of prompt statements

[0746] "Explain how the system works when a user scans the barcode '1234567890123' on a chocolate bar and wants to know the nutritional information for that food and its effects after three months."

[0747] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0748] Step 1:

[0749] The user launches the "Food Scan Assistant" app on their smartphone, which displays the app's home screen.

[0750] Input: Touch operation on smartphone

[0751] Output: App home screen

[0752] Specifically, the user selects the barcode scanning function within the app and taps the option to launch the camera, which activates the smartphone's camera module.

[0753] Step 2:

[0754] The device reads the food barcode through the camera. The camera captures the barcode and obtains its image data. Then, it uses a barcode parsing library to decode the barcode image and convert it into string data.

[0755] Input: Camera image data

[0756] Output: Stringified barcode data (e.g. "1234567890123")

[0757] Specifically, the device extracts the barcode data and generates an HTTP request to send it to the cloud server, which includes the barcode data.

[0758] Step 3:

[0759] The device generates an HTTP request and sends it to the cloud server. Use the HTTP client library to send the request, including the barcode data.

[0760] Input: HTTP request (barcode data)

[0761] Output: Sending status (e.g. 200 OK)

[0762] Specifically, the device sends a request containing barcode data to a specific API endpoint, and the process is asynchronous, waiting for a response from the server.

[0763] Step 4:

[0764] The server receives the HTTP request, extracts the barcode data, and then searches a database to retrieve the corresponding food's nutritional information, including calories, protein, fat, carbohydrates, etc.

[0765] Input: HTTP request (barcode data)

[0766] Output: Nutritional information (e.g., Calories: 200 kcal, Protein: 2 g, Fat: 12 g, Carbohydrates: 24 g)

[0767] Specifically, the server executes a database query to obtain the nutritional information of the relevant food, and prepares to pass the obtained nutritional information to the AI ​​module.

[0768] Step 5:

[0769] The server's AI module performs a detailed analysis based on the nutritional information, for example, calculating the cumulative calorie intake and predicted weight gain if the food is eaten once a day for three months.

[0770] Input: Nutritional Information

[0771] Output: Predicted data (e.g., weight gain after 3 months)

[0772] Specifically, the server's AI module uses machine learning models to simulate the long-term effects of nutritional information and returns the predictions to the server.

[0773] Step 6:

[0774] The server receives the analysis results and prediction data and generates an HTTP response to send to the user's device, which includes nutritional information and prediction results.

[0775] Input: Forecast data

[0776] Output: HTTP response (nutritional information and prediction results)

[0777] Specifically, the server generates an HTTP response and sends it to the user's device. This response includes nutritional information and prediction data.

[0778] Step 7:

[0779] The user device receives the HTTP response from the server and activates the emotion engine, which uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize their current emotional state.

[0780] Input: HTTP response (nutritional information and prediction results), user input, facial expression, and voice data

[0781] Output: Perceived emotional state (e.g., stress)

[0782] Specifically, the user device runs an emotion engine to analyze various data about the user, and classifies the user's emotional state based on the analysis results.

[0783] Step 8:

[0784] The image generation AI on the user's device adjusts its predictions based on the perceived emotional state, and uses the generative AI model to generate a simulated image of what the user will look like in, say, three months.

[0785] Input: Emotional state, prediction data

[0786] Output: Adjusted simulation image

[0787] Specifically, the image generation AI on the user device adjusts the parameters of the image to be generated based on the output of the emotion engine, and generates a visual simulation image.

[0788] Step 9:

[0789] The user device visually displays the generated image and predicted data to the user, and also displays food selection recommendations and advice according to the emotional state recognized by the emotion engine.

[0790] Input: simulated image, predicted data, recognized emotional state

[0791] Output: Images and advice visually presented to the user

[0792] Specifically, the user terminal displays data on a user interface and provides information in a format that the user can intuitively understand.

[0793] (Application example 2)

[0794] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0795] Conventional nutrition information acquisition systems can analyze the nutritional information of foods, but they do not provide comprehensive health advice that takes into account the user's emotional state. As a result, users find it difficult to make food choices that take into account their emotional state and stress level, resulting in insufficient long-term health management. To solve this problem, a system is needed that recognizes the user's emotional state and provides accurate advice on food selection based on that state.

[0796] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for scanning the barcode of a food product and acquiring nutritional information about the food product, means for performing analysis based on the acquired nutritional information and predicting the impact that food intake will have on the user, means for recognizing the user's emotional state, image generation means for generating an image of the user several months from now based on the prediction, and means for displaying advice based on the generated image of the user and their emotional state. This enables the user to receive comprehensive and personalized support regarding food selection and health management.

[0797] "Nutritional information" refers to data on the ingredients contained in food, such as calories, protein, fat, and carbohydrates.

[0798] "Emotional state" is information that represents the psychological state of the user, and includes specific emotions such as joy, sadness, and stress.

[0799] "Means for scanning barcodes" refers to the ability to use the camera of a device such as a smartphone or smart glasses to read the barcode of a food item and obtain its data.

[0800] The "means for performing analysis" is a function that performs calculations and data processing to predict the user's health condition and future weight changes based on the acquired nutritional information.

[0801] A "predictor" is an algorithm that estimates the impact of food intake on a user based on nutritional information and multiple future scenarios.

[0802] The "image generation means" is a combination of software and hardware that uses the acquired data and analysis results to visually reproduce the user's future appearance.

[0803] "Display means" is a function for informing the user of the generated images, nutritional information, analysis results, and advice using a display such as a user terminal or smart glasses.

[0804] A "cloud server" is a remote server that stores, processes, and transfers data over the Internet, and supports data processing for the entire system.

[0805] The "means of acquisition" is a function that scans the barcode, sends the data obtained to a cloud server, and retrieves the corresponding nutritional information.

[0806] "Means of recognition" refers to a function that analyzes the user's facial expressions and voice through a camera or microphone to determine their current emotional state.

[0807] This invention is a system that supports health management by allowing users to scan food barcodes to obtain nutritional information, conducting analysis based on that information, and taking into account the user's emotional state. The system includes the following components: a user terminal, a cloud server, a database, an image generation AI, and an emotion engine. This allows users to make more effective food choices and manage their health.

[0808] System Components

[0809] 1. User Device

[0810] The user uses a smartphone or smart glasses as the device, launches the "Food Scan Assistant" application, and scans the food barcode.

[0811] 2. Cloud Server

[0812] It receives data and searches for nutritional information in a database, analyzes the nutritional content of multiple foods in detail, and works with an emotion engine to make predictions based on the user's emotions.

[0813] 3. Database

[0814] It stores nutritional information for foods and responds to searches from a cloud server.

[0815] 4. Image generation AI

[0816] The system simulates the user's future appearance based on the user's emotional state and nutritional information.

[0817] 5. Emotion Engine

[0818] It analyzes the user's facial expressions and voice to recognize their current emotional state.

[0819] Implementation details

[0820] User terminal processing

[0821] When a user scans a food barcode in a physical store, the camera in the smart glasses or smartphone reads it and sends the barcode data to a cloud server, which also uses the camera and microphone in the smart glasses to analyze the user's emotional state.

[0822] Cloud server processing

[0823] The cloud server receives the barcode data and retrieves nutritional information from the database. Based on this information, the AI ​​module then makes predictions, analyzing the impact of calorie accumulation and weight gain over the next three months, for example. Furthermore, the emotion engine determines the user's emotional state and generates corresponding data.

[0824] Emotion engine processing

[0825] The emotion engine analyzes the user's emotional state and adjusts the nutritional information analysis accordingly—for example, if the user is stressed, the weight gain prediction will be emphasized.

[0826] Image generation AI processing

[0827] The image generation AI simulates the user's future appearance based on the adjusted nutritional information analysis results, and the generated image is displayed on the smart glasses or smartphone screen for easy visual understanding.

[0828] Specific examples

[0829] As a concrete use case, consider the scenario where a user scans a chocolate bar in a supermarket.

[0830] 1. A user scans the barcode on a chocolate bar using smart glasses.

[0831] 2. The barcode data is sent to a cloud server and nutritional information is retrieved.

[0832] 3. The server analyzes calories and ingredients based on nutritional information and generates a prediction result.

[0833] 4. The emotion engine recognizes the user's emotional state and adjusts predictions accordingly.

[0834] 5. Image generation AI generates a future image of the user based on the adjusted predictive data.

[0835] 6. The generated image is displayed on the smart glasses display, and advice on food choices is provided.

[0836] Prompt Sentence Examples

[0837] A user picks up a chocolate bar at the supermarket. The camera in the eyewear scans the barcode and retrieves nutritional information. At the same time, the emotion engine recognizes that the user is feeling stressed. Visually display this information to simulate what the user would look like if they continued to eat a chocolate bar every day for three months.

[0838] This allows users to receive comprehensive and personalized support regarding food choices and health management.

[0839] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0840] Step 1:

[0841] A user scans a food barcode with smart glasses in a brick-and-mortar store.

[0842] Input: Barcode data read using the smart glasses camera.

[0843] Output: HTTP request for barcode data to be sent to the cloud server.

[0844] The smart glasses activate their camera to scan the barcode of the food item selected by the user and send the data to a cloud server.

[0845] Step 2:

[0846] The cloud server receives the barcode data and searches the database to obtain the corresponding nutritional information.

[0847] Input: Barcode data received by the cloud server.

[0848] Output: Nutritional information (calories, protein, fat, carbohydrates, etc.).

[0849] The cloud server searches a database based on the barcode data and obtains nutritional information for related foods.

[0850] Step 3:

[0851] The acquired nutritional information is passed to an AI module to predict the long-term effects of food intake.

[0852] Input: Nutritional information.

[0853] Output: Data predicting the long-term effects of food intake (total calorie intake, predicted weight gain, etc.).

[0854] The AI ​​module in the server analyzes the data based on nutritional information and predicts, for example, accumulated calories and weight gain three months from now.

[0855] Step 4:

[0856] The server sends the analysis results to the user's device.

[0857] Input: Data projecting long-term effects.

[0858] Output: The HTTP response sent to the user device.

[0859] The cloud server compiles the prediction results and generates an HTTP response to send to the user's device.

[0860] Step 5:

[0861] The user terminal activates an emotion engine to recognize the user's emotional state.

[0862] Input: User's facial and voice data.

[0863] Output: Perceived emotional state (e.g., happy, sad, stressed).

[0864] The smart glasses' cameras and microphones are used to analyze the user's facial expressions and voice to recognize their emotional state.

[0865] Step 6:

[0866] Based on the recognized emotional state, image generation AI simulates the user's future appearance.

[0867] Input: Emotional state and prediction data.

[0868] Output: A representation of what the future user will look like.

[0869] The image generation AI simulates the user's future appearance based on their emotional state and predictive data, and generates an image to visually display.

[0870] Step 7:

[0871] The user's device displays the generated image and predicted data and provides health advice according to the user's emotional state.

[0872] Input: Generated image, predicted data, emotional state.

[0873] Output: Images and advice displayed to the user.

[0874] The generated images and predicted data are displayed on the smart glasses' display, and health advice is provided based on the emotional state recognized by the emotion engine.

[0875] Step 8:

[0876] The user makes food choices based on the information provided.

[0877] Input: The displayed image and health advice.

[0878] Output: The food choices selected by the user.

[0879] Users are provided with visual information and advice to help them make healthier food choices.

[0880] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0881] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0882] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0883] [Third embodiment]

[0884] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0885] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0886] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0887] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0888] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0889] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0890] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0891] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0892] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0893] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0894] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0895] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0896] The present invention is a system that uses image generation technology to display what the user will look like in a few months' time based on the nutritional information of a food item that the user scans by scanning its barcode. This system mainly consists of the following components:

[0897] System Components

[0898] 1. User Device

[0899] 2. Cloud Server

[0900] 3. Database

[0901] 4. Image generation AI

[0902] User terminal processing

[0903] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When a barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[0904] Cloud server processing

[0905] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[0906] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0907] Image generation AI processing

[0908] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI generates a realistic image of what the user will look like in a few months' time based on the predicted weight gain and changes in body fat percentage. The generated image is displayed on the device screen in a format that is easy for the user to view.

[0909] Specific examples

[0910] As a concrete example, the sequence of events when a user scans a chocolate bar is shown below.

[0911] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[0912] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[0913] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[0914] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[0915] 5. The server sends the analysis results and prediction data to the device.

[0916] 6. The device uses image generation AI to generate and display a simulated image of the user's appearance based on the predicted results three months from now, allowing the user to visualize what they will look like in a few months. Through this visualization, users can intuitively understand the impact of their dietary choices in the future and receive support to lead a healthier diet.

[0917] In this way, this system is a powerful tool that allows users to easily obtain nutritional information about foods and use that information to help manage their current and future health.

[0918] The processing flow will be explained below.

[0919] Step 1:

[0920] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scanning function. The device's camera then starts up, ready to scan the barcode.

[0921] Step 2:

[0922] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[0923] Step 3:

[0924] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[0925] Step 4:

[0926] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[0927] Step 5:

[0928] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[0929] Step 6:

[0930] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[0931] Step 7:

[0932] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[0933] Step 8:

[0934] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[0935] Step 9:

[0936] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[0937] Step 10:

[0938] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[0939] Step 11:

[0940] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[0941] Step 12:

[0942] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[0943] Step 13:

[0944] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[0945] Step 14:

[0946] Users can view simulated images displayed on their device, along with nutritional information and predictions, to visually understand how their dietary choices affect their body, helping them make healthier eating choices going forward.

[0947] Example 1

[0948] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0949] In recent years, health management based on food nutritional information has become increasingly important. However, conventional systems do not simply acquire food nutritional information, but also lack the means to predict and visually confirm future health status based on that information. This makes health management difficult, and users have difficulty intuitively understanding future changes in their body.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0951] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information for that food product, means for transmitting the obtained nutritional information to a cloud server, means for searching a database based on the nutritional information received by the cloud server and obtaining the nutritional information for the corresponding food product, means for predicting the impact of consuming the food product on the user based on the obtained nutritional information, means for generating a prompt message based on the prediction result and using image generation technology to generate an image of the user several months from now, and means for displaying the generated image of the user. This allows users to easily obtain nutritional information for foods and visually check their future health status based on that information.

[0952] A "means for scanning food barcodes" is a process or device that reads food barcodes using a user terminal such as a smartphone or tablet.

[0953] The "means for obtaining nutritional information" refers to a process or device that scans a barcode and then obtains the nutritional information of the food item corresponding to the barcode from a database.

[0954] A "cloud server" is a remote server accessible via the Internet, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[0955] A "means for searching a database" is a process or device that queries a database in a cloud server and retrieves nutrition information corresponding to a particular barcode.

[0956] A "means for predicting the impact of food intake on a user" is a process or device that calculates and predicts the impact of food intake on a user's health status, such as weight and body fat percentage, based on acquired nutritional information.

[0957] A "means for generating a prompt sentence" is a process or device that generates textual instructions to be input to the image generation AI based on the prediction results.

[0958] "Means for generating what a user will look like in a few months' time using image generation technology" refers to technology or a device that generates an image that realistically simulates what the user will look like in a few months' time based on the generated prompt text.

[0959] The "means for displaying the generated image of the user" is a process or device for displaying the generated image on the screen of the user terminal.

[0960] This invention is a system that allows a user to scan a food's barcode to obtain nutritional information for that food, and then uses image generation technology to display what the user will look like several months from now. This system mainly consists of the following components: a user terminal, a cloud server, a database, and an image generation AI. How the system is specifically implemented using these components is described below.

[0961] User terminal processing

[0962] The user launches the "Food Scan Assistant" app using a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When the barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server. The user's device can be a standard smartphone (for example, an iPhone or Android device).

[0963] Cloud server processing

[0964] The cloud server receives the barcode data sent from the device. It searches its internal database based on the received barcode data to obtain the nutritional information for the corresponding food. This nutritional information includes calories, protein, fat, carbohydrates, etc. The cloud server uses a remote server such as an EC2 instance on Amazon Web Services (AWS). The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[0965] Image generation AI processing

[0966] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI uses a generative model such as Stable Diffusion or DALL-E to generate a realistic image of what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage. The generated image is then displayed on the device screen in a format that is easy for the user to view.

[0967] Specific examples

[0968] As a concrete example, the following shows the sequence of events when a user scans a chocolate bar. The user scans the barcode on the chocolate bar with their smartphone. The device reads the barcode "1234567890123" and sends the data to a cloud server. The server receives the barcode "1234567890123" and retrieves the corresponding chocolate bar's nutritional information (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g) from a database. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months. The server then sends the analysis results and predicted data to the device. The device then uses image generation AI to generate and display an image simulating what the user will look like three months from now based on the predicted results.

[0969] Prompt Sentence Examples

[0970] Here are some examples of specific prompts to input to a generative AI model:

[0971] "Generate an image of the user based on the following data:

[0972] Current weight: 70kg

[0973] Predicted weight after 3 months: 75kg

[0974] Increase in body fat percentage: 3%

[0975] Display the user's appearance after 3 months based on these predictions."

[0976] According to this invention, users can easily obtain nutritional information about foods and use that information to help manage their current and future health.

[0977] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0978] Step 1:

[0979] The user launches the "Food Scan Assistant" app.

[0980] Input: Launching an app on a smartphone or tablet.

[0981] What happens: The user selects barcode scan mode within the app and launches the camera.

[0982] Output: The camera is activated and ready to scan a barcode.

[0983] Step 2:

[0984] The user scans the food barcode with the camera.

[0985] Input: Barcode displayed on your smartphone or tablet camera.

[0986] How it works: The camera reads barcode data in real time and converts it into text.

[0987] Output: Stringified barcode data.

[0988] Step 3:

[0989] The terminal transmits the barcode data converted into a string to the cloud server.

[0990] Input: The string data of the scanned barcode.

[0991] Operation: The device generates an HTTP request and sends it to the cloud server.

[0992] Output: The HTTP request sent to the cloud server.

[0993] Step 4:

[0994] The cloud server receives the barcode data and searches the database.

[0995] Input: Barcode data in the HTTP request.

[0996] How it works: The cloud server queries the AWS RDS database and retrieves the corresponding nutrition information.

[0997] Output: Nutritional information (calories, protein, fat, carbohydrates) for the food.

[0998] Step 5:

[0999] The cloud server uses an AI module to perform analysis based on nutritional information.

[1000] Input: Nutritional information of the retrieved food.

[1001] How it works: The AI ​​module predicts the effects of eating a food once a day for three months, calculating total calorie intake and predicted weight gain.

[1002] Output: Analysis results and forecast data.

[1003] Step 6:

[1004] The cloud server sends the analysis results and prediction data to the user's device.

[1005] Input: Analysis results and forecast data.

[1006] Operation: The cloud server generates an HTTP response and sends it to the user device.

[1007] Output: The HTTP response sent to the user's device.

[1008] Step 7:

[1009] The image generation AI is activated based on the analysis results and prediction data received by the user device.

[1010] Input: Analysis results and prediction data received from the cloud server.

[1011] How it works: The device generates a prompt and inputs it into the image generation AI.

[1012] Output: The prompt and the generated image.

[1013] Step 8:

[1014] The user terminal displays the generated image.

[1015] Input: An image generated based on the prompt sentence.

[1016] Action: The device visually displays the image to the user.

[1017] Output: A simulated image after 3 months displayed on the user's screen.

[1018] Through the above steps, the user can easily obtain nutritional information about foods and visually check their future health status based on that information.

[1019] (Application example 1)

[1020] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1021] Previously, when shopping in a physical store, there was no way to know in real time the impact that the food being purchased would have on the user's health. This made it difficult for users to accurately understand the nutritional value of foods and make choices that took future health effects into account. Furthermore, existing systems only provided abstract nutritional information, and were insufficient in providing information in a visually easy-to-understand format. To solve this problem, a system is needed that obtains the nutritional information of foods when purchasing food in a physical store and visualizes the user's future state.

[1022] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1023] In this invention, the server includes means for identifying a food identifier and acquiring nutritional information for the food, means for analyzing the acquired nutritional information and predicting the impact that food intake will have on the user, means for generating an image of the user several months from now based on the prediction, means for displaying the generated image of the user, and means for scanning the identifier with the user's mobile device while shopping in a physical store. This allows users to obtain detailed nutritional information for the food they purchase while shopping in a physical store and visualize their appearance several months from now, enabling them to make healthier food choices.

[1024] A "food identifier" is information such as a barcode or QR code that identifies a specific food item.

[1025] "Nutritional information" refers to data on nutrients such as energy, protein, fat, and carbohydrates contained in food.

[1026] "Performing an analysis" refers to the process of conducting mathematical or statistical processing on the nutritional information obtained to assess the impact of food intake.

[1027] "Predictive methods" refer to algorithms or software that use nutritional information to predict future changes in a user's health and body shape.

[1028] "Image generation means" refers to computer graphics or generative AI models that visually represent the future user's appearance based on the prediction results.

[1029] "Means for displaying" refers to a display or application for outputting the generated image to a terminal screen in a format that can be viewed by the user.

[1030] "Brick and mortar store" refers to a physical retail store where customers can physically visit and purchase products.

[1031] A "mobile terminal" is a device with communication capabilities that can be carried by a user, and in this context it mainly refers to a smartphone.

[1032] System Overview

[1033] This system uses a smartphone to obtain nutritional information about food when purchasing it in a physical store, and visualizes future health effects based on that information. The system consists of the following main components: a mobile device, a cloud server, a database, and an image-generating AI.

[1034] Processing steps

[1035] 1. Mobile device operation:

[1036] Users can launch the Smart Nutritional Shopping Assistant application at a physical store and scan the barcode of the food they plan to purchase using the camera function on their mobile device.

[1037] 2. Data transmission:

[1038] The mobile device sends the scanned barcode data to the cloud server, generating an HTTP request to send the barcode data to the server.

[1039] 3. Cloud Server Processing:

[1040] The cloud server searches the database based on the received barcode data and retrieves the nutritional information for the food, including energy, protein, fat, carbohydrates, etc.

[1041] 4. Prediction of health effects:

[1042] The cloud server uses the acquired nutritional information to instruct the AI ​​module to analyze and predict the health effects of consuming that food over a certain period of time. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain.

[1043] 5. Send prediction results:

[1044] The cloud server sends the predicted health effects to a mobile device, which provides the results as predicted data on calorie intake and weight gain.

[1045] 6. Image generation:

[1046] The mobile device launches an image generation AI based on the predicted data received from the cloud server, which then simulates what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage.

[1047] 7. Displaying images:

[1048] The mobile device displays the generated images in an easy-to-view format, allowing users to intuitively understand the impact of the foods they purchase on their future health.

[1049] Hardware and software used

[1050] Mobile device: smartphone.

[1051] Cloud server: Data processing server (e.g., Amazon Web Services, Google Cloud Platform).

[1052] Database: A database system (e.g., MySQL, MongoDB) that stores nutrition information.

[1053] Image generation AI: AI modules for image generation (e.g., TensorFlow, PyTorch).

[1054] Specific examples

[1055] Let's take the example of a user purchasing potato chips at a physical store. The user uses their smartphone to scan the barcode on the potato chips. The mobile device sends the barcode "0123456789012" to a cloud server, which retrieves nutritional information (energy: 150 kcal, fat: 10 g, carbohydrates: 15 g) from a database. The server uses an AI module to predict weight gain and other factors if the user consumes this daily for three months, and sends the prediction results to the mobile device. The mobile device then activates an image generation AI to simulate and display what the user will look like in a few months' time.

[1056] Prompt Sentence Examples

[1057] "Scan food barcodes to get nutritional information and simulate what you'll look like in three months if you eat that food every day."

[1058] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1059] Step 1:

[1060] A user uses a smartphone to scan the barcode of a food item they plan to purchase in a physical store. The input is an image of the barcode captured by the smartphone's camera. The smartphone uses the pyzbar library to decode the barcode from the image and outputs the barcode data as a string.

[1061] Step 2:

[1062] The device sends the acquired barcode data to the cloud server. The input is the barcode data from step 1, and the output is the result of sending an HTTP request to the cloud server. Using the requests library, an HTTP POST request is generated to send the barcode data to the server.

[1063] Step 3:

[1064] The server searches a database based on the received barcode data to obtain the nutritional information of the corresponding food. The input is the received barcode data, and the output is the nutritional information of the corresponding food. The server uses SQL queries to search for the nutritional information of the food from a database (e.g., MySQL, MongoDB) and obtains data such as calories, protein, fat, and carbohydrates.

[1065] Step 4:

[1066] The cloud server then instructs the AI ​​module to analyze the nutritional information and predict the health effects of consuming the food over a certain period of time. The input is the nutritional information obtained from Step 3, and the output is predicted data on health effects. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain. For example, TensorFlow or PyTorch can be used for the analysis.

[1067] Step 5:

[1068] The cloud server sends the predicted health impact results to the mobile device. The input is the predicted data from step 4, and the output is the analysis results sent as an HTTP response to the mobile device. The server generates an HTTP response and sends the predicted data to the device in JSON format.

[1069] Step 6:

[1070] The mobile device launches an image generation AI based on the predicted data received from the cloud server. The input is the received predicted data, and the output is an image that simulates the generated future appearance of the user. The image generation AI generates the user's future appearance taking into account predicted weight gain and changes in body fat percentage. This process uses models such as GAN (generative adversarial networks).

[1071] Step 7:

[1072] The mobile device displays the generated image to the user. The input is the image data generated in step 6, and the output is the image of the future user displayed on the smartphone display. Through this visual information, the user can intuitively understand the impact of the food they plan to purchase on their future health.

[1073] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1074] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[1075] System Components

[1076] 1. User Device

[1077] 2. Cloud Server

[1078] 3. Database

[1079] 4. Image generation AI

[1080] 5. Emotion Engine

[1081] User terminal processing

[1082] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When a barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[1083] Cloud server processing

[1084] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[1085] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[1086] Emotion engine processing

[1087] The user device uses the analysis results and prediction data received from the server to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, input, etc., and recognizes the user's current emotional state. The recognized emotional state is classified as, for example, "happiness," "sadness," or "stress."

[1088] Image generation AI processing

[1089] The image generation AI adjusts the parameters used to simulate the user's appearance depending on the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gradually.

[1090] Displaying Information

[1091] The user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[1092] Specific examples

[1093] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[1094] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[1095] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[1096] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[1097] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[1098] 5. The server sends the analysis results and prediction data to the device.

[1099] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[1100] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[1101] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[1102] The system allows users to receive specific advice based on their dietary habits and emotional state, providing support to lead a healthier life.

[1103] The processing flow will be explained below.

[1104] Step 1:

[1105] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scanning function. The device's camera then starts up, ready to scan the barcode.

[1106] Step 2:

[1107] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[1108] Step 3:

[1109] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[1110] Step 4:

[1111] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[1112] Step 5:

[1113] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[1114] Step 6:

[1115] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[1116] Step 7:

[1117] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[1118] Step 8:

[1119] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[1120] Step 9:

[1121] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[1122] Step 10:

[1123] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[1124] Step 11:

[1125] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[1126] Step 12:

[1127] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[1128] Step 13:

[1129] The device activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize their current emotional state, such as "happiness," "sadness," or "stress."

[1130] Step 14:

[1131] The parameters of the image generation AI are adjusted based on the emotional state recognized by the device's emotion engine. For example, if "stress" is detected, the settings are changed to emphasize the simulation of weight gain.

[1132] Step 15:

[1133] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[1134] Step 16:

[1135] Along with the generated image, the device will display advice and food choice recommendations based on the user's emotional state, such as "You're under a lot of stress, so avoid sweets."

[1136] Step 17:

[1137] Users can view simulated images displayed on their device, along with nutritional information, predictions, and emotional advice, providing specific guidance on diet and health management. This understanding helps users make healthier eating choices going forward.

[1138] Example 2

[1139] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1140] While conventional food nutrition information acquisition systems allow users to obtain nutritional information about foods, it is difficult for them to obtain long-term predictions based on that information or advice tailored to their emotional state. Furthermore, since these systems do not simply display nutritional information but also provide insufficient support for health management that takes into account the user's emotions, it is difficult to maintain the user's motivation.

[1141] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1142] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information about the food product; means for analyzing the obtained nutritional information and predicting the impact of consuming the food product on the user; image generation means for generating an image of the user several months from now based on the prediction; means for displaying the generated image of the user; means for recognizing the user's emotional state; and image generation means for adjusting the prediction result based on the user's emotional state. This allows the user to not only obtain nutritional information about the food product but also receive health management advice that predicts the long-term impact of the food product. Furthermore, the simulation and advice provided reflect the user's emotions, thereby maintaining the user's motivation and improving the effectiveness of health management.

[1143] "Scanning food barcodes" means using an image capture device such as a camera to read the barcode attached to the food and obtain the information as digital data.

[1144] "Nutritional information" refers to data about the calories, protein, fat, carbohydrates, and other components contained in food.

[1145] "Performing analysis" means using specific algorithms and AI modules to process the data based on the acquired nutritional information and calculate the impact on the user.

[1146] "Predicting" means estimating future changes in the user's weight and health condition based on past data and current nutritional information.

[1147] An "image generation means" is a device or software module that uses a specific algorithm or AI technology to visually represent the user's future appearance based on a prediction.

[1148] "Display" means visually presenting the generated image or predicted data on the screen of the user terminal.

[1149] A "means for recognizing emotional state" is a device or software module that uses a camera or sensor to analyze a user's facial expressions, voice, input, etc., to identify their current emotional state.

[1150] "Adjusting prediction results based on emotional state" means changing or adjusting prediction results or generated images by taking into account the recognized emotional state of the user.

[1151] A "cloud server" is a remote computing resource accessible via a network, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[1152] MODE FOR CARRYING OUT THE INVENTION

[1153] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[1154] 1. User device: smartphone, tablet, etc.

[1155] 2. Cloud server: A remote server that performs analysis and stores data.

[1156] 3. Database: Data storage for storing nutrition information.

[1157] 4. Image generation AI: An algorithm for generating the user's future appearance.

[1158] 5. Emotion Engine: A module for analyzing user emotions.

[1159] System Operation Overview

[1160] First, the user launches the "Food Scan Assistant" app on their smartphone and uses the function to scan the food barcode. The device's camera reads the barcode, converts it into a string, and creates and sends an HTTP request to the cloud server.

[1161] The cloud server searches its internal database based on the received barcode data and retrieves the nutritional information for the food in question. This information includes calories, protein, fat, carbohydrates, etc. This nutritional information is then passed to an AI module for detailed analysis, which predicts, for example, the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[1162] The user device uses the received analysis results and prediction data to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize the user's current emotional state. For example, emotional states are classified as "joy," "sadness," "stress," etc.

[1163] The image generation AI adjusts simulation parameters depending on the user's perceived emotional state. For example, if the user is stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gently.

[1164] Finally, the user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[1165] Specific examples

[1166] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[1167] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[1168] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[1169] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[1170] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[1171] 5. The server sends the analysis results and prediction data to the device.

[1172] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[1173] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[1174] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[1175] Examples of prompt statements

[1176] "Explain how the system works when a user scans the barcode '1234567890123' on a chocolate bar and wants to know the nutritional information for that food and its effects after three months."

[1177] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1178] Step 1:

[1179] The user launches the "Food Scan Assistant" app on their smartphone, which displays the app's home screen.

[1180] Input: Touch operation on smartphone

[1181] Output: App home screen

[1182] Specifically, the user selects the barcode scanning function within the app and taps the option to launch the camera, which activates the smartphone's camera module.

[1183] Step 2:

[1184] The device reads the food barcode through the camera. The camera captures the barcode and obtains its image data. Then, it uses a barcode parsing library to decode the barcode image and convert it into string data.

[1185] Input: Camera image data

[1186] Output: Stringified barcode data (e.g. "1234567890123")

[1187] Specifically, the device extracts the barcode data and generates an HTTP request to send it to the cloud server, which includes the barcode data.

[1188] Step 3:

[1189] The device generates an HTTP request and sends it to the cloud server. Use the HTTP client library to send the request, including the barcode data.

[1190] Input: HTTP request (barcode data)

[1191] Output: Sending status (e.g. 200 OK)

[1192] Specifically, the device sends a request containing barcode data to a specific API endpoint, and the process is asynchronous, waiting for a response from the server.

[1193] Step 4:

[1194] The server receives the HTTP request, extracts the barcode data, and then searches a database to retrieve the corresponding food's nutritional information, including calories, protein, fat, carbohydrates, etc.

[1195] Input: HTTP request (barcode data)

[1196] Output: Nutritional information (e.g., Calories: 200 kcal, Protein: 2 g, Fat: 12 g, Carbohydrates: 24 g)

[1197] Specifically, the server executes a database query to obtain the nutritional information of the relevant food, and prepares to pass the obtained nutritional information to the AI ​​module.

[1198] Step 5:

[1199] The server's AI module performs a detailed analysis based on the nutritional information, for example, calculating the cumulative calorie intake and predicted weight gain if the food is eaten once a day for three months.

[1200] Input: Nutritional Information

[1201] Output: Predicted data (e.g., weight gain after 3 months)

[1202] Specifically, the server's AI module uses machine learning models to simulate the long-term effects of nutritional information and returns the predictions to the server.

[1203] Step 6:

[1204] The server receives the analysis results and prediction data and generates an HTTP response to send to the user's device, which includes nutritional information and prediction results.

[1205] Input: Forecast data

[1206] Output: HTTP response (nutritional information and prediction results)

[1207] Specifically, the server generates an HTTP response and sends it to the user's device. This response includes nutritional information and prediction data.

[1208] Step 7:

[1209] The user device receives the HTTP response from the server and activates the emotion engine, which uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize their current emotional state.

[1210] Input: HTTP response (nutritional information and prediction results), user input, facial expression, and voice data

[1211] Output: Perceived emotional state (e.g., stress)

[1212] Specifically, the user device runs an emotion engine to analyze various data about the user, and classifies the user's emotional state based on the analysis results.

[1213] Step 8:

[1214] The image generation AI on the user's device adjusts its predictions based on the perceived emotional state, and uses the generative AI model to generate a simulated image of what the user will look like in, say, three months.

[1215] Input: Emotional state, prediction data

[1216] Output: Adjusted simulation image

[1217] Specifically, the image generation AI on the user device adjusts the parameters of the image to be generated based on the output of the emotion engine, and generates a visual simulation image.

[1218] Step 9:

[1219] The user device visually displays the generated image and predicted data to the user, and also displays food selection recommendations and advice according to the emotional state recognized by the emotion engine.

[1220] Input: simulated image, predicted data, recognized emotional state

[1221] Output: Images and advice visually presented to the user

[1222] Specifically, the user terminal displays data on a user interface and provides information in a format that the user can intuitively understand.

[1223] (Application example 2)

[1224] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1225] Conventional nutrition information acquisition systems can analyze the nutritional information of foods, but they do not provide comprehensive health advice that takes into account the user's emotional state. As a result, users find it difficult to make food choices that take into account their emotional state and stress level, resulting in insufficient long-term health management. To solve this problem, a system is needed that recognizes the user's emotional state and provides accurate advice on food selection based on that state.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for scanning the barcode of a food product and acquiring nutritional information about the food product, means for performing analysis based on the acquired nutritional information and predicting the impact that food intake will have on the user, means for recognizing the user's emotional state, image generation means for generating an image of the user several months from now based on the prediction, and means for displaying advice based on the generated image of the user and their emotional state. This enables the user to receive comprehensive and personalized support regarding food selection and health management.

[1227] "Nutritional information" refers to data on the ingredients contained in food, such as calories, protein, fat, and carbohydrates.

[1228] "Emotional state" is information that represents the psychological state of the user, and includes specific emotions such as joy, sadness, and stress.

[1229] "Means for scanning barcodes" refers to the ability to use the camera of a device such as a smartphone or smart glasses to read the barcode of a food item and obtain its data.

[1230] The "means for performing analysis" is a function that performs calculations and data processing to predict the user's health condition and future weight changes based on the acquired nutritional information.

[1231] A "predictor" is an algorithm that estimates the impact of food intake on a user based on nutritional information and multiple future scenarios.

[1232] The "image generation means" is a combination of software and hardware that uses the acquired data and analysis results to visually reproduce the user's future appearance.

[1233] "Display means" is a function for informing the user of the generated images, nutritional information, analysis results, and advice using a display such as a user terminal or smart glasses.

[1234] A "cloud server" is a remote server that stores, processes, and transfers data over the Internet, and supports data processing for the entire system.

[1235] The "means of acquisition" is a function that scans the barcode, sends the data obtained to a cloud server, and retrieves the corresponding nutritional information.

[1236] "Means of recognition" refers to a function that analyzes the user's facial expressions and voice through a camera or microphone to determine their current emotional state.

[1237] This invention is a system that supports health management by allowing users to scan food barcodes to obtain nutritional information, conducting analysis based on that information, and taking into account the user's emotional state. The system includes the following components: a user terminal, a cloud server, a database, an image generation AI, and an emotion engine. This allows users to make more effective food choices and manage their health.

[1238] System Components

[1239] 1. User Device

[1240] The user uses a smartphone or smart glasses as the device, launches the "Food Scan Assistant" application, and scans the food barcode.

[1241] 2. Cloud Server

[1242] It receives data and searches for nutritional information in a database, analyzes the nutritional content of multiple foods in detail, and works with an emotion engine to make predictions based on the user's emotions.

[1243] 3. Database

[1244] It stores nutritional information for foods and responds to searches from a cloud server.

[1245] 4. Image generation AI

[1246] The system simulates the user's future appearance based on the user's emotional state and nutritional information.

[1247] 5. Emotion Engine

[1248] It analyzes the user's facial expressions and voice to recognize their current emotional state.

[1249] Implementation details

[1250] User terminal processing

[1251] When a user scans a food barcode in a physical store, the camera in the smart glasses or smartphone reads it and sends the barcode data to a cloud server, which also uses the camera and microphone in the smart glasses to analyze the user's emotional state.

[1252] Cloud server processing

[1253] The cloud server receives the barcode data and retrieves nutritional information from the database. Based on this information, the AI ​​module then makes predictions, analyzing the impact of calorie accumulation and weight gain over the next three months, for example. Furthermore, the emotion engine determines the user's emotional state and generates corresponding data.

[1254] Emotion engine processing

[1255] The emotion engine analyzes the user's emotional state and adjusts the nutritional information analysis accordingly—for example, if the user is stressed, the weight gain prediction will be emphasized.

[1256] Image generation AI processing

[1257] The image generation AI simulates the user's future appearance based on the adjusted nutritional information analysis results, and the generated image is displayed on the smart glasses or smartphone screen for easy visual understanding.

[1258] Specific examples

[1259] As a concrete use case, consider the scenario where a user scans a chocolate bar in a supermarket.

[1260] 1. A user scans the barcode on a chocolate bar using smart glasses.

[1261] 2. The barcode data is sent to a cloud server and nutritional information is retrieved.

[1262] 3. The server analyzes calories and ingredients based on nutritional information and generates a prediction result.

[1263] 4. The emotion engine recognizes the user's emotional state and adjusts predictions accordingly.

[1264] 5. Image generation AI generates a future image of the user based on the adjusted predictive data.

[1265] 6. The generated image is displayed on the smart glasses display, and advice on food choices is provided.

[1266] Prompt Sentence Examples

[1267] A user picks up a chocolate bar at the supermarket. The camera in the eyewear scans the barcode and retrieves nutritional information. At the same time, the emotion engine recognizes that the user is feeling stressed. Visually display this information to simulate what the user would look like if they continued to eat a chocolate bar every day for three months.

[1268] This allows users to receive comprehensive and personalized support regarding food choices and health management.

[1269] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1270] Step 1:

[1271] A user scans a food barcode with smart glasses in a brick-and-mortar store.

[1272] Input: Barcode data read using the smart glasses camera.

[1273] Output: HTTP request for barcode data to be sent to the cloud server.

[1274] The smart glasses activate their camera to scan the barcode of the food item selected by the user and send the data to a cloud server.

[1275] Step 2:

[1276] The cloud server receives the barcode data and searches the database to obtain the corresponding nutritional information.

[1277] Input: Barcode data received by the cloud server.

[1278] Output: Nutritional information (calories, protein, fat, carbohydrates, etc.).

[1279] The cloud server searches a database based on the barcode data and obtains nutritional information for related foods.

[1280] Step 3:

[1281] The acquired nutritional information is passed to an AI module to predict the long-term effects of food intake.

[1282] Input: Nutritional information.

[1283] Output: Data predicting the long-term effects of food intake (total calorie intake, predicted weight gain, etc.).

[1284] The AI ​​module in the server analyzes the data based on nutritional information and predicts, for example, accumulated calories and weight gain three months from now.

[1285] Step 4:

[1286] The server sends the analysis results to the user's device.

[1287] Input: Data projecting long-term effects.

[1288] Output: The HTTP response sent to the user device.

[1289] The cloud server compiles the prediction results and generates an HTTP response to send to the user's device.

[1290] Step 5:

[1291] The user terminal activates an emotion engine to recognize the user's emotional state.

[1292] Input: User's facial and voice data.

[1293] Output: Perceived emotional state (e.g., happy, sad, stressed).

[1294] The smart glasses' cameras and microphones are used to analyze the user's facial expressions and voice to recognize their emotional state.

[1295] Step 6:

[1296] Based on the recognized emotional state, image generation AI simulates the user's future appearance.

[1297] Input: Emotional state and prediction data.

[1298] Output: A representation of what the future user will look like.

[1299] The image generation AI simulates the user's future appearance based on their emotional state and predictive data, and generates an image to visually display.

[1300] Step 7:

[1301] The user's device displays the generated image and predicted data and provides health advice according to the user's emotional state.

[1302] Input: Generated image, predicted data, emotional state.

[1303] Output: Images and advice displayed to the user.

[1304] The generated images and predicted data are displayed on the smart glasses' display, and health advice is provided based on the emotional state recognized by the emotion engine.

[1305] Step 8:

[1306] The user makes food choices based on the information provided.

[1307] Input: The displayed image and health advice.

[1308] Output: The food choices selected by the user.

[1309] Users are provided with visual information and advice to help them make healthier food choices.

[1310] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1311] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1312] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1313] [Fourth embodiment]

[1314] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1315] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1316] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1317] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1318] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1319] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1320] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1321] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1322] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1323] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1324] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1325] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1326] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1327] The present invention is a system that uses image generation technology to display what the user will look like in a few months' time based on the nutritional information of a food item that the user scans by scanning its barcode. This system mainly consists of the following components:

[1328] System Components

[1329] 1. User Device

[1330] 2. Cloud Server

[1331] 3. Database

[1332] 4. Image Generation AI

[1333] User terminal processing

[1334] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When the barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[1335] Cloud server processing

[1336] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[1337] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[1338] Image generation AI processing

[1339] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI generates a realistic image of what the user will look like in a few months' time based on the predicted weight gain and changes in body fat percentage. The generated image is displayed on the device screen in a format that is easy for the user to view.

[1340] Specific examples

[1341] As a concrete example, the sequence of events when a user scans a chocolate bar is shown below.

[1342] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[1343] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[1344] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[1345] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[1346] 5. The server sends the analysis results and prediction data to the device.

[1347] 6. The device uses image generation AI to generate and display a simulated image of the user's appearance based on the predicted results three months from now, allowing the user to visualize what they will look like in a few months. Through this visualization, users can intuitively understand the impact of their dietary choices in the future and receive support to lead a healthier diet.

[1348] In this way, this system is a powerful tool that allows users to easily obtain nutritional information about foods and use that information to help manage their current and future health.

[1349] The processing flow will be explained below.

[1350] Step 1:

[1351] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scan function. The device's camera then activates, ready to scan the barcode.

[1352] Step 2:

[1353] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[1354] Step 3:

[1355] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[1356] Step 4:

[1357] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[1358] Step 5:

[1359] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[1360] Step 6:

[1361] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[1362] Step 7:

[1363] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[1364] Step 8:

[1365] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[1366] Step 9:

[1367] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[1368] Step 10:

[1369] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[1370] Step 11:

[1371] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[1372] Step 12:

[1373] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[1374] Step 13:

[1375] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[1376] Step 14:

[1377] Users can view simulated images displayed on their device along with nutritional information and predictions to visually understand how their dietary choices affect their body, helping them make healthier eating choices going forward.

[1378] Example 1

[1379] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1380] In recent years, health management based on food nutritional information has become increasingly important. However, conventional systems do not simply acquire food nutritional information, but also lack the means to predict and visually confirm future health status based on that information. This makes health management difficult, and users have difficulty intuitively understanding future changes in their body.

[1381] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1382] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information for that food product, means for transmitting the obtained nutritional information to a cloud server, means for searching a database based on the nutritional information received by the cloud server and obtaining the nutritional information for the corresponding food product, means for predicting the impact of consuming the food product on the user based on the obtained nutritional information, means for generating a prompt message based on the prediction result and using image generation technology to generate an image of the user several months from now, and means for displaying the generated image of the user. This allows users to easily obtain nutritional information for foods and visually check their future health status based on that information.

[1383] A "means for scanning food barcodes" is a process or device that reads food barcodes using a user terminal such as a smartphone or tablet.

[1384] The "means for obtaining nutritional information" refers to a process or device that scans a barcode and then obtains the nutritional information of the food item corresponding to the barcode from a database.

[1385] A "cloud server" is a remote server accessible via the Internet, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[1386] A "means for searching a database" is a process or device that queries a database in a cloud server and retrieves nutrition information corresponding to a particular barcode.

[1387] A "means for predicting the impact of food intake on a user" is a process or device that calculates and predicts the impact of food intake on a user's health status, such as weight and body fat percentage, based on acquired nutritional information.

[1388] A "means for generating a prompt sentence" is a process or device that generates textual instructions to be input to the image generation AI based on the prediction results.

[1389] "Means for generating what a user will look like in a few months' time using image generation technology" refers to technology or a device that generates an image that realistically simulates what the user will look like in a few months' time based on the generated prompt text.

[1390] The "means for displaying the generated image of the user" is a process or device for displaying the generated image on the screen of the user terminal.

[1391] This invention is a system that allows a user to scan a food's barcode to obtain nutritional information for that food, and then uses image generation technology to display what the user will look like several months from now. This system mainly consists of the following components: a user terminal, a cloud server, a database, and an image generation AI. How the system is specifically implemented using these components is described below.

[1392] User terminal processing

[1393] The user launches the "Food Scan Assistant" app using a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When the barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server. The user's device can be a standard smartphone (for example, an iPhone or Android device).

[1394] Cloud server processing

[1395] The cloud server receives the barcode data sent from the device. It searches its internal database based on the received barcode data to obtain the nutritional information for the corresponding food. This nutritional information includes calories, protein, fat, carbohydrates, etc. The cloud server uses a remote server such as an EC2 instance on Amazon Web Services (AWS). The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[1396] Image generation AI processing

[1397] The user's device uses the analysis results and prediction data received from the server to launch the image generation AI. The image generation AI uses a generative model such as Stable Diffusion or DALL-E to generate a realistic image of what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage. The generated image is then displayed on the device screen in a format that is easy for the user to view.

[1398] Specific examples

[1399] As a concrete example, the following shows the sequence of events when a user scans a chocolate bar. The user scans the barcode on the chocolate bar with their smartphone. The device reads the barcode "1234567890123" and sends the data to a cloud server. The server receives the barcode "1234567890123" and retrieves the corresponding chocolate bar's nutritional information (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g) from a database. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months. The server then sends the analysis results and predicted data to the device. The device then uses image generation AI to generate and display an image simulating what the user will look like three months from now based on the predicted results.

[1400] Prompt Sentence Examples

[1401] Here are some examples of specific prompts to input to a generative AI model:

[1402] "Generate an image of the user based on the following data:

[1403] Current weight: 70kg

[1404] Predicted weight after 3 months: 75kg

[1405] Increase in body fat percentage: 3%

[1406] Display the user's appearance after 3 months based on these predictions."

[1407] According to this invention, users can easily obtain nutritional information about foods and use that information to help manage their current and future health.

[1408] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1409] Step 1:

[1410] The user launches the "Food Scan Assistant" app.

[1411] Input: Launching an app on a smartphone or tablet.

[1412] What happens: The user selects barcode scan mode within the app and launches the camera.

[1413] Output: The camera is activated and ready to scan a barcode.

[1414] Step 2:

[1415] The user scans the food barcode with the camera.

[1416] Input: Barcode displayed on your smartphone or tablet camera.

[1417] How it works: The camera reads barcode data in real time and converts it into text.

[1418] Output: Stringified barcode data.

[1419] Step 3:

[1420] The terminal transmits the barcode data converted into a string to the cloud server.

[1421] Input: The string data of the scanned barcode.

[1422] Operation: The device generates an HTTP request and sends it to the cloud server.

[1423] Output: The HTTP request sent to the cloud server.

[1424] Step 4:

[1425] The cloud server receives the barcode data and searches the database.

[1426] Input: Barcode data in the HTTP request.

[1427] How it works: The cloud server queries the AWS RDS database and retrieves the corresponding nutrition information.

[1428] Output: Nutritional information (calories, protein, fat, carbohydrates) for the food.

[1429] Step 5:

[1430] The cloud server uses an AI module to perform analysis based on nutritional information.

[1431] Input: Nutritional information of the retrieved food.

[1432] How it works: The AI ​​module predicts the effects of eating a food once a day for three months, calculating total calorie intake and predicted weight gain.

[1433] Output: Analysis results and forecast data.

[1434] Step 6:

[1435] The cloud server sends the analysis results and prediction data to the user's device.

[1436] Input: Analysis results and forecast data.

[1437] Operation: The cloud server generates an HTTP response and sends it to the user device.

[1438] Output: The HTTP response sent to the user's device.

[1439] Step 7:

[1440] The image generation AI is activated based on the analysis results and prediction data received by the user device.

[1441] Input: Analysis results and prediction data received from the cloud server.

[1442] How it works: The device generates a prompt and inputs it into the image generation AI.

[1443] Output: The prompt and the generated image.

[1444] Step 8:

[1445] The user terminal displays the generated image.

[1446] Input: An image generated based on the prompt sentence.

[1447] Action: The device visually displays the image to the user.

[1448] Output: A simulated image after 3 months displayed on the user's screen.

[1449] Through the above steps, the user can easily obtain nutritional information about foods and visually check their future health status based on that information.

[1450] (Application example 1)

[1451] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1452] Previously, when shopping in a physical store, there was no way to know in real time the impact that the food being purchased would have on the user's health. This made it difficult for users to accurately understand the nutritional value of foods and make choices that took future health effects into account. Furthermore, existing systems only provided abstract nutritional information, and were insufficient in providing information in a visually easy-to-understand format. To solve this problem, a system is needed that obtains the nutritional information of foods when purchasing food in a physical store and visualizes the user's future state.

[1453] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1454] In this invention, the server includes means for identifying a food identifier and acquiring nutritional information for the food, means for analyzing the acquired nutritional information and predicting the impact that food intake will have on the user, means for generating an image of the user several months from now based on the prediction, means for displaying the generated image of the user, and means for scanning the identifier with the user's mobile device while shopping in a physical store. This allows users to obtain detailed nutritional information for the food they purchase while shopping in a physical store and visualize their appearance several months from now, enabling them to make healthier food choices.

[1455] A "food identifier" is information such as a barcode or QR code that identifies a specific food item.

[1456] "Nutritional information" refers to data on nutrients such as energy, protein, fat, and carbohydrates contained in food.

[1457] "Performing an analysis" refers to the process of conducting mathematical or statistical processing on the nutritional information obtained to assess the impact of food intake.

[1458] "Predictive methods" refer to algorithms or software that use nutritional information to predict future changes in a user's health and body shape.

[1459] "Image generation means" refers to computer graphics or generative AI models that visually represent the future user's appearance based on the prediction results.

[1460] "Means for displaying" refers to a display or application for outputting the generated image to a terminal screen in a format that can be viewed by the user.

[1461] "Brick and mortar store" refers to a physical retail store where customers can physically visit and purchase products.

[1462] A "mobile terminal" is a device with communication capabilities that can be carried by a user, and in this context it mainly refers to a smartphone.

[1463] System Overview

[1464] This invention is a system that uses a smartphone to obtain nutritional information about food when purchasing it in a physical store, and visualizes future health effects based on that information. The system consists of the following main components: a mobile device, a cloud server, a database, and an image-generating AI.

[1465] Processing steps

[1466] 1. Mobile device operation:

[1467] Users can launch the Smart Nutritional Shopping Assistant application at a physical store and scan the barcode of the food they plan to purchase using the camera function on their mobile device.

[1468] 2. Data transmission:

[1469] The mobile device sends the scanned barcode data to the cloud server, generating an HTTP request to send the barcode data to the server.

[1470] 3. Cloud Server Processing:

[1471] The cloud server searches the database based on the received barcode data and retrieves the nutritional information for the food, including energy, protein, fat, carbohydrates, etc.

[1472] 4. Prediction of health effects:

[1473] The cloud server uses the acquired nutritional information to instruct the AI ​​module to analyze and predict the health effects of consuming that food over a certain period of time. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain.

[1474] 5. Send prediction results:

[1475] The cloud server sends the predicted health effects to a mobile device, which provides the results as predicted data on calorie intake and weight gain.

[1476] 6. Image generation:

[1477] The mobile device launches an image generation AI based on the predicted data received from the cloud server, which then simulates what the user will look like in a few months' time based on predicted weight gain and changes in body fat percentage.

[1478] 7. Displaying images:

[1479] The mobile device displays the generated images in an easy-to-view format, allowing users to intuitively understand the impact of the foods they purchase on their future health.

[1480] Hardware and software used

[1481] Mobile device: smartphone.

[1482] Cloud server: Data processing server (e.g., Amazon Web Services, Google Cloud Platform).

[1483] Database: A database system (e.g., MySQL, MongoDB) that stores nutrition information.

[1484] Image generation AI: AI modules for image generation (e.g., TensorFlow, PyTorch).

[1485] Specific examples

[1486] Let's take the example of a user purchasing potato chips at a physical store. The user uses their smartphone to scan the barcode on the potato chips. The mobile device sends the barcode "0123456789012" to a cloud server, which retrieves nutritional information (energy: 150 kcal, fat: 10 g, carbohydrates: 15 g) from a database. The server uses an AI module to predict weight gain and other factors if the user consumes this daily for three months, and sends the prediction results to the mobile device. The mobile device then activates an image generation AI to simulate and display what the user will look like in a few months' time.

[1487] Prompt Sentence Examples

[1488] "Scan food barcodes to get nutritional information and simulate what you'll look like in three months if you eat that food every day."

[1489] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1490] Step 1:

[1491] A user uses a smartphone to scan the barcode of a food item they plan to purchase in a physical store. The input is an image of the barcode captured by the smartphone's camera. The smartphone uses the pyzbar library to decode the barcode from the image and outputs the barcode data as a string.

[1492] Step 2:

[1493] The device sends the acquired barcode data to the cloud server. The input is the barcode data from step 1, and the output is the result of sending an HTTP request to the cloud server. Using the requests library, an HTTP POST request is generated to send the barcode data to the server.

[1494] Step 3:

[1495] The server searches a database based on the received barcode data to obtain the nutritional information of the corresponding food. The input is the received barcode data, and the output is the nutritional information of the corresponding food. The server uses SQL queries to search for the nutritional information of the food from a database (e.g., MySQL, MongoDB) and obtains data such as calories, protein, fat, and carbohydrates.

[1496] Step 4:

[1497] The cloud server then instructs the AI ​​module to analyze the nutritional information and predict the health effects of consuming the food over a certain period of time. The input is the nutritional information obtained from Step 3, and the output is predicted data on health effects. The AI ​​module uses mathematical algorithms and statistical models to predict calorie accumulation and weight gain. For example, TensorFlow or PyTorch can be used for the analysis.

[1498] Step 5:

[1499] The cloud server sends the predicted health impact results to the mobile device. The input is the predicted data from step 4, and the output is the analysis results sent as an HTTP response to the mobile device. The server generates an HTTP response and sends the predicted data to the device in JSON format.

[1500] Step 6:

[1501] The mobile device launches an image generation AI based on the predicted data received from the cloud server. The input is the received predicted data, and the output is an image that simulates the generated future appearance of the user. The image generation AI generates the user's future appearance taking into account predicted weight gain and changes in body fat percentage. This process uses models such as GAN (generative adversarial networks).

[1502] Step 7:

[1503] The mobile device displays the generated image to the user. The input is the image data generated in step 6, and the output is the image of the future user displayed on the smartphone display. Through this visual information, the user can intuitively understand the impact of the food they plan to purchase on their future health.

[1504] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1505] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[1506] System Components

[1507] 1. User Device

[1508] 2. Cloud Server

[1509] 3. Database

[1510] 4. Image Generation AI

[1511] 5. Emotion Engine

[1512] User terminal processing

[1513] The user launches the "Food Scan Assistant" app on a device such as a smartphone. Within the app, they select the function to scan food barcodes and activate the camera. When the barcode comes into the camera's field of view, the device reads the barcode data and converts it into a string. An HTTP request is then generated and sent to send this barcode data to the cloud server.

[1514] Cloud server processing

[1515] The cloud server receives the barcode data sent from the device, searches its internal database based on the received barcode data, and obtains the nutritional information for the corresponding food, including calories, protein, fat, carbohydrates, etc.

[1516] The nutritional information is then passed to an AI module for detailed analysis. Specifically, it predicts the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[1517] Emotion engine processing

[1518] The user device uses the analysis results and prediction data received from the server to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, input, etc., and recognizes the user's current emotional state. The recognized emotional state is classified as, for example, "happiness," "sadness," or "stress."

[1519] Image generation AI processing

[1520] The image generation AI adjusts the parameters used to simulate the user's appearance depending on the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gradually.

[1521] Displaying Information

[1522] The user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[1523] Specific examples

[1524] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[1525] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[1526] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[1527] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[1528] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[1529] 5. The server sends the analysis results and prediction data to the device.

[1530] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[1531] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[1532] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[1533] The system allows users to receive specific advice based on their dietary habits and emotional state, providing support to lead a healthier life.

[1534] The processing flow will be explained below.

[1535] Step 1:

[1536] The user launches the "Food Scan Assistant" app on their smartphone and selects the barcode scan function. The device's camera then activates, ready to scan the barcode.

[1537] Step 2:

[1538] The user points the barcode on the chocolate bar at the camera and scans it. The device's camera reads the barcode "1234567890123."

[1539] Step 3:

[1540] The terminal analyzes the barcode data read by the camera using an internal algorithm and converts it into text data. The converted barcode data is temporarily stored.

[1541] Step 4:

[1542] The device creates an HTTP request to send the converted barcode data to the cloud server. The request includes the barcode data.

[1543] Step 5:

[1544] The device sends the HTTP request it created to the cloud server, and may retry until the request is successful.

[1545] Step 6:

[1546] The server receives the HTTP request sent from the terminal and extracts the barcode data "1234567890123" from the request.

[1547] Step 7:

[1548] The server searches the internal nutrition information database based on the extracted barcode data, retrieves the record of the corresponding chocolate bar, and obtains its nutrition information (calories, protein, fat, carbohydrates, etc.).

[1549] Step 8:

[1550] The server passes the acquired nutritional information to an AI module for detailed analysis, which then predicts the cumulative calorie intake and weight gain if the individual eats a chocolate bar once a day for three months.

[1551] Step 9:

[1552] The server's AI module generates analysis results as data based on various predictions (e.g., total calories, weight gain), and formats the generated analysis results as an HTTP response.

[1553] Step 10:

[1554] The server sends a formatted HTTP response to the user device, which includes nutritional information, prediction data, and analysis results.

[1555] Step 11:

[1556] The device receives the HTTP response from the server, obtains the analysis results and prediction data, and temporarily stores the obtained data.

[1557] Step 12:

[1558] The device passes the analysis results to an image generation AI, which then uses the predicted weight gain and changes in body fat percentage to create a realistic simulation of what the user will look like in a few months' time.

[1559] Step 13:

[1560] The device activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize their current emotional state, such as "happiness," "sadness," or "stress."

[1561] Step 14:

[1562] The parameters of the image generation AI are adjusted based on the emotional state recognized by the device's emotion engine. For example, if "stress" is detected, the settings are changed to emphasize the simulation of weight gain.

[1563] Step 15:

[1564] The device's image generation AI generates an image of the simulated user, which is then displayed in a format that is easy for the user to understand.

[1565] Step 16:

[1566] Along with the generated image, the device will display advice and food choice recommendations based on the user's emotional state, such as "You're under a lot of stress, so avoid sweets."

[1567] Step 17:

[1568] Users can view simulated images displayed on their device, along with nutritional information, predictions, and emotional advice, providing specific guidance on diet and health management. This understanding helps users make healthier eating choices going forward.

[1569] Example 2

[1570] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1571] While conventional food nutrition information acquisition systems allow users to obtain nutritional information about foods, it is difficult for them to obtain long-term predictions based on that information or advice tailored to their emotional state. Furthermore, since these systems do not simply display nutritional information but also provide insufficient support for health management that takes into account the user's emotions, it is difficult to maintain the user's motivation.

[1572] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1573] In this invention, the server includes means for scanning the barcode of a food product and obtaining nutritional information about the food product; means for analyzing the obtained nutritional information and predicting the impact of consuming the food product on the user; image generation means for generating an image of the user several months from now based on the prediction; means for displaying the generated image of the user; means for recognizing the user's emotional state; and image generation means for adjusting the prediction result based on the user's emotional state. This allows the user to not only obtain nutritional information about the food product but also receive health management advice that predicts the long-term impact of the food product. Furthermore, the simulation and advice provided reflect the user's emotions, thereby maintaining the user's motivation and improving the effectiveness of health management.

[1574] "Scanning food barcodes" means using an image capture device such as a camera to read the barcode attached to the food and obtain the information as digital data.

[1575] "Nutritional information" refers to data about the calories, protein, fat, carbohydrates, and other components contained in food.

[1576] "Performing analysis" means using specific algorithms and AI modules to process the data based on the acquired nutritional information and calculate the impact on the user.

[1577] "Predicting" means estimating future changes in the user's weight and health condition based on past data and current nutritional information.

[1578] An "image generation means" is a device or software module that uses a specific algorithm or AI technology to visually represent the user's future appearance based on a prediction.

[1579] "Display" means visually presenting the generated image or predicted data on the screen of the user terminal.

[1580] A "means for recognizing emotional state" is a device or software module that uses a camera or sensor to analyze a user's facial expressions, voice, input, etc., to identify their current emotional state.

[1581] "Adjusting prediction results based on emotional state" means changing or adjusting prediction results or generated images by taking into account the recognized emotional state of the user.

[1582] A "cloud server" is a remote computing resource accessible via a network, and is a computer system that stores, processes, analyzes, and otherwise processes data.

[1583] MODE FOR CARRYING OUT THE INVENTION

[1584] The present invention combines a system that scans food barcodes and acquires nutritional information for those foods with an emotion engine that recognizes the user's emotions, thereby more effectively supporting users in their food selection and health management. This system is composed of the following components:

[1585] 1. User device: smartphone, tablet, etc.

[1586] 2. Cloud server: A remote server that performs analysis and stores data.

[1587] 3. Database: Data storage for storing nutrition information.

[1588] 4. Image generation AI: An algorithm for generating the user's future appearance.

[1589] 5. Emotion Engine: A module for analyzing user emotions.

[1590] System Operation Overview

[1591] First, the user launches the "Food Scan Assistant" app on their smartphone and uses the function to scan the food barcode. The device's camera reads the barcode, converts it into a string, and creates and sends an HTTP request to the cloud server.

[1592] The cloud server searches its internal database based on the received barcode data and retrieves the nutritional information for the food in question. This information includes calories, protein, fat, carbohydrates, etc. This nutritional information is then passed to an AI module for detailed analysis, which predicts, for example, the cumulative calorie intake and weight gain if the food is eaten once a day for three months. The prediction results are compiled and an HTTP response is generated to be sent to the user's device.

[1593] The user device uses the received analysis results and prediction data to activate the emotion engine. The emotion engine uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize the user's current emotional state. For example, emotional states are classified as "joy," "sadness," "stress," etc.

[1594] The image generation AI adjusts simulation parameters depending on the user's perceived emotional state. For example, if the user is stressed, weight gain or changes in appearance may be more emphasized. Conversely, if the user is in a positive emotional state, changes may be shown more gently.

[1595] Finally, the user device visually displays the generated images and prediction data to the user, and can also display advice and food selection recommendations according to the emotional state recognized by the emotion engine.

[1596] Specific examples

[1597] As a concrete example, consider a scenario in which a user scans a chocolate bar.

[1598] 1. The user scans the barcode of the chocolate bar they plan to eat with their smartphone.

[1599] 2. The device reads the barcode "1234567890123" and sends the data to the cloud server.

[1600] 3. The server receives the barcode "1234567890123" and retrieves the nutritional information of the corresponding chocolate bar from the database (calories: 200 kcal, protein: 2 g, fat: 12 g, carbohydrates: 24 g).

[1601] 4. The server's AI module begins analysis based on the nutritional information and predicts the impact on the user (total calorie intake, predicted weight gain, etc.) if they continue to eat one chocolate bar per day for three months.

[1602] 5. The server sends the analysis results and prediction data to the device.

[1603] 6. The device activates the emotion engine to recognize the user's emotional state. For example, the emotion engine recognizes that the user is feeling stressed.

[1604] 7. The device uses image generation AI to generate an image that simulates what the user will look like in three months, adjusting the predictions based on the perceived emotional state.

[1605] 8. The device displays the generated image to the user and provides advice based on the user's emotional state along with the predicted data.

[1606] Examples of prompt statements

[1607] "Explain how the system works when a user scans the barcode '1234567890123' on a chocolate bar and wants to know the nutritional information and effects of that food after three months."

[1608] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1609] Step 1:

[1610] The user launches the "Food Scan Assistant" app on their smartphone, which displays the app's home screen.

[1611] Input: Touch operation on smartphone

[1612] Output: App home screen

[1613] Specifically, the user selects the barcode scanning function within the app and taps the option to launch the camera, which activates the smartphone's camera module.

[1614] Step 2:

[1615] The device reads the food barcode through the camera. The camera captures the barcode and obtains its image data. Then, it uses a barcode parsing library to decode the barcode image and convert it into string data.

[1616] Input: Camera image data

[1617] Output: Stringified barcode data (e.g. "1234567890123")

[1618] Specifically, the device extracts the barcode data and generates an HTTP request to send it to the cloud server, which includes the barcode data.

[1619] Step 3:

[1620] The device generates an HTTP request and sends it to the cloud server. Use the HTTP client library to send the request, including the barcode data.

[1621] Input: HTTP request (barcode data)

[1622] Output: Sending status (e.g. 200 OK)

[1623] Specifically, the device sends a request containing barcode data to a specific API endpoint, and the process is asynchronous, waiting for a response from the server.

[1624] Step 4:

[1625] The server receives the HTTP request, extracts the barcode data, and then searches a database to retrieve the corresponding food's nutritional information, including calories, protein, fat, carbohydrates, etc.

[1626] Input: HTTP request (barcode data)

[1627] Output: Nutritional information (e.g., Calories: 200kcal, Protein: 2g, Fat: 12g, Carbohydrates: 24g)

[1628] Specifically, the server executes a database query to obtain the nutritional information of the relevant food, and prepares to pass the obtained nutritional information to the AI ​​module.

[1629] Step 5:

[1630] The server's AI module performs a detailed analysis based on the nutritional information, for example, calculating the cumulative calorie intake and predicted weight gain if the food is eaten once a day for three months.

[1631] Input: Nutritional Information

[1632] Output: Predicted data (e.g., weight gain after 3 months)

[1633] Specifically, the server's AI module uses machine learning models to simulate the long-term effects of nutritional information and returns the predictions to the server.

[1634] Step 6:

[1635] The server receives the analysis results and prediction data and generates an HTTP response to send to the user's device, which includes nutritional information and prediction results.

[1636] Input: Forecast data

[1637] Output: HTTP response (nutritional information and prediction results)

[1638] Specifically, the server generates an HTTP response and sends it to the user's device. This response includes nutritional information and prediction data.

[1639] Step 7:

[1640] The user device receives the HTTP response from the server and activates the emotion engine, which uses the device's camera and sensors to analyze the user's facial expressions, voice, and input to recognize their current emotional state.

[1641] Input: HTTP response (nutritional information and prediction results), user input, facial expression, and voice data

[1642] Output: Perceived emotional state (e.g., stress)

[1643] Specifically, the user device runs an emotion engine to analyze various data about the user, and classifies the user's emotional state based on the analysis results.

[1644] Step 8:

[1645] The image generation AI on the user's device adjusts its predictions based on the perceived emotional state, and uses the generative AI model to generate a simulated image of what the user will look like in, say, three months.

[1646] Input: Emotional state, prediction data

[1647] Output: Adjusted simulation image

[1648] Specifically, the image generation AI on the user device adjusts the parameters of the image to be generated based on the output of the emotion engine, and generates a visual simulation image.

[1649] Step 9:

[1650] The user device visually displays the generated image and predicted data to the user, and also displays food selection recommendations and advice according to the emotional state recognized by the emotion engine.

[1651] Input: simulated image, predicted data, recognized emotional state

[1652] Output: Images and advice visually presented to the user

[1653] Specifically, the user terminal displays data on a user interface and provides information in a format that the user can intuitively understand.

[1654] (Application example 2)

[1655] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1656] Conventional nutrition information acquisition systems can analyze the nutritional information of foods, but they do not provide comprehensive health advice that takes into account the user's emotional state. As a result, users find it difficult to make food choices that take into account their emotional state and stress level, resulting in insufficient long-term health management. To solve this problem, a system is needed that recognizes the user's emotional state and provides accurate advice on food selection based on that state.

[1657] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for scanning the barcode of a food product and acquiring nutritional information about the food product, means for performing analysis based on the acquired nutritional information and predicting the impact that food intake will have on the user, means for recognizing the user's emotional state, image generation means for generating an image of the user several months from now based on the prediction, and means for displaying advice based on the generated image of the user and their emotional state. This enables the user to receive comprehensive and personalized support regarding food selection and health management.

[1658] "Nutritional information" refers to data on the ingredients contained in food, such as calories, protein, fat, and carbohydrates.

[1659] "Emotional state" is information that represents the psychological state of the user, and includes specific emotions such as joy, sadness, and stress.

[1660] "Means for scanning barcodes" refers to the ability to use the camera of a device such as a smartphone or smart glasses to read the barcode of a food item and obtain its data.

[1661] The "means for performing analysis" is a function that performs calculations and data processing to predict the user's health condition and future weight changes based on the acquired nutritional information.

[1662] A "predictor" is an algorithm that estimates the impact of food intake on a user based on nutritional information and multiple future scenarios.

[1663] The "image generation means" is a combination of software and hardware that uses the acquired data and analysis results to visually reproduce the user's future appearance.

[1664] "Display means" is a function for informing the user of the generated images, nutritional information, analysis results, and advice using a display such as a user terminal or smart glasses.

[1665] A "cloud server" is a remote server that stores, processes, and transfers data over the Internet, and supports data processing for the entire system.

[1666] The "means of acquisition" is a function that scans the barcode, sends the data obtained to a cloud server, and retrieves the corresponding nutritional information.

[1667] "Means of recognition" refers to a function that analyzes the user's facial expressions and voice through a camera or microphone to determine their current emotional state.

[1668] This invention is a system that supports health management by allowing users to scan food barcodes to obtain nutritional information, conducting analysis based on that information, and taking into account the user's emotional state. The system includes the following components: a user terminal, a cloud server, a database, an image generation AI, and an emotion engine. This allows users to make more effective food choices and manage their health.

[1669] System Components

[1670] 1. User Device

[1671] The user uses a smartphone or smart glasses as the device, launches the "Food Scan Assistant" application, and scans the food barcode.

[1672] 2. Cloud Server

[1673] It receives data and searches for nutritional information in a database, analyzes the nutritional content of multiple foods in detail, and works with an emotion engine to make predictions based on the user's emotions.

[1674] 3. Database

[1675] It stores nutritional information for foods and responds to searches from a cloud server.

[1676] 4. Image Generation AI

[1677] The system simulates the user's future appearance based on the user's emotional state and nutritional information.

[1678] 5. Emotion Engine

[1679] It analyzes the user's facial expressions and voice to recognize their current emotional state.

[1680] Implementation details

[1681] User terminal processing

[1682] When a user scans a food barcode in a physical store, the camera in the smart glasses or smartphone reads it and sends the barcode data to a cloud server, which also uses the camera and microphone in the smart glasses to analyze the user's emotional state.

[1683] Cloud server processing

[1684] The cloud server receives the barcode data and retrieves nutritional information from the database. Based on this information, the AI ​​module then makes predictions, analyzing the impact of calorie accumulation and weight gain over the next three months, for example. Furthermore, the emotion engine determines the user's emotional state and generates corresponding data.

[1685] Emotion engine processing

[1686] The emotion engine analyzes the user's emotional state and adjusts the nutritional information analysis accordingly—for example, if the user is stressed, the weight gain prediction will be emphasized.

[1687] Image generation AI processing

[1688] The image generation AI simulates the user's future appearance based on the adjusted nutritional information analysis results, and the generated image is displayed on the smart glasses or smartphone screen for easy visual understanding.

[1689] Specific examples

[1690] As a concrete use case, consider the scenario where a user scans a chocolate bar in a supermarket.

[1691] 1. A user scans the barcode on a chocolate bar using smart glasses.

[1692] 2. The barcode data is sent to a cloud server and nutritional information is retrieved.

[1693] 3. The server analyzes calories and ingredients based on nutritional information and generates a prediction result.

[1694] 4. The emotion engine recognizes the user's emotional state and adjusts predictions accordingly.

[1695] 5. Image generation AI generates a future image of the user based on the adjusted predictive data.

[1696] 6. The generated image is displayed on the smart glasses display, and advice on food choices is provided.

[1697] Prompt Sentence Examples

[1698] A user picks up a chocolate bar at the supermarket. The camera in the eyewear scans the barcode and retrieves nutritional information. At the same time, the emotion engine recognizes that the user is feeling stressed. Visually display this information to simulate what the user would look like if they continued to eat a chocolate bar every day for three months.

[1699] This allows users to receive comprehensive and personalized support regarding food choices and health management.

[1700] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1701] Step 1:

[1702] A user scans the barcode of a food item in a brick-and-mortar store using smart glasses.

[1703] Input: Barcode data read using the smart glasses camera.

[1704] Output: HTTP request for barcode data to be sent to the cloud server.

[1705] The smart glasses activate their camera to scan the barcode of the food item selected by the user and send the data to a cloud server.

[1706] Step 2:

[1707] The cloud server receives the barcode data and searches the database to obtain the corresponding nutritional information.

[1708] Input: Barcode data received by the cloud server.

[1709] Output: Nutritional information (calories, protein, fat, carbohydrates, etc.).

[1710] The cloud server searches a database based on the barcode data and obtains nutritional information for related foods.

[1711] Step 3:

[1712] The acquired nutritional information is passed to an AI module to predict the long-term effects of food intake.

[1713] Input: Nutritional information.

[1714] Output: Data predicting the long-term effects of food intake (total calorie intake, predicted weight gain, etc.).

[1715] The AI ​​module in the server analyzes the data based on nutritional information and predicts, for example, accumulated calories and weight gain three months from now.

[1716] Step 4:

[1717] The server sends the analysis results to the user's device.

[1718] Input: Data projecting long-term effects.

[1719] Output: The HTTP response sent to the user device.

[1720] The cloud server compiles the prediction results and generates an HTTP response to send to the user's device.

[1721] Step 5:

[1722] The user terminal activates an emotion engine to recognize the user's emotional state.

[1723] Input: User facial and voice data.

[1724] Output: Perceived emotional state (e.g., happy, sad, stressed).

[1725] The smart glasses' cameras and microphones are used to analyze the user's facial expressions and voice to recognize their emotional state.

[1726] Step 6:

[1727] Based on the recognized emotional state, image generation AI simulates the user's future appearance.

[1728] Input: Emotional state and prediction data.

[1729] Output: A representation of what the future user will look like.

[1730] The image generation AI simulates the user's future appearance based on their emotional state and predictive data, and generates an image to visually display.

[1731] Step 7:

[1732] The user's device displays the generated image and predicted data and provides health advice according to the user's emotional state.

[1733] Input: Generated image, predicted data, emotional state.

[1734] Output: Images and advice displayed to the user.

[1735] The generated images and predicted data are displayed on the smart glasses' display, and health advice is provided based on the emotional state recognized by the emotion engine.

[1736] Step 8:

[1737] The user makes food choices based on the information provided.

[1738] Input: The displayed image and health advice.

[1739] Output: The food choices selected by the user.

[1740] Users are provided with visual information and advice to help them make healthier food choices.

[1741] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1742] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1743] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1744] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1745] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1746] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1747] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1748] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, motorcycles, and other devices, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1749] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1750] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1751] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1752] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1753] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1754] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1755] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1756] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1757] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1758] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1759] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1760] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1761] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1762] The following is further disclosed regarding the above embodiment.

[1763] (Claim 1)

[1764] means for scanning a barcode of a food product to obtain nutritional information for the food product;

[1765] A means for analyzing the acquired nutritional information and predicting the effect of food intake on the user;

[1766] an image generation means for generating an image of the user several months later based on the prediction;

[1767] a means for displaying the generated user appearance;

[1768] A system including:

[1769] (Claim 2)

[1770] A means for performing detailed analysis of calories, protein, fat, and carbohydrate components based on the acquired nutritional information;

[1771] 10. The system of claim 1, further comprising means for notifying a user of an impact based on the prediction.

[1772] (Claim 3)

[1773] 10. The system of claim 1, further comprising means for transmitting information about the food to a cloud server and receiving analysis results from the cloud server.

[1774] "Example 1"

[1775] (Claim 1)

[1776] means for scanning a barcode of a food product to obtain nutritional information for the food product;

[1777] means for transmitting the acquired nutritional information to a cloud server;

[1778] A means for searching a database based on the nutritional information received by the cloud server and acquiring nutritional information of the corresponding food;

[1779] A means for predicting the effect of food intake on a user based on the acquired nutritional information;

[1780] A means for generating prompt sentences based on the prediction results and generating an image of the user several months later using image generation technology;

[1781] a means for displaying the generated user appearance;

[1782] A system including:

[1783] (Claim 2)

[1784] A means for performing detailed analysis of calories, protein, fat, and carbohydrate components based on the acquired nutritional information;

[1785] 10. The system of claim 1, further comprising means for notifying a user of an impact based on the prediction.

[1786] (Claim 3)

[1787] 10. The system of claim 1, further comprising means for transmitting information about the food to a cloud server and receiving analysis results from the cloud server.

[1788] "Application Example 1"

[1789] (Claim 1)

[1790] means for identifying a food product identifier and obtaining nutritional information for the food product;

[1791] A method for analyzing the acquired nutritional information and predicting the impact of food intake on the user;

[1792] an image generation means for generating an image of the user several months later based on the prediction;

[1793] A means for displaying the generated user's appearance;

[1794] means for scanning the identifier with a user's mobile device when shopping at a physical store;

[1795] A system including:

[1796] (Claim 2)

[1797] a means for performing a detailed analysis of energy, protein, lipid, and carbohydrate components based on the obtained nutritional information;

[1798] 10. The system of claim 1, further comprising means for notifying a user of the impact based on the prediction.

[1799] (Claim 3)

[1800] 10. The system of claim 1, further comprising means for transmitting information about the food product to a remote server and receiving analysis results from the remote server.

[1801] "Example 2: Combining Emotion Engines"

[1802] (Claim 1)

[1803] means for scanning a barcode of a food product to obtain nutritional information for the food product;

[1804] A means for analyzing the acquired nutritional information and predicting the effect of food intake on the user;

[1805] an image generation means for generating an image of the user several months later based on the prediction;

[1806] a means for displaying the generated user appearance;

[1807] means for recognizing the emotional state of a user;

[1808] an image generation means for adjusting the prediction result based on the emotional state;

[1809] A system including:

[1810] (Claim 2)

[1811] A means for performing detailed analysis of calories, protein, fat, and carbohydrate components based on the acquired nutritional information;

[1812] 10. The system of claim 1, further comprising means for notifying a user of an impact based on the prediction.

[1813] (Claim 3)

[1814] 10. The system of claim 1, further comprising means for transmitting information about the food to a cloud server and receiving analysis results from the cloud server.

[1815] "Application example 2 when combining emotion engines"

[1816] (Claim 1)

[1817] means for scanning a barcode of a food product to obtain nutritional information for the food product;

[1818] A means for analyzing the acquired nutritional information and predicting the effect of food intake on the user;

[1819] means for recognizing the emotional state of a user;

[1820] an image generation means for generating an image of the user several months later based on the prediction;

[1821] a means for displaying advice based on the generated user's appearance and emotional state;

[1822] A system including:

[1823] (Claim 2)

[1824] A means for performing detailed analysis of calories, protein, fat, and carbohydrate components based on the acquired nutritional information;

[1825] 10. The system of claim 1, further comprising means for informing the user of the impact based on the prediction and providing health advice based on the emotional state.

[1826] (Claim 3)

[1827] 10. The system of claim 1, further comprising means for transmitting information about the food to a cloud server and receiving data based on the analysis results and the emotional state from the cloud server. [Explanation of symbols]

[1828] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for scanning a barcode of a food product to obtain nutritional information for the food product; A means for analyzing the acquired nutritional information and predicting the effect of food intake on the user; an image generation means for generating an image of the user several months later based on the prediction; a means for displaying the generated user appearance; A system including:

2. A means for performing detailed analysis of calories, protein, fat, and carbohydrate components based on the acquired nutritional information; The system of claim 1 further comprising means for notifying a user of the impact based on the prediction.

3. The system of claim 1 , further comprising means for transmitting information about the food to a cloud server and receiving analysis results from the cloud server.

Images