System

Abstract

A system is provided.SOLUTION: A system including means for recording an image of a meal with a camera, means for analyzing meal contents by image recognition, means for calculating ingested nutrients on the basis of an analysis result, means for storing the analysis result in a cloud server, means for measuring the number of steps, means for transmitting step count data to the cloud server, means for calculating consumed calories on the cloud server, means for comparing ingested calories and consumed calories and generating advice, and means for notifying a user of the generated advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Lifestyle-related diseases have become a serious problem in modern society. It is difficult for individuals to accurately manage their diet and exercise habits in their busy daily lives, and the resources of registered dietitians are limited. As a result, there is a lack of tools for managing health comfortably and efficiently. Therefore, there is a need for a system that allows people to easily record their daily meals and exercise and provide healthy advice. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system including means for recording images of meals with a camera, means for analyzing meal contents using image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results on a cloud server, means for counting steps, means for transmitting step count data to the cloud server, means for calculating calories burned on the cloud server, means for generating advice by comparing calories consumed with calories burned, and means for notifying the user of the generated advice, thereby enabling users to automatically record their own diet and exercise habits and receive appropriate health advice.

[0006] A "camera" is a device that converts visual information into digital data and records it.

[0007] "Meal images" are digital data captured by a camera that contain visual information of the food and drink consumed.

[0008] "Image recognition" is the technology of identifying and classifying specific objects or patterns from digital images.

[0009] "Dietary content" refers to the types and amounts of food and drink, and is information that determines the nutrients ingested.

[0010] The "analysis results" are information about the meal contents obtained using image recognition technology.

[0011] "Nutrients ingested" refers to the amount of calories, salt, fat, and alcohol calculated based on the diet.

[0012] A "cloud server" is a remote server available over the Internet that provides computing resources for storing and processing data.

[0013] "Step count" refers to the number of steps taken by the user, and is data measured by the built-in sensor.

[0014] "Calories burned" refers to the amount of energy consumed in a particular activity and is calculated from data such as the number of steps taken.

[0015] "Advice" is a suggestion for improving the user's lifestyle habits that is generated based on the analysis results and calories burned.

[0016] "Means for notifying the user" refers to a method for communicating the generated advice to the user, including an electronic display, audio output, or other notification means. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases. The system aims to provide users with appropriate advice by recording and analyzing dietary information, measuring steps, and managing the balance between calorie intake and calorie expenditure.

[0039] System configuration

[0040] The system of the present invention includes the following major components:

[0041] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[0042] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[0043] 3. Cloud Server - A remote server for storing analysis results and performing necessary calculations and data storage.

[0044] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[0045] 5. Notification means - includes a display and audio output device to notify the user of advice.

[0046] Specific explanation of program processing

[0047] Food recording and analysis

[0048] 1. The user prepares breakfast on the table, and the camera in the AI ​​glasses takes pictures of the meal.

[0049] 2. The device (AI glasses) sends the captured image data to the cloud server.

[0050] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[0051] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[0052] Step counting

[0053] 1. When the user commutes to work or exercises, the AI ​​glasses measure the number of steps taken using the built-in sensor.

[0054] 2. The device sends step count data to the cloud server at regular intervals.

[0055] 3. The server analyzes the received step count data and calculates the calories burned.

[0056] Data integration and advice generation

[0057] 1. The server compares the daily calorie intake with the calorie expenditure.

[0058] 2. The server generates advice based on the user's health status, for example, to improve diet and exercise.

[0059] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[0060] Specific examples

[0061] Example 1: Breakfast recording and analysis

[0062] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[0063] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[0064] The server stores this information and updates the user database.

[0065] Example 2: Measuring steps during commute

[0066] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[0067] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[0068] Example 3: Providing advice

[0069] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[0070] The device will notify the user of this advice and display it on the AI ​​glasses' display.

[0071] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] The user prepares a meal, for example, bread and orange juice for breakfast.

[0075] Step 2:

[0076] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[0077] Step 3:

[0078] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[0079] Step 4:

[0080] The server adds the received image data to a processing queue.

[0081] Step 5:

[0082] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[0083] Step 6:

[0084] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[0085] Step 7:

[0086] The server organizes and stores the calculated nutrient data in a database for each user.

[0087] Step 8:

[0088] The user begins their daily routine or exercise, for example, walking to the station for work.

[0089] Step 9:

[0090] The device's built-in sensor measures the user's step count data.

[0091] Step 10:

[0092] The device sends step count data to the server at regular intervals.

[0093] Step 11:

[0094] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[0095] Step 12:

[0096] The server aggregates daily calorie, salt, fat, alcohol, and step count data.

[0097] Step 13:

[0098] The server uses machine learning models to analyze the user's health status and generate data-based advice, such as "Today's breakfast was too calorie-dense, so choose a vegetable-based lunch."

[0099] Step 14:

[0100] The server sends the advice to the notification system.

[0101] Step 15:

[0102] The terminal receives the advice from the server.

[0103] Step 16:

[0104] The device will then display the received advice in the user's field of vision, for example, "Eat mostly vegetables for lunch" on the Glasses' display.

[0105] Step 17:

[0106] The user follows the advice and practices a healthy diet.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] In conventional systems for preventing and managing lifestyle-related diseases, recording meals and counting steps are often done manually, placing a heavy burden on users. Furthermore, manually calculating calories consumed and calories burned can lead to a decrease in accuracy. Furthermore, it is difficult to provide appropriate advice tailored to each individual user's health condition.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes means for a user to prepare a meal and record an image of the meal with a camera, means for the terminal to send the image data to a cloud server, means for the cloud server to analyze the meal contents using an image recognition algorithm, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a database, means for using a built-in sensor to count steps, means for sending the step count data to the cloud server, means for calculating calories burned on the cloud server, means for comparing calories consumed with calories burned to generate advice, and means for notifying the user of the generated advice, thereby enabling users to efficiently and accurately manage their own dietary and exercise habits.

[0112] A "user" is a person who uses the system to manage their own eating and exercise habits and prevent and manage lifestyle-related diseases.

[0113] "Camera" refers to a photographic device used to record images of a meal.

[0114] A "terminal" is a device used to send captured image data and step count data to a cloud server.

[0115] A "cloud server" is an infrastructure for storing and analyzing data on a remote server.

[0116] An "image recognition algorithm" is a technology that analyzes food contents from captured images and identifies the type and amount of food.

[0117] A "database" is a digital recording medium for storing analysis results and nutrient information.

[0118] "Built-in sensor" refers to a sensor device built into a device to measure steps.

[0119] "Calories burned" refers to the amount of energy consumed calculated based on step count data.

[0120] "Advice" refers to dietary and exercise recommendations for the user that are generated by comparing calories ingested with calories burned.

[0121] The present invention relates to a system that supports the prevention and management of lifestyle-related diseases. This system aims to provide users with appropriate advice by recording and analyzing dietary content, measuring the number of steps taken, and managing the balance between calorie intake and calorie expenditure.

[0122] System configuration

[0123] The system of the present invention includes the following major components:

[0124] 1. Camera - A photographic device used to record images of meals.

[0125] 2. Device - A device that sends captured image data and step count data to a cloud server. It has an internal sensor and also measures the number of steps.

[0126] 3. Cloud Server - A remote server that stores and analyzes data.

[0127] 4. Image recognition algorithms - These are technologies used to analyze food content and identify the types and quantities of food. For example, software such as TensorFlow and OpenCV are examples of such algorithms.

[0128] 5. Database - A digital recording medium for storing analysis results and nutritional information.

[0129] 6. Built-in sensors - Includes an accelerometer and gyro sensor for counting steps.

[0130] 7. Notification means - A display or audio output device for notifying the user of advice.

[0131] Specific processing of the program

[0132] Food recording and analysis

[0133] 1. The user prepares breakfast and takes a picture of the meal using the AI ​​glasses. For example, if the user prepares bread and milk, they press the camera button to take a picture.

[0134] 2. The device sends the captured image data to a cloud server using Wi-Fi or mobile data.

[0135] 3. The server receives the image data and analyzes the meal contents using an image recognition algorithm (e.g., YOLO or ResNet). It identifies bread and milk and calculates the nutritional value of each.

[0136] 4. The server stores the analysis results in the user's database, specifically recording information such as the bread's calories, carbohydrates, and fat content.

[0137] Step counting

[0138] 1. The user begins a daily activity, for example, leaving home for their morning commute.

[0139] 2. The device uses a built-in sensor to count steps and detects walking movements in real time.

[0140] 3. The device sends the measured data at regular intervals (for example, every 10 minutes) to the cloud server.

[0141] 4. The server analyzes the step count data and calculates the calories burned. For example, if 3,000 steps result in 150 calories burned, the server records this data in the user's database.

[0142] Data integration and advice generation

[0143] 1. The server aggregates the daily calorie intake and calorie expenditure data. For example, it aggregates data showing that you ate 500 calories at breakfast and burned 150 calories by walking.

[0144] 2. The server generates diet and exercise advice based on the user's individual profile (e.g., age, gender, weight, activity level, etc.). If the calorie intake is high, advice such as "eat a lighter lunch" or "exercise more" is generated.

[0145] 3. The device receives the advice generated by the server and notifies the user, either by displaying it on the AI ​​glasses display or by voice notification.

[0146] Examples of concrete examples and prompts

[0147] Specific examples

[0148] The user prepares yogurt and a banana for breakfast, and the AI ​​glasses take an image of the meal and send the data to a cloud server.

[0149] The server analyzes the image and calculates the nutritional value of yogurt (100 calories) and a banana (90 calories).

[0150] The server stores this information and updates the user database.

[0151] Prompt Sentence Examples

[0152] "How many calories did you have at dinner last night?"

[0153] "How many calories are in my breakfast today?"

[0154] "Based on your step count data this week, do you need additional exercise?"

[0155] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[0156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0157] Step 1:

[0158] The user prepares breakfast and uses the AI ​​glasses to take pictures of the meal.

[0159] Input: User prepares meal and presses camera button.

[0160] Output: The camera generates image data of the meal.

[0161] Specific operation: When the user presses the camera button, the camera on the AI ​​glasses takes a photo of the meal and the image data is saved on the device.

[0162] Step 2:

[0163] The device sends the captured image data to a cloud server.

[0164] Input: The captured image data is saved on the device.

[0165] Output: The image data is sent to the cloud server.

[0166] Specific operation: The device sends image data to the cloud server using Wi-Fi or mobile data.

[0167] Step 3:

[0168] The server applies image recognition algorithms to analyze the meal contents.

[0169] Input: Image data stored on a cloud server.

[0170] Output: Analysis results regarding food type, quantity, and nutritional value.

[0171] Specific operation: The server analyzes the image using an image recognition algorithm (e.g., TensorFlow or OpenCV), identifies the type and quantity of food, and calculates nutritional value.

[0172] Step 4:

[0173] The server stores the analysis results in a database.

[0174] Input: Analysis results (food type, amount, and nutritional value information).

[0175] Output: Analysis results stored in a database.

[0176] Specific operation: The server records the analysis results in the user's database, and nutritional information (calories, carbohydrates, fats, etc.) is saved.

[0177] Step 5:

[0178] The user begins their daily activities.

[0179] Input: The user starts walking.

[0180] Output: Real-time step count data is generated by sensors in the AI ​​glasses.

[0181] Specific operation: The user starts commuting or exercising, and the AI ​​glasses detect walking movements and count the number of steps.

[0182] Step 6:

[0183] The device periodically transmits step count data to a cloud server.

[0184] Input: Measured step count data.

[0185] Output: Step count data sent to cloud server.

[0186] Specific operation: The device sends step count data to the cloud server at regular intervals (for example, every 10 minutes).

[0187] Step 7:

[0188] The server analyzes the received step count data and calculates the calories burned.

[0189] Input: Step count data stored on a cloud server.

[0190] Output: Calculated calories burned.

[0191] Specific operation: The server analyzes the step count data, calculates the calories burned, and records them in a database.

[0192] Step 8:

[0193] The server aggregates the daily calorie intake and expenditure data.

[0194] Input: Stored calorie intake and calorie expenditure data.

[0195] Output: Consolidated calorie data.

[0196] Specific operation: The server integrates the calorie intake and calorie expenditure information obtained from the database and calculates the daily calorie balance.

[0197] Step 9:

[0198] The server generates advice based on the user's profile.

[0199] Input: Consolidated calorie data and user profile information (age, gender, weight, activity level).

[0200] Output: The generated advice.

[0201] Specific operation: The server uses the generated AI model to generate advice (recommended dietary restrictions and exercise) based on the user's health condition.

[0202] Step 10:

[0203] The terminal notifies the user of the generated advice.

[0204] Input: Advice data from the server.

[0205] Output: Advice given to the user.

[0206] Specific operation: The device notifies the user of the advice using a display or voice output.

[0207] (Application example 1)

[0208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0209] Modern brick-and-mortar stores lack an effective system to support customers in making healthy dietary choices. They also need a way to provide personalized advice in real time based on each customer's health status. This makes it difficult for customers to prevent and manage lifestyle-related diseases without stress.

[0210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0211] In this invention, the server includes means for recording images of meals using a camera, means for analyzing meal contents using image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a cloud server, means for counting steps, means for transmitting step count data to the cloud server, means for calculating calories burned on the cloud server, means for generating advice by comparing calories consumed with calories burned, means for notifying the user of the generated advice, means for analyzing food calories and nutrients in real time and suggesting nutritionally balanced meals to support customers' healthy dietary choices in the physical store, and means for generating and presenting personalized health advice based on each customer's health condition, thereby enabling customers to receive information supporting healthy lifestyle habits in real time in the physical store.

[0212] A "camera" is a photographic device for recording images of meals.

[0213] "Image recognition" is a technology that analyzes recorded meal contents.

[0214] A "cloud server" is a remote server that stores data and performs calculations such as calorie consumption.

[0215] "Step counting" is a means for counting the number of steps.

[0216] "Calculating calories burned" is a method for calculating calories burned from step count data.

[0217] "Personalized health advice" is a means of generating specific recommendations based on each customer's health status.

[0218] "Nutritionally balanced meal suggestions" is a method of analyzing the calories and nutrients of food and supporting healthy dietary choices.

[0219] The present invention provides a system for supporting health management of customers in brick-and-mortar stores, and in particular provides a specific configuration for analyzing dietary content and providing advice. This system includes the following main hardware and software:

[0220] System configuration

[0221] 1. Camera

[0222] The camera is built into the AI ​​glasses and is a device for capturing images of the food. The camera takes pictures of the dishes selected by the customer in real time and sends the image data to the system.

[0223] 2. Image Recognition Algorithm

[0224] The image recognition algorithm is a technology that analyzes photographed images of food and identifies the type and amount of food. This algorithm is implemented using machine learning libraries such as TensorFlow and Keras.

[0225] 3. Cloud Server

[0226] A cloud server is a remote server that stores analysis results and manages the necessary calculations and data. AWS (Amazon Web Services) and Google Cloud Platform are often used.

[0227] 4. Built-in sensor

[0228] The built-in sensor is a device that measures the number of steps taken by customers, and the step count data is sent to a cloud server via Bluetooth or other means.

[0229] 5. Means of notification

[0230] The notification means includes a display and an audio output device for notifying the user of the advice, which allows healthy eating suggestions and advice to be provided to customers in real time within the physical store.

[0231] Program processing explanation

[0232] Camera Function:

[0233] A camera is built into the AI ​​glasses and takes a picture of the customer's dish selection, which is then instantly sent to an image recognition algorithm.

[0234] Image Recognition:

[0235] Image recognition algorithms analyze captured images to identify the type and quantity of food, using a pre-trained neural network model to identify nutritional and calorie information for each food item.

[0236] Cloud server processing:

[0237] The detected meal content data is sent to a cloud server and stored there, where the cloud server also calculates the calories burned and compares them with the user's calorie intake.

[0238] Advice Generation:

[0239] The cloud server generates personalized health advice based on the user's health status, such as nutritionally balanced meal suggestions and exercise instructions.

[0240] notification:

[0241] The generated advice is sent to the user in real time either as a display on the AI ​​glasses or as a voice prompt.

[0242] Specific examples

[0243] Example 1: Use in a restaurant

[0244] When a user orders a steak at a restaurant and takes a picture of the meal with the AI ​​glasses, the system identifies it as a steak and instantly obtains calorie information, which can be used as a guide for the next meal or exercise.

[0245] Example prompt: "From this image, identify the type of food and its calories, and analyze whether it is nutritionally balanced."

[0246] Example 2: Use in a cafe

[0247] When a user orders a salad and soup at a cafe, the AI ​​glasses recognize this and suggest healthy eating choices, informing them of any food or exercise they should avoid later.

[0248] Example prompt: "Calculate the nutritional value of a salad (200 calories) and a soup (150 calories) and store that information in a database."

[0249] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0250] Step 1:

[0251] The user takes a photo of the dish they have selected with a camera, which takes image data of the dish as input and sends the image to an image recognition algorithm.

[0252] Step 2:

[0253] The device runs an image recognition algorithm to analyze the input image data. Specifically, after preprocessing the image, it inputs it into a neural network model to identify the type and quantity of food. The output is the type of food and its nutritional information.

[0254] Step 3:

[0255] The server receives the analysis results and retrieves food types and nutrient information from the nutrition database. It takes food types as input, executes a database query, and obtains nutrient data as output.

[0256] Step 4:

[0257] The server saves the nutrient data on the cloud server. It receives the analysis results and nutrient data as input and stores them in the database. The output confirms that the data is persisted.

[0258] Step 5:

[0259] When a user walks, the device's built-in sensor measures the number of steps taken. The device receives step count data in real time as input and sends the data to a cloud server.

[0260] Step 6:

[0261] The server calculates calories burned based on the step count data received. It receives step count data as input and calculates calories burned using a predefined algorithm. It obtains the calorie burned value as output.

[0262] Step 7:

[0263] The server compares daily calorie intake and expenditure and generates personalized advice. It receives calorie intake and expenditure data as input and generates advice using a health management algorithm. The output is advice text.

[0264] Step 8:

[0265] The device notifies the user of the generated advice. It receives the advice text as input and displays it on the AI ​​glasses display or notifies the user by voice.

[0266] Step 9:

[0267] The server analyzes food calories and nutrients in real time and suggests nutritionally balanced meals to support healthy meal choices for users in physical stores. It receives data on in-store dishes as input and analyzes it in real time. It outputs healthy meal suggestions to be presented to customers.

[0268] Step 10:

[0269] The server generates and presents personalized health advice based on each customer's health status. It receives each customer's health data as input, analyzes it using a machine learning model, and obtains personalized advice as output.

[0270] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0271] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[0272] System configuration

[0273] The system of the present invention includes the following major components:

[0274] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[0275] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[0276] 3. Cloud Server - A remote server that stores analysis results and emotion data, and performs necessary calculations and data storage.

[0277] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[0278] 5. Emotion Engine - Contains technology to identify the user's emotional state.

[0279] 6. Notification means - includes a display and audio output device to notify the user of advice.

[0280] Specific explanation of program processing

[0281] Food recording and analysis

[0282] 1. The user prepares a meal, and the camera in the AI ​​glasses takes a picture of the meal.

[0283] 2. The device (AI glasses) sends the captured image data to the cloud server.

[0284] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[0285] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[0286] Step counting

[0287] 1. The user begins their daily routine or exercise, for example, walking to the station for work.

[0288] 2. The device's built-in sensor measures the user's step data.

[0289] 3. The device sends step count data to the cloud server at regular intervals.

[0290] 4. The server analyzes the received step count data and calculates the calories burned.

[0291] Emotion recognition and analysis

[0292] 1. The emotion engine identifies the user's emotional state. For example, it determines whether the user is feeling stressed based on facial expressions and voice data.

[0293] 2. The server sends the emotional data to the cloud, where it integrates and analyzes past emotional data and health data.

[0294] Data integration and advice generation

[0295] 1. The server compares and integrates the daily calorie intake, calorie expenditure, and emotional data.

[0296] 2. The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[0297] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[0298] Specific examples

[0299] Example 1: Breakfast recording and analysis

[0300] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[0301] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[0302] The server stores this information and updates the user database.

[0303] Example 2: Measuring steps during commute

[0304] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[0305] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[0306] Example 3: Providing advice

[0307] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[0308] The device will notify the user of this advice and display it on the AI ​​glasses' display. If the user's emotional state is high, the device will also suggest taking a short break to relax.

[0309] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[0310] The processing flow will be explained below.

[0311] Step 1:

[0312] The user prepares a meal, for example, bread and orange juice for breakfast.

[0313] Step 2:

[0314] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[0315] Step 3:

[0316] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[0317] Step 4:

[0318] The server adds the received image data to a processing queue.

[0319] Step 5:

[0320] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[0321] Step 6:

[0322] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[0323] Step 7:

[0324] The server organizes and stores the calculated nutrient data in a database for each user.

[0325] Step 8:

[0326] The user begins their daily routine or exercise, for example, walking to the station for work.

[0327] Step 9:

[0328] The device's built-in sensor measures the user's step count data.

[0329] Step 10:

[0330] The device sends step count data to the server at regular intervals.

[0331] Step 11:

[0332] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[0333] Step 12:

[0334] The emotion engine recognizes the user's facial expressions and voice to identify their emotional state, for example, determining whether they are stressed.

[0335] Step 13:

[0336] The device transmits the emotion data to the cloud server.

[0337] Step 14:

[0338] The server receives the emotional data and integrates and analyzes the past emotional data and health data.

[0339] Step 15:

[0340] The server compares and integrates daily calorie intake, calorie expenditure, and emotional data.

[0341] Step 16:

[0342] The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[0343] Step 17:

[0344] The server sends the advice to the notification system.

[0345] Step 18:

[0346] The terminal receives the advice from the server.

[0347] Step 19:

[0348] The device will then display the received advice in the user's field of vision. For example, the Glasses' display will say, "Eat a vegetable-based lunch." If the user is feeling stressed, the device will suggest, "Take a short break to relax."

[0349] Step 20:

[0350] Users follow advice to implement healthy eating, exercise, and stress management.

[0351] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[0352] Example 2

[0353] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] Achieving a balance between daily calorie intake and calorie expenditure is important for preventing and managing lifestyle-related diseases, but it is difficult for users to consistently maintain this balance in their daily lives. Furthermore, a user's emotional state is also an important factor in health management, and it is widely known that stress, in particular, has a negative impact on health. However, current systems only record dietary habits and measure steps, and are unable to provide comprehensive health management that also takes emotional state into account. Therefore, there is a need for a system that can comprehensively manage a user's health and emotional state and provide individually optimized advice.

[0355] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0356] In this invention, the server includes means for recording images of meals, means for analyzing meal contents by image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results on a remote server, means for counting steps, means for transmitting step count data to the remote server, means for calculating calories burned on the remote server, means for comparing calories consumed with calories burned to generate advice, means for identifying the user's emotional state and transmitting the identified emotional data to the remote server, means for integrating and analyzing the user's emotional data and health data, and means for notifying the user of the generated advice. This enables the user to comprehensively manage their health throughout the day and receive individually optimized advice that also takes into account their emotional state.

[0357] The "means for recording images of meals" refers to a device that uses a photographing device such as a camera to record images of meals consumed by the user as digital data.

[0358] "Means for analyzing meal contents using image recognition" refers to technologies and algorithms that analyze image data of photographed meals and identify the foods contained in the meal and their amounts.

[0359] The "means for calculating the nutrients ingested based on the analysis results" refers to a computing device or program that calculates the calories ingested and various nutrients from the dietary content data obtained by image recognition.

[0360] The "means for saving the analysis results on a remote server" is a function for saving the calculated nutrient data and other analysis results on a remote server via the Internet.

[0361] The "means for measuring the number of steps" refers to a device or sensor for measuring the walking activity of a user, and includes a pedometer.

[0362] The "means for transmitting step count data to a remote server" refers to a communication device or function for transmitting the measured step count data to a remote server via the Internet.

[0363] The "means for calculating calories burned on a remote server" refers to an algorithm or program that calculates calories burned based on the step count data received on the remote server.

[0364] The "means for generating advice by comparing calorie intake with calorie expenditure" refers to a device or software that compares the balance between calorie intake and calorie expenditure and automatically generates advice for health management.

[0365] "Means for identifying the user's emotional state and transmitting the identified emotional data to a remote server" refers to technology or devices for identifying the user's emotions through facial expression recognition, voice analysis, etc., and transmitting that data to a remote server.

[0366] "Means for integrating and analyzing a user's emotional data and health data" refers to an algorithm or program that integrates and analyzes a user's emotional data and health data (calories ingested, calories burned, etc.) to assess their overall health status.

[0367] The "means for notifying the user of the generated advice" refers to a notification device such as a display or audio output device for conveying the generated health management advice to the user.

[0368] This invention relates to an AI system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[0369] Hardware and software used

[0370] Camera: A device built into the AI ​​glasses that takes pictures of meals.

[0371] Built-in sensor: A sensor for measuring steps.

[0372] Image recognition algorithm: Technologies such as TensorFlow and OpenCV are used for image recognition.

[0373] Emotion Engine: This engine identifies emotional states through facial expression recognition and voice analysis, using machine learning models with Pytorch and Keras.

[0374] Cloud Server: The remote server is responsible for data analysis and storage, where it calculates calories ingested and burned and generates advice.

[0375] Notification means: Includes a display and an audio output device for notifying the user of the generated advice.

[0376] Explanation of program processing

[0377] Food recording and analysis

[0378] When a user prepares a meal, the camera in the AI ​​glasses automatically captures an image of the meal. The device then sends the captured image data to a cloud server via Wi-Fi or Bluetooth. The server then analyzes the received image data and runs an image recognition algorithm using TensorFlow or OpenCV to identify the food. Based on the analysis results, the nutrient intake is calculated and stored in a database.

[0379] Step counting

[0380] When a user starts moving, the built-in sensor measures the number of steps. This data is sent to a cloud server at regular intervals, and the server calculates the calories burned. Specifically, a calculation algorithm is used based on the step count data to calculate the calories burned.

[0381] Emotion recognition and analysis

[0382] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. This data is then used to identify their emotional state using facial and voice recognition algorithms. The emotion data is then sent to a remote server, which then combines it with past health data for analysis. Machine learning models using Pytorch and Keras are used for the analysis.

[0383] Data integration and advice generation

[0384] The server analyzes calorie intake, calorie expenditure, and emotional data, and then comprehensively evaluates the user's health and emotional state to generate appropriate advice. The advice is displayed in text format on the screen, and voice notification is also available if necessary.

[0385] Specific examples

[0386] For example, if a user has toast and black coffee for breakfast, the AI ​​Glasses will take a photo of this and send the image data to a cloud server. The server will analyze the image, calculate the nutritional value of each food item, and save the results. If the user then takes 2,000 steps on their way to work, the built-in sensor will measure the number of steps and send it to the cloud server. Based on the data received by the server, it will determine that the user has burned approximately 100 calories in 2,000 steps.

[0387] Furthermore, if the emotion engine analyzes the user's facial expressions and voice and determines that they are under high stress, it will use this data to comprehensively evaluate their health condition. Based on this, it will generate advice such as "Today's breakfast was too high in calories, so eat a lighter lunch and create a relaxing environment," and display it on the screen.

[0388] Prompt Sentence Examples

[0389] Here are some example prompts to input to a generative AI model:

[0390] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[0391] In this way, the system of the present invention allows users to manage their diet and exercise habits without straining themselves, and provides appropriate advice according to their emotional state.

[0392] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0393] Specific flow of program processing

[0394] Food recording and analysis

[0395] Step 1:

[0396] The user prepares a meal. At this time, the camera in the AI ​​glasses automatically captures images of the meal. The input is the scene of the user eating, and the output is image data of the meal.

[0397] Step 2:

[0398] The device uses Wi-Fi or Bluetooth to send captured image data to a cloud server. The input is the image data captured by the camera, and the output is the image data sent to the cloud server.

[0399] Step 3:

[0400] The server receives the image data and runs an image recognition algorithm (e.g., TensorFlow or OpenCV). The input is the image data sent to the cloud server, and the output is the analyzed food data.

[0401] Step 4:

[0402] The server identifies foods in the image and calculates the nutritional value (calories, protein, fat, etc.) of each food. The input is the analyzed food data, and the output is the calculated nutritional value information.

[0403] Step 5:

[0404] The server stores the analysis results in a database. The input is the calculated nutritional value information, and the output is the nutritional data stored in the database.

[0405] Step counting

[0406] Step 1:

[0407] The user starts daily activities or exercise. For example, walking to the station for work. The input is the user's movement activity, and the output is walking data.

[0408] Step 2:

[0409] The device's built-in sensor measures walking data in real time. The input is the user's physical steps, and the output is step count data.

[0410] Step 3:

[0411] The device periodically (e.g., every hour) sends the cumulative step count data to the cloud server. The input is the step count data accumulated on the device, and the output is the data sent to the cloud server.

[0412] Step 4:

[0413] The server analyzes the received step count data and calculates the calories burned. The input is the step count data sent to the cloud server, and the output is the calculated calories burned.

[0414] Emotion recognition and analysis

[0415] Step 1:

[0416] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. The input is the user's facial expressions and voice data, and the output is the captured emotion data.

[0417] Step 2:

[0418] The device analyzes these emotion data in real time and sends the identified emotion data to the cloud server. The input is the captured emotion data, and the output is the emotion data sent to the cloud server.

[0419] Step 3:

[0420] The server receives the emotional data and integrates and analyzes the past emotional data and health data. The input is the emotional data sent to the cloud server, and the output is the integrated analysis result.

[0421] Data integration and advice generation

[0422] Step 1:

[0423] The server integrates and analyzes daily calorie intake, calorie expenditure, and emotional data. The input is calorie intake, calorie expenditure, and emotional data, and the output is integrated health evaluation data.

[0424] Step 2:

[0425] The server uses machine learning models (e.g., Pytorch or Keras) to analyze the user's health and emotional state and generate advice. The input is the integrated health assessment data, and the output is the generated advice.

[0426] Step 3:

[0427] The terminal notifies the user of the generated advice. The input is the generated advice, and the output is text displayed on the screen or a voice notification.

[0428] Prompt Sentence Examples

[0429] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[0430] This prompt enables the generative AI model to generate advice based on the user's health and emotional state.

[0431] (Application example 2)

[0432] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0433] Health management is becoming increasingly important in modern society. While there are a wide variety of technologies for preventing and managing lifestyle-related diseases, there is still a lack of systems that provide personalized advice based on individual health conditions. In particular, there is a need for comprehensive support that promotes healthy choices and also includes mental health when choosing food when eating out or at a cafe. However, current systems lack a system that integrates multiple functions, such as analyzing food content, managing the balance of calorie intake and expenditure, and recognizing emotional states. Therefore, new technologies that can achieve these functions are needed.

[0434] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is a system including: means for recording images of meals with a camera; means for analyzing meal contents by image recognition; means for calculating nutrient intake based on the analysis results; means for saving the analysis results on a cloud server; means for counting steps; means for transmitting step count data to the cloud server; means for calculating calories burned on the cloud server; means for comparing calories consumed with calories burned to generate advice; and means for notifying the user of the generated advice. This makes it possible to analyze restaurant menus when dining out, provide nutritional information and recommended meal plans based on individual health conditions, and provide comprehensive mental health support that also takes emotional states into account.

[0435] A "camera" is a device for recording images.

[0436] "Means" refers to the methods and techniques used to achieve a goal.

[0437] "Image recognition" is a technology that extracts, identifies, and classifies objects and features from captured images.

[0438] "Analysis results" refers to the data obtained after processing such as image recognition.

[0439] "Intaken nutrients" refers to the nutrients taken into the body through meals and food.

[0440] A "cloud server" is a device for storing data and performing computations on a remote server accessible via the Internet.

[0441] "Steps" refers to the number of times your feet move when walking a certain distance.

[0442] "Calories consumed" refers to the energy consumed through physical activity and basal metabolism.

[0443] "Advice" refers to instructions or guidance given to another person based on particular knowledge or information.

[0444] "User" refers to a person who uses this system.

[0445] "Store menu" refers to a list of foods and drinks offered at a restaurant or other establishment.

[0446] "Nutritional information" refers to data about the nutritional components and amounts contained in food.

[0447] A "meal plan" refers to a combination of meals designed to achieve a specific health goal.

[0448] "Emotional state" refers to an individual's temporary psychological or mental state.

[0449] "Mental health" refers to an individual's mental and psychological well-being.

[0450] "Comprehensive support" refers to a wide range of assistance and support that encompasses multiple aspects.

[0451] This invention relates to the "AI Glass Guide" system, which is used in health-conscious cafes and other places, and aims to provide optimal health advice by simultaneously analyzing dietary content and emotional state.

[0452] System configuration

[0453] The system of the present invention uses the following hardware and software.

[0454] Hardware:

[0455] 1. AI Glasses - Includes a camera (e.g., Sony IMX camera), a display (OLED display), a microphone, and a built-in sensor (MPU6050 sensor for counting steps).

[0456] 2. Cloud Server - A server accessible via the Internet.

[0457] software:

[0458] 1. Image Recognition Algorithms - Custom deep learning models using TensorFlow and PyTorch.

[0459] 2. Emotion recognition engine - A proprietary library called EmotionEngine.

[0460] 3. Cloud communication - Uses APIs from AWS and Google Cloud Platform.

[0461] 4. Health Advisor - A program that generates advice based on health and emotional data.

[0462] The role of each hardware and software

[0463] 1. A user visits a cafe, puts on the AI ​​glasses, and uses the camera in the glasses to scan the menu.

[0464] 2. The device (AI glasses) takes a picture of the menu and sends it to the cloud server.

[0465] 3. The server uses image recognition algorithms to identify the foods on the menu and parse the nutritional information.

[0466] 4. The server stores the analysis results in a database.

[0467] 5. When the user selects a meal, the AI ​​glasses will generate advice about it and display it on the screen or notify them via voice.

[0468] 6. The device's built-in sensor measures the user's steps and sends them to the cloud server.

[0469] 7. The server calculates the calories burned based on the step count data and compares this with the calories ingested to generate optimal advice.

[0470] 8. The emotion engine analyzes the user's emotional state and provides information about their mental health.

[0471] Program processing explanation

[0472] 1. Image processing: The image data of the menu captured by the AI ​​glasses' camera is preprocessed using OpenCV, and then an image recognition algorithm trained in TensorFlow or PyTorch is run to identify the food.

[0473] 2. Cloud communication: Analysis results and meal content data are sent in real time to a cloud server via the AWS or Google Cloud Platform API.

[0474] 3. Data Analysis: The cloud server uses EmotionEngine to analyze the user's facial expression data to identify their emotional state, and then integrates it with their health data to generate advice.

[0475] 4. Notification: The generated advice is notified to the user through the AI ​​glasses' OLED display and audio output.

[0476] Examples and prompts

[0477] Specific examples

[0478] 1. When visiting the cafe: The user enters the cafe, borrows and puts on the AI ​​glasses. "Hello, welcome to Health Cafe. What would you like to order today?"

[0479] 2. When selecting a meal: The user scans the menu with their camera and the analysis results are displayed. "This chicken salad is 120 calories. It's recommended considering today's calorie intake."

[0480] Prompt Sentence Examples

[0481] "The camera on the AI ​​glasses takes pictures of meals, sends them to the cloud, and analyzes their nutritional value. Based on the analysis results, the system will suggest the optimal meal plan for the user based on their health condition."

[0482] With specific technical configurations and operating procedures, users can manage their health even when eating out, and can also receive comprehensive support that takes into account their emotional state.

[0483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0484] Step 1:

[0485] A user visits a cafe, borrows the AI ​​glasses, and puts them on. Putting on the AI ​​glasses is the input, and the system becomes ready for use.

[0486] Step 2:

[0487] The user scans the menu by pointing it at the camera on the AI ​​glasses. The menu image is then input. The camera captures the image of the menu, which is then acquired by the device. The acquired image data is then sent to the next processing step.

[0488] Step 3:

[0489] The device sends the captured image data to the cloud server. The input is the image data of the menu, and the output is the image data sent to the cloud server. This is done using cloud communication.

[0490] Step 4:

[0491] The server uses an image recognition algorithm to analyze the menu image data sent. The input is the menu image data, and the image recognition algorithm extracts image features to identify the food items. The output is a list of identified foods.

[0492] Step 5:

[0493] The server analyzes the nutritional value information based on the food list. The input is the identified food list, and the server obtains the nutritional value information by referencing the nutrient database. The output is the analysis result including the nutritional value information.

[0494] Step 6:

[0495] The server stores the analysis results in a database. The input is the analysis results including nutritional information, and storing this data in storage allows for future reference. The output is an updated database.

[0496] Step 7:

[0497] The server generates dietary advice and sends it to the device via the cloud. The input is nutritional information and the user's health data, which the machine learning model analyzes to generate advice. The output is the generated advice.

[0498] Step 8:

[0499] The device displays the generated advice on the AI ​​glasses display or notifies the user by voice. The input is advice data sent from the server, which is notified to the user. The output is a visual or audio notification to the user.

[0500] Step 9:

[0501] The user selects a meal and then begins to move. The built-in sensor in the AI ​​glasses measures the user's steps. The input is the user's movement data, which the sensor measures to generate step count data. The output is the measured step count data.

[0502] Step 10:

[0503] The device sends step count data to the cloud server. The input is the measured step count data, and the output is the step count data sent to the cloud server.

[0504] Step 11:

[0505] The server calculates the calories burned based on the step count data. The input is the step count data, and the server calculates the calories burned as a data operation. The output is the calculated calories burned.

[0506] Step 12:

[0507] The server compares the calories ingested and burned, and the emotion engine analyzes the user's emotional state. The inputs are calories ingested, calories burned, and facial expression data, which are analyzed by the machine learning model and the Emotion Engine. The output is comprehensive health and mental health advice.

[0508] Step 13:

[0509] The server sends the generated advice to the terminal, which then notifies the user. The input is the generated advice, and the output is the advice notified to the user.

[0510] Through these processing steps, the system of the present invention provides comprehensive health management support that integrates nutritional analysis of food, step counting, and emotional state recognition.

[0511] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0512] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0513] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0514] [Second embodiment]

[0515] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0516] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0517] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0518] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0519] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0520] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0521] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0522] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0523] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0524] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0525] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0526] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0527] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases. The system aims to provide users with appropriate advice by recording and analyzing dietary information, measuring steps, and managing the balance between calorie intake and calorie expenditure.

[0528] System configuration

[0529] The system of the present invention includes the following major components:

[0530] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[0531] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[0532] 3. Cloud Server - A remote server for storing analysis results and performing necessary calculations and data storage.

[0533] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[0534] 5. Notification means - includes a display and audio output device to notify the user of advice.

[0535] Specific explanation of program processing

[0536] Food recording and analysis

[0537] 1. The user prepares breakfast on the table, and the camera in the AI ​​glasses takes pictures of the meal.

[0538] 2. The device (AI glasses) sends the captured image data to the cloud server.

[0539] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[0540] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[0541] Step counting

[0542] 1. When the user commutes to work or exercises, the AI ​​glasses measure the number of steps taken using the built-in sensor.

[0543] 2. The device sends step count data to the cloud server at regular intervals.

[0544] 3. The server analyzes the received step count data and calculates the calories burned.

[0545] Data integration and advice generation

[0546] 1. The server compares the daily calorie intake with the calorie expenditure.

[0547] 2. The server generates advice based on the user's health status, for example, to improve diet and exercise.

[0548] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[0549] Specific examples

[0550] Example 1: Breakfast recording and analysis

[0551] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[0552] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[0553] The server stores this information and updates the user database.

[0554] Example 2: Measuring steps during commute

[0555] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[0556] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[0557] Example 3: Providing advice

[0558] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[0559] The device will notify the user of this advice and display it on the AI ​​glasses' display.

[0560] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[0561] The processing flow will be explained below.

[0562] Step 1:

[0563] The user prepares a meal, for example, bread and orange juice for breakfast.

[0564] Step 2:

[0565] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[0566] Step 3:

[0567] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[0568] Step 4:

[0569] The server adds the received image data to a processing queue.

[0570] Step 5:

[0571] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[0572] Step 6:

[0573] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[0574] Step 7:

[0575] The server organizes and stores the calculated nutrient data in a database for each user.

[0576] Step 8:

[0577] The user begins their daily routine or exercise, for example, walking to the station for work.

[0578] Step 9:

[0579] The device's built-in sensor measures the user's step count data.

[0580] Step 10:

[0581] The device sends step count data to the server at regular intervals.

[0582] Step 11:

[0583] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[0584] Step 12:

[0585] The server aggregates daily calorie, salt, fat, alcohol, and step count data.

[0586] Step 13:

[0587] The server uses machine learning models to analyze the user's health status and generate data-based advice, such as "Today's breakfast was too calorie-dense, so choose a vegetable-based lunch."

[0588] Step 14:

[0589] The server sends the advice to the notification system.

[0590] Step 15:

[0591] The terminal receives the advice from the server.

[0592] Step 16:

[0593] The device will then display the received advice in the user's field of vision, for example, "Eat mostly vegetables for lunch" on the Glasses' display.

[0594] Step 17:

[0595] The user follows the advice and practices a healthy diet.

[0596] Example 1

[0597] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0598] In conventional systems for preventing and managing lifestyle-related diseases, recording meals and counting steps are often done manually, placing a heavy burden on users. Furthermore, manually calculating calories consumed and calories burned can lead to a decrease in accuracy. Furthermore, it is difficult to provide appropriate advice tailored to each individual user's health condition.

[0599] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0600] In this invention, the server includes means for a user to prepare a meal and record an image of the meal with a camera, means for the terminal to send the image data to a cloud server, means for the cloud server to analyze the meal contents using an image recognition algorithm, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a database, means for using a built-in sensor to count steps, means for sending the step count data to the cloud server, means for calculating calories burned on the cloud server, means for comparing calories consumed with calories burned to generate advice, and means for notifying the user of the generated advice, thereby enabling users to efficiently and accurately manage their own dietary and exercise habits.

[0601] A "user" is a person who uses the system to manage their own eating and exercise habits and prevent and manage lifestyle-related diseases.

[0602] "Camera" refers to a photographic device used to record images of a meal.

[0603] A "terminal" is a device used to send captured image data and step count data to a cloud server.

[0604] A "cloud server" is an infrastructure for storing and analyzing data on a remote server.

[0605] An "image recognition algorithm" is a technology that analyzes food contents from captured images and identifies the type and amount of food.

[0606] A "database" is a digital recording medium for storing analysis results and nutrient information.

[0607] "Built-in sensor" refers to a sensor device built into a device to measure steps.

[0608] "Calories burned" refers to the amount of energy consumed calculated based on step count data.

[0609] "Advice" refers to dietary and exercise recommendations for the user that are generated by comparing calories ingested with calories burned.

[0610] The present invention relates to a system that supports the prevention and management of lifestyle-related diseases. This system aims to provide users with appropriate advice by recording and analyzing dietary content, measuring the number of steps taken, and managing the balance between calorie intake and calorie expenditure.

[0611] System configuration

[0612] The system of the present invention includes the following major components:

[0613] 1. Camera - A photographic device used to record images of meals.

[0614] 2. Device - A device that sends captured image data and step count data to a cloud server. It has an internal sensor and also measures the number of steps.

[0615] 3. Cloud Server - A remote server that stores and analyzes data.

[0616] 4. Image recognition algorithms - These are technologies used to analyze food content and identify the types and quantities of food. For example, software such as TensorFlow and OpenCV are examples of such algorithms.

[0617] 5. Database - A digital recording medium for storing analysis results and nutritional information.

[0618] 6. Built-in sensors - Includes an accelerometer and gyro sensor for counting steps.

[0619] 7. Notification means - A display or audio output device for notifying the user of advice.

[0620] Specific processing of the program

[0621] Food recording and analysis

[0622] 1. The user prepares breakfast and takes a picture of the meal using the AI ​​glasses. For example, if the user prepares bread and milk, they press the camera button to take a picture.

[0623] 2. The device sends the captured image data to a cloud server using Wi-Fi or mobile data.

[0624] 3. The server receives the image data and analyzes the meal contents using an image recognition algorithm (e.g., YOLO or ResNet). It identifies bread and milk and calculates the nutritional value of each.

[0625] 4. The server stores the analysis results in the user's database, specifically recording information such as the bread's calories, carbohydrates, and fat content.

[0626] Step counting

[0627] 1. The user begins a daily activity, for example, leaving home for their morning commute.

[0628] 2. The device uses a built-in sensor to count steps and detects walking movements in real time.

[0629] 3. The device sends the measured data at regular intervals (for example, every 10 minutes) to the cloud server.

[0630] 4. The server analyzes the step count data and calculates the calories burned. For example, if 3,000 steps result in 150 calories burned, the server records this data in the user's database.

[0631] Data integration and advice generation

[0632] 1. The server aggregates the daily calorie intake and calorie expenditure data. For example, it aggregates data showing that you ate 500 calories at breakfast and burned 150 calories by walking.

[0633] 2. The server generates diet and exercise advice based on the user's individual profile (e.g., age, gender, weight, activity level, etc.). If the calorie intake is high, advice such as "eat a lighter lunch" or "exercise more" is generated.

[0634] 3. The device receives the advice generated by the server and notifies the user, either by displaying it on the AI ​​glasses display or by voice notification.

[0635] Examples of concrete examples and prompts

[0636] Specific examples

[0637] The user prepares yogurt and a banana for breakfast, and the AI ​​glasses take an image of the meal and send the data to a cloud server.

[0638] The server analyzes the image and calculates the nutritional value of yogurt (100 calories) and a banana (90 calories).

[0639] The server stores this information and updates the user database.

[0640] Prompt Sentence Examples

[0641] "How many calories did you have at dinner last night?"

[0642] "How many calories are in my breakfast today?"

[0643] "Based on your step count data this week, do you need additional exercise?"

[0644] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[0645] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0646] Step 1:

[0647] The user prepares breakfast and uses the AI ​​glasses to take pictures of the meal.

[0648] Input: User prepares meal and presses camera button.

[0649] Output: The camera generates image data of the meal.

[0650] Specific operation: When the user presses the camera button, the camera on the AI ​​glasses takes a photo of the meal and the image data is saved on the device.

[0651] Step 2:

[0652] The device sends the captured image data to a cloud server.

[0653] Input: The captured image data is saved on the device.

[0654] Output: The image data is sent to the cloud server.

[0655] Specific operation: The device sends image data to the cloud server using Wi-Fi or mobile data.

[0656] Step 3:

[0657] The server applies image recognition algorithms to analyze the meal contents.

[0658] Input: Image data stored on a cloud server.

[0659] Output: Analysis results regarding food type, quantity, and nutritional value.

[0660] Specific operation: The server analyzes the image using an image recognition algorithm (e.g., TensorFlow or OpenCV), identifies the type and quantity of food, and calculates nutritional value.

[0661] Step 4:

[0662] The server stores the analysis results in a database.

[0663] Input: Analysis results (food type, amount, and nutritional value information).

[0664] Output: Analysis results stored in a database.

[0665] Specific operation: The server records the analysis results in the user's database, and nutritional information (calories, carbohydrates, fats, etc.) is saved.

[0666] Step 5:

[0667] The user begins their daily activities.

[0668] Input: The user starts walking.

[0669] Output: Real-time step count data is generated by sensors in the AI ​​glasses.

[0670] Specific operation: The user starts commuting or exercising, and the AI ​​glasses detect walking movements and count the number of steps.

[0671] Step 6:

[0672] The device periodically transmits step count data to a cloud server.

[0673] Input: Measured step count data.

[0674] Output: Step count data sent to cloud server.

[0675] Specific operation: The device sends step count data to the cloud server at regular intervals (for example, every 10 minutes).

[0676] Step 7:

[0677] The server analyzes the received step count data and calculates the calories burned.

[0678] Input: Step count data stored on a cloud server.

[0679] Output: Calculated calories burned.

[0680] Specific operation: The server analyzes the step count data, calculates the calories burned, and records them in a database.

[0681] Step 8:

[0682] The server aggregates the daily calorie intake and expenditure data.

[0683] Input: Stored calorie intake and calorie expenditure data.

[0684] Output: Consolidated calorie data.

[0685] Specific operation: The server integrates the calorie intake and calorie expenditure information obtained from the database and calculates the daily calorie balance.

[0686] Step 9:

[0687] The server generates advice based on the user's profile.

[0688] Input: Consolidated calorie data and user profile information (age, gender, weight, activity level).

[0689] Output: The generated advice.

[0690] Specific operation: The server uses the generated AI model to generate advice (recommended dietary restrictions and exercise) based on the user's health condition.

[0691] Step 10:

[0692] The terminal notifies the user of the generated advice.

[0693] Input: Advice data from the server.

[0694] Output: Advice given to the user.

[0695] Specific operation: The device notifies the user of the advice using a display or voice output.

[0696] (Application example 1)

[0697] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0698] Modern brick-and-mortar stores lack an effective system to support customers in making healthy dietary choices. They also need a way to provide personalized advice in real time based on each customer's health status. This makes it difficult for customers to prevent and manage lifestyle-related diseases without stress.

[0699] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0700] In this invention, the server includes means for recording images of meals using a camera, means for analyzing meal contents using image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a cloud server, means for counting steps, means for transmitting step count data to the cloud server, means for calculating calories burned on the cloud server, means for generating advice by comparing calories consumed with calories burned, means for notifying the user of the generated advice, means for analyzing food calories and nutrients in real time and suggesting nutritionally balanced meals to support customers' healthy dietary choices in the physical store, and means for generating and presenting personalized health advice based on each customer's health condition, thereby enabling customers to receive information supporting healthy lifestyle habits in real time in the physical store.

[0701] A "camera" is a photographic device for recording images of meals.

[0702] "Image recognition" is a technology that analyzes recorded meal contents.

[0703] A "cloud server" is a remote server that stores data and performs calculations such as calorie consumption.

[0704] "Step counting" is a means for counting the number of steps.

[0705] "Calculating calories burned" is a method for calculating calories burned from step count data.

[0706] "Personalized health advice" is a means of generating specific recommendations based on each customer's health status.

[0707] "Nutritionally balanced meal suggestions" is a method of analyzing the calories and nutrients of food and supporting healthy dietary choices.

[0708] The present invention provides a system for supporting health management of customers in brick-and-mortar stores, and in particular provides a specific configuration for analyzing dietary content and providing advice. This system includes the following main hardware and software:

[0709] System configuration

[0710] 1. Camera

[0711] The camera is built into the AI ​​glasses and is a device for capturing images of the food. The camera takes pictures of the dishes selected by the customer in real time and sends the image data to the system.

[0712] 2. Image Recognition Algorithm

[0713] The image recognition algorithm is a technology that analyzes photographed images of food and identifies the type and amount of food. This algorithm is implemented using machine learning libraries such as TensorFlow and Keras.

[0714] 3. Cloud Server

[0715] A cloud server is a remote server that stores analysis results and manages the necessary calculations and data. AWS (Amazon Web Services) and Google Cloud Platform are often used.

[0716] 4. Built-in sensor

[0717] The built-in sensor is a device that measures the number of steps taken by customers, and the step count data is sent to a cloud server via Bluetooth or other means.

[0718] 5. Means of notification

[0719] The notification means includes a display and an audio output device for notifying the user of the advice, which allows healthy eating suggestions and advice to be provided to customers in real time within the physical store.

[0720] Program processing explanation

[0721] Camera Function:

[0722] A camera is built into the AI ​​glasses and takes a picture of the customer's dish selection, which is then instantly sent to an image recognition algorithm.

[0723] Image Recognition:

[0724] Image recognition algorithms analyze captured images to identify the type and quantity of food, using a pre-trained neural network model to identify nutritional and calorie information for each food item.

[0725] Cloud server processing:

[0726] The detected meal content data is sent to a cloud server and stored there, where the cloud server also calculates the calories burned and compares them with the user's calorie intake.

[0727] Advice Generation:

[0728] The cloud server generates personalized health advice based on the user's health status, such as nutritionally balanced meal suggestions and exercise instructions.

[0729] notification:

[0730] The generated advice is sent to the user in real time either as a display on the AI ​​glasses or as a voice prompt.

[0731] Specific examples

[0732] Example 1: Use in a restaurant

[0733] When a user orders a steak at a restaurant and takes a picture of the meal with the AI ​​glasses, the system identifies it as a steak and instantly obtains calorie information, which can be used as a guide for the next meal or exercise.

[0734] Example prompt: "From this image, identify the type of food and its calories, and analyze whether it is nutritionally balanced."

[0735] Example 2: Use in a cafe

[0736] When a user orders a salad and soup at a cafe, the AI ​​glasses recognize this and suggest healthy eating choices, informing them of any food or exercise they should avoid later.

[0737] Example prompt: "Calculate the nutritional value of a salad (200 calories) and a soup (150 calories) and store that information in a database."

[0738] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0739] Step 1:

[0740] The user takes a photo of the dish they have selected with a camera, which takes image data of the dish as input and sends the image to an image recognition algorithm.

[0741] Step 2:

[0742] The device runs an image recognition algorithm to analyze the input image data. Specifically, after preprocessing the image, it inputs it into a neural network model to identify the type and quantity of food. The output is the type of food and its nutritional information.

[0743] Step 3:

[0744] The server receives the analysis results and retrieves food types and nutrient information from the nutrition database. It takes food types as input, executes a database query, and obtains nutrient data as output.

[0745] Step 4:

[0746] The server saves the nutrient data on the cloud server. It receives the analysis results and nutrient data as input and stores them in the database. The output confirms that the data is persisted.

[0747] Step 5:

[0748] When a user walks, the device's built-in sensor measures the number of steps taken. The device receives step count data in real time as input and sends the data to a cloud server.

[0749] Step 6:

[0750] The server calculates calories burned based on the step count data received. It receives step count data as input and calculates calories burned using a predefined algorithm. It obtains the calorie burned value as output.

[0751] Step 7:

[0752] The server compares daily calorie intake and expenditure and generates personalized advice. It receives calorie intake and expenditure data as input and generates advice using a health management algorithm. The output is advice text.

[0753] Step 8:

[0754] The device notifies the user of the generated advice. It receives the advice text as input and displays it on the AI ​​glasses display or notifies the user by voice.

[0755] Step 9:

[0756] The server analyzes food calories and nutrients in real time and suggests nutritionally balanced meals to support healthy meal choices for users in physical stores. It receives data on in-store dishes as input and analyzes it in real time. It outputs healthy meal suggestions to be presented to customers.

[0757] Step 10:

[0758] The server generates and presents personalized health advice based on each customer's health status. It receives each customer's health data as input, analyzes it using a machine learning model, and obtains personalized advice as output.

[0759] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0760] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[0761] System configuration

[0762] The system of the present invention includes the following major components:

[0763] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[0764] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[0765] 3. Cloud Server - A remote server that stores analysis results and emotion data, and performs necessary calculations and data storage.

[0766] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[0767] 5. Emotion Engine - Contains technology to identify the user's emotional state.

[0768] 6. Notification means - includes a display and audio output device to notify the user of advice.

[0769] Specific explanation of program processing

[0770] Food recording and analysis

[0771] 1. The user prepares a meal, and the camera in the AI ​​glasses takes a picture of the meal.

[0772] 2. The device (AI glasses) sends the captured image data to the cloud server.

[0773] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[0774] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[0775] Step counting

[0776] 1. The user begins their daily routine or exercise, for example, walking to the station for work.

[0777] 2. The device's built-in sensor measures the user's step data.

[0778] 3. The device sends step count data to the cloud server at regular intervals.

[0779] 4. The server analyzes the received step count data and calculates the calories burned.

[0780] Emotion recognition and analysis

[0781] 1. The emotion engine identifies the user's emotional state. For example, it determines whether the user is feeling stressed based on facial expressions and voice data.

[0782] 2. The server sends the emotional data to the cloud, where it integrates and analyzes past emotional data and health data.

[0783] Data integration and advice generation

[0784] 1. The server compares and integrates the daily calorie intake, calorie expenditure, and emotional data.

[0785] 2. The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[0786] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[0787] Specific examples

[0788] Example 1: Breakfast recording and analysis

[0789] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[0790] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[0791] The server stores this information and updates the user database.

[0792] Example 2: Measuring steps during commute

[0793] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[0794] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[0795] Example 3: Providing advice

[0796] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[0797] The device will notify the user of this advice and display it on the AI ​​glasses' display. If the user's emotional state is high, the device will also suggest taking a short break to relax.

[0798] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[0799] The processing flow will be explained below.

[0800] Step 1:

[0801] The user prepares a meal, for example, bread and orange juice for breakfast.

[0802] Step 2:

[0803] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[0804] Step 3:

[0805] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[0806] Step 4:

[0807] The server adds the received image data to a processing queue.

[0808] Step 5:

[0809] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[0810] Step 6:

[0811] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[0812] Step 7:

[0813] The server organizes and stores the calculated nutrient data in a database for each user.

[0814] Step 8:

[0815] The user begins their daily routine or exercise, for example, walking to the station for work.

[0816] Step 9:

[0817] The device's built-in sensor measures the user's step count data.

[0818] Step 10:

[0819] The device sends step count data to the server at regular intervals.

[0820] Step 11:

[0821] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[0822] Step 12:

[0823] The emotion engine recognizes the user's facial expressions and voice to identify their emotional state, for example, determining whether they are stressed.

[0824] Step 13:

[0825] The device transmits the emotion data to the cloud server.

[0826] Step 14:

[0827] The server receives the emotional data and integrates and analyzes the past emotional data and health data.

[0828] Step 15:

[0829] The server compares and integrates daily calorie intake, calorie expenditure, and emotional data.

[0830] Step 16:

[0831] The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[0832] Step 17:

[0833] The server sends the advice to the notification system.

[0834] Step 18:

[0835] The terminal receives the advice from the server.

[0836] Step 19:

[0837] The device will then display the received advice in the user's field of vision. For example, the Glasses' display will say, "Eat a vegetable-based lunch." If the user is feeling stressed, the device will suggest, "Take a short break to relax."

[0838] Step 20:

[0839] Users follow advice to implement healthy eating, exercise, and stress management.

[0840] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[0841] Example 2

[0842] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0843] Achieving a balance between daily calorie intake and calorie expenditure is important for preventing and managing lifestyle-related diseases, but it is difficult for users to consistently maintain this balance in their daily lives. Furthermore, a user's emotional state is also an important factor in health management, and it is widely known that stress, in particular, has a negative impact on health. However, current systems only record dietary habits and measure steps, and are unable to provide comprehensive health management that also takes emotional state into account. Therefore, there is a need for a system that can comprehensively manage a user's health and emotional state and provide individually optimized advice.

[0844] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0845] In this invention, the server includes means for recording images of meals, means for analyzing meal contents by image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results on a remote server, means for counting steps, means for transmitting step count data to the remote server, means for calculating calories burned on the remote server, means for comparing calories consumed with calories burned to generate advice, means for identifying the user's emotional state and transmitting the identified emotional data to the remote server, means for integrating and analyzing the user's emotional data and health data, and means for notifying the user of the generated advice. This enables the user to comprehensively manage their health throughout the day and receive individually optimized advice that also takes into account their emotional state.

[0846] The "means for recording images of meals" refers to a device that uses a photographing device such as a camera to record images of meals consumed by the user as digital data.

[0847] "Means for analyzing meal contents using image recognition" refers to technologies and algorithms that analyze image data of photographed meals and identify the foods contained in the meal and their amounts.

[0848] The "means for calculating the nutrients ingested based on the analysis results" refers to a computing device or program that calculates the calories ingested and various nutrients from the dietary content data obtained by image recognition.

[0849] The "means for saving the analysis results on a remote server" is a function for saving the calculated nutrient data and other analysis results on a remote server via the Internet.

[0850] The "means for measuring the number of steps" refers to a device or sensor for measuring the walking activity of a user, and includes a pedometer.

[0851] The "means for transmitting step count data to a remote server" refers to a communication device or function for transmitting the measured step count data to a remote server via the Internet.

[0852] The "means for calculating calories burned on a remote server" refers to an algorithm or program that calculates calories burned based on the step count data received on the remote server.

[0853] The "means for generating advice by comparing calorie intake with calorie expenditure" refers to a device or software that compares the balance between calorie intake and calorie expenditure and automatically generates advice for health management.

[0854] "Means for identifying the user's emotional state and transmitting the identified emotional data to a remote server" refers to technology or devices for identifying the user's emotions through facial expression recognition, voice analysis, etc., and transmitting that data to a remote server.

[0855] "Means for integrating and analyzing a user's emotional data and health data" refers to an algorithm or program that integrates and analyzes a user's emotional data and health data (calories ingested, calories burned, etc.) to assess their overall health status.

[0856] The "means for notifying the user of the generated advice" refers to a notification device such as a display or audio output device for conveying the generated health management advice to the user.

[0857] This invention relates to an AI system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[0858] Hardware and software used

[0859] Camera: A device built into the AI ​​glasses that takes pictures of meals.

[0860] Built-in sensor: A sensor for measuring steps.

[0861] Image recognition algorithm: Technologies such as TensorFlow and OpenCV are used for image recognition.

[0862] Emotion Engine: This engine identifies emotional states through facial expression recognition and voice analysis, using machine learning models with Pytorch and Keras.

[0863] Cloud Server: The remote server is responsible for data analysis and storage, where it calculates calories ingested and burned and generates advice.

[0864] Notification means: Includes a display and an audio output device for notifying the user of the generated advice.

[0865] Explanation of program processing

[0866] Food recording and analysis

[0867] When a user prepares a meal, the camera in the AI ​​glasses automatically captures an image of the meal. The device then sends the captured image data to a cloud server via Wi-Fi or Bluetooth. The server then analyzes the received image data and runs an image recognition algorithm using TensorFlow or OpenCV to identify the food. Based on the analysis results, the nutrient intake is calculated and stored in a database.

[0868] Step counting

[0869] When a user starts moving, the built-in sensor measures the number of steps. This data is sent to a cloud server at regular intervals, and the server calculates the calories burned. Specifically, a calculation algorithm is used based on the step count data to calculate the calories burned.

[0870] Emotion recognition and analysis

[0871] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. This data is then used to identify their emotional state using facial and voice recognition algorithms. The emotion data is then sent to a remote server, which then combines it with past health data for analysis. Machine learning models using Pytorch and Keras are used for the analysis.

[0872] Data integration and advice generation

[0873] The server analyzes calorie intake, calorie expenditure, and emotional data, and then comprehensively evaluates the user's health and emotional state to generate appropriate advice. The advice is displayed in text format on the screen, and voice notification is also available if necessary.

[0874] Specific examples

[0875] For example, if a user has toast and black coffee for breakfast, the AI ​​Glasses will take a photo of this and send the image data to a cloud server. The server will analyze the image, calculate the nutritional value of each food item, and save the results. If the user then takes 2,000 steps on their way to work, the built-in sensor will measure the number of steps and send it to the cloud server. Based on the data received by the server, it will determine that the user has burned approximately 100 calories in 2,000 steps.

[0876] Furthermore, if the emotion engine analyzes the user's facial expressions and voice and determines that they are under high stress, it will use this data to comprehensively evaluate their health condition. Based on this, it will generate advice such as "Today's breakfast was too high in calories, so eat a lighter lunch and create a relaxing environment," and display it on the screen.

[0877] Prompt Sentence Examples

[0878] Here are some example prompts to input to a generative AI model:

[0879] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[0880] In this way, the system of the present invention allows users to manage their diet and exercise habits without straining themselves, and provides appropriate advice according to their emotional state.

[0881] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0882] Specific flow of program processing

[0883] Food recording and analysis

[0884] Step 1:

[0885] The user prepares a meal. At this time, the camera in the AI ​​glasses automatically captures images of the meal. The input is the scene of the user eating, and the output is image data of the meal.

[0886] Step 2:

[0887] The device uses Wi-Fi or Bluetooth to send captured image data to a cloud server. The input is the image data captured by the camera, and the output is the image data sent to the cloud server.

[0888] Step 3:

[0889] The server receives the image data and runs an image recognition algorithm (e.g., TensorFlow or OpenCV). The input is the image data sent to the cloud server, and the output is the analyzed food data.

[0890] Step 4:

[0891] The server identifies foods in the image and calculates the nutritional value (calories, protein, fat, etc.) of each food. The input is the analyzed food data, and the output is the calculated nutritional value information.

[0892] Step 5:

[0893] The server stores the analysis results in a database. The input is the calculated nutritional value information, and the output is the nutritional data stored in the database.

[0894] Step counting

[0895] Step 1:

[0896] The user starts daily activities or exercise. For example, walking to the station for work. The input is the user's movement activity, and the output is walking data.

[0897] Step 2:

[0898] The device's built-in sensor measures walking data in real time. The input is the user's physical steps, and the output is step count data.

[0899] Step 3:

[0900] The device periodically (e.g., every hour) sends the cumulative step count data to the cloud server. The input is the step count data accumulated on the device, and the output is the data sent to the cloud server.

[0901] Step 4:

[0902] The server analyzes the received step count data and calculates the calories burned. The input is the step count data sent to the cloud server, and the output is the calculated calories burned.

[0903] Emotion recognition and analysis

[0904] Step 1:

[0905] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. The input is the user's facial expressions and voice data, and the output is the captured emotion data.

[0906] Step 2:

[0907] The device analyzes these emotion data in real time and sends the identified emotion data to the cloud server. The input is the captured emotion data, and the output is the emotion data sent to the cloud server.

[0908] Step 3:

[0909] The server receives the emotional data and integrates and analyzes the past emotional data and health data. The input is the emotional data sent to the cloud server, and the output is the integrated analysis result.

[0910] Data integration and advice generation

[0911] Step 1:

[0912] The server integrates and analyzes daily calorie intake, calorie expenditure, and emotional data. The input is calorie intake, calorie expenditure, and emotional data, and the output is integrated health evaluation data.

[0913] Step 2:

[0914] The server uses machine learning models (e.g., Pytorch or Keras) to analyze the user's health and emotional state and generate advice. The input is the integrated health assessment data, and the output is the generated advice.

[0915] Step 3:

[0916] The terminal notifies the user of the generated advice. The input is the generated advice, and the output is text displayed on the screen or a voice notification.

[0917] Prompt Sentence Examples

[0918] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[0919] This prompt enables the generative AI model to generate advice based on the user's health and emotional state.

[0920] (Application example 2)

[0921] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0922] Health management is becoming increasingly important in modern society. While there are a wide variety of technologies for preventing and managing lifestyle-related diseases, there is still a lack of systems that provide personalized advice based on individual health conditions. In particular, there is a need for comprehensive support that promotes healthy choices and also includes mental health when choosing food when eating out or at a cafe. However, current systems lack a system that integrates multiple functions, such as analyzing food content, managing the balance of calorie intake and expenditure, and recognizing emotional states. Therefore, new technologies that can achieve these functions are needed.

[0923] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is a system including: means for recording images of meals with a camera; means for analyzing meal contents by image recognition; means for calculating nutrient intake based on the analysis results; means for saving the analysis results on a cloud server; means for counting steps; means for transmitting step count data to the cloud server; means for calculating calories burned on the cloud server; means for comparing calories consumed with calories burned to generate advice; and means for notifying the user of the generated advice. This makes it possible to analyze restaurant menus when dining out, provide nutritional information and recommended meal plans based on individual health conditions, and provide comprehensive mental health support that also takes emotional states into account.

[0924] A "camera" is a device for recording images.

[0925] "Means" refers to the methods and techniques used to achieve a goal.

[0926] "Image recognition" is a technology that extracts, identifies, and classifies objects and features from captured images.

[0927] "Analysis results" refers to the data obtained after processing such as image recognition.

[0928] "Intaken nutrients" refers to the nutrients taken into the body through meals and food.

[0929] A "cloud server" is a device for storing data and performing computations on a remote server accessible via the Internet.

[0930] "Steps" refers to the number of times your feet move when walking a certain distance.

[0931] "Calories consumed" refers to the energy consumed through physical activity and basal metabolism.

[0932] "Advice" refers to instructions or guidance given to another person based on particular knowledge or information.

[0933] "User" refers to a person who uses this system.

[0934] "Store menu" refers to a list of foods and drinks offered at a restaurant or other establishment.

[0935] "Nutritional information" refers to data about the nutritional components and amounts contained in food.

[0936] A "meal plan" refers to a combination of meals designed to achieve a specific health goal.

[0937] "Emotional state" refers to an individual's temporary psychological or mental state.

[0938] "Mental health" refers to an individual's mental and psychological well-being.

[0939] "Comprehensive support" refers to a wide range of assistance and support that encompasses multiple aspects.

[0940] This invention relates to the "AI Glass Guide" system, which is used in health-conscious cafes and other places, and aims to provide optimal health advice by simultaneously analyzing dietary content and emotional state.

[0941] System configuration

[0942] The system of the present invention uses the following hardware and software.

[0943] Hardware:

[0944] 1. AI Glasses - Includes a camera (e.g., Sony IMX camera), a display (OLED display), a microphone, and a built-in sensor (MPU6050 sensor for counting steps).

[0945] 2. Cloud Server - A server accessible via the Internet.

[0946] software:

[0947] 1. Image Recognition Algorithms - Custom deep learning models using TensorFlow and PyTorch.

[0948] 2. Emotion recognition engine - A proprietary library called EmotionEngine.

[0949] 3. Cloud communication - Uses APIs from AWS and Google Cloud Platform.

[0950] 4. Health Advisor - A program that generates advice based on health and emotional data.

[0951] The role of each hardware and software

[0952] 1. A user visits a cafe, puts on the AI ​​glasses, and uses the camera in the glasses to scan the menu.

[0953] 2. The device (AI glasses) takes a picture of the menu and sends it to the cloud server.

[0954] 3. The server uses image recognition algorithms to identify the foods on the menu and parse the nutritional information.

[0955] 4. The server stores the analysis results in a database.

[0956] 5. When the user selects a meal, the AI ​​glasses will generate advice about it and display it on the screen or notify them via voice.

[0957] 6. The device's built-in sensor measures the user's steps and sends them to the cloud server.

[0958] 7. The server calculates the calories burned based on the step count data and compares this with the calories ingested to generate optimal advice.

[0959] 8. The emotion engine analyzes the user's emotional state and provides information about their mental health.

[0960] Program processing explanation

[0961] 1. Image processing: The image data of the menu captured by the AI ​​glasses' camera is preprocessed using OpenCV, and then an image recognition algorithm trained in TensorFlow or PyTorch is run to identify the food.

[0962] 2. Cloud communication: Analysis results and meal content data are sent in real time to a cloud server via the AWS or Google Cloud Platform API.

[0963] 3. Data Analysis: The cloud server uses EmotionEngine to analyze the user's facial expression data to identify their emotional state, and then integrates it with their health data to generate advice.

[0964] 4. Notification: The generated advice is notified to the user through the AI ​​glasses' OLED display and audio output.

[0965] Examples and prompts

[0966] Specific examples

[0967] 1. When visiting the cafe: The user enters the cafe, borrows and puts on the AI ​​glasses. "Hello, welcome to Health Cafe. What would you like to order today?"

[0968] 2. When selecting a meal: The user scans the menu with their camera and the analysis results are displayed. "This chicken salad is 120 calories. It's recommended considering today's calorie intake."

[0969] Prompt Sentence Examples

[0970] "The camera on the AI ​​glasses takes pictures of meals, sends them to the cloud, and analyzes their nutritional value. Based on the analysis results, the system will suggest the optimal meal plan for the user based on their health condition."

[0971] With specific technical configurations and operating procedures, users can manage their health even when eating out, and can also receive comprehensive support that takes into account their emotional state.

[0972] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0973] Step 1:

[0974] A user visits a cafe, borrows the AI ​​glasses, and puts them on. Putting on the AI ​​glasses is the input, and the system becomes ready for use.

[0975] Step 2:

[0976] The user scans the menu by pointing it at the camera on the AI ​​glasses. The menu image is then input. The camera captures the image of the menu, which is then acquired by the device. The acquired image data is then sent to the next processing step.

[0977] Step 3:

[0978] The device sends the captured image data to the cloud server. The input is the image data of the menu, and the output is the image data sent to the cloud server. This is done using cloud communication.

[0979] Step 4:

[0980] The server uses an image recognition algorithm to analyze the menu image data sent. The input is the menu image data, and the image recognition algorithm extracts image features to identify the food items. The output is a list of identified foods.

[0981] Step 5:

[0982] The server analyzes the nutritional value information based on the food list. The input is the identified food list, and the server obtains the nutritional value information by referencing the nutrient database. The output is the analysis result including the nutritional value information.

[0983] Step 6:

[0984] The server stores the analysis results in a database. The input is the analysis results including nutritional information, and storing this data in storage allows for future reference. The output is an updated database.

[0985] Step 7:

[0986] The server generates dietary advice and sends it to the device via the cloud. The input is nutritional information and the user's health data, which the machine learning model analyzes to generate advice. The output is the generated advice.

[0987] Step 8:

[0988] The device displays the generated advice on the AI ​​glasses display or notifies the user by voice. The input is advice data sent from the server, which is notified to the user. The output is a visual or audio notification to the user.

[0989] Step 9:

[0990] The user selects a meal and then begins to move. The built-in sensor in the AI ​​glasses measures the user's steps. The input is the user's movement data, which the sensor measures to generate step count data. The output is the measured step count data.

[0991] Step 10:

[0992] The device sends step count data to the cloud server. The input is the measured step count data, and the output is the step count data sent to the cloud server.

[0993] Step 11:

[0994] The server calculates the calories burned based on the step count data. The input is the step count data, and the server calculates the calories burned as a data operation. The output is the calculated calories burned.

[0995] Step 12:

[0996] The server compares the calories ingested and burned, and the emotion engine analyzes the user's emotional state. The inputs are calories ingested, calories burned, and facial expression data, which are analyzed by the machine learning model and the Emotion Engine. The output is comprehensive health and mental health advice.

[0997] Step 13:

[0998] The server sends the generated advice to the terminal, which then notifies the user. The input is the generated advice, and the output is the advice notified to the user.

[0999] Through these processing steps, the system of the present invention provides comprehensive health management support that integrates nutritional analysis of food, step counting, and emotional state recognition.

[1000] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1001] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1002] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1003] [Third embodiment]

[1004] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1005] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1006] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1007] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1008] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1009] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1010] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1011] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1012] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1013] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1014] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1015] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1016] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases. The system aims to provide users with appropriate advice by recording and analyzing dietary information, measuring steps, and managing the balance between calorie intake and calorie expenditure.

[1017] System configuration

[1018] The system of the present invention includes the following major components:

[1019] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[1020] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[1021] 3. Cloud Server - A remote server for storing analysis results and performing necessary calculations and data storage.

[1022] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[1023] 5. Notification means - includes a display and audio output device to notify the user of advice.

[1024] Specific explanation of program processing

[1025] Food recording and analysis

[1026] 1. The user prepares breakfast on the table, and the camera in the AI ​​glasses takes pictures of the meal.

[1027] 2. The device (AI glasses) sends the captured image data to the cloud server.

[1028] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[1029] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[1030] Step counting

[1031] 1. When the user commutes to work or exercises, the AI ​​glasses measure the number of steps taken using the built-in sensor.

[1032] 2. The device sends step count data to the cloud server at regular intervals.

[1033] 3. The server analyzes the received step count data and calculates the calories burned.

[1034] Data integration and advice generation

[1035] 1. The server compares the daily calorie intake with the calorie expenditure.

[1036] 2. The server generates advice based on the user's health status, for example, to improve diet and exercise.

[1037] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[1038] Specific examples

[1039] Example 1: Breakfast recording and analysis

[1040] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[1041] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[1042] The server stores this information and updates the user database.

[1043] Example 2: Measuring steps during commute

[1044] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[1045] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[1046] Example 3: Providing advice

[1047] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[1048] The device will notify the user of this advice and display it on the AI ​​glasses' display.

[1049] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[1050] The processing flow will be explained below.

[1051] Step 1:

[1052] The user prepares a meal, for example, bread and orange juice for breakfast.

[1053] Step 2:

[1054] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[1055] Step 3:

[1056] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[1057] Step 4:

[1058] The server adds the received image data to a processing queue.

[1059] Step 5:

[1060] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[1061] Step 6:

[1062] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[1063] Step 7:

[1064] The server organizes and stores the calculated nutrient data in a database for each user.

[1065] Step 8:

[1066] The user begins their daily routine or exercise, for example, walking to the station for work.

[1067] Step 9:

[1068] The device's built-in sensor measures the user's step count data.

[1069] Step 10:

[1070] The device sends step count data to the server at regular intervals.

[1071] Step 11:

[1072] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[1073] Step 12:

[1074] The server aggregates daily calorie, salt, fat, alcohol, and step count data.

[1075] Step 13:

[1076] The server uses machine learning models to analyze the user's health status and generate data-based advice, such as "Today's breakfast was too calorie-dense, so choose a vegetable-based lunch."

[1077] Step 14:

[1078] The server sends the advice to the notification system.

[1079] Step 15:

[1080] The terminal receives the advice from the server.

[1081] Step 16:

[1082] The device will then display the received advice in the user's field of vision, for example, "Eat mostly vegetables for lunch" on the Glasses' display.

[1083] Step 17:

[1084] The user follows the advice and practices a healthy diet.

[1085] Example 1

[1086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1087] In conventional systems for preventing and managing lifestyle-related diseases, recording meals and counting steps are often done manually, placing a heavy burden on users. Furthermore, manually calculating calories consumed and calories burned can lead to a decrease in accuracy. Furthermore, it is difficult to provide appropriate advice tailored to each individual user's health condition.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1089] In this invention, the server includes means for a user to prepare a meal and record an image of the meal with a camera, means for the terminal to send the image data to a cloud server, means for the cloud server to analyze the meal contents using an image recognition algorithm, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a database, means for using a built-in sensor to count steps, means for sending the step count data to the cloud server, means for calculating calories burned on the cloud server, means for comparing calories consumed with calories burned to generate advice, and means for notifying the user of the generated advice, thereby enabling users to efficiently and accurately manage their own dietary and exercise habits.

[1090] A "user" is a person who uses the system to manage their own eating and exercise habits and prevent and manage lifestyle-related diseases.

[1091] "Camera" refers to a photographic device used to record images of a meal.

[1092] A "terminal" is a device used to send captured image data and step count data to a cloud server.

[1093] A "cloud server" is an infrastructure for storing and analyzing data on a remote server.

[1094] An "image recognition algorithm" is a technology that analyzes food contents from captured images and identifies the type and amount of food.

[1095] A "database" is a digital recording medium for storing analysis results and nutrient information.

[1096] "Built-in sensor" refers to a sensor device built into a device to measure steps.

[1097] "Calories burned" refers to the amount of energy consumed calculated based on step count data.

[1098] "Advice" refers to dietary and exercise recommendations for the user that are generated by comparing calories ingested with calories burned.

[1099] The present invention relates to a system that supports the prevention and management of lifestyle-related diseases. This system aims to provide users with appropriate advice by recording and analyzing dietary content, measuring the number of steps taken, and managing the balance between calorie intake and calorie expenditure.

[1100] System configuration

[1101] The system of the present invention includes the following major components:

[1102] 1. Camera - A photographic device used to record images of meals.

[1103] 2. Device - A device that sends captured image data and step count data to a cloud server. It has an internal sensor and also measures the number of steps.

[1104] 3. Cloud Server - A remote server that stores and analyzes data.

[1105] 4. Image recognition algorithms - These are technologies used to analyze food content and identify the types and quantities of food. For example, software such as TensorFlow and OpenCV are examples of such algorithms.

[1106] 5. Database - A digital recording medium for storing analysis results and nutritional information.

[1107] 6. Built-in sensors - Includes an accelerometer and gyro sensor for counting steps.

[1108] 7. Notification means - A display or audio output device for notifying the user of advice.

[1109] Specific processing of the program

[1110] Food recording and analysis

[1111] 1. The user prepares breakfast and takes a picture of the meal using the AI ​​glasses. For example, if the user prepares bread and milk, they press the camera button to take a picture.

[1112] 2. The device sends the captured image data to a cloud server using Wi-Fi or mobile data.

[1113] 3. The server receives the image data and analyzes the meal contents using an image recognition algorithm (e.g., YOLO or ResNet). It identifies bread and milk and calculates the nutritional value of each.

[1114] 4. The server stores the analysis results in the user's database, specifically recording information such as the bread's calories, carbohydrates, and fat content.

[1115] Step counting

[1116] 1. The user begins a daily activity, for example, leaving home for their morning commute.

[1117] 2. The device uses a built-in sensor to count steps and detects walking movements in real time.

[1118] 3. The device sends the measured data at regular intervals (for example, every 10 minutes) to the cloud server.

[1119] 4. The server analyzes the step count data and calculates the calories burned. For example, if 3,000 steps result in 150 calories burned, the server records this data in the user's database.

[1120] Data integration and advice generation

[1121] 1. The server aggregates the daily calorie intake and calorie expenditure data. For example, it aggregates data showing that you ate 500 calories at breakfast and burned 150 calories by walking.

[1122] 2. The server generates diet and exercise advice based on the user's individual profile (e.g., age, gender, weight, activity level, etc.). If the calorie intake is high, advice such as "eat a lighter lunch" or "exercise more" is generated.

[1123] 3. The device receives the advice generated by the server and notifies the user, either by displaying it on the AI ​​glasses display or by voice notification.

[1124] Examples of concrete examples and prompts

[1125] Specific examples

[1126] The user prepares yogurt and a banana for breakfast, and the AI ​​glasses take an image of the meal and send the data to a cloud server.

[1127] The server analyzes the image and calculates the nutritional value of yogurt (100 calories) and a banana (90 calories).

[1128] The server stores this information and updates the user database.

[1129] Prompt Sentence Examples

[1130] "How many calories did you have at dinner last night?"

[1131] "How many calories are in my breakfast today?"

[1132] "Based on your step count data this week, do you need additional exercise?"

[1133] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[1134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1135] Step 1:

[1136] The user prepares breakfast and uses the AI ​​glasses to take pictures of the meal.

[1137] Input: User prepares meal and presses camera button.

[1138] Output: The camera generates image data of the meal.

[1139] Specific operation: When the user presses the camera button, the camera on the AI ​​glasses takes a photo of the meal and the image data is saved on the device.

[1140] Step 2:

[1141] The device sends the captured image data to a cloud server.

[1142] Input: The captured image data is saved on the device.

[1143] Output: The image data is sent to the cloud server.

[1144] Specific operation: The device sends image data to the cloud server using Wi-Fi or mobile data.

[1145] Step 3:

[1146] The server applies image recognition algorithms to analyze the meal contents.

[1147] Input: Image data stored on a cloud server.

[1148] Output: Analysis results regarding food type, quantity, and nutritional value.

[1149] Specific operation: The server analyzes the image using an image recognition algorithm (e.g., TensorFlow or OpenCV), identifies the type and quantity of food, and calculates nutritional value.

[1150] Step 4:

[1151] The server stores the analysis results in a database.

[1152] Input: Analysis results (food type, amount, and nutritional value information).

[1153] Output: Analysis results stored in a database.

[1154] Specific operation: The server records the analysis results in the user's database, and nutritional information (calories, carbohydrates, fats, etc.) is saved.

[1155] Step 5:

[1156] The user begins their daily activities.

[1157] Input: The user starts walking.

[1158] Output: Real-time step count data is generated by sensors in the AI ​​glasses.

[1159] Specific operation: The user starts commuting or exercising, and the AI ​​glasses detect walking movements and count the number of steps.

[1160] Step 6:

[1161] The device periodically transmits step count data to a cloud server.

[1162] Input: Measured step count data.

[1163] Output: Step count data sent to cloud server.

[1164] Specific operation: The device sends step count data to the cloud server at regular intervals (for example, every 10 minutes).

[1165] Step 7:

[1166] The server analyzes the received step count data and calculates the calories burned.

[1167] Input: Step count data stored on a cloud server.

[1168] Output: Calculated calories burned.

[1169] Specific operation: The server analyzes the step count data, calculates the calories burned, and records them in a database.

[1170] Step 8:

[1171] The server aggregates the daily calorie intake and expenditure data.

[1172] Input: Stored calorie intake and calorie expenditure data.

[1173] Output: Consolidated calorie data.

[1174] Specific operation: The server integrates the calorie intake and calorie expenditure information obtained from the database and calculates the daily calorie balance.

[1175] Step 9:

[1176] The server generates advice based on the user's profile.

[1177] Input: Consolidated calorie data and user profile information (age, gender, weight, activity level).

[1178] Output: The generated advice.

[1179] Specific operation: The server uses the generated AI model to generate advice (recommended dietary restrictions and exercise) based on the user's health condition.

[1180] Step 10:

[1181] The terminal notifies the user of the generated advice.

[1182] Input: Advice data from the server.

[1183] Output: Advice given to the user.

[1184] Specific operation: The device notifies the user of the advice using a display or voice output.

[1185] (Application example 1)

[1186] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1187] Modern brick-and-mortar stores lack an effective system to support customers in making healthy dietary choices. They also need a way to provide personalized advice in real time based on each customer's health status. This makes it difficult for customers to prevent and manage lifestyle-related diseases without stress.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1189] In this invention, the server includes means for recording images of meals using a camera, means for analyzing meal contents using image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a cloud server, means for counting steps, means for transmitting step count data to the cloud server, means for calculating calories burned on the cloud server, means for generating advice by comparing calories consumed with calories burned, means for notifying the user of the generated advice, means for analyzing food calories and nutrients in real time and suggesting nutritionally balanced meals to support customers' healthy dietary choices in the physical store, and means for generating and presenting personalized health advice based on each customer's health condition, thereby enabling customers to receive information supporting healthy lifestyle habits in real time in the physical store.

[1190] A "camera" is a photographic device for recording images of meals.

[1191] "Image recognition" is a technology that analyzes recorded meal contents.

[1192] A "cloud server" is a remote server that stores data and performs calculations such as calorie consumption.

[1193] "Step counting" is a means for counting the number of steps.

[1194] "Calculating calories burned" is a method for calculating calories burned from step count data.

[1195] "Personalized health advice" is a means of generating specific recommendations based on each customer's health status.

[1196] "Nutritionally balanced meal suggestions" is a method of analyzing the calories and nutrients of food and supporting healthy dietary choices.

[1197] The present invention provides a system for supporting health management of customers in brick-and-mortar stores, and in particular provides a specific configuration for analyzing dietary content and providing advice. This system includes the following main hardware and software:

[1198] System configuration

[1199] 1. Camera

[1200] The camera is built into the AI ​​glasses and is a device for capturing images of the food. The camera takes pictures of the dishes selected by the customer in real time and sends the image data to the system.

[1201] 2. Image Recognition Algorithm

[1202] The image recognition algorithm is a technology that analyzes photographed images of food and identifies the type and amount of food. This algorithm is implemented using machine learning libraries such as TensorFlow and Keras.

[1203] 3. Cloud Server

[1204] A cloud server is a remote server that stores analysis results and manages the necessary calculations and data. AWS (Amazon Web Services) and Google Cloud Platform are often used.

[1205] 4. Built-in sensor

[1206] The built-in sensor is a device that measures the number of steps taken by customers, and the step count data is sent to a cloud server via Bluetooth or other means.

[1207] 5. Means of notification

[1208] The notification means includes a display and an audio output device for notifying the user of the advice, which allows healthy eating suggestions and advice to be provided to customers in real time within the physical store.

[1209] Program processing explanation

[1210] Camera Function:

[1211] A camera is built into the AI ​​glasses and takes a picture of the customer's dish selection, which is then instantly sent to an image recognition algorithm.

[1212] Image Recognition:

[1213] Image recognition algorithms analyze captured images to identify the type and quantity of food, using a pre-trained neural network model to identify nutritional and calorie information for each food item.

[1214] Cloud server processing:

[1215] The detected meal content data is sent to a cloud server and stored there, where the cloud server also calculates the calories burned and compares them with the user's calorie intake.

[1216] Advice Generation:

[1217] The cloud server generates personalized health advice based on the user's health status, such as nutritionally balanced meal suggestions and exercise instructions.

[1218] notification:

[1219] The generated advice is sent to the user in real time either as a display on the AI ​​glasses or as a voice prompt.

[1220] Specific examples

[1221] Example 1: Use in a restaurant

[1222] When a user orders a steak at a restaurant and takes a picture of the meal with the AI ​​glasses, the system identifies it as a steak and instantly obtains calorie information, which can be used as a guide for the next meal or exercise.

[1223] Example prompt: "From this image, identify the type of food and its calories, and analyze whether it is nutritionally balanced."

[1224] Example 2: Use in a cafe

[1225] When a user orders a salad and soup at a cafe, the AI ​​glasses recognize this and suggest healthy eating choices, informing them of any food or exercise they should avoid later.

[1226] Example prompt: "Calculate the nutritional value of a salad (200 calories) and a soup (150 calories) and store that information in a database."

[1227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1228] Step 1:

[1229] The user takes a photo of the dish they have selected with a camera, which takes image data of the dish as input and sends the image to an image recognition algorithm.

[1230] Step 2:

[1231] The device runs an image recognition algorithm to analyze the input image data. Specifically, after preprocessing the image, it inputs it into a neural network model to identify the type and quantity of food. The output is the type of food and its nutritional information.

[1232] Step 3:

[1233] The server receives the analysis results and retrieves food types and nutrient information from the nutrition database. It takes food types as input, executes a database query, and obtains nutrient data as output.

[1234] Step 4:

[1235] The server saves the nutrient data on the cloud server. It receives the analysis results and nutrient data as input and stores them in the database. The output confirms that the data is persisted.

[1236] Step 5:

[1237] When a user walks, the device's built-in sensor measures the number of steps taken. The device receives step count data in real time as input and sends the data to a cloud server.

[1238] Step 6:

[1239] The server calculates calories burned based on the step count data received. It receives step count data as input and calculates calories burned using a predefined algorithm. It obtains the calorie burned value as output.

[1240] Step 7:

[1241] The server compares daily calorie intake and expenditure and generates personalized advice. It receives calorie intake and expenditure data as input and generates advice using a health management algorithm. The output is advice text.

[1242] Step 8:

[1243] The device notifies the user of the generated advice. It receives the advice text as input and displays it on the AI ​​glasses display or notifies the user by voice.

[1244] Step 9:

[1245] The server analyzes food calories and nutrients in real time and suggests nutritionally balanced meals to support healthy meal choices for users in physical stores. It receives data on in-store dishes as input and analyzes it in real time. It outputs healthy meal suggestions to be presented to customers.

[1246] Step 10:

[1247] The server generates and presents personalized health advice based on each customer's health status. It receives each customer's health data as input, analyzes it using a machine learning model, and obtains personalized advice as output.

[1248] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1249] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[1250] System configuration

[1251] The system of the present invention includes the following major components:

[1252] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[1253] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[1254] 3. Cloud Server - A remote server that stores analysis results and emotion data, and performs necessary calculations and data storage.

[1255] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[1256] 5. Emotion Engine - Contains technology to identify the user's emotional state.

[1257] 6. Notification means - includes a display and audio output device to notify the user of advice.

[1258] Specific explanation of program processing

[1259] Food recording and analysis

[1260] 1. The user prepares a meal, and the camera in the AI ​​glasses takes a picture of the meal.

[1261] 2. The device (AI glasses) sends the captured image data to the cloud server.

[1262] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[1263] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[1264] Step counting

[1265] 1. The user begins their daily routine or exercise, for example, walking to the station for work.

[1266] 2. The device's built-in sensor measures the user's step data.

[1267] 3. The device sends step count data to the cloud server at regular intervals.

[1268] 4. The server analyzes the received step count data and calculates the calories burned.

[1269] Emotion recognition and analysis

[1270] 1. The emotion engine identifies the user's emotional state. For example, it determines whether the user is feeling stressed based on facial expressions and voice data.

[1271] 2. The server sends the emotional data to the cloud, where it integrates and analyzes past emotional data and health data.

[1272] Data integration and advice generation

[1273] 1. The server compares and integrates the daily calorie intake, calorie expenditure, and emotional data.

[1274] 2. The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[1275] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[1276] Specific examples

[1277] Example 1: Breakfast recording and analysis

[1278] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[1279] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[1280] The server stores this information and updates the user database.

[1281] Example 2: Measuring steps during commute

[1282] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[1283] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[1284] Example 3: Providing advice

[1285] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[1286] The device will notify the user of this advice and display it on the AI ​​glasses' display. If the user's emotional state is high, the device will also suggest taking a short break to relax.

[1287] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[1288] The processing flow will be explained below.

[1289] Step 1:

[1290] The user prepares a meal, for example, bread and orange juice for breakfast.

[1291] Step 2:

[1292] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[1293] Step 3:

[1294] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[1295] Step 4:

[1296] The server adds the received image data to a processing queue.

[1297] Step 5:

[1298] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[1299] Step 6:

[1300] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[1301] Step 7:

[1302] The server organizes and stores the calculated nutrient data in a database for each user.

[1303] Step 8:

[1304] The user begins their daily routine or exercise, for example, walking to the station for work.

[1305] Step 9:

[1306] The device's built-in sensor measures the user's step count data.

[1307] Step 10:

[1308] The device sends step count data to the server at regular intervals.

[1309] Step 11:

[1310] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[1311] Step 12:

[1312] The emotion engine recognizes the user's facial expressions and voice to identify their emotional state, for example, determining whether they are stressed.

[1313] Step 13:

[1314] The device transmits the emotion data to the cloud server.

[1315] Step 14:

[1316] The server receives the emotional data and integrates and analyzes the past emotional data and health data.

[1317] Step 15:

[1318] The server compares and integrates daily calorie intake, calorie expenditure, and emotional data.

[1319] Step 16:

[1320] The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[1321] Step 17:

[1322] The server sends the advice to the notification system.

[1323] Step 18:

[1324] The terminal receives the advice from the server.

[1325] Step 19:

[1326] The device will then display the received advice in the user's field of vision. For example, the Glasses' display will say, "Eat a vegetable-based lunch." If the user is feeling stressed, the device will suggest, "Take a short break to relax."

[1327] Step 20:

[1328] Users follow advice to implement healthy eating, exercise, and stress management.

[1329] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[1330] Example 2

[1331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1332] Achieving a balance between daily calorie intake and calorie expenditure is important for preventing and managing lifestyle-related diseases, but it is difficult for users to consistently maintain this balance in their daily lives. Furthermore, a user's emotional state is also an important factor in health management, and it is widely known that stress, in particular, has a negative impact on health. However, current systems only record dietary habits and measure steps, and are unable to provide comprehensive health management that also takes emotional state into account. Therefore, there is a need for a system that can comprehensively manage a user's health and emotional state and provide individually optimized advice.

[1333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1334] In this invention, the server includes means for recording images of meals, means for analyzing meal contents by image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results on a remote server, means for counting steps, means for transmitting step count data to the remote server, means for calculating calories burned on the remote server, means for comparing calories consumed with calories burned to generate advice, means for identifying the user's emotional state and transmitting the identified emotional data to the remote server, means for integrating and analyzing the user's emotional data and health data, and means for notifying the user of the generated advice. This enables the user to comprehensively manage their health throughout the day and receive individually optimized advice that also takes into account their emotional state.

[1335] The "means for recording images of meals" refers to a device that uses a photographing device such as a camera to record images of meals consumed by the user as digital data.

[1336] "Means for analyzing meal contents using image recognition" refers to technologies and algorithms that analyze image data of photographed meals and identify the foods contained in the meal and their amounts.

[1337] The "means for calculating the nutrients ingested based on the analysis results" refers to a computing device or program that calculates the calories ingested and various nutrients from the dietary content data obtained by image recognition.

[1338] The "means for saving the analysis results on a remote server" is a function for saving the calculated nutrient data and other analysis results on a remote server via the Internet.

[1339] The "means for measuring the number of steps" refers to a device or sensor for measuring the walking activity of a user, and includes a pedometer.

[1340] The "means for transmitting step count data to a remote server" refers to a communication device or function for transmitting the measured step count data to a remote server via the Internet.

[1341] The "means for calculating calories burned on a remote server" refers to an algorithm or program that calculates calories burned based on the step count data received on the remote server.

[1342] The "means for generating advice by comparing calorie intake with calorie expenditure" refers to a device or software that compares the balance between calorie intake and calorie expenditure and automatically generates advice for health management.

[1343] "Means for identifying the user's emotional state and transmitting the identified emotional data to a remote server" refers to technology or devices for identifying the user's emotions through facial expression recognition, voice analysis, etc., and transmitting that data to a remote server.

[1344] "Means for integrating and analyzing a user's emotional data and health data" refers to an algorithm or program that integrates and analyzes a user's emotional data and health data (calories ingested, calories burned, etc.) to assess their overall health status.

[1345] The "means for notifying the user of the generated advice" refers to a notification device such as a display or audio output device for conveying the generated health management advice to the user.

[1346] This invention relates to an AI system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[1347] Hardware and software used

[1348] Camera: A device built into the AI ​​glasses that takes pictures of meals.

[1349] Built-in sensor: A sensor for measuring steps.

[1350] Image recognition algorithm: Technologies such as TensorFlow and OpenCV are used for image recognition.

[1351] Emotion Engine: This engine identifies emotional states through facial expression recognition and voice analysis, using machine learning models with Pytorch and Keras.

[1352] Cloud Server: The remote server is responsible for data analysis and storage, where it calculates calories ingested and burned and generates advice.

[1353] Notification means: Includes a display and an audio output device for notifying the user of the generated advice.

[1354] Explanation of program processing

[1355] Food recording and analysis

[1356] When a user prepares a meal, the camera in the AI ​​glasses automatically captures an image of the meal. The device then sends the captured image data to a cloud server via Wi-Fi or Bluetooth. The server then analyzes the received image data and runs an image recognition algorithm using TensorFlow or OpenCV to identify the food. Based on the analysis results, the nutrient intake is calculated and stored in a database.

[1357] Step counting

[1358] When a user starts moving, the built-in sensor measures the number of steps. This data is sent to a cloud server at regular intervals, and the server calculates the calories burned. Specifically, a calculation algorithm is used based on the step count data to calculate the calories burned.

[1359] Emotion recognition and analysis

[1360] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. This data is then used to identify their emotional state using facial and voice recognition algorithms. The emotion data is then sent to a remote server, which then combines it with past health data for analysis. Machine learning models using Pytorch and Keras are used for the analysis.

[1361] Data integration and advice generation

[1362] The server analyzes calorie intake, calorie expenditure, and emotional data, and then comprehensively evaluates the user's health and emotional state to generate appropriate advice. The advice is displayed in text format on the screen, and voice notification is also available if necessary.

[1363] Specific examples

[1364] For example, if a user has toast and black coffee for breakfast, the AI ​​Glasses will take a photo of this and send the image data to a cloud server. The server will analyze the image, calculate the nutritional value of each food item, and save the results. If the user then takes 2,000 steps on their way to work, the built-in sensor will measure the number of steps and send it to the cloud server. Based on the data received by the server, it will determine that the user has burned approximately 100 calories in 2,000 steps.

[1365] Furthermore, if the emotion engine analyzes the user's facial expressions and voice and determines that they are under high stress, it will use this data to comprehensively evaluate their health condition. Based on this, it will generate advice such as "Today's breakfast was too high in calories, so eat a lighter lunch and create a relaxing environment," and display it on the screen.

[1366] Prompt Sentence Examples

[1367] Here are some example prompts to input to a generative AI model:

[1368] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[1369] In this way, the system of the present invention allows users to manage their diet and exercise habits without straining themselves, and provides appropriate advice according to their emotional state.

[1370] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1371] Specific flow of program processing

[1372] Food recording and analysis

[1373] Step 1:

[1374] The user prepares a meal. At this time, the camera in the AI ​​glasses automatically captures images of the meal. The input is the scene of the user eating, and the output is image data of the meal.

[1375] Step 2:

[1376] The device uses Wi-Fi or Bluetooth to send captured image data to a cloud server. The input is the image data captured by the camera, and the output is the image data sent to the cloud server.

[1377] Step 3:

[1378] The server receives the image data and runs an image recognition algorithm (e.g., TensorFlow or OpenCV). The input is the image data sent to the cloud server, and the output is the analyzed food data.

[1379] Step 4:

[1380] The server identifies foods in the image and calculates the nutritional value (calories, protein, fat, etc.) of each food. The input is the analyzed food data, and the output is the calculated nutritional value information.

[1381] Step 5:

[1382] The server stores the analysis results in a database. The input is the calculated nutritional value information, and the output is the nutritional data stored in the database.

[1383] Step counting

[1384] Step 1:

[1385] The user starts daily activities or exercise. For example, walking to the station for work. The input is the user's movement activity, and the output is walking data.

[1386] Step 2:

[1387] The device's built-in sensor measures walking data in real time. The input is the user's physical steps, and the output is step count data.

[1388] Step 3:

[1389] The device periodically (e.g., every hour) sends the cumulative step count data to the cloud server. The input is the step count data accumulated on the device, and the output is the data sent to the cloud server.

[1390] Step 4:

[1391] The server analyzes the received step count data and calculates the calories burned. The input is the step count data sent to the cloud server, and the output is the calculated calories burned.

[1392] Emotion recognition and analysis

[1393] Step 1:

[1394] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. The input is the user's facial expressions and voice data, and the output is the captured emotion data.

[1395] Step 2:

[1396] The device analyzes these emotion data in real time and sends the identified emotion data to the cloud server. The input is the captured emotion data, and the output is the emotion data sent to the cloud server.

[1397] Step 3:

[1398] The server receives the emotional data and integrates and analyzes the past emotional data and health data. The input is the emotional data sent to the cloud server, and the output is the integrated analysis result.

[1399] Data integration and advice generation

[1400] Step 1:

[1401] The server integrates and analyzes daily calorie intake, calorie expenditure, and emotional data. The input is calorie intake, calorie expenditure, and emotional data, and the output is integrated health evaluation data.

[1402] Step 2:

[1403] The server uses machine learning models (e.g., Pytorch or Keras) to analyze the user's health and emotional state and generate advice. The input is the integrated health assessment data, and the output is the generated advice.

[1404] Step 3:

[1405] The terminal notifies the user of the generated advice. The input is the generated advice, and the output is text displayed on the screen or a voice notification.

[1406] Prompt Sentence Examples

[1407] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[1408] This prompt enables the generative AI model to generate advice based on the user's health and emotional state.

[1409] (Application example 2)

[1410] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1411] Health management is becoming increasingly important in modern society. While there are a wide variety of technologies for preventing and managing lifestyle-related diseases, there is still a lack of systems that provide personalized advice based on individual health conditions. In particular, there is a need for comprehensive support that promotes healthy choices and also includes mental health when choosing food when eating out or at a cafe. However, current systems lack a system that integrates multiple functions, such as analyzing food content, managing the balance of calorie intake and expenditure, and recognizing emotional states. Therefore, new technologies that can achieve these functions are needed.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is a system including: means for recording images of meals with a camera; means for analyzing meal contents by image recognition; means for calculating nutrient intake based on the analysis results; means for saving the analysis results on a cloud server; means for counting steps; means for transmitting step count data to the cloud server; means for calculating calories burned on the cloud server; means for comparing calories consumed with calories burned to generate advice; and means for notifying the user of the generated advice. This makes it possible to analyze restaurant menus when dining out, provide nutritional information and recommended meal plans based on individual health conditions, and provide comprehensive mental health support that also takes emotional states into account.

[1413] A "camera" is a device for recording images.

[1414] "Means" refers to the methods and techniques used to achieve a goal.

[1415] "Image recognition" is a technology that extracts, identifies, and classifies objects and features from captured images.

[1416] "Analysis results" refers to the data obtained after processing such as image recognition.

[1417] "Intaken nutrients" refers to the nutrients taken into the body through meals and food.

[1418] A "cloud server" is a device for storing data and performing computations on a remote server accessible via the Internet.

[1419] "Steps" refers to the number of times your feet move when walking a certain distance.

[1420] "Calories consumed" refers to the energy consumed through physical activity and basal metabolism.

[1421] "Advice" refers to instructions or guidance given to another person based on particular knowledge or information.

[1422] "User" refers to a person who uses this system.

[1423] "Store menu" refers to a list of foods and drinks offered at a restaurant or other establishment.

[1424] "Nutritional information" refers to data about the nutritional components and amounts contained in food.

[1425] A "meal plan" refers to a combination of meals designed to achieve a specific health goal.

[1426] "Emotional state" refers to an individual's temporary psychological or mental state.

[1427] "Mental health" refers to an individual's mental and psychological well-being.

[1428] "Comprehensive support" refers to a wide range of assistance and support that encompasses multiple aspects.

[1429] This invention relates to the "AI Glass Guide" system, which is used in health-conscious cafes and other places, and aims to provide optimal health advice by simultaneously analyzing dietary content and emotional state.

[1430] System configuration

[1431] The system of the present invention uses the following hardware and software.

[1432] Hardware:

[1433] 1. AI Glasses - Includes a camera (e.g., Sony IMX camera), a display (OLED display), a microphone, and a built-in sensor (MPU6050 sensor for counting steps).

[1434] 2. Cloud Server - A server accessible via the Internet.

[1435] software:

[1436] 1. Image Recognition Algorithms - Custom deep learning models using TensorFlow and PyTorch.

[1437] 2. Emotion recognition engine - A proprietary library called EmotionEngine.

[1438] 3. Cloud communication - Uses APIs from AWS and Google Cloud Platform.

[1439] 4. Health Advisor - A program that generates advice based on health and emotional data.

[1440] The role of each hardware and software

[1441] 1. A user visits a cafe, puts on the AI ​​glasses, and uses the camera in the glasses to scan the menu.

[1442] 2. The device (AI glasses) takes a picture of the menu and sends it to the cloud server.

[1443] 3. The server uses image recognition algorithms to identify the foods on the menu and parse the nutritional information.

[1444] 4. The server stores the analysis results in a database.

[1445] 5. When the user selects a meal, the AI ​​glasses will generate advice about it and display it on the screen or notify them via voice.

[1446] 6. The device's built-in sensor measures the user's steps and sends them to the cloud server.

[1447] 7. The server calculates the calories burned based on the step count data and compares this with the calories ingested to generate optimal advice.

[1448] 8. The emotion engine analyzes the user's emotional state and provides information about their mental health.

[1449] Program processing explanation

[1450] 1. Image processing: The image data of the menu captured by the AI ​​glasses' camera is preprocessed using OpenCV, and then an image recognition algorithm trained in TensorFlow or PyTorch is run to identify the food.

[1451] 2. Cloud communication: Analysis results and meal content data are sent in real time to a cloud server via the AWS or Google Cloud Platform API.

[1452] 3. Data Analysis: The cloud server uses EmotionEngine to analyze the user's facial expression data to identify their emotional state, and then integrates it with their health data to generate advice.

[1453] 4. Notification: The generated advice is notified to the user through the AI ​​glasses' OLED display and audio output.

[1454] Examples and prompts

[1455] Specific examples

[1456] 1. When visiting the cafe: The user enters the cafe, borrows and puts on the AI ​​glasses. "Hello, welcome to Health Cafe. What would you like to order today?"

[1457] 2. When selecting a meal: The user scans the menu with their camera and the analysis results are displayed. "This chicken salad is 120 calories. It's recommended considering today's calorie intake."

[1458] Prompt Sentence Examples

[1459] "The camera on the AI ​​glasses takes pictures of meals, sends them to the cloud, and analyzes their nutritional value. Based on the analysis results, the system will suggest the optimal meal plan for the user based on their health condition."

[1460] With specific technical configurations and operating procedures, users can manage their health even when eating out, and can also receive comprehensive support that takes into account their emotional state.

[1461] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1462] Step 1:

[1463] A user visits a cafe, borrows the AI ​​glasses, and puts them on. Putting on the AI ​​glasses is the input, and the system becomes ready for use.

[1464] Step 2:

[1465] The user scans the menu by pointing it at the camera on the AI ​​glasses. The menu image is then input. The camera captures the image of the menu, which is then acquired by the device. The acquired image data is then sent to the next processing step.

[1466] Step 3:

[1467] The device sends the captured image data to the cloud server. The input is the image data of the menu, and the output is the image data sent to the cloud server. This is done using cloud communication.

[1468] Step 4:

[1469] The server uses an image recognition algorithm to analyze the menu image data sent. The input is the menu image data, and the image recognition algorithm extracts image features to identify the food items. The output is a list of identified foods.

[1470] Step 5:

[1471] The server analyzes the nutritional value information based on the food list. The input is the identified food list, and the server obtains the nutritional value information by referencing the nutrient database. The output is the analysis result including the nutritional value information.

[1472] Step 6:

[1473] The server stores the analysis results in a database. The input is the analysis results including nutritional information, and storing this data in storage allows for future reference. The output is an updated database.

[1474] Step 7:

[1475] The server generates dietary advice and sends it to the device via the cloud. The input is nutritional information and the user's health data, which the machine learning model analyzes to generate advice. The output is the generated advice.

[1476] Step 8:

[1477] The device displays the generated advice on the AI ​​glasses display or notifies the user by voice. The input is advice data sent from the server, which is notified to the user. The output is a visual or audio notification to the user.

[1478] Step 9:

[1479] The user selects a meal and then begins to move. The built-in sensor in the AI ​​glasses measures the user's steps. The input is the user's movement data, which the sensor measures to generate step count data. The output is the measured step count data.

[1480] Step 10:

[1481] The device sends step count data to the cloud server. The input is the measured step count data, and the output is the step count data sent to the cloud server.

[1482] Step 11:

[1483] The server calculates the calories burned based on the step count data. The input is the step count data, and the server calculates the calories burned as a data operation. The output is the calculated calories burned.

[1484] Step 12:

[1485] The server compares the calories ingested and burned, and the emotion engine analyzes the user's emotional state. The inputs are calories ingested, calories burned, and facial expression data, which are analyzed by the machine learning model and the Emotion Engine. The output is comprehensive health and mental health advice.

[1486] Step 13:

[1487] The server sends the generated advice to the terminal, which then notifies the user. The input is the generated advice, and the output is the advice notified to the user.

[1488] Through these processing steps, the system of the present invention provides comprehensive health management support that integrates nutritional analysis of food, step counting, and emotional state recognition.

[1489] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1490] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1491] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1492] [Fourth embodiment]

[1493] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1494] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1495] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1496] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1497] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1498] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1499] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1500] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1501] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1502] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1503] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1504] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1505] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1506] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases. The system aims to provide users with appropriate advice by recording and analyzing dietary information, measuring steps, and managing the balance between calorie intake and calorie expenditure.

[1507] System configuration

[1508] The system of the present invention includes the following major components:

[1509] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[1510] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[1511] 3. Cloud Server - A remote server for storing analysis results and performing necessary calculations and data storage.

[1512] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[1513] 5. Notification means - includes a display and audio output device to notify the user of advice.

[1514] Specific explanation of program processing

[1515] Food recording and analysis

[1516] 1. The user prepares breakfast on the table, and the camera in the AI ​​glasses takes pictures of the meal.

[1517] 2. The device (AI glasses) sends the captured image data to the cloud server.

[1518] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[1519] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[1520] Step counting

[1521] 1. When the user commutes to work or exercises, the AI ​​glasses measure the number of steps taken using the built-in sensor.

[1522] 2. The device sends step count data to the cloud server at regular intervals.

[1523] 3. The server analyzes the received step count data and calculates the calories burned.

[1524] Data integration and advice generation

[1525] 1. The server compares the daily calorie intake with the calorie expenditure.

[1526] 2. The server generates advice based on the user's health status, for example, to improve diet and exercise.

[1527] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[1528] Specific examples

[1529] Example 1: Breakfast recording and analysis

[1530] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[1531] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[1532] The server stores this information and updates the user database.

[1533] Example 2: Measuring steps during commute

[1534] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[1535] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[1536] Example 3: Providing advice

[1537] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[1538] The device will notify the user of this advice and display it on the AI ​​glasses' display.

[1539] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[1540] The processing flow will be explained below.

[1541] Step 1:

[1542] The user prepares a meal, for example, bread and orange juice for breakfast.

[1543] Step 2:

[1544] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[1545] Step 3:

[1546] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[1547] Step 4:

[1548] The server adds the received image data to a processing queue.

[1549] Step 5:

[1550] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[1551] Step 6:

[1552] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[1553] Step 7:

[1554] The server organizes and stores the calculated nutrient data in a database for each user.

[1555] Step 8:

[1556] The user begins their daily routine or exercise, for example, walking to the station for work.

[1557] Step 9:

[1558] The device's built-in sensor measures the user's step count data.

[1559] Step 10:

[1560] The device sends step count data to the server at regular intervals.

[1561] Step 11:

[1562] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[1563] Step 12:

[1564] The server aggregates daily calorie, salt, fat, alcohol, and step count data.

[1565] Step 13:

[1566] The server uses machine learning models to analyze the user's health status and generate data-based advice, such as "Today's breakfast was too calorie-dense, so choose a vegetable-based lunch."

[1567] Step 14:

[1568] The server sends the advice to the notification system.

[1569] Step 15:

[1570] The terminal receives the advice from the server.

[1571] Step 16:

[1572] The device will then display the received advice in the user's field of vision, for example, "Eat mostly vegetables for lunch" on the Glasses' display.

[1573] Step 17:

[1574] The user follows the advice and practices a healthy diet.

[1575] Example 1

[1576] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1577] In conventional systems for preventing and managing lifestyle-related diseases, recording meals and counting steps are often done manually, placing a heavy burden on users. Furthermore, manually calculating calories consumed and calories burned can lead to a decrease in accuracy. Furthermore, it is difficult to provide appropriate advice tailored to each individual user's health condition.

[1578] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1579] In this invention, the server includes means for a user to prepare a meal and record an image of the meal with a camera, means for the terminal to send the image data to a cloud server, means for the cloud server to analyze the meal contents using an image recognition algorithm, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a database, means for using a built-in sensor to count steps, means for sending the step count data to the cloud server, means for calculating calories burned on the cloud server, means for comparing calories consumed with calories burned to generate advice, and means for notifying the user of the generated advice, thereby enabling users to efficiently and accurately manage their own dietary and exercise habits.

[1580] A "user" is a person who uses the system to manage their own eating and exercise habits and prevent and manage lifestyle-related diseases.

[1581] "Camera" refers to a photographic device used to record images of a meal.

[1582] A "terminal" is a device used to send captured image data and step count data to a cloud server.

[1583] A "cloud server" is an infrastructure for storing and analyzing data on a remote server.

[1584] An "image recognition algorithm" is a technology that analyzes food contents from captured images and identifies the type and amount of food.

[1585] A "database" is a digital recording medium for storing analysis results and nutrient information.

[1586] "Built-in sensor" refers to a sensor device built into a device to measure steps.

[1587] "Calories burned" refers to the amount of energy consumed calculated based on step count data.

[1588] "Advice" refers to dietary and exercise recommendations for the user that are generated by comparing calories ingested with calories burned.

[1589] The present invention relates to a system that supports the prevention and management of lifestyle-related diseases. This system aims to provide users with appropriate advice by recording and analyzing dietary content, measuring the number of steps taken, and managing the balance between calorie intake and calorie expenditure.

[1590] System configuration

[1591] The system of the present invention includes the following major components:

[1592] 1. Camera - A photographic device used to record images of meals.

[1593] 2. Device - A device that sends captured image data and step count data to a cloud server. It has an internal sensor and also measures the number of steps.

[1594] 3. Cloud Server - A remote server that stores and analyzes data.

[1595] 4. Image recognition algorithms - These are technologies used to analyze food content and identify the types and quantities of food. For example, software such as TensorFlow and OpenCV are examples of such algorithms.

[1596] 5. Database - A digital recording medium for storing analysis results and nutritional information.

[1597] 6. Built-in sensors - Includes an accelerometer and gyro sensor for counting steps.

[1598] 7. Notification means - A display or audio output device for notifying the user of advice.

[1599] Specific processing of the program

[1600] Food recording and analysis

[1601] 1. The user prepares breakfast and takes a picture of the meal using the AI ​​glasses. For example, if the user prepares bread and milk, they press the camera button to take a picture.

[1602] 2. The device sends the captured image data to a cloud server using Wi-Fi or mobile data.

[1603] 3. The server receives the image data and analyzes the meal contents using an image recognition algorithm (e.g., YOLO or ResNet). It identifies bread and milk and calculates the nutritional value of each.

[1604] 4. The server stores the analysis results in the user's database, specifically recording information such as the bread's calories, carbohydrates, and fat content.

[1605] Step counting

[1606] 1. The user begins a daily activity, for example, leaving home for their morning commute.

[1607] 2. The device uses a built-in sensor to count steps and detects walking movements in real time.

[1608] 3. The device sends the measured data at regular intervals (for example, every 10 minutes) to the cloud server.

[1609] 4. The server analyzes the step count data and calculates the calories burned. For example, if 3,000 steps result in 150 calories burned, the server records this data in the user's database.

[1610] Data integration and advice generation

[1611] 1. The server aggregates the daily calorie intake and calorie expenditure data. For example, it aggregates data showing that you ate 500 calories at breakfast and burned 150 calories by walking.

[1612] 2. The server generates diet and exercise advice based on the user's individual profile (e.g., age, gender, weight, activity level, etc.). If the calorie intake is high, advice such as "eat a lighter lunch" or "exercise more" is generated.

[1613] 3. The device receives the advice generated by the server and notifies the user, either by displaying it on the AI ​​glasses display or by voice notification.

[1614] Examples of concrete examples and prompts

[1615] Specific examples

[1616] The user prepares yogurt and a banana for breakfast, and the AI ​​glasses take an image of the meal and send the data to a cloud server.

[1617] The server analyzes the image and calculates the nutritional value of yogurt (100 calories) and a banana (90 calories).

[1618] The server stores this information and updates the user database.

[1619] Prompt Sentence Examples

[1620] "How many calories did you have at dinner last night?"

[1621] "How many calories are in my breakfast today?"

[1622] "Based on your step count data this week, do you need additional exercise?"

[1623] In this way, the system of the present invention provides advanced support for users to manage their dietary and exercise habits without straining themselves.

[1624] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1625] Step 1:

[1626] The user prepares breakfast and uses the AI ​​glasses to take pictures of the meal.

[1627] Input: User prepares meal and presses camera button.

[1628] Output: The camera generates image data of the meal.

[1629] Specific operation: When the user presses the camera button, the camera on the AI ​​glasses takes a photo of the meal and the image data is saved on the device.

[1630] Step 2:

[1631] The device sends the captured image data to a cloud server.

[1632] Input: The captured image data is saved on the device.

[1633] Output: The image data is sent to the cloud server.

[1634] Specific operation: The device sends image data to the cloud server using Wi-Fi or mobile data.

[1635] Step 3:

[1636] The server applies image recognition algorithms to analyze the meal contents.

[1637] Input: Image data stored on a cloud server.

[1638] Output: Analysis results regarding food type, quantity, and nutritional value.

[1639] Specific operation: The server analyzes the image using an image recognition algorithm (e.g., TensorFlow or OpenCV), identifies the type and quantity of food, and calculates nutritional value.

[1640] Step 4:

[1641] The server stores the analysis results in a database.

[1642] Input: Analysis results (food type, amount, and nutritional value information).

[1643] Output: Analysis results stored in a database.

[1644] Specific operation: The server records the analysis results in the user's database, and nutritional information (calories, carbohydrates, fats, etc.) is saved.

[1645] Step 5:

[1646] The user begins their daily activities.

[1647] Input: The user starts walking.

[1648] Output: Real-time step count data is generated by sensors in the AI ​​glasses.

[1649] Specific operation: The user starts commuting or exercising, and the AI ​​glasses detect walking movements and count the number of steps.

[1650] Step 6:

[1651] The device periodically transmits step count data to a cloud server.

[1652] Input: Measured step count data.

[1653] Output: Step count data sent to cloud server.

[1654] Specific operation: The device sends step count data to the cloud server at regular intervals (for example, every 10 minutes).

[1655] Step 7:

[1656] The server analyzes the received step count data and calculates the calories burned.

[1657] Input: Step count data stored on a cloud server.

[1658] Output: Calculated calories burned.

[1659] Specific operation: The server analyzes the step count data, calculates the calories burned, and records them in a database.

[1660] Step 8:

[1661] The server aggregates the daily calorie intake and expenditure data.

[1662] Input: Stored calorie intake and calorie expenditure data.

[1663] Output: Consolidated calorie data.

[1664] Specific operation: The server integrates the calorie intake and calorie expenditure information obtained from the database and calculates the daily calorie balance.

[1665] Step 9:

[1666] The server generates advice based on the user's profile.

[1667] Input: Consolidated calorie data and user profile information (age, gender, weight, activity level).

[1668] Output: The generated advice.

[1669] Specific operation: The server uses the generated AI model to generate advice (recommended dietary restrictions and exercise) based on the user's health condition.

[1670] Step 10:

[1671] The terminal notifies the user of the generated advice.

[1672] Input: Advice data from the server.

[1673] Output: Advice given to the user.

[1674] Specific operation: The device notifies the user of the advice using a display or voice output.

[1675] (Application example 1)

[1676] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1677] Modern brick-and-mortar stores lack an effective system to support customers in making healthy dietary choices. They also need a way to provide personalized advice in real time based on each customer's health status. This makes it difficult for customers to prevent and manage lifestyle-related diseases without stress.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1679] In this invention, the server includes means for recording images of meals using a camera, means for analyzing meal contents using image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results in a cloud server, means for counting steps, means for transmitting step count data to the cloud server, means for calculating calories burned on the cloud server, means for generating advice by comparing calories consumed with calories burned, means for notifying the user of the generated advice, means for analyzing food calories and nutrients in real time and suggesting nutritionally balanced meals to support customers' healthy dietary choices in the physical store, and means for generating and presenting personalized health advice based on each customer's health condition, thereby enabling customers to receive information supporting healthy lifestyle habits in real time in the physical store.

[1680] A "camera" is a photographic device for recording images of meals.

[1681] "Image recognition" is a technology that analyzes recorded meal contents.

[1682] A "cloud server" is a remote server that stores data and performs calculations such as calorie consumption.

[1683] "Step counting" is a means for counting the number of steps.

[1684] "Calculating calories burned" is a method for calculating calories burned from step count data.

[1685] "Personalized health advice" is a means of generating specific recommendations based on each customer's health status.

[1686] "Nutritionally balanced meal suggestions" is a method of analyzing the calories and nutrients of food and supporting healthy dietary choices.

[1687] The present invention provides a system for supporting health management of customers in brick-and-mortar stores, and in particular provides a specific configuration for analyzing dietary content and providing advice. This system includes the following main hardware and software:

[1688] System configuration

[1689] 1. Camera

[1690] The camera is built into the AI ​​glasses and is a device for capturing images of the food. The camera takes pictures of the dishes selected by the customer in real time and sends the image data to the system.

[1691] 2. Image Recognition Algorithm

[1692] The image recognition algorithm is a technology that analyzes photographed images of food and identifies the type and amount of food. This algorithm is implemented using machine learning libraries such as TensorFlow and Keras.

[1693] 3. Cloud Server

[1694] A cloud server is a remote server that stores analysis results and manages the necessary calculations and data. AWS (Amazon Web Services) and Google Cloud Platform are often used.

[1695] 4. Built-in sensor

[1696] The built-in sensor is a device that measures the number of steps taken by customers, and the step count data is sent to a cloud server via Bluetooth or other means.

[1697] 5. Means of notification

[1698] The notification means includes a display and an audio output device for notifying the user of the advice, which allows healthy eating suggestions and advice to be provided to customers in real time within the physical store.

[1699] Program processing explanation

[1700] Camera Function:

[1701] A camera is built into the AI ​​glasses and takes a picture of the customer's dish selection, which is then instantly sent to an image recognition algorithm.

[1702] Image Recognition:

[1703] Image recognition algorithms analyze captured images to identify the type and quantity of food, using a pre-trained neural network model to identify nutritional and calorie information for each food item.

[1704] Cloud server processing:

[1705] The detected meal content data is sent to a cloud server and stored there, where the cloud server also calculates the calories burned and compares them with the user's calorie intake.

[1706] Advice Generation:

[1707] The cloud server generates personalized health advice based on the user's health status, such as nutritionally balanced meal suggestions and exercise instructions.

[1708] notification:

[1709] The generated advice is sent to the user in real time either as a display on the AI ​​glasses or as a voice prompt.

[1710] Specific examples

[1711] Example 1: Use in a restaurant

[1712] When a user orders a steak at a restaurant and takes a picture of the meal with the AI ​​glasses, the system identifies it as a steak and instantly obtains calorie information, which can be used as a guide for the next meal or exercise.

[1713] Example prompt: "From this image, identify the type of food and its calories, and analyze whether it is nutritionally balanced."

[1714] Example 2: Use in a cafe

[1715] When a user orders a salad and soup at a cafe, the AI ​​glasses recognize this and suggest healthy eating choices, informing them of any food or exercise they should avoid later.

[1716] Example prompt: "Calculate the nutritional value of a salad (200 calories) and a soup (150 calories) and store that information in a database."

[1717] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1718] Step 1:

[1719] The user takes a photo of the dish they have selected with a camera, which takes image data of the dish as input and sends the image to an image recognition algorithm.

[1720] Step 2:

[1721] The device runs an image recognition algorithm to analyze the input image data. Specifically, after preprocessing the image, it inputs it into a neural network model to identify the type and quantity of food. The output is the type of food and its nutritional information.

[1722] Step 3:

[1723] The server receives the analysis results and retrieves food types and nutrient information from the nutrition database. It takes food types as input, executes a database query, and obtains nutrient data as output.

[1724] Step 4:

[1725] The server saves the nutrient data on the cloud server. It receives the analysis results and nutrient data as input and stores them in the database. The output confirms that the data is persisted.

[1726] Step 5:

[1727] When a user walks, the device's built-in sensor measures the number of steps taken. The device receives step count data in real time as input and sends the data to a cloud server.

[1728] Step 6:

[1729] The server calculates calories burned based on the step count data received. It receives step count data as input and calculates calories burned using a predefined algorithm. It obtains the calorie burned value as output.

[1730] Step 7:

[1731] The server compares daily calorie intake and expenditure and generates personalized advice. It receives calorie intake and expenditure data as input and generates advice using a health management algorithm. The output is advice text.

[1732] Step 8:

[1733] The device notifies the user of the generated advice. It receives the advice text as input and displays it on the AI ​​glasses display or notifies the user by voice.

[1734] Step 9:

[1735] The server analyzes food calories and nutrients in real time and suggests nutritionally balanced meals to support healthy meal choices for users in physical stores. It receives data on in-store dishes as input and analyzes it in real time. It outputs healthy meal suggestions to be presented to customers.

[1736] Step 10:

[1737] The server generates and presents personalized health advice based on each customer's health status. It receives each customer's health data as input, analyzes it using a machine learning model, and obtains personalized advice as output.

[1738] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1739] This invention relates to an AI glasses system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[1740] System configuration

[1741] The system of the present invention includes the following major components:

[1742] 1. Camera - A device built into the AI ​​glasses that takes pictures of the food.

[1743] 2. Image recognition algorithm - A technology that identifies foods from photographed images of food and analyzes the type and quantity of food.

[1744] 3. Cloud Server - A remote server that stores analysis results and emotion data, and performs necessary calculations and data storage.

[1745] 4. Built-in sensor - A sensor for measuring the number of steps taken by the user.

[1746] 5. Emotion Engine - Contains technology to identify the user's emotional state.

[1747] 6. Notification means - includes a display and audio output device to notify the user of advice.

[1748] Specific explanation of program processing

[1749] Food recording and analysis

[1750] 1. The user prepares a meal, and the camera in the AI ​​glasses takes a picture of the meal.

[1751] 2. The device (AI glasses) sends the captured image data to the cloud server.

[1752] 3. The server runs an image recognition algorithm to analyze the foods in the photo, identifying bread and orange juice, for example, and calculating their nutritional values.

[1753] 4. The server stores the analysis results in a database, along with nutritional information (calories, salt, fat, etc.).

[1754] Step counting

[1755] 1. The user begins their daily routine or exercise, for example, walking to the station for work.

[1756] 2. The device's built-in sensor measures the user's step data.

[1757] 3. The device sends step count data to the cloud server at regular intervals.

[1758] 4. The server analyzes the received step count data and calculates the calories burned.

[1759] Emotion recognition and analysis

[1760] 1. The emotion engine identifies the user's emotional state. For example, it determines whether the user is feeling stressed based on facial expressions and voice data.

[1761] 2. The server sends the emotional data to the cloud, where it integrates and analyzes past emotional data and health data.

[1762] Data integration and advice generation

[1763] 1. The server compares and integrates the daily calorie intake, calorie expenditure, and emotional data.

[1764] 2. The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[1765] 3. The device sends the generated advice to the user, and displays it on the AI ​​glasses display or notifies them via voice.

[1766] Specific examples

[1767] Example 1: Breakfast recording and analysis

[1768] The user prepares toast and black coffee for breakfast, and the AI ​​glasses take a picture of the meal and send the data to a cloud server.

[1769] The server analyzes the image and calculates the nutritional value of toast (150 calories) and coffee (5 calories).

[1770] The server stores this information and updates the user database.

[1771] Example 2: Measuring steps during commute

[1772] The user walks to the station to commute to work. The AI ​​glasses measure the number of steps and send the data to a cloud server.

[1773] The server analyzes the step count data it receives and calculates that 2,000 steps burned approximately 100 calories.

[1774] Example 3: Providing advice

[1775] If the daily calorie intake exceeds the target, the server generates advice such as "eat a lighter lunch" or "exercise more."

[1776] The device will notify the user of this advice and display it on the AI ​​glasses' display. If the user's emotional state is high, the device will also suggest taking a short break to relax.

[1777] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[1778] The processing flow will be explained below.

[1779] Step 1:

[1780] The user prepares a meal, for example, bread and orange juice for breakfast.

[1781] Step 2:

[1782] The device (AI Glasses) instructs the user to take a photo. The user presses the capture button, and the AI ​​Glasses take a picture of the meal.

[1783] Step 3:

[1784] The image data captured by the device is temporarily stored locally and then sent to a cloud server via the Internet.

[1785] Step 4:

[1786] The server adds the received image data to a processing queue.

[1787] Step 5:

[1788] The server runs an image recognition algorithm to identify food items in the photo, for example, bread and orange juice.

[1789] Step 6:

[1790] The server retrieves the nutritional information (calories, salt, fat, and alcohol) of each food item from the database. For example, bread has 150 calories, and orange juice has 80 calories, 0g salt, and 1g fat.

[1791] Step 7:

[1792] The server organizes and stores the calculated nutrient data in a database for each user.

[1793] Step 8:

[1794] The user begins their daily routine or exercise, for example, walking to the station for work.

[1795] Step 9:

[1796] The device's built-in sensor measures the user's step count data.

[1797] Step 10:

[1798] The device sends step count data to the server at regular intervals.

[1799] Step 11:

[1800] The server analyzes the received step count data and calculates the calorie consumption. For example, 2000 steps will burn approximately 100 calories.

[1801] Step 12:

[1802] The emotion engine recognizes the user's facial expressions and voice to identify their emotional state, for example, determining whether they are stressed.

[1803] Step 13:

[1804] The device transmits the emotion data to the cloud server.

[1805] Step 14:

[1806] The server receives the emotional data and integrates and analyzes the past emotional data and health data.

[1807] Step 15:

[1808] The server compares and integrates daily calorie intake, calorie expenditure, and emotional data.

[1809] Step 16:

[1810] The server uses machine learning models to analyze the user's health and emotional state and generate data-based advice, such as "Today's breakfast was too calorie-dense, so eat a lighter lunch and create a more relaxing environment."

[1811] Step 17:

[1812] The server sends the advice to the notification system.

[1813] Step 18:

[1814] The terminal receives the advice from the server.

[1815] Step 19:

[1816] The device will then display the received advice in the user's field of vision. For example, the Glasses' display will say, "Eat a vegetable-based lunch." If the user is feeling stressed, the device will suggest, "Take a short break to relax."

[1817] Step 20:

[1818] Users follow advice to implement healthy eating, exercise, and stress management.

[1819] In this way, the system of the present invention provides appropriate advice according to the user's emotional state while allowing the user to manage their dietary and exercise habits without straining themselves.

[1820] Example 2

[1821] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1822] Achieving a balance between daily calorie intake and calorie expenditure is important for preventing and managing lifestyle-related diseases, but it is difficult for users to consistently maintain this balance in their daily lives. Furthermore, a user's emotional state is also an important factor in health management, and it is widely known that stress, in particular, has a negative impact on health. However, current systems only record dietary habits and measure steps, and are unable to provide comprehensive health management that also takes emotional state into account. Therefore, there is a need for a system that can comprehensively manage a user's health and emotional state and provide individually optimized advice.

[1823] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1824] In this invention, the server includes means for recording images of meals, means for analyzing meal contents by image recognition, means for calculating nutrient intake based on the analysis results, means for saving the analysis results on a remote server, means for counting steps, means for transmitting step count data to the remote server, means for calculating calories burned on the remote server, means for comparing calories consumed with calories burned to generate advice, means for identifying the user's emotional state and transmitting the identified emotional data to the remote server, means for integrating and analyzing the user's emotional data and health data, and means for notifying the user of the generated advice. This enables the user to comprehensively manage their health throughout the day and receive individually optimized advice that also takes into account their emotional state.

[1825] The "means for recording images of meals" refers to a device that uses a photographing device such as a camera to record images of meals consumed by the user as digital data.

[1826] "Means for analyzing meal contents using image recognition" refers to technologies and algorithms that analyze image data of photographed meals and identify the foods contained in the meal and their amounts.

[1827] The "means for calculating the nutrients ingested based on the analysis results" refers to a computing device or program that calculates the calories ingested and various nutrients from the dietary content data obtained by image recognition.

[1828] The "means for saving the analysis results on a remote server" is a function for saving the calculated nutrient data and other analysis results on a remote server via the Internet.

[1829] The "means for measuring the number of steps" refers to a device or sensor for measuring the walking activity of a user, and includes a pedometer.

[1830] The "means for transmitting step count data to a remote server" refers to a communication device or function for transmitting the measured step count data to a remote server via the Internet.

[1831] The "means for calculating calories burned on a remote server" refers to an algorithm or program that calculates calories burned based on the step count data received on the remote server.

[1832] The "means for generating advice by comparing calorie intake with calorie expenditure" refers to a device or software that compares the balance between calorie intake and calorie expenditure and automatically generates advice for health management.

[1833] "Means for identifying the user's emotional state and transmitting the identified emotional data to a remote server" refers to technology or devices for identifying the user's emotions through facial expression recognition, voice analysis, etc., and transmitting that data to a remote server.

[1834] "Means for integrating and analyzing a user's emotional data and health data" refers to an algorithm or program that integrates and analyzes a user's emotional data and health data (calories ingested, calories burned, etc.) to assess their overall health status.

[1835] The "means for notifying the user of the generated advice" refers to a notification device such as a display or audio output device for conveying the generated health management advice to the user.

[1836] This invention relates to an AI system that supports the prevention and management of lifestyle-related diseases, and is equipped with functions to record and analyze dietary content, measure steps, manage the balance of calorie intake and expenditure, and recognize the user's emotions and reflect them in advice. The system aims to comprehensively manage health and emotional states and provide appropriate advice.

[1837] Hardware and software used

[1838] Camera: A device built into the AI ​​glasses that takes pictures of meals.

[1839] Built-in sensor: A sensor for measuring steps.

[1840] Image recognition algorithm: Technologies such as TensorFlow and OpenCV are used for image recognition.

[1841] Emotion Engine: This engine identifies emotional states through facial expression recognition and voice analysis, using machine learning models with Pytorch and Keras.

[1842] Cloud Server: The remote server is responsible for data analysis and storage, where it calculates calories ingested and burned and generates advice.

[1843] Notification means: Includes a display and an audio output device for notifying the user of the generated advice.

[1844] Explanation of program processing

[1845] Food recording and analysis

[1846] When a user prepares a meal, the camera in the AI ​​glasses automatically captures an image of the meal. The device then sends the captured image data to a cloud server via Wi-Fi or Bluetooth. The server then analyzes the received image data and runs an image recognition algorithm using TensorFlow or OpenCV to identify the food. Based on the analysis results, the nutrient intake is calculated and stored in a database.

[1847] Step counting

[1848] When a user starts moving, the built-in sensor measures the number of steps. This data is sent to a cloud server at regular intervals, and the server calculates the calories burned. Specifically, a calculation algorithm is used based on the step count data to calculate the calories burned.

[1849] Emotion recognition and analysis

[1850] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. This data is then used to identify their emotional state using facial and voice recognition algorithms. The emotion data is then sent to a remote server, which then combines it with past health data for analysis. Machine learning models using Pytorch and Keras are used for the analysis.

[1851] Data integration and advice generation

[1852] The server analyzes calorie intake, calorie expenditure, and emotional data, and then comprehensively evaluates the user's health and emotional state to generate appropriate advice. The advice is displayed in text format on the screen, and voice notification is also available if necessary.

[1853] Specific examples

[1854] For example, if a user has toast and black coffee for breakfast, the AI ​​Glasses will take a photo of this and send the image data to a cloud server. The server will analyze the image, calculate the nutritional value of each food item, and save the results. If the user then takes 2,000 steps on their way to work, the built-in sensor will measure the number of steps and send it to the cloud server. Based on the data received by the server, it will determine that the user has burned approximately 100 calories in 2,000 steps.

[1855] Furthermore, if the emotion engine analyzes the user's facial expressions and voice and determines that they are under high stress, it will use this data to comprehensively evaluate their health condition. Based on this, it will generate advice such as "Today's breakfast was too high in calories, so eat a lighter lunch and create a relaxing environment," and display it on the screen.

[1856] Prompt Sentence Examples

[1857] Here are some example prompts to input to a generative AI model:

[1858] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[1859] In this way, the system of the present invention allows users to manage their diet and exercise habits without straining themselves, and provides appropriate advice according to their emotional state.

[1860] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1861] Specific flow of program processing

[1862] Food recording and analysis

[1863] Step 1:

[1864] The user prepares a meal. At this time, the camera in the AI ​​glasses automatically captures images of the meal. The input is the scene of the user eating, and the output is image data of the meal.

[1865] Step 2:

[1866] The device uses Wi-Fi or Bluetooth to send captured image data to a cloud server. The input is the image data captured by the camera, and the output is the image data sent to the cloud server.

[1867] Step 3:

[1868] The server receives the image data and runs an image recognition algorithm (e.g., TensorFlow or OpenCV). The input is the image data sent to the cloud server, and the output is the analyzed food data.

[1869] Step 4:

[1870] The server identifies foods in the image and calculates the nutritional value (calories, protein, fat, etc.) of each food. The input is the analyzed food data, and the output is the calculated nutritional value information.

[1871] Step 5:

[1872] The server stores the analysis results in a database. The input is the calculated nutritional value information, and the output is the nutritional data stored in the database.

[1873] Step counting

[1874] Step 1:

[1875] The user starts daily activities or exercise. For example, walking to the station for work. The input is the user's movement activity, and the output is walking data.

[1876] Step 2:

[1877] The device's built-in sensor measures walking data in real time. The input is the user's physical steps, and the output is step count data.

[1878] Step 3:

[1879] The device periodically (e.g., every hour) sends the cumulative step count data to the cloud server. The input is the step count data accumulated on the device, and the output is the data sent to the cloud server.

[1880] Step 4:

[1881] The server analyzes the received step count data and calculates the calories burned. The input is the step count data sent to the cloud server, and the output is the calculated calories burned.

[1882] Emotion recognition and analysis

[1883] Step 1:

[1884] The emotion engine captures the user's facial expressions with a camera and simultaneously records their voice. The input is the user's facial expressions and voice data, and the output is the captured emotion data.

[1885] Step 2:

[1886] The device analyzes these emotion data in real time and sends the identified emotion data to the cloud server. The input is the captured emotion data, and the output is the emotion data sent to the cloud server.

[1887] Step 3:

[1888] The server receives the emotional data and integrates and analyzes the past emotional data and health data. The input is the emotional data sent to the cloud server, and the output is the integrated analysis result.

[1889] Data integration and advice generation

[1890] Step 1:

[1891] The server integrates and analyzes daily calorie intake, calorie expenditure, and emotional data. The input is calorie intake, calorie expenditure, and emotional data, and the output is integrated health evaluation data.

[1892] Step 2:

[1893] The server uses machine learning models (e.g., Pytorch or Keras) to analyze the user's health and emotional state and generate advice. The input is the integrated health assessment data, and the output is the generated advice.

[1894] Step 3:

[1895] The terminal notifies the user of the generated advice. The input is the generated advice, and the output is text displayed on the screen or a voice notification.

[1896] Prompt Sentence Examples

[1897] "User had toast and black coffee for breakfast and walked 2000 steps on their commute. Their current emotional state is stressed. Use this information to provide advice on how to achieve both health and emotional balance."

[1898] This prompt enables the generative AI model to generate advice based on the user's health and emotional state.

[1899] (Application example 2)

[1900] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1901] Health management is becoming increasingly important in modern society. While there are a wide variety of technologies for preventing and managing lifestyle-related diseases, there is still a lack of systems that provide personalized advice based on individual health conditions. In particular, there is a need for comprehensive support that promotes healthy choices and also includes mental health when choosing food when eating out or at a cafe. However, current systems lack a system that integrates multiple functions, such as analyzing food content, managing the balance of calorie intake and expenditure, and recognizing emotional states. Therefore, new technologies that can achieve these functions are needed.

[1902] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is a system including: means for recording images of meals with a camera; means for analyzing meal contents by image recognition; means for calculating nutrient intake based on the analysis results; means for saving the analysis results on a cloud server; means for counting steps; means for transmitting step count data to the cloud server; means for calculating calories burned on the cloud server; means for comparing calories consumed with calories burned to generate advice; and means for notifying the user of the generated advice. This makes it possible to analyze restaurant menus when dining out, provide nutritional information and recommended meal plans based on individual health conditions, and provide comprehensive mental health support that also takes emotional states into account.

[1903] A "camera" is a device for recording images.

[1904] "Means" refers to the methods and techniques used to achieve a goal.

[1905] "Image recognition" is a technology that extracts, identifies, and classifies objects and features from captured images.

[1906] "Analysis results" refers to the data obtained after processing such as image recognition.

[1907] "Intaken nutrients" refers to the nutrients taken into the body through meals and food.

[1908] A "cloud server" is a device for storing data and performing computations on a remote server accessible via the Internet.

[1909] "Steps" refers to the number of times your feet move when walking a certain distance.

[1910] "Calories consumed" refers to the energy consumed through physical activity and basal metabolism.

[1911] "Advice" refers to instructions or guidance given to another person based on particular knowledge or information.

[1912] "User" refers to a person who uses this system.

[1913] "Store menu" refers to a list of foods and drinks offered at a restaurant or other establishment.

[1914] "Nutritional information" refers to data about the nutritional components and amounts contained in food.

[1915] A "meal plan" refers to a combination of meals designed to achieve a specific health goal.

[1916] "Emotional state" refers to an individual's temporary psychological or mental state.

[1917] "Mental health" refers to an individual's mental and psychological well-being.

[1918] "Comprehensive support" refers to a wide range of assistance and support that encompasses multiple aspects.

[1919] This invention relates to the "AI Glass Guide" system, which is used in health-conscious cafes and other places, and aims to provide optimal health advice by simultaneously analyzing dietary content and emotional state.

[1920] System configuration

[1921] The system of the present invention uses the following hardware and software.

[1922] Hardware:

[1923] 1. AI Glasses - Includes a camera (e.g., Sony IMX camera), a display (OLED display), a microphone, and a built-in sensor (MPU6050 sensor for counting steps).

[1924] 2. Cloud Server - A server accessible via the Internet.

[1925] software:

[1926] 1. Image Recognition Algorithms - Custom deep learning models using TensorFlow and PyTorch.

[1927] 2. Emotion recognition engine - A proprietary library called EmotionEngine.

[1928] 3. Cloud communication - Uses APIs from AWS and Google Cloud Platform.

[1929] 4. Health Advisor - A program that generates advice based on health and emotional data.

[1930] The role of each hardware and software

[1931] 1. A user visits a cafe, puts on the AI ​​glasses, and uses the camera in the glasses to scan the menu.

[1932] 2. The device (AI glasses) takes a picture of the menu and sends it to the cloud server.

[1933] 3. The server uses image recognition algorithms to identify the foods on the menu and parse the nutritional information.

[1934] 4. The server stores the analysis results in a database.

[1935] 5. When the user selects a meal, the AI ​​glasses will generate advice about it and display it on the screen or notify them via voice.

[1936] 6. The device's built-in sensor measures the user's steps and sends them to the cloud server.

[1937] 7. The server calculates the calories burned based on the step count data and compares this with the calories ingested to generate optimal advice.

[1938] 8. The emotion engine analyzes the user's emotional state and provides information about their mental health.

[1939] Program processing explanation

[1940] 1. Image processing: The image data of the menu captured by the AI ​​glasses' camera is preprocessed using OpenCV, and then an image recognition algorithm trained in TensorFlow or PyTorch is run to identify the food.

[1941] 2. Cloud communication: Analysis results and meal content data are sent in real time to a cloud server via the AWS or Google Cloud Platform API.

[1942] 3. Data Analysis: The cloud server uses EmotionEngine to analyze the user's facial expression data to identify their emotional state, and then integrates it with their health data to generate advice.

[1943] 4. Notification: The generated advice is notified to the user through the AI ​​glasses' OLED display and audio output.

[1944] Examples and prompts

[1945] Specific examples

[1946] 1. When visiting the cafe: The user enters the cafe, borrows and puts on the AI ​​glasses. "Hello, welcome to Health Cafe. What would you like to order today?"

[1947] 2. When selecting a meal: The user scans the menu with their camera and the analysis results are displayed. "This chicken salad is 120 calories. It's recommended considering today's calorie intake."

[1948] Prompt Sentence Examples

[1949] "The camera on the AI ​​glasses takes pictures of meals, sends them to the cloud, and analyzes their nutritional value. Based on the analysis results, the system will suggest the optimal meal plan for the user based on their health condition."

[1950] With specific technical configurations and operating procedures, users can manage their health even when eating out, and can also receive comprehensive support that takes into account their emotional state.

[1951] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1952] Step 1:

[1953] A user visits a cafe, borrows the AI ​​glasses, and puts them on. Putting on the AI ​​glasses is the input, and the system becomes ready for use.

[1954] Step 2:

[1955] The user scans the menu by pointing it at the camera on the AI ​​glasses. The menu image is then input. The camera captures the image of the menu, which is then acquired by the device. The acquired image data is then sent to the next processing step.

[1956] Step 3:

[1957] The device sends the captured image data to the cloud server. The input is the image data of the menu, and the output is the image data sent to the cloud server. This is done using cloud communication.

[1958] Step 4:

[1959] The server uses an image recognition algorithm to analyze the menu image data sent. The input is the menu image data, and the image recognition algorithm extracts image features to identify the food items. The output is a list of identified foods.

[1960] Step 5:

[1961] The server analyzes the nutritional value information based on the food list. The input is the identified food list, and the server obtains the nutritional value information by referencing the nutrient database. The output is the analysis result including the nutritional value information.

[1962] Step 6:

[1963] The server stores the analysis results in a database. The input is the analysis results including nutritional information, and storing this data in storage allows for future reference. The output is an updated database.

[1964] Step 7:

[1965] The server generates dietary advice and sends it to the device via the cloud. The input is nutritional information and the user's health data, which the machine learning model analyzes to generate advice. The output is the generated advice.

[1966] Step 8:

[1967] The device displays the generated advice on the AI ​​glasses display or notifies the user by voice. The input is advice data sent from the server, which is notified to the user. The output is a visual or audio notification to the user.

[1968] Step 9:

[1969] The user selects a meal and then begins to move. The built-in sensor in the AI ​​glasses measures the user's steps. The input is the user's movement data, which the sensor measures to generate step count data. The output is the measured step count data.

[1970] Step 10:

[1971] The device sends step count data to the cloud server. The input is the measured step count data, and the output is the step count data sent to the cloud server.

[1972] Step 11:

[1973] The server calculates the calories burned based on the step count data. The input is the step count data, and the server calculates the calories burned as a data operation. The output is the calculated calories burned.

[1974] Step 12:

[1975] The server compares the calories ingested and burned, and the emotion engine analyzes the user's emotional state. The inputs are calories ingested, calories burned, and facial expression data, which are analyzed by the machine learning model and the Emotion Engine. The output is comprehensive health and mental health advice.

[1976] Step 13:

[1977] The server sends the generated advice to the terminal, which then notifies the user. The input is the generated advice, and the output is the advice notified to the user.

[1978] Through these processing steps, the system of the present invention provides comprehensive health management support that integrates nutritional analysis of food, step counting, and emotional state recognition.

[1979] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1980] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1981] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1982] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1983] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1984] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1985] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1986] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1987] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1988] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1989] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1990] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1991] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1992] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1993] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1994] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1995] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1996] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1997] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1998] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1999] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2000] The following is further disclosed regarding the above embodiment.

[2001] (Claim 1)

[2002] means for recording images of the meal by a camera;

[2003] A means for analyzing meal contents using image recognition;

[2004] A means for calculating nutrient intake based on the analysis results;

[2005] A means for storing the analysis results on a cloud server;

[2006] A means for measuring the number of steps;

[2007] means for transmitting step count data to a cloud server;

[2008] A means for calculating calorie consumption on a cloud server;

[2009] a means for comparing calories in with calories out to generate advice;

[2010] The system includes a means for notifying the user of the generated advice.

[2011] (Claim 2)

[2012] The system according to claim 1, further comprising means for acquiring the meal contents detected by image recognition from a nutrient database.

[2013] (Claim 3)

[2014] The system according to claim 1, further comprising means for analyzing the health condition of a user using a machine learning model on a cloud server.

[2015] "Example 1"

[2016] (Claim 1)

[2017] means for a user to prepare a meal and record images of the meal with a camera;

[2018] A means for the terminal to transmit image data to a cloud server;

[2019] A cloud server applies an image recognition algorithm to analyze the meal contents;

[2020] A means for calculating nutrient intake based on the analysis results;

[2021] a means for storing the analysis results in a database;

[2022] using a built-in sensor to count steps;

[2023] means for transmitting step count data to a cloud server;

[2024] A means for calculating calorie consumption on a cloud server;

[2025] a means for comparing calories in with calories out to generate advice;

[2026] The system includes a means for notifying the user of the generated advice.

[2027] (Claim 2)

[2028] The system according to claim 1, further comprising means for acquiring the meal contents detected by image recognition from a nutrient database.

[2029] (Claim 3)

[2030] The system according to claim 1, further comprising means for analyzing the health condition of a user using a machine learning model on a cloud server.

[2031] "Application Example 1"

[2032] (Claim 1)

[2033] means for recording images of the meal by a camera;

[2034] A means for analyzing meal contents using image recognition;

[2035] A means for calculating nutrient intake based on the analysis results;

[2036] A means for storing the analysis results on a cloud server;

[2037] A means for measuring the number of steps;

[2038] means for transmitting step count data to a cloud server;

[2039] A means for calculating calorie consumption on a cloud server;

[2040] a means for comparing calories in with calories out to generate advice;

[2041] means for notifying a user of the generated advice;

[2042] In order to support customers' healthy meal choices in physical stores, a means of analyzing food calories and nutrients in real time and suggesting nutritionally balanced meals is provided.

[2043] and a means for generating and presenting personalized health advice based on each customer's health status.

[2044] (Claim 2)

[2045] The system according to claim 1, further comprising means for acquiring the meal contents detected by image recognition from a nutrient database, and means for providing health support for the customer's meal selection.

[2046] (Claim 3)

[2047] The system according to claim 1, further comprising means for analyzing the health condition of a user using a machine learning model on a cloud server and providing personalized health management advice.

[2048] "Example 2: Combining Emotion Engines"

[2049] (Claim 1)

[2050] a means for recording images of meals;

[2051] A means for analyzing meal contents using image recognition;

[2052] A means for calculating nutrient intake based on the analysis results;

[2053] a means for storing the analysis results on a remote server;

[2054] A means for measuring the number of steps;

[2055] means for transmitting the step count data to a remote server;

[2056] means for calculating calories burned on a remote server;

[2057] a means for comparing calories in with calories out to generate advice;

[2058] means for identifying an emotional state of the user and transmitting the identified emotional data to a remote server;

[2059] A means for integrating and analyzing the user's emotional data and health data;

[2060] The system includes a means for notifying the user of the generated advice.

[2061] (Claim 2)

[2062] The system according to claim 1, further comprising means for acquiring the meal contents detected by image recognition from a nutrient database.

[2063] (Claim 3)

[2064] 10. The system of claim 1, further comprising means for analyzing the health and emotional state of the user using machine learning models on a remote server.

[2065] "Application example 2 when combining emotion engines"

[2066] (Claim 1)

[2067] means for recording images of the meal by a camera;

[2068] A means for analyzing meal contents using image recognition;

[2069] A means for calculating nutrient intake based on the analysis results;

[2070] A means for storing the analysis results on a cloud server;

[2071] A means for measuring the number of steps;

[2072] means for transmitting step count data to a cloud server;

[2073] A means for calculating calorie consumption on a cloud server;

[2074] a means for comparing calories in with calories out to generate advice;

[2075] means for notifying a user of the generated advice;

[2076] A means to analyze restaurant menus and provide nutritional information and recommended meal plans,

[2077] A system including a means of recognizing emotional states and providing information about mental health.

[2078] (Claim 2)

[2079] The system according to claim 1, further comprising means for acquiring the meal contents detected by image recognition from a nutrient database.

[2080] (Claim 3)

[2081] 2. The system of claim 1, further comprising: means for analyzing a user's health status using a machine learning model on a cloud server; and means for generating mental health advice based on the user's emotional state. [Explanation of symbols]

[2082] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for recording images of the meal by a camera; A means for analyzing meal contents using image recognition; A means for calculating nutrient intake based on the analysis results; A means for storing the analysis results on a cloud server; A means for measuring the number of steps; means for transmitting step count data to a cloud server; A means for calculating calorie consumption on a cloud server; a means for comparing calories in with calories out to generate advice; The system includes a means for notifying the user of the generated advice.

2. The system according to claim 1, further comprising means for acquiring the meal contents detected by image recognition from a nutrient database.

3. The system according to claim 1 , further comprising means for analyzing the health condition of a user using a machine learning model on a cloud server.

Images