System

Abstract

A system is provided.SOLUTION: A system including means for a user to capture an image of a meal and upload the image on the spot, means for analyzing the uploaded image and calculating ingredients, three major nutrients, and calories of food materials, means for analyzing meal contents and tendencies registered in the past and proposing a meal menu necessary for health maintenance, means for acquiring exercise information and sleeping information from a smartphone or a smartwatch, and means for analyzing an amount of exercise necessary for health maintenance on the basis of the acquired information and proposing an appropriate exercise menu.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's busy lifestyles, many people's health often deteriorates due to irregular eating habits and lack of exercise. In particular, it is difficult to easily obtain specific advice based on individual health conditions, making it difficult to maintain health management. This invention aims to solve this problem by easily recording and analyzing daily diet and exercise data and providing appropriate advice for maintaining individual health conditions. [Means for solving the problem]

[0005] This invention provides a system that includes the following components: a means for a user to photograph a meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients; a means for analyzing previously registered meal contents and trends and proposing a meal menu necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or smartwatch; and a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and proposing an appropriate exercise menu. The system also includes a means for using the user's location information to link inventory information of nearby commercial facilities and menus offered by restaurants to support the implementation of the proposed menu. This system generates optimal suggestions based on the user's individual health status, making it easier for users to continue managing their health.

[0006] "User" refers to a person who uses the system to record their diet, obtain exercise information, and receive health management suggestions.

[0007] "Meal details" refers to information about the foods and dishes consumed by the user, and this information is uploaded to the system as images.

[0008] "Means for uploading images" refers to the operation and function for transferring images of meal contents taken by the user to the server.

[0009] "Means for analyzing uploaded images" refers to the process by which the system decodes the received meal images and calculates the ingredients, three major nutrients, and calories of the ingredients.

[0010] The "three macronutrients" are proteins, fats, and carbohydrates, and refer to the main nutrients we ingest from our diet.

[0011] "Food ingredients" refers to the specific ingredients and nutrients contained in a meal.

[0012] "Previously registered meal content" refers to meal images previously recorded by the user and the analysis results thereof.

[0013] "Trend analysis methods" refers to the process of finding patterns and trends in collected historical data.

[0014] "Means for suggesting a meal menu necessary for maintaining health" refers to a process of providing a meal plan for improving or maintaining the user's health based on analytical results and trend analysis.

[0015] "Smartphone" or "smartwatch" refers to an electronic device that captures a user's exercise and sleep information.

[0016] "Exercise information" refers to data regarding the type, duration, intensity, etc. of exercise performed by the user.

[0017] "Sleep Information" refers to data regarding a user's sleep patterns, quality, and duration.

[0018] "Means for analyzing the amount of exercise required to maintain health" refers to a process for calculating the optimal amount of exercise for the user's health condition based on the acquired exercise information and sleep information.

[0019] The "means for proposing an appropriate exercise menu" refers to a process for proposing an exercise plan that the user should implement based on the analysis results.

[0020] "Location information" refers to geographic location data obtained from a user's smartphone or smartwatch.

[0021] "Inventory information of commercial facilities" refers to information about the inventory status of products offered by supermarkets and other stores located near the user.

[0022] "Restaurant menu" refers to the list of food and beverages offered by a restaurant. [Brief explanation of the drawings]

[0023] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] This invention relates to a system that allows users to easily record their dietary and exercise information and provides necessary advice for health management. Specifically, it utilizes a smartphone or smartwatch to automatically collect dietary and exercise information, analyzes it using artificial intelligence (AI), and proposes an appropriate health maintenance menu.

[0045] System configuration and operation

[0046] Input and analysis of dietary information

[0047] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[0048] The server then passes the image file to an artificial intelligence module, which analyzes the image and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The results of this analysis are stored in a database for later analysis.

[0049] Acquiring exercise and sleep information

[0050] The device (smartphone or smartwatch) automatically collects data about the user's exercise and sleep, such as the number of steps, distance, heart rate, and sleep time, and periodically transmits this data to a server.

[0051] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[0052] Suggestions for maintaining good health

[0053] The server uses AI to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to evaluate the user's health. Based on this evaluation, it proposes specific diet and exercise menus to help the user maintain their health.

[0054] For example, if the server determines that the user has not been exercising recently, it can suggest that the user "go for a 30-minute jog next weekend." If the user's nutritional intake is unbalanced, the server can recommend that the user "eat more fish and vegetables at your next dinner."

[0055] Support for implementing proposals (future enhancements)

[0056] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0057] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[0061] Step 2:

[0062] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[0063] Step 3:

[0064] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[0065] Step 4:

[0066] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[0067] Step 5:

[0068] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[0069] Step 6:

[0070] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[0071] Step 7:

[0072] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[0073] Step 8:

[0074] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[0075] Step 9:

[0076] The server passes the accumulated dietary, exercise, and sleep data to the AI ​​for analysis, which then analyzes past data and trends to assess the user's health.

[0077] Step 10:

[0078] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "go for a 30-minute jog next weekend" or "eat more fish and vegetables at your next dinner."

[0079] Step 11:

[0080] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[0081] Step 12:

[0082] (as a future extension)

[0083] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[0084] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management.

[0085] Example 1

[0086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0087] In modern society, health management has become an important issue, but it is difficult to accurately record diet and exercise habits in a busy lifestyle. Furthermore, there are only a limited number of systems that comprehensively evaluate health status and provide specific recommendations tailored to individual users. In conventional systems, analyzing dietary content and collecting exercise and sleep information is time-consuming, and it is difficult to comprehensively analyze this data and provide appropriate advice. In particular, flexible recommendations utilizing the user's location information have not been realized. There is a need to improve these issues and provide a system that allows users to easily manage their health.

[0088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0089] In this invention, the server includes a means for users to photograph their meal and upload the images on the spot; a means for analyzing the uploaded images and calculating the ingredients, three major nutrients, and calories; a means for analyzing previously registered meal contents and trends and suggesting meal menus necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or wearable device; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for analyzing images using an artificial intelligence module and storing them in a database; and a means for comprehensively analyzing the user's data, evaluating their health status, and providing advice. This allows users to easily analyze the photographed meal contents, centrally manage their exercise and sleep information, and receive specific suggestions based on their individual health status. Furthermore, a function for assisting users in implementing the suggested menus can be provided using the user's location information, making it easier for users to carry out health-maintaining activities.

[0090] "User" refers to an individual who uses this system to record dietary, exercise, and sleep information and manage their health.

[0091] "Means for taking photos of meal contents and uploading images on the spot" refers to a function that allows users to take photos of their meal using a smartphone camera or a dedicated app and send the image data to a server in real time.

[0092] "Means for analyzing uploaded images and calculating the ingredients, three major nutrients, and calories" refers to an algorithm in which an artificial intelligence module running on a server analyzes the meal images sent by users and calculates the ingredients, proteins, fats, carbohydrates, and calories contained in the food.

[0093] "A means for analyzing previously registered dietary content and trends and proposing dietary menus necessary for maintaining health" refers to a function that suggests dietary menus suited to the user's current health condition based on the user's past dietary data stored in a database.

[0094] "Means for acquiring exercise and sleep information from a smartphone or wearable device" refers to the function of a smartphone or smartwatch worn by a user to collect data such as the number of steps taken during exercise, distance traveled, heart rate, and sleep time, and transmit this data to a server.

[0095] "Means of analyzing the amount of exercise required to maintain health based on the acquired information and proposing an appropriate exercise menu" refers to the function in which the server analyzes exercise and sleep information, and generates and presents the optimal exercise menu for the user.

[0096] "Image analysis using an artificial intelligence module and storage in a database" refers to the function in which an AI running on a server analyzes food images and stores the results in a database.

[0097] "Means for comprehensively analyzing user data, assessing health status, and providing advice" refers to the function of comprehensively analyzing dietary information, exercise information, and sleep information collected by the server, assessing the user's health status, and providing specific advice for maintaining good health.

[0098] MODE FOR CARRYING OUT THE INVENTION

[0099] The present invention relates to a system that allows users to efficiently and easily record their daily diet and exercise information and perform comprehensive health management. This system utilizes smartphones and wearable devices, analyzes data using artificial intelligence (AI), and proposes appropriate health maintenance menus to users.

[0100] Hardware and software used

[0101] Hardware:

[0102] Smartphone

[0103] Wearable devices such as smartwatches

[0104] server

[0105] software:

[0106] Dedicated application (for smartphones)

[0107] Server-side artificial intelligence module

[0108] Database System

[0109] Specific explanation of the system's operation

[0110] Users use a smartphone with a dedicated app installed to take pictures of their daily meals. At this time, they can also enter detailed information about the meal and the types of ingredients. The photographed food images are uploaded to a server via the smartphone. At this time, metadata such as the date and time of the photo and location information are also sent along with the image.

[0111] The server passes the received image file to an artificial intelligence module. The AI ​​analyzes the image, identifies the ingredients in the image, and calculates the protein, fat, carbohydrates, and calories of each ingredient. This data is stored in a database. For example, if you take a photo and upload a meal of bread, eggs, and bananas for breakfast, the AI ​​will automatically generate and record the following: "Bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "Egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "Banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[0112] The devices, such as smartphones and smartwatches, automatically collect users' exercise information (e.g., number of steps, distance traveled, heart rate) and sleep information. This data is periodically sent to a server and stored in a database.

[0113] The server comprehensively analyzes the collected dietary, exercise, and sleep data to evaluate the user's health condition. Based on this evaluation, it suggests appropriate meal and exercise menus to the user. For example, if it determines that the user has not been exercising recently, the server will suggest "go for a 30-minute jog next weekend." If the user's nutrition is unbalanced, it will recommend "eat more fish and vegetables at your next dinner."

[0114] Some examples of specific prompts include:

[0115] "I had bread, eggs, and a banana for breakfast. What are the calories and nutrients in this meal?"

[0116] In the future, the server may use the user's location information to link inventory information from nearby commercial facilities and restaurant menus to support the implementation of suggested menus, making it easier for users to implement suggested health maintenance menus and supporting continuous health management.

[0117] As described above, the system of the present invention enables the user to efficiently manage their health and supports continuous health maintenance activities.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] The user launches the dedicated application and takes a photo of the meal with their smartphone. The user also has the option to input ingredients and meal details. The input data is the image of the meal and text information about the ingredients.

[0121] Input: Meal image, ingredient information

[0122] Output: Food image data and ingredient information stored on the smartphone

[0123] Specific behavior:

[0124] For breakfast, users take a photo of bread, eggs, and bananas and upload it to the app.

[0125] Step 2:

[0126] The device (smartphone) transmits the captured food image and the input ingredient information to the server. This transmission includes metadata such as the date and time of the image capture and location information along with the image data.

[0127] Input: Food image data, ingredient information, metadata

[0128] Output: Food image data, ingredient information, and metadata sent to the server

[0129] Specific behavior:

[0130] The smartphone captures a photo of the meal and information about the ingredients, and then sends the photo, information, and metadata all at once to the server.

[0131] Step 3:

[0132] The server then passes the image files to an artificial intelligence module, which uses image analysis technology to identify the ingredients in the image and calculate the protein, fat, carbohydrates, and calories of each ingredient. The analysis results are then stored in a database.

[0133] Input: Submitted food image data, metadata

[0134] Output: Ingredient data, nutrient data, and calorie data for each ingredient stored in the database

[0135] Specific behavior:

[0136] The AI ​​analyzes the image and generates and stores data such as "bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[0137] Step 4:

[0138] The device (smartphone or wearable device) automatically collects the user's exercise and sleep information. The collected data includes the number of steps taken, distance traveled, heart rate, and sleep time. This data is sent to a server at regular intervals.

[0139] Input: User's exercise and sleep data

[0140] Output: Exercise and sleep data sent to the server

[0141] Specific behavior:

[0142] The smartwatch monitors your heart rate 24 hours a day and sends your step count and sleep data to a server at the end of the day.

[0143] Step 5:

[0144] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[0145] Input: exercise data, sleep data sent

[0146] Output: Exercise data and sleep data stored in a database

[0147] Specific behavior:

[0148] The server records the received exercise data in a database and stores the data by day, week, or month.

[0149] Step 6:

[0150] The server uses AI to perform a comprehensive analysis of the collected dietary, exercise, and sleep data. Based on the results of this analysis, the server evaluates the user's health status and suggests specific diet and exercise menus.

[0151] Input: Food data, exercise data, and sleep data in the database

[0152] Output: Proposal of a health maintenance menu for the user

[0153] Specific behavior:

[0154] The server generates a health maintenance menu such as, "Based on your recent data, you have continued to lack exercise, so we recommend that you go for a 30-minute jog next weekend. Also, your nutritional intake tends to be unbalanced, so we recommend that you eat more fish and vegetables for your next dinner," and notifies the user.

[0155] Step 7:

[0156] In the future, the server will use the user's location information to provide a function that links inventory information from nearby commercial facilities and menus offered by restaurants to support the implementation of suggested menus.

[0157] Input: User's location information, suggested health menu

[0158] Output: Information about commercial facilities and restaurants based on the user's location

[0159] Specific behavior:

[0160] The server obtains the user's location information and provides information on stock availability at nearby supermarkets where the suggested ingredients can be purchased, as well as information on restaurants that offer the suggested menu items.

[0161] (Application example 1)

[0162] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] Conventional health management systems require users to manually input information, posing challenges in terms of time and accuracy. Furthermore, there was a lack of systems that could comprehensively grasp the health status of employees in specific work environments, such as factories, and provide optimal health management plans. As a result, there is a demand for ways to maintain employee health and improve work efficiency.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0165] In this invention, the server includes: a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of ingredients; a means for analyzing previously registered meal contents and trends and proposing a meal menu necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and proposing an appropriate exercise menu; a means for automatically collecting health data including the user's physiological data and analyzing it in real time; a means for sending alerts and providing appropriate health advice based on the results of the health data analysis; and a means for evaluating the health status of factory workers and supporting appropriate health management. This eliminates the need for users to manually input information and enables real-time understanding and evaluation of health status, thereby improving the health of factory workers and improving work efficiency.

[0166] "User" refers to a person who uses the system to manage their own health condition.

[0167] "Meal contents" refers to all of the foods and menu items consumed by the user.

[0168] "Image" refers to a photograph of the meal contents taken by the user using a smartphone or other photographic device.

[0169] "Upload" refers to the act of sending an image taken by a user to a server.

[0170] "Food ingredients" refers to the nutritional and chemical constituents of the individual foods in a meal.

[0171] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[0172] A "calorie" is a unit of measurement that indicates the amount of energy contained in food, and refers to the energy provided by food.

[0173] "Exercise information" refers to exercise data during the user's daily activities, including the number of steps, heart rate, distance, and the like.

[0174] "Sleep information" refers to information such as the user's sleep patterns and sleep duration.

[0175] An "exercise menu" refers to an exercise plan proposed for the purpose of maintaining the user's health.

[0176] "Physiological data" refers to data on the user's physical physiological condition, such as heart rate, body temperature, and blood pressure.

[0177] "Health Data" refers to data reflecting a user's overall health, including diet, exercise, sleep, and physiological data.

[0178] "Real-time" refers to data being collected and analyzed immediately, with results being provided almost instantly.

[0179] "Alert" refers to a notification sent to alert you based on the results of an analysis of your health data.

[0180] "Health advice" refers to specific advice for maintaining health provided to a user based on analyzed health data.

[0181] "Factory workers" refers to employees who work in a particular industrial facility or manufacturing site.

[0182] "Health status" refers to the overall physical and mental health of a user.

[0183] "Health management" refers to actions and plans to maintain and improve a user's health.

[0184] "Server" refers to a computer system that collects and analyzes data and provides information to users.

[0185] System Overview

[0186] This system allows users to easily record their diet and exercise information and provides them with advice on health management. By using a smartphone or smartwatch, the system automatically collects diet and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus.

[0187] Hardware and software used

[0188] Smartphone: A device that allows users to take pictures of their meals and upload the images to a server.

[0189] Smartwatch: A device that automatically collects a user's exercise and sleep information.

[0190] Server: A central computer system that performs image analysis, stores data, and performs AI-based analysis and proposals.

[0191] AI model: A generative AI model is used to analyze food images and generate optimal health maintenance menus based on individual health conditions.

[0192] Program processing overview

[0193] Input and analysis of dietary information

[0194] Users take photos of their daily meals with their smartphones and use a dedicated app to upload the images to a server. The server also receives image metadata (such as the date and time the photo was taken, location information, etc.) and passes the image files to an AI model. The AI ​​model analyzes the images and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The results of this analysis are stored in a database for later analysis.

[0195] Acquiring exercise and sleep information

[0196] Smartwatches automatically collect data on the user's exercise and sleep. For example, they record information such as the number of steps taken, distance traveled, heart rate, and sleep time, and periodically send it to a server. The server stores the received exercise and sleep information in a database, enabling the user to grasp their overall health status.

[0197] Suggestions for maintaining good health

[0198] The server uses an AI model to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to assess the user's health status. Based on this assessment, it proposes specific diet and exercise menus to help the user maintain their health.

[0199] Support for implementing proposals (future enhancements)

[0200] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0201] Specific examples

[0202] Assume an employee is wearing a smartwatch, which collects data such as heart rate, steps, and sleep time, and sends it to a server as follows:

[0203] Heart rate: 80 BPM

[0204] Steps: 6700

[0205] Sleep time: 7.2 hours

[0206] Based on this data, the user's health status can be assessed and prompts such as the following can be fed into a generative AI model:

[0207] The user has a heart rate of 80, steps taken 6700, and 7.2 hours of sleep. What health advice should be provided?

[0208] Based on the analysis results, the generative AI model provides specific advice such as, "Today, we recommend that you maintain moderate exercise and eat a vegetable-based diet."

[0209] As described above, this system makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] Users take photos of their meals and upload the images on the spot.

[0213] Input: A user takes a photo of their meal using their smartphone.

[0214] How it works: The user launches a camera app and takes a picture of their meal. After taking the picture, they upload the image to the server via a dedicated app. The server receives the image along with metadata such as the date and time the photo was taken and its location.

[0215] Output: Image data and metadata uploaded to the server.

[0216] Step 2:

[0217] The server analyzes the received image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[0218] Input: Image data uploaded to the server.

[0219] How it works: The server passes the image to the AI ​​model, which recognizes the food and calculates its ingredients, the three major nutrients (protein, fat, carbohydrates) and calories.

[0220] Output: Ingredients, macronutrients, and calorie information for analyzed ingredients.

[0221] Step 3:

[0222] The server stores the analysis results in a database.

[0223] Input: Ingredients, macronutrients, and calorie information.

[0224] Specific operation: The server writes the analysis results to a database, which is used for future analyses required for health management.

[0225] Output: Analysis results stored in a database.

[0226] Step 4:

[0227] The smartwatch collects the user's exercise and sleep information.

[0228] Input: User's exercise and sleep information.

[0229] How it works: The smartwatch uses sensors to record information such as heart rate, steps, distance traveled, and sleep time in real time, and periodically transmits the recorded data to a server via the smartphone.

[0230] Output: Exercise and sleep data sent to the server.

[0231] Step 5:

[0232] The server stores the collected exercise and sleep information in a database.

[0233] Input: Exercise and sleep information sent from your smartwatch.

[0234] What it does: The server stores the received exercise and sleep information in a database, which allows it to understand the user's overall health.

[0235] Output: Exercise and sleep data stored in a database.

[0236] Step 6:

[0237] Based on the data collected by the server, the system evaluates the user's health condition and suggests an appropriate health maintenance menu.

[0238] Input: Food data, exercise data, sleep data.

[0239] Specific operation: The server uses an AI model to analyze past data and trends stored in the database and evaluate the user's health condition. The AI ​​model uses a generative AI model to generate optimal meal and exercise menus based on the user's health condition. It also generates specific health advice by inputting a prompt: "The user's heart rate is X, number of steps is Y, and sleep time is Z hours. What kind of health advice should be provided?"

[0240] Output: Health maintenance menu and advice suggested to the user.

[0241] Step 7:

[0242] The server sends alerts based on the analysis of health data and provides appropriate health advice.

[0243] Input: Alert information and advice generated based on the user's health status.

[0244] How it works: The server generates alerts when user-defined criteria are exceeded and sends notifications via smartphone, including alerts for excessive heart rate or insufficient sleep, along with health advice generated by the AI ​​model.

[0245] Output: Alert notification and health advice to the user.

[0246] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0247] The present invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice necessary for health management. Furthermore, the present invention also includes a feature that incorporates an emotion engine that recognizes the user's emotions. Specifically, the system utilizes a smartphone or smartwatch to automatically acquire dietary and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus. The emotion engine also recognizes the user's emotions and improves the suggestions based on those information.

[0248] System configuration and operation

[0249] Input and analysis of dietary information

[0250] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[0251] The server then passes the image file to the analysis module, where AI analyzes the image to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The analysis results are stored in a database for later analysis.

[0252] Acquiring exercise and sleep information

[0253] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information, such as the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server.

[0254] The server stores the received exercise and sleep information in a database, allowing the user to understand their overall health condition.

[0255] Acquisition and analysis of emotional information

[0256] Users wear smartphones or smartwatches and record emotional information. In particular, emotions are recognized in real time through facial recognition and voice analysis. For example, facial expressions are read using a smartphone camera, and the emotion engine analyzes the emotional state based on that data.

[0257] The device transmits emotional information to a server, so that the user's emotional state is also used as part of the health analysis.

[0258] Suggestions for maintaining good health

[0259] The server passes the accumulated dietary, exercise, sleep, and emotional data to the AI ​​for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, the AI ​​proposes specific dietary and exercise menus for the user to maintain their health.

[0260] For example, if the server determines that the user has not been exercising recently and is under stress, it can suggest that the user "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in the next dinner." If the user's nutritional intake is unbalanced, the server can also suggest "a balanced dish for the next meal."

[0261] Support for implementing proposals (future enhancements)

[0262] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0263] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management that also takes emotional states into consideration.

[0264] The processing flow will be explained below.

[0265] Step 1:

[0266] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[0267] Step 2:

[0268] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[0269] Step 3:

[0270] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[0271] Step 4:

[0272] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[0273] Step 5:

[0274] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[0275] Step 6:

[0276] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[0277] Step 7:

[0278] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[0279] Step 8:

[0280] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[0281] Step 9:

[0282] Users wear smartphones or smartwatches that record emotional information, and in particular, recognize emotions in real time through facial recognition and voice analysis.

[0283] Step 10:

[0284] The device sends the emotion information to the server, along with the emotion data processed by the emotion engine.

[0285] Step 11:

[0286] The server integrates dietary, exercise, and sleep data, including emotional information, and passes it on to the AI ​​for analysis. The AI ​​analyzes past data and trends to provide a comprehensive assessment of the user's health and emotional state.

[0287] Step 12:

[0288] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in your next dinner."

[0289] Step 13:

[0290] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[0291] Step 14:

[0292] (as a future extension)

[0293] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[0294] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management that also takes emotional state into account.

[0295] Example 2

[0296] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] Modern society demands comprehensive health management that takes into account individual lifestyle habits and emotional states. However, conventional approaches only collect and analyze individual data (e.g., diet, exercise, sleep), making it difficult to achieve comprehensive health management that also includes emotional information. Furthermore, there is a lack of implementation support to determine whether health management proposals are feasible.

[0298] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of their meal and upload the image on the spot, a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients, and a means for recording the user's emotional information in real time using a smartphone or smartwatch and analyzing it with an emotion engine. This allows for appropriate health management suggestions that take into account the user's overall health condition and also include emotional information. Furthermore, by providing support for implementing the suggested menu, more effective health management is achieved.

[0299] A "user" is an individual who uses this system to record and manage information about their diet, exercise, sleep, and emotions.

[0300] A "terminal" is a device (e.g., a smartphone or smartwatch) that a user uses to take photos of their meals and record exercise information, sleep information, and emotional information.

[0301] The "server" is a central system that receives data sent from the terminals, analyzes it, stores it in a database, and makes suggestions for health management.

[0302] "Upload" is the operation or process of sending data from a terminal to a server.

[0303] "Image analysis" is a process that uses AI technology to analyze uploaded food images and calculate the ingredients, three major nutrients, and calories of the ingredients.

[0304] The "three macronutrients" are protein, fat, and carbohydrates, which are the main components of food.

[0305] A "health maintenance menu" is a meal and exercise plan proposed to maintain or improve the user's health.

[0306] An "emotion engine" is an algorithm or software for analyzing a user's emotional information.

[0307] "Emotional information" is data that indicates the user's current mood or emotional state, and is collected, for example, by facial expression recognition or voice analysis.

[0308] The "database" is a collection of structured data that allows for efficient storage and management of collected dietary, exercise, sleep, and emotional information.

[0309] "Location information" is data that indicates a user's current geographic location.

[0310] "Health status" is comprehensive information that indicates the physical and mental state of the user.

[0311] "Artificial intelligence" is a technology that learns from a user's past health and emotional state data and generates optimal suggestions.

[0312] "Commercial facilities" are stores that sell food ingredients and health-related products, and facilities that provide food and beverage services.

[0313] This invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice on health management. This system also includes a function for recording and analyzing emotional information in real time. Using devices such as smartphones and smartwatches, the system automatically acquires dietary and exercise information and analyzes it using artificial intelligence (AI) to suggest appropriate health maintenance menus. It also uses an emotion engine to recognize the user's emotions and improve the suggestions based on those emotions.

[0314] Input and analysis of dietary information

[0315] Users take photos of their daily meals with their smartphone and upload the images to a server using a dedicated app. When the smartphone transfers the captured image to the server, it also sends image metadata (e.g., the date and time the image was taken, location information, etc.). The server then passes the received image file to an AI analysis module. The AI ​​analysis module uses image recognition technology (e.g., YOLO or ResNet) to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The analysis results are stored in a database and can be used for later analysis.

[0316] Example: A user takes a photo of the bread and coffee they had for breakfast with their smartphone and uploads the image. The server receives the image, and an AI analysis module automatically analyzes the type of bread and coffee, their nutritional content, and their calories. The analysis results are stored in a database.

[0317] Example prompt sentence:

[0318] Can you analyze a photo of my breakfast and give me calorie and nutrition information?

[0319] Acquiring exercise and sleep information

[0320] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information. The collected data includes the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server. The server stores the received exercise and sleep information in a database and uses it as basic data for understanding the user's overall health condition.

[0321] Example: A user wears a smartwatch and goes for a run. After the run, the smartwatch records the number of steps, distance traveled, and heart rate, and automatically sends this data to a server. The server stores this data in a database for later analysis.

[0322] Example prompt sentence:

[0323] Record your running data today and let me know your total steps and calories burned.

[0324] Acquisition and analysis of emotional information

[0325] Users record emotional information using their smartphones or smartwatches. In particular, emotions are recognized in real time through facial recognition and voice analysis. Using the smartphone's camera and microphone, the emotion engine analyzes facial expressions and vocal states to obtain emotional state data. The emotional information collected by the device is sent to a server, and the user's emotional state is also used as part of health analysis.

[0326] Example: A user uses a smartphone and the emotion engine analyzes facial expressions captured by the camera during a video chat, records the user's emotional state (e.g., joy, sadness, stress), and sends it to a server.

[0327] Example prompt sentence:

[0328] Please analyze my current emotional state and let me know.

[0329] Suggestions for maintaining good health

[0330] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI analysis module for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, it suggests specific meal and exercise menus to maintain the user's health. For example, if it determines that the user has not been exercising recently and is under stress, the server might suggest "doing 30 minutes of yoga to relieve stress" or "having some relaxing herbal tea with your next dinner." It can also suggest "a balanced meal for your next meal" if the user's nutrition is unbalanced.

[0331] Example: The server determines that the user is under stress based on dietary, exercise, and emotional data. The AI ​​generates a suggestion, such as "Drink some relaxing herbal tea with your next dinner," and notifies the user.

[0332] Example prompt sentence:

[0333] Based on your recent exercise and emotional data, we'll suggest what you should do to take care of your health.

[0334] Support for implementing proposals (future enhancements)

[0335] The server uses the user's location information to support the execution of the suggestions by providing information on nearby commercial facilities and restaurants. For example, it provides the user with information on inventory at the nearest commercial facility where the suggested ingredients can be purchased, or information on restaurants that offer the suggested menu.

[0336] Example: When a user tries to purchase suggested ingredients, the server provides inventory information from the nearest supermarket, helping the user to shop efficiently.

[0337] Example prompt sentence:

[0338] Could you please tell me the nearest supermarket where I can buy the ingredients you suggested?

[0339] In this way, this system is closely integrated into the user's daily life, collecting and analyzing data on diet, exercise, sleep, and emotions, and supporting comprehensive, personalized health management.

[0340] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0341] Step 1: Photograph and upload your meal

[0342] Users take pictures of their daily meals with their smartphones and upload the images to a server using a dedicated app.

[0343] Input: Photographed food image, metadata (photo date and time, location information)

[0344] Output: Food images and metadata sent to the server

[0345] How it works: The user takes a photo of their breakfast and uploads it through a dedicated app. The image is accompanied by information about the date, time, and location of the photo.

[0346] Step 2: Analyze the images

[0347] The server passes the uploaded meal image to an AI analysis module, which analyzes the image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[0348] Input: Uploaded food image and metadata

[0349] Output: Analysis results of ingredients, three major nutrients, and calories

[0350] Specific operation: The server passes the image to the AI ​​analysis module, which analyzes the type of ingredients and their nutritional content.

[0351] Step 3: Saving to the database

[0352] The server stores the analysis results in a database, which includes information such as the ingredients, the ratio of the three major nutrients, and calories.

[0353] Input: Analysis results of ingredients, three major nutrients, and calories

[0354] Output: Analysis data stored in a database

[0355] Specific operation: The server stores the analysis results in a database, making them available for later analysis.

[0356] Step 4: Record your exercise and sleep information

[0357] The device (smartwatch) periodically collects the user's exercise data (number of steps, distance, heart rate) and sleep data (sleep time, sleep quality).

[0358] Input: User's exercise data, sleep data

[0359] Output: Collected exercise data, sleep data

[0360] Specific operation: The user wears the smartwatch and goes out for exercise. The smartwatch automatically records the number of steps taken and the amount of sleep.

[0361] Step 5: Send your exercise and sleep data

[0362] The terminal periodically transmits the collected exercise and sleep data to the server.

[0363] Input: Collected exercise data, sleep data

[0364] Output: Exercise data and sleep data sent to the server

[0365] Specific operation: The smartwatch periodically sends exercise and sleep data to the server.

[0366] Step 6: Save to database

[0367] The server stores the received exercise and sleep data in a database, allowing the user to understand their overall health status.

[0368] Input: Exercise data and sleep data sent to the server

[0369] Output: Exercise data and sleep data stored in a database

[0370] Specific operation: The server stores the received exercise and sleep data in a database and uses it to comprehensively understand the user's health condition.

[0371] Step 7: Recording Emotional Information

[0372] Users record their emotional information using their smartphones or smartwatches, and emotions are recognized in real time through facial recognition and voice analysis.

[0373] Input: User's facial expression data, voice data

[0374] Output: Recorded emotion information

[0375] Specific operation: The user engages in a video chat, and facial expressions are captured on camera and analyzed by the emotion engine.

[0376] Step 8: Sending Emotional Information

[0377] The terminal transmits the collected emotion information to the server.

[0378] Input: Recorded emotion information

[0379] Output: Emotion information sent to the server

[0380] Specific operation: The smartphone sends the recorded emotional information to the server.

[0381] Step 9: Analyze and store emotional information

[0382] The server passes the received emotional information to an analysis module and stores the emotional state in a database.

[0383] Input: Emotion information sent to the server

[0384] Output: Emotional state data stored in a database

[0385] Specific operation: The emotion engine analyzes the received emotion information and stores the results in a database.

[0386] Step 10: Comprehensive data analysis

[0387] The server passes dietary, exercise, sleep, and emotional data to an AI analysis module to evaluate the user's health and emotional state.

[0388] Input: Various data stored in the database (diet, exercise, sleep, emotions)

[0389] Output: Assessment of the user's health and emotional state

[0390] Specific operation: The server analyzes data on diet, exercise, sleep, and emotions to assess overall health.

[0391] Step 11: Proposal for a health maintenance menu

[0392] Based on the analysis results, the server suggests specific meal and exercise menus to the user to maintain their health.

[0393] Input: User's health and emotional state assessment results

[0394] Output: Generated health maintenance menu

[0395] Specific behavior: Based on the user's health and emotional state, the AI ​​generates suggestions such as "do 30 minutes of yoga to relieve stress" or "have some relaxing herbal tea with dinner."

[0396] Step 12: Submit your proposal

[0397] The server transmits the generated proposal content to the terminal and notifies the user.

[0398] Input: Generated health maintenance menu

[0399] Output: The suggestion sent to the user

[0400] Specific operation: The generated health suggestions are displayed on the smartphone and the user is notified.

[0401] Step 13: Proposal implementation support

[0402] The server provides information on nearby commercial facilities and restaurants based on the user's location information, and supports the implementation of the suggestions.

[0403] Input: User's location information, suggestions

[0404] Output: Information to support the execution of the proposal (e.g., food inventory information and restaurant information)

[0405] Specific operation: The server provides inventory information of the nearest commercial establishment where the suggested ingredients can be purchased, helping the user to efficiently execute the suggested menu.

[0406] (Application example 2)

[0407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0408] In modern self-driving vehicles, health management of drivers and passengers is not sufficiently considered, making it difficult to maintain good health, especially during long periods of driving or riding. In addition, conventional health management systems do not take emotional states into account, which means they are unable to provide health recommendations that address stress and emotional fluctuations. The present invention aims to provide a comfortable and healthy driving environment by efficiently managing the user's health in a self-driving vehicle and providing health recommendations that address emotional fluctuations.

[0409] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients; a means for analyzing previously registered meal contents and trends and suggesting a meal menu necessary for maintaining health; a means for acquiring exercise information and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for integrating food records, exercise information, and emotional information into the internal information system of the autonomous vehicle to monitor the health status in real time and provide personalized health advice; and a means for acquiring the emotional state of passengers using an in-vehicle camera and voice recognition system and adjusting health advice based on the emotions. This not only enables health management of users in the vehicle but also makes it possible to provide advice that takes emotional state into consideration.

[0410] "User" refers to an individual who uses the system, particularly for the purpose of health management within an autonomous vehicle.

[0411] "Meal contents" refers to the overall meal that the user ingests, and specifically includes information such as ingredients, the three major nutrients, and calories.

[0412] "Images" refers to visual data such as photos or videos taken by the user of their meal.

[0413] "Means for uploading" refers to the function for sending images taken by the user to the cloud or server.

[0414] "Means for analyzing" refers to the function of analyzing uploaded images and calculating the ingredients, three major nutrients, and calories of ingredients.

[0415] "Ingredients" refers to the nutrients and components contained in each food, specifically vitamins and minerals.

[0416] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[0417] A "calorie" is a unit that indicates the amount of energy contained in food.

[0418] "Means for suggesting meal menus" refers to a function that presents meal menus suitable for the user based on the analyzed meal contents.

[0419] "Exercise information" refers to information such as the amount of exercise, type of exercise, and time of exercise performed by the user.

[0420] "Sleep information" refers to information related to the user's sleep time, sleep quality, and the like.

[0421] "Means of acquisition" refers to the ability to collect data from devices such as smartphones and smartwatches.

[0422] The "means for suggesting an exercise menu" refers to a function that suggests the type and amount of exercise suitable for the user based on the acquired exercise information.

[0423] "Autonomous vehicle" refers to a vehicle that has the ability to drive itself.

[0424] "Internal information system" refers to a system installed within an autonomous vehicle for managing and analyzing healthcare-related data.

[0425] "Means for monitoring health status" refers to a function that integrates food records, exercise information, and emotional information to monitor the user's health status in real time.

[0426] "Means for providing personalized health advice" refers to a function that makes personalized health management suggestions based on user data.

[0427] An "in-car camera" refers to a camera installed inside a vehicle that captures the user's facial expressions and behavior.

[0428] A "voice recognition system" refers to a system that has the function of analyzing a user's speech and extracting linguistic information.

[0429] "Means for acquiring emotional state" refers to the function of assessing the user's emotions using an in-car camera or voice recognition system.

[0430] "Means for adjusting health suggestions based on emotions" refers to a function that uses the results of emotion analysis to modify the suggestions to suit the user's emotional state.

[0431] The present invention relates to a system for managing the health of users in autonomous vehicles. Specifically, the system allows users to photograph and record their meals, acquire exercise information, sleep information, and emotional state, and provide health advice based on this data. This allows users to maintain their health even during long trips in the car.

[0432] Hardware Configuration

[0433] The main hardware configuration of this system is as follows:

[0434] Autonomous vehicle internal information system: This system comprehensively manages various data necessary for health management.

[0435] Smartphones and smartwatches: Users use these devices to capture photos of their meals and to obtain exercise and sleep information.

[0436] In-car camera: Used to analyze the user's facial expressions and capture their emotional state.

[0437] Speech recognition system: Analyzes the user's voice and complements their emotional state.

[0438] Software Configuration

[0439] The software configuration is as follows:

[0440] AI-based analysis module: The present invention uses machine learning frameworks such as TensorFlow and Keras.

[0441] Emotion recognition engine: A model (e.g. TensorFlow / Keras) for analyzing the user's facial expressions and recognizing their emotional state.

[0442] Nutrition Analysis API: Analyzes uploaded food images and extracts nutritional information.

[0443] How it works

[0444] The operation of this system is as follows.

[0445] 1. Entering and analyzing dietary information

[0446] The user takes a photo of their meal using a smartphone, and the image is automatically uploaded to the autonomous vehicle's internal information system.

[0447] The uploaded image is passed to an AI-based analysis module that calculates the ingredients, macronutrients, and calories of the food, and the results are stored in a database for later analysis.

[0448] 2. Acquisition of exercise and sleep information

[0449] Smartphones and smartwatches automatically collect users' exercise and sleep information, including steps, distance, heart rate, and sleep duration.

[0450] The collected data is periodically transmitted to the autonomous vehicle's internal information system and stored in a database.

[0451] 3. Acquisition and analysis of emotional information

[0452] The user's emotional state is captured in real time using an in-car camera and a voice recognition system: the camera reads the user's facial expressions, and the voice recognition system analyzes the user's voice.

[0453] An emotion recognition engine evaluates the user's emotional state and stores it in a database.

[0454] 4. Suggestions for maintaining good health

[0455] The internal information system of the autonomous vehicle passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis.

[0456] The AI ​​analyzes past data and trends to assess the user's health and emotional state, and based on this assessment, suggests specific diet and exercise regimes to help maintain health.

[0457] Specific examples

[0458] For example, if a user is eating lunch in the car, an image of the meal is taken and a nutritional analysis is performed. If the analysis results indicate that the meal is high in calories, appropriate nutritional supplements and advice about the next meal are provided. At the same time, if facial recognition detects that the user's emotion is "sad," suggestions for breaks and relaxation methods to relieve stress are made.

[0459] Prompt Sentence Examples

[0460] "Take a picture of the user eating in the car and provide health advice based on the analysis results."

[0461] In this way, this system provides comprehensive support for health management in the car and offers advice that takes into account the driver's emotional state.

[0462] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0463] Step 1:

[0464] A user takes a photo of their meal using their smartphone and uploads the image to the server. The input is the image of the meal, and the output is the uploaded image data. Specifically, the user launches the smartphone's camera app, takes a photo of the meal, and then transfers the image file to the cloud server via a dedicated app.

[0465] Step 2:

[0466] The server receives the uploaded images and passes them to the analysis module. The input is the uploaded image data, and the output is the transmission of the image data to obtain the analysis results. The server also sends metadata (e.g., shooting date and time, location information, etc.) along with the image file to the analysis module.

[0467] Step 3:

[0468] The analysis module uses the received image to calculate the ingredients, three major nutrients, and calories of the ingredients. The input is image data, and the output is ingredient information, nutrient information, and calorie calculation data. Specifically, it uses AI frameworks such as TensorFlow and Keras to recognize ingredients from the image and calculate the corresponding ingredients and nutritional information based on a database.

[0469] Step 4:

[0470] The server stores the nutrition information received from the analysis module in a database. The input is the analysis result data, and the output is a new entry in the database. The server organizes this data by user and stores it for later analysis.

[0471] Step 5:

[0472] The device acquires exercise and sleep information from a smartphone or smartwatch and sends it to a server. The input is exercise and sleep information from the device, and the output is data sent to the server. Specifically, the device periodically collects data such as the number of steps taken, heart rate, and sleep status, and sends it to a cloud server.

[0473] Step 6:

[0474] The server stores the exercise and sleep information in a database. The input is the submitted exercise and sleep information, and the output is a new entry in the database. The server manages this data for each user and uses it for comprehensive health assessment.

[0475] Step 7:

[0476] The in-car camera and voice recognition system capture the user's emotional information in real time and send it to the server. The input is the user's facial expression and voice data, and the output is the analysis result of the emotional state. Specifically, the camera captures the facial expression, the voice recognition system analyzes the voice, and the data is sent to the emotion recognition engine.

[0477] Step 8:

[0478] The server receives the emotional state from the emotion recognition engine and stores it in a database. The input is the emotional state data, and the output is a new entry in the database. This allows emotional information to be used as part of health analysis.

[0479] Step 9:

[0480] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis. These data are the input, and the output is a comprehensive health assessment and advice suggestion data. The AI ​​module analyzes this data comprehensively and evaluates the user's health condition.

[0481] Step 10:

[0482] Based on the results of the AI ​​analysis, the server proposes specific meal and exercise menus to the user to maintain their health. The input is the AI ​​analysis results, and the output is the proposals. The server displays these proposals on the user's smartphone or in-car display, providing them in an easy-to-follow format.

[0483] In this way, the system of the present invention automates the entire process from data collection to analysis and proposals, thereby supporting the user's health management.

[0484] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0485] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0486] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0487] [Second embodiment]

[0488] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0489] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0490] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0491] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0492] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0493] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0494] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0495] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0496] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0497] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0498] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0499] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0500] This invention relates to a system that allows users to easily record their dietary and exercise information and provides necessary advice for health management. Specifically, it utilizes a smartphone or smartwatch to automatically collect dietary and exercise information, analyzes it using artificial intelligence (AI), and proposes an appropriate health maintenance menu.

[0501] System configuration and operation

[0502] Input and analysis of dietary information

[0503] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[0504] The server then passes the image file to an artificial intelligence module, which analyzes the image and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The results of this analysis are stored in a database for later analysis.

[0505] Acquiring exercise and sleep information

[0506] The device (smartphone or smartwatch) automatically collects data about the user's exercise and sleep, such as the number of steps, distance, heart rate, and sleep time, and periodically transmits this data to a server.

[0507] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[0508] Suggestions for maintaining good health

[0509] The server uses AI to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to evaluate the user's health. Based on this evaluation, it proposes specific diet and exercise menus to help the user maintain their health.

[0510] For example, if the server determines that the user has not been exercising recently, it can suggest that the user "go for a 30-minute jog next weekend." If the user's nutritional intake is unbalanced, the server can recommend that the user "eat more fish and vegetables at your next dinner."

[0511] Support for implementing proposals (future enhancements)

[0512] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0513] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[0514] The processing flow will be explained below.

[0515] Step 1:

[0516] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[0517] Step 2:

[0518] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[0519] Step 3:

[0520] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[0521] Step 4:

[0522] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[0523] Step 5:

[0524] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[0525] Step 6:

[0526] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[0527] Step 7:

[0528] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[0529] Step 8:

[0530] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[0531] Step 9:

[0532] The server passes the accumulated dietary, exercise, and sleep data to the AI ​​for analysis, which then analyzes past data and trends to assess the user's health.

[0533] Step 10:

[0534] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "go for a 30-minute jog next weekend" or "eat more fish and vegetables at your next dinner."

[0535] Step 11:

[0536] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[0537] Step 12:

[0538] (as a future extension)

[0539] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[0540] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management.

[0541] Example 1

[0542] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0543] In modern society, health management has become an important issue, but it is difficult to accurately record diet and exercise habits in a busy lifestyle. Furthermore, there are only a limited number of systems that comprehensively evaluate health status and provide specific recommendations tailored to individual users. In conventional systems, analyzing dietary content and collecting exercise and sleep information is time-consuming, and it is difficult to comprehensively analyze this data and provide appropriate advice. In particular, flexible recommendations utilizing the user's location information have not been realized. There is a need to improve these issues and provide a system that allows users to easily manage their health.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0545] In this invention, the server includes a means for users to photograph their meal and upload the images on the spot; a means for analyzing the uploaded images and calculating the ingredients, three major nutrients, and calories; a means for analyzing previously registered meal contents and trends and suggesting meal menus necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or wearable device; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for analyzing images using an artificial intelligence module and storing them in a database; and a means for comprehensively analyzing the user's data, evaluating their health status, and providing advice. This allows users to easily analyze the photographed meal contents, centrally manage their exercise and sleep information, and receive specific suggestions based on their individual health status. Furthermore, a function for assisting users in implementing the suggested menus can be provided using the user's location information, making it easier for users to carry out health-maintaining activities.

[0546] "User" refers to an individual who uses this system to record dietary, exercise, and sleep information and manage their health.

[0547] "Means for taking photos of meal contents and uploading images on the spot" refers to a function that allows users to take photos of their meal using a smartphone camera or a dedicated app and send the image data to a server in real time.

[0548] "Means for analyzing uploaded images and calculating the ingredients, three major nutrients, and calories" refers to an algorithm in which an artificial intelligence module running on a server analyzes the meal images sent by users and calculates the ingredients, proteins, fats, carbohydrates, and calories contained in the food.

[0549] "A means for analyzing previously registered dietary content and trends and proposing dietary menus necessary for maintaining health" refers to a function that suggests dietary menus suited to the user's current health condition based on the user's past dietary data stored in a database.

[0550] "Means for acquiring exercise and sleep information from a smartphone or wearable device" refers to the function of a smartphone or smartwatch worn by a user to collect data such as the number of steps taken during exercise, distance traveled, heart rate, and sleep time, and transmit this data to a server.

[0551] "Means of analyzing the amount of exercise required to maintain health based on the acquired information and proposing an appropriate exercise menu" refers to the function in which the server analyzes exercise and sleep information, and generates and presents the optimal exercise menu for the user.

[0552] "Image analysis using an artificial intelligence module and storage in a database" refers to the function in which an AI running on a server analyzes food images and stores the results in a database.

[0553] "Means for comprehensively analyzing user data, assessing health status, and providing advice" refers to the function of comprehensively analyzing dietary information, exercise information, and sleep information collected by the server, assessing the user's health status, and providing specific advice for maintaining good health.

[0554] MODE FOR CARRYING OUT THE INVENTION

[0555] The present invention relates to a system that allows users to efficiently and easily record their daily diet and exercise information and perform comprehensive health management. This system utilizes smartphones and wearable devices, analyzes data using artificial intelligence (AI), and proposes appropriate health maintenance menus to users.

[0556] Hardware and software used

[0557] Hardware:

[0558] Smartphone

[0559] Wearable devices such as smartwatches

[0560] server

[0561] software:

[0562] Dedicated application (for smartphones)

[0563] Server-side artificial intelligence module

[0564] Database System

[0565] Specific explanation of the system's operation

[0566] Users use a smartphone with a dedicated app installed to take pictures of their daily meals. At this time, they can also enter detailed information about the meal and the types of ingredients. The photographed food images are uploaded to a server via the smartphone. At this time, metadata such as the date and time of the photo and location information are also sent along with the image.

[0567] The server passes the received image file to an artificial intelligence module. The AI ​​analyzes the image, identifies the ingredients in the image, and calculates the protein, fat, carbohydrates, and calories of each ingredient. This data is stored in a database. For example, if you take a photo and upload a meal of bread, eggs, and bananas for breakfast, the AI ​​will automatically generate and record the following: "Bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "Egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "Banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[0568] The devices, such as smartphones and smartwatches, automatically collect users' exercise information (e.g., number of steps, distance traveled, heart rate) and sleep information. This data is periodically sent to a server and stored in a database.

[0569] The server comprehensively analyzes the collected dietary, exercise, and sleep data to evaluate the user's health condition. Based on this evaluation, it suggests appropriate meal and exercise menus to the user. For example, if it determines that the user has not been exercising recently, the server will suggest "go for a 30-minute jog next weekend." If the user's nutrition is unbalanced, it will recommend "eat more fish and vegetables at your next dinner."

[0570] Some examples of specific prompts include:

[0571] "I had bread, eggs, and a banana for breakfast. What are the calories and nutrients in this meal?"

[0572] In the future, the server may use the user's location information to link inventory information from nearby commercial facilities and restaurant menus to support the implementation of suggested menus, making it easier for users to implement suggested health maintenance menus and supporting continuous health management.

[0573] As described above, the system of the present invention enables the user to efficiently manage their health and supports continuous health maintenance activities.

[0574] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0575] Step 1:

[0576] The user launches the dedicated application and takes a photo of the meal with their smartphone. The user also has the option to input ingredients and meal details. The input data is the image of the meal and text information about the ingredients.

[0577] Input: Meal image, ingredient information

[0578] Output: Food image data and ingredient information stored on the smartphone

[0579] Specific behavior:

[0580] For breakfast, users take a photo of bread, eggs, and bananas and upload it to the app.

[0581] Step 2:

[0582] The device (smartphone) transmits the captured food image and the input ingredient information to the server. This transmission includes metadata such as the date and time of the image capture and location information along with the image data.

[0583] Input: Food image data, ingredient information, metadata

[0584] Output: Food image data, ingredient information, and metadata sent to the server

[0585] Specific behavior:

[0586] The smartphone captures a photo of the meal and information about the ingredients, and then sends the photo, information, and metadata all at once to the server.

[0587] Step 3:

[0588] The server then passes the image files to an artificial intelligence module, which uses image analysis technology to identify the ingredients in the image and calculate the protein, fat, carbohydrates, and calories of each ingredient. The analysis results are then stored in a database.

[0589] Input: Submitted food image data, metadata

[0590] Output: Ingredient data, nutrient data, and calorie data for each ingredient stored in the database

[0591] Specific behavior:

[0592] The AI ​​analyzes the image and generates and stores data such as "bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[0593] Step 4:

[0594] The device (smartphone or wearable device) automatically collects the user's exercise and sleep information. The collected data includes the number of steps taken, distance traveled, heart rate, and sleep time. This data is sent to a server at regular intervals.

[0595] Input: User's exercise and sleep data

[0596] Output: Exercise and sleep data sent to the server

[0597] Specific behavior:

[0598] The smartwatch monitors your heart rate 24 hours a day and sends your step count and sleep data to a server at the end of the day.

[0599] Step 5:

[0600] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[0601] Input: exercise data, sleep data sent

[0602] Output: Exercise data and sleep data stored in a database

[0603] Specific behavior:

[0604] The server records the received exercise data in a database and stores the data by day, week, or month.

[0605] Step 6:

[0606] The server uses AI to perform a comprehensive analysis of the collected dietary, exercise, and sleep data. Based on the results of this analysis, the server evaluates the user's health status and suggests specific diet and exercise menus.

[0607] Input: Food data, exercise data, and sleep data in the database

[0608] Output: Proposal of a health maintenance menu for the user

[0609] Specific behavior:

[0610] The server generates a health maintenance menu such as, "Based on your recent data, you have continued to lack exercise, so we recommend that you go for a 30-minute jog next weekend. Also, your nutritional intake tends to be unbalanced, so we recommend that you eat more fish and vegetables for your next dinner," and notifies the user.

[0611] Step 7:

[0612] In the future, the server will use the user's location information to provide a function that links inventory information from nearby commercial facilities and menus offered by restaurants to support the implementation of suggested menus.

[0613] Input: User's location information, suggested health menu

[0614] Output: Information about commercial facilities and restaurants based on the user's location

[0615] Specific behavior:

[0616] The server obtains the user's location information and provides information on stock availability at nearby supermarkets where the suggested ingredients can be purchased, as well as information on restaurants that offer the suggested menu items.

[0617] (Application example 1)

[0618] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] Conventional health management systems require users to manually input information, posing challenges in terms of time and accuracy. Furthermore, there was a lack of systems that could comprehensively grasp the health status of employees in specific work environments, such as factories, and provide optimal health management plans. As a result, there is a demand for ways to maintain employee health and improve work efficiency.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0621] In this invention, the server includes: a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of ingredients; a means for analyzing previously registered meal contents and trends and proposing a meal menu necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and proposing an appropriate exercise menu; a means for automatically collecting health data including the user's physiological data and analyzing it in real time; a means for sending alerts and providing appropriate health advice based on the results of the health data analysis; and a means for evaluating the health status of factory workers and supporting appropriate health management. This eliminates the need for users to manually input information and enables real-time understanding and evaluation of health status, thereby improving the health of factory workers and improving work efficiency.

[0622] "User" refers to a person who uses the system to manage their own health condition.

[0623] "Meal contents" refers to all of the foods and menu items consumed by the user.

[0624] "Image" refers to a photograph of the meal contents taken by the user using a smartphone or other photographic device.

[0625] "Upload" refers to the act of sending an image taken by a user to a server.

[0626] "Food ingredients" refers to the nutritional and chemical constituents of the individual foods in a meal.

[0627] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[0628] A "calorie" is a unit of measurement that indicates the amount of energy contained in food, and refers to the energy provided by food.

[0629] "Exercise information" refers to exercise data during the user's daily activities, including the number of steps, heart rate, distance, and the like.

[0630] "Sleep information" refers to information such as the user's sleep patterns and sleep duration.

[0631] An "exercise menu" refers to an exercise plan proposed for the purpose of maintaining the user's health.

[0632] "Physiological data" refers to data on the user's physical physiological condition, such as heart rate, body temperature, and blood pressure.

[0633] "Health Data" refers to data reflecting a user's overall health, including diet, exercise, sleep, and physiological data.

[0634] "Real-time" refers to data being collected and analyzed immediately, with results being provided almost instantly.

[0635] "Alert" refers to a notification sent to alert you based on the results of an analysis of your health data.

[0636] "Health advice" refers to specific advice for maintaining health provided to a user based on analyzed health data.

[0637] "Factory workers" refers to employees who work in a particular industrial facility or manufacturing site.

[0638] "Health status" refers to the overall physical and mental health of a user.

[0639] "Health management" refers to actions and plans to maintain and improve a user's health.

[0640] "Server" refers to a computer system that collects and analyzes data and provides information to users.

[0641] System Overview

[0642] This system allows users to easily record their diet and exercise information and provides them with advice on health management. By using a smartphone or smartwatch, the system automatically collects diet and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus.

[0643] Hardware and software used

[0644] Smartphone: A device that allows users to take pictures of their meals and upload the images to a server.

[0645] Smartwatch: A device that automatically collects a user's exercise and sleep information.

[0646] Server: A central computer system that performs image analysis, stores data, and performs AI-based analysis and proposals.

[0647] AI model: A generative AI model is used to analyze food images and generate optimal health maintenance menus based on individual health conditions.

[0648] Program processing overview

[0649] Input and analysis of dietary information

[0650] Users take photos of their daily meals with their smartphones and use a dedicated app to upload the images to a server. The server also receives image metadata (such as the date and time the photo was taken, location information, etc.) and passes the image files to an AI model. The AI ​​model analyzes the images and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The results of this analysis are stored in a database for later analysis.

[0651] Acquiring exercise and sleep information

[0652] Smartwatches automatically collect data on the user's exercise and sleep. For example, they record information such as the number of steps taken, distance traveled, heart rate, and sleep time, and periodically send it to a server. The server stores the received exercise and sleep information in a database, enabling the user to grasp their overall health status.

[0653] Suggestions for maintaining good health

[0654] The server uses an AI model to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to assess the user's health status. Based on this assessment, it proposes specific diet and exercise menus to help the user maintain their health.

[0655] Support for implementing proposals (future enhancements)

[0656] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0657] Specific examples

[0658] Assume an employee is wearing a smartwatch, which collects data such as heart rate, steps, and sleep time, and sends it to a server as follows:

[0659] Heart rate: 80 BPM

[0660] Steps: 6700

[0661] Sleep time: 7.2 hours

[0662] Based on this data, the user's health status can be assessed and prompts such as the following can be fed into a generative AI model:

[0663] The user has a heart rate of 80, steps taken 6700, and 7.2 hours of sleep. What health advice should be provided?

[0664] Based on the analysis results, the generative AI model provides specific advice such as, "Today, we recommend that you maintain moderate exercise and eat a vegetable-based diet."

[0665] As described above, this system makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[0666] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0667] Step 1:

[0668] Users take photos of their meals and upload the images on the spot.

[0669] Input: A user takes a photo of their meal using their smartphone.

[0670] How it works: The user launches a camera app and takes a picture of their meal. After taking the picture, they upload the image to the server via a dedicated app. The server receives the image along with metadata such as the date and time the photo was taken and its location.

[0671] Output: Image data and metadata uploaded to the server.

[0672] Step 2:

[0673] The server analyzes the received image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[0674] Input: Image data uploaded to the server.

[0675] How it works: The server passes the image to the AI ​​model, which recognizes the food and calculates its ingredients, the three major nutrients (protein, fat, carbohydrates) and calories.

[0676] Output: Ingredients, macronutrients, and calorie information for analyzed ingredients.

[0677] Step 3:

[0678] The server stores the analysis results in a database.

[0679] Input: Ingredients, macronutrients, and calorie information.

[0680] Specific operation: The server writes the analysis results to a database, which is used for future analyses required for health management.

[0681] Output: Analysis results stored in a database.

[0682] Step 4:

[0683] The smartwatch collects the user's exercise and sleep information.

[0684] Input: User's exercise and sleep information.

[0685] How it works: The smartwatch uses sensors to record information such as heart rate, steps, distance traveled, and sleep time in real time, and periodically transmits the recorded data to a server via the smartphone.

[0686] Output: Exercise and sleep data sent to the server.

[0687] Step 5:

[0688] The server stores the collected exercise and sleep information in a database.

[0689] Input: Exercise and sleep information sent from your smartwatch.

[0690] What it does: The server stores the received exercise and sleep information in a database, which allows it to understand the user's overall health.

[0691] Output: Exercise and sleep data stored in a database.

[0692] Step 6:

[0693] Based on the data collected by the server, the system evaluates the user's health condition and suggests an appropriate health maintenance menu.

[0694] Input: Food data, exercise data, sleep data.

[0695] Specific operation: The server uses an AI model to analyze past data and trends stored in the database and evaluate the user's health condition. The AI ​​model uses a generative AI model to generate optimal meal and exercise menus based on the user's health condition. It also generates specific health advice by inputting a prompt: "The user's heart rate is X, number of steps is Y, and sleep time is Z hours. What kind of health advice should be provided?"

[0696] Output: Health maintenance menu and advice suggested to the user.

[0697] Step 7:

[0698] The server sends alerts based on the analysis of health data and provides appropriate health advice.

[0699] Input: Alert information and advice generated based on the user's health status.

[0700] How it works: The server generates alerts when user-defined criteria are exceeded and sends notifications via smartphone, including alerts for excessive heart rate or insufficient sleep, along with health advice generated by the AI ​​model.

[0701] Output: Alert notification and health advice to the user.

[0702] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0703] The present invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice necessary for health management. Furthermore, the present invention also includes a feature that incorporates an emotion engine that recognizes the user's emotions. Specifically, the system utilizes a smartphone or smartwatch to automatically acquire dietary and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus. The emotion engine also recognizes the user's emotions and improves the suggestions based on those information.

[0704] System configuration and operation

[0705] Input and analysis of dietary information

[0706] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[0707] The server then passes the image file to the analysis module, where AI analyzes the image to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The analysis results are stored in a database for later analysis.

[0708] Acquiring exercise and sleep information

[0709] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information, such as the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server.

[0710] The server stores the received exercise and sleep information in a database, allowing the user to understand their overall health condition.

[0711] Acquisition and analysis of emotional information

[0712] Users wear smartphones or smartwatches and record emotional information. In particular, emotions are recognized in real time through facial recognition and voice analysis. For example, facial expressions are read using a smartphone camera, and the emotion engine analyzes the emotional state based on that data.

[0713] The device transmits emotional information to a server, so that the user's emotional state is also used as part of the health analysis.

[0714] Suggestions for maintaining good health

[0715] The server passes the accumulated dietary, exercise, sleep, and emotional data to the AI ​​for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, the AI ​​proposes specific dietary and exercise menus for the user to maintain their health.

[0716] For example, if the server determines that the user has not been exercising recently and is under stress, it can suggest that the user "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in the next dinner." If the user's nutritional intake is unbalanced, the server can also suggest "a balanced dish for the next meal."

[0717] Support for implementing proposals (future enhancements)

[0718] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0719] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management that also takes emotional states into consideration.

[0720] The processing flow will be explained below.

[0721] Step 1:

[0722] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[0723] Step 2:

[0724] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[0725] Step 3:

[0726] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[0727] Step 4:

[0728] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[0729] Step 5:

[0730] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[0731] Step 6:

[0732] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[0733] Step 7:

[0734] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[0735] Step 8:

[0736] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[0737] Step 9:

[0738] Users wear smartphones or smartwatches that record emotional information, and in particular, recognize emotions in real time through facial recognition and voice analysis.

[0739] Step 10:

[0740] The device sends the emotion information to the server, along with the emotion data processed by the emotion engine.

[0741] Step 11:

[0742] The server integrates dietary, exercise, and sleep data, including emotional information, and passes it on to the AI ​​for analysis. The AI ​​analyzes past data and trends to provide a comprehensive assessment of the user's health and emotional state.

[0743] Step 12:

[0744] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in your next dinner."

[0745] Step 13:

[0746] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[0747] Step 14:

[0748] (as a future extension)

[0749] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[0750] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management that also takes emotional state into account.

[0751] Example 2

[0752] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] Modern society demands comprehensive health management that takes into account individual lifestyle habits and emotional states. However, conventional approaches only collect and analyze individual data (e.g., diet, exercise, sleep), making it difficult to achieve comprehensive health management that also includes emotional information. Furthermore, there is a lack of implementation support to determine whether health management proposals are feasible.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of their meal and upload the image on the spot, a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients, and a means for recording the user's emotional information in real time using a smartphone or smartwatch and analyzing it with an emotion engine. This allows for appropriate health management suggestions that take into account the user's overall health condition and also include emotional information. Furthermore, by providing support for implementing the suggested menu, more effective health management is achieved.

[0755] A "user" is an individual who uses this system to record and manage information about their diet, exercise, sleep, and emotions.

[0756] A "terminal" is a device (e.g., a smartphone or smartwatch) that a user uses to take photos of their meals and record exercise information, sleep information, and emotional information.

[0757] The "server" is a central system that receives data sent from the terminals, analyzes it, stores it in a database, and makes suggestions for health management.

[0758] "Upload" is the operation or process of sending data from a terminal to a server.

[0759] "Image analysis" is a process that uses AI technology to analyze uploaded food images and calculate the ingredients, three major nutrients, and calories of the ingredients.

[0760] The "three macronutrients" are protein, fat, and carbohydrates, which are the main components of food.

[0761] A "health maintenance menu" is a meal and exercise plan proposed to maintain or improve the user's health.

[0762] An "emotion engine" is an algorithm or software for analyzing a user's emotional information.

[0763] "Emotional information" is data that indicates the user's current mood or emotional state, and is collected, for example, by facial expression recognition or voice analysis.

[0764] The "database" is a collection of structured data that allows for efficient storage and management of collected dietary, exercise, sleep, and emotional information.

[0765] "Location information" is data that indicates a user's current geographic location.

[0766] "Health status" is comprehensive information that indicates the physical and mental state of the user.

[0767] "Artificial intelligence" is a technology that learns from a user's past health and emotional state data and generates optimal suggestions.

[0768] "Commercial facilities" are stores that sell food ingredients and health-related products, and facilities that provide food and beverage services.

[0769] This invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice on health management. This system also includes a function for recording and analyzing emotional information in real time. Using devices such as smartphones and smartwatches, the system automatically acquires dietary and exercise information and analyzes it using artificial intelligence (AI) to suggest appropriate health maintenance menus. It also uses an emotion engine to recognize the user's emotions and improve the suggestions based on those emotions.

[0770] Input and analysis of dietary information

[0771] Users take photos of their daily meals with their smartphone and upload the images to a server using a dedicated app. When the smartphone transfers the captured image to the server, it also sends image metadata (e.g., the date and time the image was taken, location information, etc.). The server then passes the received image file to an AI analysis module. The AI ​​analysis module uses image recognition technology (e.g., YOLO or ResNet) to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The analysis results are stored in a database and can be used for later analysis.

[0772] Example: A user takes a photo of the bread and coffee they had for breakfast with their smartphone and uploads the image. The server receives the image, and an AI analysis module automatically analyzes the type of bread and coffee, their nutritional content, and their calories. The analysis results are stored in a database.

[0773] Example prompt sentence:

[0774] Can you analyze a photo of my breakfast and give me calorie and nutrition information?

[0775] Acquiring exercise and sleep information

[0776] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information. The collected data includes the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server. The server stores the received exercise and sleep information in a database and uses it as basic data for understanding the user's overall health condition.

[0777] Example: A user wears a smartwatch and goes for a run. After the run, the smartwatch records the number of steps, distance traveled, and heart rate, and automatically sends this data to a server. The server stores this data in a database for later analysis.

[0778] Example prompt sentence:

[0779] Record your running data today and let me know your total steps and calories burned.

[0780] Acquisition and analysis of emotional information

[0781] Users record emotional information using their smartphones or smartwatches. In particular, emotions are recognized in real time through facial recognition and voice analysis. Using the smartphone's camera and microphone, the emotion engine analyzes facial expressions and vocal states to obtain emotional state data. The emotional information collected by the device is sent to a server, and the user's emotional state is also used as part of health analysis.

[0782] Example: A user uses a smartphone and the emotion engine analyzes facial expressions captured by the camera during a video chat, records the user's emotional state (e.g., joy, sadness, stress), and sends it to a server.

[0783] Example prompt sentence:

[0784] Please analyze my current emotional state and let me know.

[0785] Suggestions for maintaining good health

[0786] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI analysis module for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, it suggests specific meal and exercise menus to maintain the user's health. For example, if it determines that the user has not been exercising recently and is under stress, the server might suggest "doing 30 minutes of yoga to relieve stress" or "having some relaxing herbal tea with your next dinner." It can also suggest "a balanced meal for your next meal" if the user's nutrition is unbalanced.

[0787] Example: The server determines that the user is under stress based on dietary, exercise, and emotional data. The AI ​​generates a suggestion, such as "Drink some relaxing herbal tea with your next dinner," and notifies the user.

[0788] Example prompt sentence:

[0789] Based on your recent exercise and emotional data, we'll suggest what you should do to take care of your health.

[0790] Support for implementing proposals (future enhancements)

[0791] The server uses the user's location information to support the execution of the suggestions by providing information on nearby commercial facilities and restaurants. For example, it provides the user with information on inventory at the nearest commercial facility where the suggested ingredients can be purchased, or information on restaurants that offer the suggested menu.

[0792] Example: When a user tries to purchase suggested ingredients, the server provides inventory information from the nearest supermarket, helping the user to shop efficiently.

[0793] Example prompt sentence:

[0794] Could you please tell me the nearest supermarket where I can buy the ingredients you suggested?

[0795] In this way, this system is closely integrated into the user's daily life, collecting and analyzing data on diet, exercise, sleep, and emotions, and supporting comprehensive, personalized health management.

[0796] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0797] Step 1: Photograph and upload your meal

[0798] Users take pictures of their daily meals with their smartphones and upload the images to a server using a dedicated app.

[0799] Input: Photographed food image, metadata (photo date and time, location information)

[0800] Output: Food images and metadata sent to the server

[0801] How it works: The user takes a photo of their breakfast and uploads it through a dedicated app. The image is accompanied by information about the date, time, and location of the photo.

[0802] Step 2: Analyze the images

[0803] The server passes the uploaded meal image to an AI analysis module, which analyzes the image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[0804] Input: Uploaded food image and metadata

[0805] Output: Analysis results of ingredients, three major nutrients, and calories

[0806] Specific operation: The server passes the image to the AI ​​analysis module, which analyzes the type of ingredients and their nutritional content.

[0807] Step 3: Saving to the database

[0808] The server stores the analysis results in a database, which includes information such as the ingredients, the ratio of the three major nutrients, and calories.

[0809] Input: Analysis results of ingredients, three major nutrients, and calories

[0810] Output: Analysis data stored in a database

[0811] Specific operation: The server stores the analysis results in a database, making them available for later analysis.

[0812] Step 4: Record your exercise and sleep information

[0813] The device (smartwatch) periodically collects the user's exercise data (number of steps, distance, heart rate) and sleep data (sleep time, sleep quality).

[0814] Input: User's exercise data, sleep data

[0815] Output: Collected exercise data, sleep data

[0816] Specific operation: The user wears the smartwatch and goes out for exercise. The smartwatch automatically records the number of steps taken and the amount of sleep.

[0817] Step 5: Send your exercise and sleep data

[0818] The terminal periodically transmits the collected exercise and sleep data to the server.

[0819] Input: Collected exercise data, sleep data

[0820] Output: Exercise data and sleep data sent to the server

[0821] Specific operation: The smartwatch periodically sends exercise and sleep data to the server.

[0822] Step 6: Save to database

[0823] The server stores the received exercise and sleep data in a database, allowing the user to understand their overall health status.

[0824] Input: Exercise data and sleep data sent to the server

[0825] Output: Exercise data and sleep data stored in a database

[0826] Specific operation: The server stores the received exercise and sleep data in a database and uses it to comprehensively understand the user's health condition.

[0827] Step 7: Recording Emotional Information

[0828] Users record their emotional information using their smartphones or smartwatches, and emotions are recognized in real time through facial recognition and voice analysis.

[0829] Input: User's facial expression data, voice data

[0830] Output: Recorded emotion information

[0831] Specific operation: The user engages in a video chat, and facial expressions are captured on camera and analyzed by the emotion engine.

[0832] Step 8: Sending Emotional Information

[0833] The terminal transmits the collected emotion information to the server.

[0834] Input: Recorded emotion information

[0835] Output: Emotion information sent to the server

[0836] Specific operation: The smartphone sends the recorded emotional information to the server.

[0837] Step 9: Analyze and store emotional information

[0838] The server passes the received emotional information to an analysis module and stores the emotional state in a database.

[0839] Input: Emotion information sent to the server

[0840] Output: Emotional state data stored in a database

[0841] Specific operation: The emotion engine analyzes the received emotion information and stores the results in a database.

[0842] Step 10: Comprehensive data analysis

[0843] The server passes dietary, exercise, sleep, and emotional data to an AI analysis module to evaluate the user's health and emotional state.

[0844] Input: Various data stored in the database (diet, exercise, sleep, emotions)

[0845] Output: Assessment of the user's health and emotional state

[0846] Specific operation: The server analyzes data on diet, exercise, sleep, and emotions to assess overall health.

[0847] Step 11: Proposal for a health maintenance menu

[0848] Based on the analysis results, the server suggests specific meal and exercise menus to the user to maintain their health.

[0849] Input: User's health and emotional state assessment results

[0850] Output: Generated health maintenance menu

[0851] Specific behavior: Based on the user's health and emotional state, the AI ​​generates suggestions such as "do 30 minutes of yoga to relieve stress" or "have some relaxing herbal tea with dinner."

[0852] Step 12: Submit your proposal

[0853] The server transmits the generated proposal content to the terminal and notifies the user.

[0854] Input: Generated health maintenance menu

[0855] Output: The suggestion sent to the user

[0856] Specific operation: The generated health suggestions are displayed on the smartphone and the user is notified.

[0857] Step 13: Proposal implementation support

[0858] The server provides information on nearby commercial facilities and restaurants based on the user's location information, and supports the implementation of the suggestions.

[0859] Input: User's location information, suggestions

[0860] Output: Information to support the execution of the proposal (e.g., food inventory information and restaurant information)

[0861] Specific operation: The server provides inventory information of the nearest commercial establishment where the suggested ingredients can be purchased, helping the user to efficiently execute the suggested menu.

[0862] (Application example 2)

[0863] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0864] In modern self-driving vehicles, health management of drivers and passengers is not sufficiently considered, making it difficult to maintain good health, especially during long periods of driving or riding. In addition, conventional health management systems do not take emotional states into account, which means they are unable to provide health recommendations that address stress and emotional fluctuations. The present invention aims to provide a comfortable and healthy driving environment by efficiently managing the user's health in a self-driving vehicle and providing health recommendations that address emotional fluctuations.

[0865] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients; a means for analyzing previously registered meal contents and trends and suggesting a meal menu necessary for maintaining health; a means for acquiring exercise information and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for integrating food records, exercise information, and emotional information into the internal information system of the autonomous vehicle to monitor the health status in real time and provide personalized health advice; and a means for acquiring the emotional state of passengers using an in-vehicle camera and voice recognition system and adjusting health advice based on the emotions. This not only enables health management of users in the vehicle but also makes it possible to provide advice that takes emotional state into consideration.

[0866] "User" refers to an individual who uses the system, particularly for the purpose of health management within an autonomous vehicle.

[0867] "Meal contents" refers to the overall meal that the user ingests, and specifically includes information such as ingredients, the three major nutrients, and calories.

[0868] "Images" refers to visual data such as photos or videos taken by the user of their meal.

[0869] "Means for uploading" refers to the function for sending images taken by the user to the cloud or server.

[0870] "Means for analyzing" refers to the function of analyzing uploaded images and calculating the ingredients, three major nutrients, and calories of ingredients.

[0871] "Ingredients" refers to the nutrients and components contained in each food, specifically vitamins and minerals.

[0872] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[0873] A "calorie" is a unit that indicates the amount of energy contained in food.

[0874] "Means for suggesting meal menus" refers to a function that presents meal menus suitable for the user based on the analyzed meal contents.

[0875] "Exercise information" refers to information such as the amount of exercise, type of exercise, and time of exercise performed by the user.

[0876] "Sleep information" refers to information related to the user's sleep time, sleep quality, and the like.

[0877] "Means of acquisition" refers to the ability to collect data from devices such as smartphones and smartwatches.

[0878] The "means for suggesting an exercise menu" refers to a function that suggests the type and amount of exercise suitable for the user based on the acquired exercise information.

[0879] "Autonomous vehicle" refers to a vehicle that has the ability to drive itself.

[0880] "Internal information system" refers to a system installed within an autonomous vehicle for managing and analyzing healthcare-related data.

[0881] "Means for monitoring health status" refers to a function that integrates food records, exercise information, and emotional information to monitor the user's health status in real time.

[0882] "Means for providing personalized health advice" refers to a function that makes personalized health management suggestions based on user data.

[0883] An "in-car camera" refers to a camera installed inside a vehicle that captures the user's facial expressions and behavior.

[0884] A "voice recognition system" refers to a system that has the function of analyzing a user's speech and extracting linguistic information.

[0885] "Means for acquiring emotional state" refers to the function of assessing the user's emotions using an in-car camera or voice recognition system.

[0886] "Means for adjusting health suggestions based on emotions" refers to a function that uses the results of emotion analysis to modify the suggestions to suit the user's emotional state.

[0887] The present invention relates to a system for managing the health of users in autonomous vehicles. Specifically, the system allows users to photograph and record their meals, acquire exercise information, sleep information, and emotional state, and provide health advice based on this data. This allows users to maintain their health even during long trips in the car.

[0888] Hardware Configuration

[0889] The main hardware configuration of this system is as follows:

[0890] Autonomous vehicle internal information system: This system comprehensively manages various data necessary for health management.

[0891] Smartphones and smartwatches: Users use these devices to capture photos of their meals and to obtain exercise and sleep information.

[0892] In-car camera: Used to analyze the user's facial expressions and capture their emotional state.

[0893] Speech recognition system: Analyzes the user's voice and complements their emotional state.

[0894] Software Configuration

[0895] The software configuration is as follows:

[0896] AI-based analysis module: The present invention uses machine learning frameworks such as TensorFlow and Keras.

[0897] Emotion recognition engine: A model (e.g. TensorFlow / Keras) for analyzing the user's facial expressions and recognizing their emotional state.

[0898] Nutrition Analysis API: Analyzes uploaded food images and extracts nutritional information.

[0899] How it works

[0900] The operation of this system is as follows.

[0901] 1. Entering and analyzing dietary information

[0902] The user takes a photo of their meal using a smartphone, and the image is automatically uploaded to the autonomous vehicle's internal information system.

[0903] The uploaded image is passed to an AI-based analysis module that calculates the ingredients, macronutrients, and calories of the food, and the results are stored in a database for later analysis.

[0904] 2. Acquisition of exercise and sleep information

[0905] Smartphones and smartwatches automatically collect users' exercise and sleep information, including steps, distance, heart rate, and sleep duration.

[0906] The collected data is periodically transmitted to the autonomous vehicle's internal information system and stored in a database.

[0907] 3. Acquisition and analysis of emotional information

[0908] The user's emotional state is captured in real time using an in-car camera and a voice recognition system: the camera reads the user's facial expressions, and the voice recognition system analyzes the user's voice.

[0909] An emotion recognition engine evaluates the user's emotional state and stores it in a database.

[0910] 4. Suggestions for maintaining good health

[0911] The internal information system of the autonomous vehicle passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis.

[0912] The AI ​​analyzes past data and trends to assess the user's health and emotional state, and based on this assessment, suggests specific diet and exercise regimes to help maintain health.

[0913] Specific examples

[0914] For example, if a user is eating lunch in the car, an image of the meal is taken and a nutritional analysis is performed. If the analysis results indicate that the meal is high in calories, appropriate nutritional supplements and advice about the next meal are provided. At the same time, if facial recognition detects that the user's emotion is "sad," suggestions for breaks and relaxation methods to relieve stress are made.

[0915] Prompt Sentence Examples

[0916] "Take a picture of the user eating in the car and provide health advice based on the analysis results."

[0917] In this way, this system provides comprehensive support for health management in the car and offers advice that takes into account the driver's emotional state.

[0918] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0919] Step 1:

[0920] A user takes a photo of their meal using their smartphone and uploads the image to the server. The input is the image of the meal, and the output is the uploaded image data. Specifically, the user launches the smartphone's camera app, takes a photo of the meal, and then transfers the image file to the cloud server via a dedicated app.

[0921] Step 2:

[0922] The server receives the uploaded images and passes them to the analysis module. The input is the uploaded image data, and the output is the transmission of the image data to obtain the analysis results. The server also sends metadata (e.g., shooting date and time, location information, etc.) along with the image file to the analysis module.

[0923] Step 3:

[0924] The analysis module uses the received image to calculate the ingredients, three major nutrients, and calories of the ingredients. The input is image data, and the output is ingredient information, nutrient information, and calorie calculation data. Specifically, it uses AI frameworks such as TensorFlow and Keras to recognize ingredients from the image and calculate the corresponding ingredients and nutritional information based on a database.

[0925] Step 4:

[0926] The server stores the nutrition information received from the analysis module in a database. The input is the analysis result data, and the output is a new entry in the database. The server organizes this data by user and stores it for later analysis.

[0927] Step 5:

[0928] The device acquires exercise and sleep information from a smartphone or smartwatch and sends it to a server. The input is exercise and sleep information from the device, and the output is data sent to the server. Specifically, the device periodically collects data such as the number of steps taken, heart rate, and sleep status, and sends it to a cloud server.

[0929] Step 6:

[0930] The server stores the exercise and sleep information in a database. The input is the submitted exercise and sleep information, and the output is a new entry in the database. The server manages this data for each user and uses it for comprehensive health assessment.

[0931] Step 7:

[0932] The in-car camera and voice recognition system capture the user's emotional information in real time and send it to the server. The input is the user's facial expression and voice data, and the output is the analysis result of the emotional state. Specifically, the camera captures the facial expression, the voice recognition system analyzes the voice, and the data is sent to the emotion recognition engine.

[0933] Step 8:

[0934] The server receives the emotional state from the emotion recognition engine and stores it in a database. The input is the emotional state data, and the output is a new entry in the database. This allows emotional information to be used as part of health analysis.

[0935] Step 9:

[0936] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis. These data are the input, and the output is a comprehensive health assessment and advice suggestion data. The AI ​​module analyzes this data comprehensively and evaluates the user's health condition.

[0937] Step 10:

[0938] Based on the results of the AI ​​analysis, the server proposes specific meal and exercise menus to the user to maintain their health. The input is the AI ​​analysis results, and the output is the proposals. The server displays these proposals on the user's smartphone or in-car display, providing them in an easy-to-follow format.

[0939] In this way, the system of the present invention automates the entire process from data collection to analysis and proposals, thereby supporting the user's health management.

[0940] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0941] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0942] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0943] [Third embodiment]

[0944] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0945] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0946] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0947] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0948] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0949] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0950] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0951] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0952] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0953] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0954] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0955] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0956] This invention relates to a system that allows users to easily record their dietary and exercise information and provides necessary advice for health management. Specifically, it utilizes a smartphone or smartwatch to automatically collect dietary and exercise information, analyzes it using artificial intelligence (AI), and proposes an appropriate health maintenance menu.

[0957] System configuration and operation

[0958] Input and analysis of dietary information

[0959] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[0960] The server then passes the image file to an artificial intelligence module, which analyzes the image and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The results of this analysis are stored in a database for later analysis.

[0961] Acquiring exercise and sleep information

[0962] The device (smartphone or smartwatch) automatically collects data about the user's exercise and sleep, such as the number of steps, distance, heart rate, and sleep time, and periodically transmits this data to a server.

[0963] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[0964] Suggestions for maintaining good health

[0965] The server uses AI to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to evaluate the user's health. Based on this evaluation, it proposes specific diet and exercise menus to help the user maintain their health.

[0966] For example, if the server determines that the user has not been exercising recently, it can suggest that the user "go for a 30-minute jog next weekend." If the user's nutritional intake is unbalanced, the server can recommend that the user "eat more fish and vegetables at your next dinner."

[0967] Support for implementing proposals (future enhancements)

[0968] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[0969] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[0970] The processing flow will be explained below.

[0971] Step 1:

[0972] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[0973] Step 2:

[0974] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[0975] Step 3:

[0976] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[0977] Step 4:

[0978] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[0979] Step 5:

[0980] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[0981] Step 6:

[0982] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[0983] Step 7:

[0984] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[0985] Step 8:

[0986] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[0987] Step 9:

[0988] The server passes the accumulated dietary, exercise, and sleep data to the AI ​​for analysis, which then analyzes past data and trends to assess the user's health.

[0989] Step 10:

[0990] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "go for a 30-minute jog next weekend" or "eat more fish and vegetables at your next dinner."

[0991] Step 11:

[0992] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[0993] Step 12:

[0994] (as a future extension)

[0995] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[0996] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management.

[0997] Example 1

[0998] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0999] In modern society, health management has become an important issue, but it is difficult to accurately record diet and exercise habits in a busy lifestyle. Furthermore, there are only a limited number of systems that comprehensively evaluate health status and provide specific recommendations tailored to individual users. In conventional systems, analyzing dietary content and collecting exercise and sleep information is time-consuming, and it is difficult to comprehensively analyze this data and provide appropriate advice. In particular, flexible recommendations utilizing the user's location information have not been realized. There is a need to improve these issues and provide a system that allows users to easily manage their health.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1001] In this invention, the server includes a means for users to photograph their meal and upload the images on the spot; a means for analyzing the uploaded images and calculating the ingredients, three major nutrients, and calories; a means for analyzing previously registered meal contents and trends and suggesting meal menus necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or wearable device; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for analyzing images using an artificial intelligence module and storing them in a database; and a means for comprehensively analyzing the user's data, evaluating their health status, and providing advice. This allows users to easily analyze the photographed meal contents, centrally manage their exercise and sleep information, and receive specific suggestions based on their individual health status. Furthermore, a function for assisting users in implementing the suggested menus can be provided using the user's location information, making it easier for users to carry out health-maintaining activities.

[1002] "User" refers to an individual who uses this system to record dietary, exercise, and sleep information and manage their health.

[1003] "Means for taking photos of meal contents and uploading images on the spot" refers to a function that allows users to take photos of their meal using a smartphone camera or a dedicated app and send the image data to a server in real time.

[1004] "Means for analyzing uploaded images and calculating the ingredients, three major nutrients, and calories" refers to an algorithm in which an artificial intelligence module running on a server analyzes the meal images sent by users and calculates the ingredients, proteins, fats, carbohydrates, and calories contained in the food.

[1005] "A means for analyzing previously registered dietary content and trends and proposing dietary menus necessary for maintaining health" refers to a function that suggests dietary menus suited to the user's current health condition based on the user's past dietary data stored in a database.

[1006] "Means for acquiring exercise and sleep information from a smartphone or wearable device" refers to the function of a smartphone or smartwatch worn by a user to collect data such as the number of steps taken during exercise, distance traveled, heart rate, and sleep time, and transmit this data to a server.

[1007] "Means of analyzing the amount of exercise required to maintain health based on the acquired information and proposing an appropriate exercise menu" refers to the function in which the server analyzes exercise and sleep information, and generates and presents the optimal exercise menu for the user.

[1008] "Image analysis using an artificial intelligence module and storage in a database" refers to the function in which an AI running on a server analyzes food images and stores the results in a database.

[1009] "Means for comprehensively analyzing user data, assessing health status, and providing advice" refers to the function of comprehensively analyzing dietary information, exercise information, and sleep information collected by the server, assessing the user's health status, and providing specific advice for maintaining good health.

[1010] MODE FOR CARRYING OUT THE INVENTION

[1011] The present invention relates to a system that allows users to efficiently and easily record their daily diet and exercise information and perform comprehensive health management. This system utilizes smartphones and wearable devices, analyzes data using artificial intelligence (AI), and proposes appropriate health maintenance menus to users.

[1012] Hardware and software used

[1013] Hardware:

[1014] Smartphone

[1015] Wearable devices such as smartwatches

[1016] server

[1017] software:

[1018] Dedicated application (for smartphones)

[1019] Server-side artificial intelligence module

[1020] Database System

[1021] Specific explanation of the system's operation

[1022] Users use a smartphone with a dedicated app installed to take pictures of their daily meals. At this time, they can also enter detailed information about the meal and the types of ingredients. The photographed food images are uploaded to a server via the smartphone. At this time, metadata such as the date and time of the photo and location information are also sent along with the image.

[1023] The server passes the received image file to an artificial intelligence module. The AI ​​analyzes the image, identifies the ingredients in the image, and calculates the protein, fat, carbohydrates, and calories of each ingredient. This data is stored in a database. For example, if you take a photo and upload a meal of bread, eggs, and bananas for breakfast, the AI ​​will automatically generate and record the following: "Bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "Egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "Banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[1024] The devices, such as smartphones and smartwatches, automatically collect users' exercise information (e.g., number of steps, distance traveled, heart rate) and sleep information. This data is periodically sent to a server and stored in a database.

[1025] The server comprehensively analyzes the collected dietary, exercise, and sleep data to evaluate the user's health condition. Based on this evaluation, it suggests appropriate meal and exercise menus to the user. For example, if it determines that the user has not been exercising recently, the server will suggest "go for a 30-minute jog next weekend." If the user's nutrition is unbalanced, it will recommend "eat more fish and vegetables at your next dinner."

[1026] Some examples of specific prompts include:

[1027] "I had bread, eggs, and a banana for breakfast. What are the calories and nutrients in this meal?"

[1028] In the future, the server may use the user's location information to link inventory information from nearby commercial facilities and restaurant menus to support the implementation of suggested menus, making it easier for users to implement suggested health maintenance menus and supporting continuous health management.

[1029] As described above, the system of the present invention enables the user to efficiently manage their health and supports continuous health maintenance activities.

[1030] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1031] Step 1:

[1032] The user launches the dedicated application and takes a photo of the meal with their smartphone. The user also has the option to input ingredients and meal details. The input data is the image of the meal and text information about the ingredients.

[1033] Input: Meal image, ingredient information

[1034] Output: Food image data and ingredient information stored on the smartphone

[1035] Specific behavior:

[1036] For breakfast, users take a photo of bread, eggs, and bananas and upload it to the app.

[1037] Step 2:

[1038] The device (smartphone) transmits the captured food image and the input ingredient information to the server. This transmission includes metadata such as the date and time of the image capture and location information along with the image data.

[1039] Input: Food image data, ingredient information, metadata

[1040] Output: Food image data, ingredient information, and metadata sent to the server

[1041] Specific behavior:

[1042] The smartphone captures a photo of the meal and information about the ingredients, and then sends the photo, information, and metadata all at once to the server.

[1043] Step 3:

[1044] The server then passes the image files to an artificial intelligence module, which uses image analysis technology to identify the ingredients in the image and calculate the protein, fat, carbohydrates, and calories of each ingredient. The analysis results are then stored in a database.

[1045] Input: Submitted food image data, metadata

[1046] Output: Ingredient data, nutrient data, and calorie data for each ingredient stored in the database

[1047] Specific behavior:

[1048] The AI ​​analyzes the image and generates and stores data such as "bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[1049] Step 4:

[1050] The device (smartphone or wearable device) automatically collects the user's exercise and sleep information. The collected data includes the number of steps taken, distance traveled, heart rate, and sleep time. This data is sent to a server at regular intervals.

[1051] Input: User's exercise and sleep data

[1052] Output: Exercise and sleep data sent to the server

[1053] Specific behavior:

[1054] The smartwatch monitors your heart rate 24 hours a day and sends your step count and sleep data to a server at the end of the day.

[1055] Step 5:

[1056] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[1057] Input: exercise data, sleep data sent

[1058] Output: Exercise data and sleep data stored in a database

[1059] Specific behavior:

[1060] The server records the received exercise data in a database and stores the data by day, week, or month.

[1061] Step 6:

[1062] The server uses AI to perform a comprehensive analysis of the collected dietary, exercise, and sleep data. Based on the results of this analysis, the server evaluates the user's health status and suggests specific diet and exercise menus.

[1063] Input: Food data, exercise data, and sleep data in the database

[1064] Output: Proposal of a health maintenance menu for the user

[1065] Specific behavior:

[1066] The server generates a health maintenance menu such as, "Based on your recent data, you have continued to lack exercise, so we recommend that you go for a 30-minute jog next weekend. Also, your nutritional intake tends to be unbalanced, so we recommend that you eat more fish and vegetables for your next dinner," and notifies the user.

[1067] Step 7:

[1068] In the future, the server will use the user's location information to provide a function that links inventory information from nearby commercial facilities and menus offered by restaurants to support the implementation of suggested menus.

[1069] Input: User's location information, suggested health menu

[1070] Output: Information about commercial facilities and restaurants based on the user's location

[1071] Specific behavior:

[1072] The server obtains the user's location information and provides information on stock availability at nearby supermarkets where the suggested ingredients can be purchased, as well as information on restaurants that offer the suggested menu items.

[1073] (Application example 1)

[1074] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1075] Conventional health management systems require users to manually input information, posing challenges in terms of time and accuracy. Furthermore, there was a lack of systems that could comprehensively grasp the health status of employees in specific work environments, such as factories, and provide optimal health management plans. As a result, there is a demand for ways to maintain employee health and improve work efficiency.

[1076] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1077] In this invention, the server includes: a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of ingredients; a means for analyzing previously registered meal contents and trends and proposing a meal menu necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and proposing an appropriate exercise menu; a means for automatically collecting health data including the user's physiological data and analyzing it in real time; a means for sending alerts and providing appropriate health advice based on the results of the health data analysis; and a means for evaluating the health status of factory workers and supporting appropriate health management. This eliminates the need for users to manually input information and enables real-time understanding and evaluation of health status, thereby improving the health of factory workers and improving work efficiency.

[1078] "User" refers to a person who uses the system to manage their own health condition.

[1079] "Meal contents" refers to all of the foods and menu items consumed by the user.

[1080] "Image" refers to a photograph of the meal contents taken by the user using a smartphone or other photographic device.

[1081] "Upload" refers to the act of sending an image taken by a user to a server.

[1082] "Food ingredients" refers to the nutritional and chemical constituents of the individual foods in a meal.

[1083] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[1084] A "calorie" is a unit of measurement that indicates the amount of energy contained in food, and refers to the energy provided by food.

[1085] "Exercise information" refers to exercise data during the user's daily activities, including the number of steps, heart rate, distance, and the like.

[1086] "Sleep information" refers to information such as the user's sleep patterns and sleep duration.

[1087] An "exercise menu" refers to an exercise plan proposed for the purpose of maintaining the user's health.

[1088] "Physiological data" refers to data on the user's physical physiological condition, such as heart rate, body temperature, and blood pressure.

[1089] "Health Data" refers to data reflecting a user's overall health, including diet, exercise, sleep, and physiological data.

[1090] "Real-time" refers to data being collected and analyzed immediately, with results being provided almost instantly.

[1091] "Alert" refers to a notification sent to alert you based on the results of an analysis of your health data.

[1092] "Health advice" refers to specific advice for maintaining health provided to a user based on analyzed health data.

[1093] "Factory workers" refers to employees who work in a particular industrial facility or manufacturing site.

[1094] "Health status" refers to the overall physical and mental health of a user.

[1095] "Health management" refers to actions and plans to maintain and improve a user's health.

[1096] "Server" refers to a computer system that collects and analyzes data and provides information to users.

[1097] System Overview

[1098] This system allows users to easily record their diet and exercise information and provides them with advice on health management. By using a smartphone or smartwatch, the system automatically collects diet and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus.

[1099] Hardware and software used

[1100] Smartphone: A device that allows users to take pictures of their meals and upload the images to a server.

[1101] Smartwatch: A device that automatically collects a user's exercise and sleep information.

[1102] Server: A central computer system that performs image analysis, stores data, and performs AI-based analysis and proposals.

[1103] AI model: A generative AI model is used to analyze food images and generate optimal health maintenance menus based on individual health conditions.

[1104] Program processing overview

[1105] Input and analysis of dietary information

[1106] Users take photos of their daily meals with their smartphones and use a dedicated app to upload the images to a server. The server also receives image metadata (such as the date and time the photo was taken, location information, etc.) and passes the image files to an AI model. The AI ​​model analyzes the images and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The results of this analysis are stored in a database for later analysis.

[1107] Acquiring exercise and sleep information

[1108] Smartwatches automatically collect data on the user's exercise and sleep. For example, they record information such as the number of steps taken, distance traveled, heart rate, and sleep time, and periodically send it to a server. The server stores the received exercise and sleep information in a database, enabling the user to grasp their overall health status.

[1109] Suggestions for maintaining good health

[1110] The server uses an AI model to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to assess the user's health status. Based on this assessment, it proposes specific diet and exercise menus to help the user maintain their health.

[1111] Support for implementing proposals (future enhancements)

[1112] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[1113] Specific examples

[1114] Assume an employee is wearing a smartwatch, which collects data such as heart rate, steps, and sleep time, and sends it to a server as follows:

[1115] Heart rate: 80 BPM

[1116] Steps: 6700

[1117] Sleep time: 7.2 hours

[1118] Based on this data, the user's health status can be assessed and prompts such as the following can be fed into a generative AI model:

[1119] The user has a heart rate of 80, steps taken 6700, and 7.2 hours of sleep. What health advice should be provided?

[1120] Based on the analysis results, the generative AI model provides specific advice such as, "Today, we recommend that you maintain moderate exercise and eat a vegetable-based diet."

[1121] As described above, this system makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[1122] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1123] Step 1:

[1124] Users take photos of their meals and upload the images on the spot.

[1125] Input: A user takes a photo of their meal using their smartphone.

[1126] How it works: The user launches a camera app and takes a picture of their meal. After taking the picture, they upload the image to the server via a dedicated app. The server receives the image along with metadata such as the date and time the photo was taken and its location.

[1127] Output: Image data and metadata uploaded to the server.

[1128] Step 2:

[1129] The server analyzes the received image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[1130] Input: Image data uploaded to the server.

[1131] How it works: The server passes the image to the AI ​​model, which recognizes the food and calculates its ingredients, the three major nutrients (protein, fat, carbohydrates) and calories.

[1132] Output: Ingredients, macronutrients, and calorie information for analyzed ingredients.

[1133] Step 3:

[1134] The server stores the analysis results in a database.

[1135] Input: Ingredients, macronutrients, and calorie information.

[1136] Specific operation: The server writes the analysis results to a database, which is used for future analyses required for health management.

[1137] Output: Analysis results stored in a database.

[1138] Step 4:

[1139] The smartwatch collects the user's exercise and sleep information.

[1140] Input: User's exercise and sleep information.

[1141] How it works: The smartwatch uses sensors to record information such as heart rate, steps, distance traveled, and sleep time in real time, and periodically transmits the recorded data to a server via the smartphone.

[1142] Output: Exercise and sleep data sent to the server.

[1143] Step 5:

[1144] The server stores the collected exercise and sleep information in a database.

[1145] Input: Exercise and sleep information sent from your smartwatch.

[1146] What it does: The server stores the received exercise and sleep information in a database, which allows it to understand the user's overall health.

[1147] Output: Exercise and sleep data stored in a database.

[1148] Step 6:

[1149] Based on the data collected by the server, the system evaluates the user's health condition and suggests an appropriate health maintenance menu.

[1150] Input: Food data, exercise data, sleep data.

[1151] Specific operation: The server uses an AI model to analyze past data and trends stored in the database and evaluate the user's health condition. The AI ​​model uses a generative AI model to generate optimal meal and exercise menus based on the user's health condition. It also generates specific health advice by inputting a prompt: "The user's heart rate is X, number of steps is Y, and sleep time is Z hours. What kind of health advice should be provided?"

[1152] Output: Health maintenance menu and advice suggested to the user.

[1153] Step 7:

[1154] The server sends alerts based on the analysis of health data and provides appropriate health advice.

[1155] Input: Alert information and advice generated based on the user's health status.

[1156] How it works: The server generates alerts when user-defined criteria are exceeded and sends notifications via smartphone, including alerts for excessive heart rate or insufficient sleep, along with health advice generated by the AI ​​model.

[1157] Output: Alert notification and health advice to the user.

[1158] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1159] The present invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice necessary for health management. Furthermore, the present invention also includes a feature that incorporates an emotion engine that recognizes the user's emotions. Specifically, the system utilizes a smartphone or smartwatch to automatically acquire dietary and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus. The emotion engine also recognizes the user's emotions and improves the suggestions based on those information.

[1160] System configuration and operation

[1161] Input and analysis of dietary information

[1162] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[1163] The server then passes the image file to the analysis module, where AI analyzes the image to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The analysis results are stored in a database for later analysis.

[1164] Acquiring exercise and sleep information

[1165] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information, such as the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server.

[1166] The server stores the received exercise and sleep information in a database, allowing the user to understand their overall health condition.

[1167] Acquisition and analysis of emotional information

[1168] Users wear smartphones or smartwatches and record emotional information. In particular, emotions are recognized in real time through facial recognition and voice analysis. For example, facial expressions are read using a smartphone camera, and the emotion engine analyzes the emotional state based on that data.

[1169] The device transmits emotional information to a server, so that the user's emotional state is also used as part of the health analysis.

[1170] Suggestions for maintaining good health

[1171] The server passes the accumulated dietary, exercise, sleep, and emotional data to the AI ​​for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, the AI ​​proposes specific dietary and exercise menus for the user to maintain their health.

[1172] For example, if the server determines that the user has not been exercising recently and is under stress, it can suggest that the user "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in the next dinner." If the user's nutritional intake is unbalanced, the server can also suggest "a balanced dish for the next meal."

[1173] Support for implementing proposals (future enhancements)

[1174] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[1175] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management that also takes emotional states into consideration.

[1176] The processing flow will be explained below.

[1177] Step 1:

[1178] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[1179] Step 2:

[1180] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[1181] Step 3:

[1182] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[1183] Step 4:

[1184] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[1185] Step 5:

[1186] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[1187] Step 6:

[1188] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[1189] Step 7:

[1190] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[1191] Step 8:

[1192] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[1193] Step 9:

[1194] Users wear smartphones or smartwatches that record emotional information, and in particular, recognize emotions in real time through facial recognition and voice analysis.

[1195] Step 10:

[1196] The device sends the emotion information to the server, along with the emotion data processed by the emotion engine.

[1197] Step 11:

[1198] The server integrates dietary, exercise, and sleep data, including emotional information, and passes it on to the AI ​​for analysis. The AI ​​analyzes past data and trends to provide a comprehensive assessment of the user's health and emotional state.

[1199] Step 12:

[1200] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in your next dinner."

[1201] Step 13:

[1202] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[1203] Step 14:

[1204] (as a future extension)

[1205] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[1206] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management that also takes emotional state into account.

[1207] Example 2

[1208] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1209] Modern society demands comprehensive health management that takes into account individual lifestyle habits and emotional states. However, conventional approaches only collect and analyze individual data (e.g., diet, exercise, sleep), making it difficult to achieve comprehensive health management that also includes emotional information. Furthermore, there is a lack of implementation support to determine whether health management proposals are feasible.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of their meal and upload the image on the spot, a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients, and a means for recording the user's emotional information in real time using a smartphone or smartwatch and analyzing it with an emotion engine. This allows for appropriate health management suggestions that take into account the user's overall health condition and also include emotional information. Furthermore, by providing support for implementing the suggested menu, more effective health management is achieved.

[1211] A "user" is an individual who uses this system to record and manage information about their diet, exercise, sleep, and emotions.

[1212] A "terminal" is a device (e.g., a smartphone or smartwatch) that a user uses to take photos of their meals and record exercise information, sleep information, and emotional information.

[1213] The "server" is a central system that receives data sent from the terminals, analyzes it, stores it in a database, and makes suggestions for health management.

[1214] "Upload" is the operation or process of sending data from a terminal to a server.

[1215] "Image analysis" is a process that uses AI technology to analyze uploaded food images and calculate the ingredients, three major nutrients, and calories of the ingredients.

[1216] The "three macronutrients" are protein, fat, and carbohydrates, which are the main components of food.

[1217] A "health maintenance menu" is a meal and exercise plan proposed to maintain or improve the user's health.

[1218] An "emotion engine" is an algorithm or software for analyzing a user's emotional information.

[1219] "Emotional information" is data that indicates the user's current mood or emotional state, and is collected, for example, by facial expression recognition or voice analysis.

[1220] The "database" is a collection of structured data that allows for efficient storage and management of collected dietary, exercise, sleep, and emotional information.

[1221] "Location information" is data that indicates a user's current geographic location.

[1222] "Health status" is comprehensive information that indicates the physical and mental state of the user.

[1223] "Artificial intelligence" is a technology that learns from a user's past health and emotional state data and generates optimal suggestions.

[1224] "Commercial facilities" are stores that sell food ingredients and health-related products, and facilities that provide food and beverage services.

[1225] This invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice on health management. This system also includes a function for recording and analyzing emotional information in real time. Using devices such as smartphones and smartwatches, the system automatically acquires dietary and exercise information and analyzes it using artificial intelligence (AI) to suggest appropriate health maintenance menus. It also uses an emotion engine to recognize the user's emotions and improve the suggestions based on those emotions.

[1226] Input and analysis of dietary information

[1227] Users take photos of their daily meals with their smartphone and upload the images to a server using a dedicated app. When the smartphone transfers the captured image to the server, it also sends image metadata (e.g., the date and time the image was taken, location information, etc.). The server then passes the received image file to an AI analysis module. The AI ​​analysis module uses image recognition technology (e.g., YOLO or ResNet) to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The analysis results are stored in a database and can be used for later analysis.

[1228] Example: A user takes a photo of the bread and coffee they had for breakfast with their smartphone and uploads the image. The server receives the image, and an AI analysis module automatically analyzes the type of bread and coffee, their nutritional content, and their calories. The analysis results are stored in a database.

[1229] Example prompt sentence:

[1230] Can you analyze a photo of my breakfast and give me calorie and nutrition information?

[1231] Acquiring exercise and sleep information

[1232] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information. The collected data includes the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server. The server stores the received exercise and sleep information in a database and uses it as basic data for understanding the user's overall health condition.

[1233] Example: A user wears a smartwatch and goes for a run. After the run, the smartwatch records the number of steps, distance traveled, and heart rate, and automatically sends this data to a server. The server stores this data in a database for later analysis.

[1234] Example prompt sentence:

[1235] Record your running data today and let me know your total steps and calories burned.

[1236] Acquisition and analysis of emotional information

[1237] Users record emotional information using their smartphones or smartwatches. In particular, emotions are recognized in real time through facial recognition and voice analysis. Using the smartphone's camera and microphone, the emotion engine analyzes facial expressions and vocal states to obtain emotional state data. The emotional information collected by the device is sent to a server, and the user's emotional state is also used as part of health analysis.

[1238] Example: A user uses a smartphone and the emotion engine analyzes facial expressions captured by the camera during a video chat, records the user's emotional state (e.g., joy, sadness, stress), and sends it to a server.

[1239] Example prompt sentence:

[1240] Please analyze my current emotional state and let me know.

[1241] Suggestions for maintaining good health

[1242] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI analysis module for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, it suggests specific meal and exercise menus to maintain the user's health. For example, if it determines that the user has not been exercising recently and is under stress, the server might suggest "doing 30 minutes of yoga to relieve stress" or "having some relaxing herbal tea with your next dinner." It can also suggest "a balanced meal for your next meal" if the user's nutrition is unbalanced.

[1243] Example: The server determines that the user is under stress based on dietary, exercise, and emotional data. The AI ​​generates a suggestion, such as "Drink some relaxing herbal tea with your next dinner," and notifies the user.

[1244] Example prompt sentence:

[1245] Based on your recent exercise and emotional data, we'll suggest what you should do to take care of your health.

[1246] Support for implementing proposals (future enhancements)

[1247] The server uses the user's location information to support the execution of the suggestions by providing information on nearby commercial facilities and restaurants. For example, it provides the user with information on inventory at the nearest commercial facility where the suggested ingredients can be purchased, or information on restaurants that offer the suggested menu.

[1248] Example: When a user tries to purchase suggested ingredients, the server provides inventory information from the nearest supermarket, helping the user to shop efficiently.

[1249] Example prompt sentence:

[1250] Could you please tell me the nearest supermarket where I can buy the ingredients you suggested?

[1251] In this way, this system is closely integrated into the user's daily life, collecting and analyzing data on diet, exercise, sleep, and emotions, and supporting comprehensive, personalized health management.

[1252] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1253] Step 1: Photograph and upload your meal

[1254] Users take pictures of their daily meals with their smartphones and upload the images to a server using a dedicated app.

[1255] Input: Photographed food image, metadata (photo date and time, location information)

[1256] Output: Food images and metadata sent to the server

[1257] How it works: The user takes a photo of their breakfast and uploads it through a dedicated app. The image is accompanied by information about the date, time, and location of the photo.

[1258] Step 2: Analyze the images

[1259] The server passes the uploaded meal image to an AI analysis module, which analyzes the image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[1260] Input: Uploaded food image and metadata

[1261] Output: Analysis results of ingredients, three major nutrients, and calories

[1262] Specific operation: The server passes the image to the AI ​​analysis module, which analyzes the type of ingredients and their nutritional content.

[1263] Step 3: Saving to the database

[1264] The server stores the analysis results in a database, which includes information such as the ingredients, the ratio of the three major nutrients, and calories.

[1265] Input: Analysis results of ingredients, three major nutrients, and calories

[1266] Output: Analysis data stored in a database

[1267] Specific operation: The server stores the analysis results in a database, making them available for later analysis.

[1268] Step 4: Record your exercise and sleep information

[1269] The device (smartwatch) periodically collects the user's exercise data (number of steps, distance, heart rate) and sleep data (sleep time, sleep quality).

[1270] Input: User's exercise data, sleep data

[1271] Output: Collected exercise data, sleep data

[1272] Specific operation: The user wears the smartwatch and goes out for exercise. The smartwatch automatically records the number of steps taken and the amount of sleep.

[1273] Step 5: Send your exercise and sleep data

[1274] The terminal periodically transmits the collected exercise and sleep data to the server.

[1275] Input: Collected exercise data, sleep data

[1276] Output: Exercise data and sleep data sent to the server

[1277] Specific operation: The smartwatch periodically sends exercise and sleep data to the server.

[1278] Step 6: Save to database

[1279] The server stores the received exercise and sleep data in a database, allowing the user to understand their overall health status.

[1280] Input: Exercise data and sleep data sent to the server

[1281] Output: Exercise data and sleep data stored in a database

[1282] Specific operation: The server stores the received exercise and sleep data in a database and uses it to comprehensively understand the user's health condition.

[1283] Step 7: Recording Emotional Information

[1284] Users record their emotional information using their smartphones or smartwatches, and emotions are recognized in real time through facial recognition and voice analysis.

[1285] Input: User's facial expression data, voice data

[1286] Output: Recorded emotion information

[1287] Specific operation: The user engages in a video chat, and facial expressions are captured on camera and analyzed by the emotion engine.

[1288] Step 8: Sending Emotional Information

[1289] The terminal transmits the collected emotion information to the server.

[1290] Input: Recorded emotion information

[1291] Output: Emotion information sent to the server

[1292] Specific operation: The smartphone sends the recorded emotional information to the server.

[1293] Step 9: Analyze and store emotional information

[1294] The server passes the received emotional information to an analysis module and stores the emotional state in a database.

[1295] Input: Emotion information sent to the server

[1296] Output: Emotional state data stored in a database

[1297] Specific operation: The emotion engine analyzes the received emotion information and stores the results in a database.

[1298] Step 10: Comprehensive data analysis

[1299] The server passes dietary, exercise, sleep, and emotional data to an AI analysis module to evaluate the user's health and emotional state.

[1300] Input: Various data stored in the database (diet, exercise, sleep, emotions)

[1301] Output: Assessment of the user's health and emotional state

[1302] Specific operation: The server analyzes data on diet, exercise, sleep, and emotions to assess overall health.

[1303] Step 11: Proposal for a health maintenance menu

[1304] Based on the analysis results, the server suggests specific meal and exercise menus to the user to maintain their health.

[1305] Input: User's health and emotional state assessment results

[1306] Output: Generated health maintenance menu

[1307] Specific behavior: Based on the user's health and emotional state, the AI ​​generates suggestions such as "do 30 minutes of yoga to relieve stress" or "have some relaxing herbal tea with dinner."

[1308] Step 12: Submit your proposal

[1309] The server transmits the generated proposal content to the terminal and notifies the user.

[1310] Input: Generated health maintenance menu

[1311] Output: The suggestion sent to the user

[1312] Specific operation: The generated health suggestions are displayed on the smartphone and the user is notified.

[1313] Step 13: Proposal implementation support

[1314] The server provides information on nearby commercial facilities and restaurants based on the user's location information, and supports the implementation of the suggestions.

[1315] Input: User's location information, suggestions

[1316] Output: Information to support the execution of the proposal (e.g., food inventory information and restaurant information)

[1317] Specific operation: The server provides inventory information of the nearest commercial establishment where the suggested ingredients can be purchased, helping the user to efficiently execute the suggested menu.

[1318] (Application example 2)

[1319] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1320] In modern self-driving vehicles, health management of drivers and passengers is not sufficiently considered, making it difficult to maintain good health, especially during long periods of driving or riding. In addition, conventional health management systems do not take emotional states into account, which means they are unable to provide health recommendations that address stress and emotional fluctuations. The present invention aims to provide a comfortable and healthy driving environment by efficiently managing the user's health in a self-driving vehicle and providing health recommendations that address emotional fluctuations.

[1321] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients; a means for analyzing previously registered meal contents and trends and suggesting a meal menu necessary for maintaining health; a means for acquiring exercise information and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for integrating food records, exercise information, and emotional information into the internal information system of the autonomous vehicle to monitor the health status in real time and provide personalized health advice; and a means for acquiring the emotional state of passengers using an in-vehicle camera and voice recognition system and adjusting health advice based on the emotions. This not only enables health management of users in the vehicle but also makes it possible to provide advice that takes emotional state into consideration.

[1322] "User" refers to an individual who uses the system, particularly for the purpose of health management within an autonomous vehicle.

[1323] "Meal contents" refers to the overall meal that the user ingests, and specifically includes information such as ingredients, the three major nutrients, and calories.

[1324] "Images" refers to visual data such as photos or videos taken by the user of their meal.

[1325] "Means for uploading" refers to the function for sending images taken by the user to the cloud or server.

[1326] "Means for analyzing" refers to the function of analyzing uploaded images and calculating the ingredients, three major nutrients, and calories of ingredients.

[1327] "Ingredients" refers to the nutrients and components contained in each food, specifically vitamins and minerals.

[1328] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[1329] A "calorie" is a unit that indicates the amount of energy contained in food.

[1330] "Means for suggesting meal menus" refers to a function that presents meal menus suitable for the user based on the analyzed meal contents.

[1331] "Exercise information" refers to information such as the amount of exercise, type of exercise, and time of exercise performed by the user.

[1332] "Sleep information" refers to information related to the user's sleep time, sleep quality, and the like.

[1333] "Means of acquisition" refers to the ability to collect data from devices such as smartphones and smartwatches.

[1334] The "means for suggesting an exercise menu" refers to a function that suggests the type and amount of exercise suitable for the user based on the acquired exercise information.

[1335] "Autonomous vehicle" refers to a vehicle that has the ability to drive itself.

[1336] "Internal information system" refers to a system installed within an autonomous vehicle for managing and analyzing healthcare-related data.

[1337] "Means for monitoring health status" refers to a function that integrates food records, exercise information, and emotional information to monitor the user's health status in real time.

[1338] "Means for providing personalized health advice" refers to a function that makes personalized health management suggestions based on user data.

[1339] An "in-car camera" refers to a camera installed inside a vehicle that captures the user's facial expressions and behavior.

[1340] A "voice recognition system" refers to a system that has the function of analyzing a user's speech and extracting linguistic information.

[1341] "Means for acquiring emotional state" refers to the function of assessing the user's emotions using an in-car camera or voice recognition system.

[1342] "Means for adjusting health suggestions based on emotions" refers to a function that uses the results of emotion analysis to modify the suggestions to suit the user's emotional state.

[1343] The present invention relates to a system for managing the health of users in autonomous vehicles. Specifically, the system allows users to photograph and record their meals, acquire exercise information, sleep information, and emotional state, and provide health advice based on this data. This allows users to maintain their health even during long trips in the car.

[1344] Hardware Configuration

[1345] The main hardware configuration of this system is as follows:

[1346] Autonomous vehicle internal information system: This system comprehensively manages various data necessary for health management.

[1347] Smartphones and smartwatches: Users use these devices to capture photos of their meals and to obtain exercise and sleep information.

[1348] In-car camera: Used to analyze the user's facial expressions and capture their emotional state.

[1349] Speech recognition system: Analyzes the user's voice and complements their emotional state.

[1350] Software Configuration

[1351] The software configuration is as follows:

[1352] AI-based analysis module: The present invention uses machine learning frameworks such as TensorFlow and Keras.

[1353] Emotion recognition engine: A model (e.g. TensorFlow / Keras) for analyzing the user's facial expressions and recognizing their emotional state.

[1354] Nutrition Analysis API: Analyzes uploaded food images and extracts nutritional information.

[1355] How it works

[1356] The operation of this system is as follows.

[1357] 1. Entering and analyzing dietary information

[1358] The user takes a photo of their meal using a smartphone, and the image is automatically uploaded to the autonomous vehicle's internal information system.

[1359] The uploaded image is passed to an AI-based analysis module that calculates the ingredients, macronutrients, and calories of the food, and the results are stored in a database for later analysis.

[1360] 2. Acquisition of exercise and sleep information

[1361] Smartphones and smartwatches automatically collect users' exercise and sleep information, including steps, distance, heart rate, and sleep duration.

[1362] The collected data is periodically transmitted to the autonomous vehicle's internal information system and stored in a database.

[1363] 3. Acquisition and analysis of emotional information

[1364] The user's emotional state is captured in real time using an in-car camera and a voice recognition system: the camera reads the user's facial expressions, and the voice recognition system analyzes the user's voice.

[1365] An emotion recognition engine evaluates the user's emotional state and stores it in a database.

[1366] 4. Suggestions for maintaining good health

[1367] The internal information system of the autonomous vehicle passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis.

[1368] The AI ​​analyzes past data and trends to assess the user's health and emotional state, and based on this assessment, suggests specific diet and exercise regimes to help maintain health.

[1369] Specific examples

[1370] For example, if a user is eating lunch in the car, an image of the meal is taken and a nutritional analysis is performed. If the analysis results indicate that the meal is high in calories, appropriate nutritional supplements and advice about the next meal are provided. At the same time, if facial recognition detects that the user's emotion is "sad," suggestions for breaks and relaxation methods to relieve stress are made.

[1371] Prompt Sentence Examples

[1372] "Take a picture of the user eating in the car and provide health advice based on the analysis results."

[1373] In this way, this system provides comprehensive support for health management in the car and offers advice that takes into account the driver's emotional state.

[1374] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1375] Step 1:

[1376] A user takes a photo of their meal using their smartphone and uploads the image to the server. The input is the image of the meal, and the output is the uploaded image data. Specifically, the user launches the smartphone's camera app, takes a photo of the meal, and then transfers the image file to the cloud server via a dedicated app.

[1377] Step 2:

[1378] The server receives the uploaded images and passes them to the analysis module. The input is the uploaded image data, and the output is the transmission of the image data to obtain the analysis results. The server also sends metadata (e.g., shooting date and time, location information, etc.) along with the image file to the analysis module.

[1379] Step 3:

[1380] The analysis module uses the received image to calculate the ingredients, three major nutrients, and calories of the ingredients. The input is image data, and the output is ingredient information, nutrient information, and calorie calculation data. Specifically, it uses AI frameworks such as TensorFlow and Keras to recognize ingredients from the image and calculate the corresponding ingredients and nutritional information based on a database.

[1381] Step 4:

[1382] The server stores the nutrition information received from the analysis module in a database. The input is the analysis result data, and the output is a new entry in the database. The server organizes this data by user and stores it for later analysis.

[1383] Step 5:

[1384] The device acquires exercise and sleep information from a smartphone or smartwatch and sends it to a server. The input is exercise and sleep information from the device, and the output is data sent to the server. Specifically, the device periodically collects data such as the number of steps taken, heart rate, and sleep status, and sends it to a cloud server.

[1385] Step 6:

[1386] The server stores the exercise and sleep information in a database. The input is the submitted exercise and sleep information, and the output is a new entry in the database. The server manages this data for each user and uses it for comprehensive health assessment.

[1387] Step 7:

[1388] The in-car camera and voice recognition system capture the user's emotional information in real time and send it to the server. The input is the user's facial expression and voice data, and the output is the analysis result of the emotional state. Specifically, the camera captures the facial expression, the voice recognition system analyzes the voice, and the data is sent to the emotion recognition engine.

[1389] Step 8:

[1390] The server receives the emotional state from the emotion recognition engine and stores it in a database. The input is the emotional state data, and the output is a new entry in the database. This allows emotional information to be used as part of health analysis.

[1391] Step 9:

[1392] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis. These data are the input, and the output is a comprehensive health assessment and advice suggestion data. The AI ​​module analyzes this data comprehensively and evaluates the user's health condition.

[1393] Step 10:

[1394] Based on the results of the AI ​​analysis, the server proposes specific meal and exercise menus to the user to maintain their health. The input is the AI ​​analysis results, and the output is the proposals. The server displays these proposals on the user's smartphone or in-car display, providing them in an easy-to-follow format.

[1395] In this way, the system of the present invention automates the entire process from data collection to analysis and proposals, thereby supporting the user's health management.

[1396] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1397] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1398] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1399] [Fourth embodiment]

[1400] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1401] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1402] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1403] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1404] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1405] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1406] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1407] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1408] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1409] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1410] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1411] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1412] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1413] This invention relates to a system that allows users to easily record their dietary and exercise information and provides necessary advice for health management. Specifically, it utilizes a smartphone or smartwatch to automatically collect dietary and exercise information, analyzes it using artificial intelligence (AI), and proposes an appropriate health maintenance menu.

[1414] System configuration and operation

[1415] Input and analysis of dietary information

[1416] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[1417] The server then passes the image file to an artificial intelligence module, which analyzes the image and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The results of this analysis are stored in a database for later analysis.

[1418] Acquiring exercise and sleep information

[1419] The device (smartphone or smartwatch) automatically collects data about the user's exercise and sleep, such as the number of steps, distance, heart rate, and sleep time, and periodically transmits this data to a server.

[1420] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[1421] Suggestions for maintaining good health

[1422] The server uses AI to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to evaluate the user's health. Based on this evaluation, it proposes specific diet and exercise menus to help the user maintain their health.

[1423] For example, if the server determines that the user has not been exercising recently, it can suggest that the user "go for a 30-minute jog next weekend." If the user's nutritional intake is unbalanced, the server can recommend that the user "eat more fish and vegetables at your next dinner."

[1424] Support for implementing proposals (future enhancements)

[1425] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[1426] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[1427] The processing flow will be explained below.

[1428] Step 1:

[1429] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[1430] Step 2:

[1431] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[1432] Step 3:

[1433] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[1434] Step 4:

[1435] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[1436] Step 5:

[1437] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[1438] Step 6:

[1439] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[1440] Step 7:

[1441] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[1442] Step 8:

[1443] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[1444] Step 9:

[1445] The server passes the accumulated dietary, exercise, and sleep data to the AI ​​for analysis, which then analyzes past data and trends to assess the user's health.

[1446] Step 10:

[1447] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "go for a 30-minute jog next weekend" or "eat more fish and vegetables at your next dinner."

[1448] Step 11:

[1449] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[1450] Step 12:

[1451] (as a future extension)

[1452] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[1453] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management.

[1454] Example 1

[1455] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1456] In modern society, health management has become an important issue, but it is difficult to accurately record diet and exercise habits in a busy lifestyle. Furthermore, there are only a limited number of systems that comprehensively evaluate health status and provide specific recommendations tailored to individual users. In conventional systems, analyzing dietary content and collecting exercise and sleep information is time-consuming, and it is difficult to comprehensively analyze this data and provide appropriate advice. In particular, flexible recommendations utilizing the user's location information have not been realized. There is a need to improve these issues and provide a system that allows users to easily manage their health.

[1457] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1458] In this invention, the server includes a means for users to photograph their meal and upload the images on the spot; a means for analyzing the uploaded images and calculating the ingredients, three major nutrients, and calories; a means for analyzing previously registered meal contents and trends and suggesting meal menus necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or wearable device; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for analyzing images using an artificial intelligence module and storing them in a database; and a means for comprehensively analyzing the user's data, evaluating their health status, and providing advice. This allows users to easily analyze the photographed meal contents, centrally manage their exercise and sleep information, and receive specific suggestions based on their individual health status. Furthermore, a function for assisting users in implementing the suggested menus can be provided using the user's location information, making it easier for users to carry out health-maintaining activities.

[1459] "User" refers to an individual who uses this system to record dietary, exercise, and sleep information and manage their health.

[1460] "Means for taking photos of meal contents and uploading images on the spot" refers to a function that allows users to take photos of their meal using a smartphone camera or a dedicated app and send the image data to a server in real time.

[1461] "Means for analyzing uploaded images and calculating the ingredients, three major nutrients, and calories" refers to an algorithm in which an artificial intelligence module running on a server analyzes the meal images sent by users and calculates the ingredients, proteins, fats, carbohydrates, and calories contained in the food.

[1462] "A means for analyzing previously registered dietary content and trends and proposing dietary menus necessary for maintaining health" refers to a function that suggests dietary menus suited to the user's current health condition based on the user's past dietary data stored in a database.

[1463] "Means for acquiring exercise and sleep information from a smartphone or wearable device" refers to the function of a smartphone or smartwatch worn by a user to collect data such as the number of steps taken during exercise, distance traveled, heart rate, and sleep time, and transmit this data to a server.

[1464] "Means of analyzing the amount of exercise required to maintain health based on the acquired information and proposing an appropriate exercise menu" refers to the function in which the server analyzes exercise and sleep information, and generates and presents the optimal exercise menu for the user.

[1465] "Image analysis using an artificial intelligence module and storage in a database" refers to the function in which an AI running on a server analyzes food images and stores the results in a database.

[1466] "Means for comprehensively analyzing user data, assessing health status, and providing advice" refers to the function of comprehensively analyzing dietary information, exercise information, and sleep information collected by the server, assessing the user's health status, and providing specific advice for maintaining good health.

[1467] MODE FOR CARRYING OUT THE INVENTION

[1468] The present invention relates to a system that allows users to efficiently and easily record their daily diet and exercise information and perform comprehensive health management. This system utilizes smartphones and wearable devices, analyzes data using artificial intelligence (AI), and proposes appropriate health maintenance menus to users.

[1469] Hardware and software used

[1470] Hardware:

[1471] Smartphone

[1472] Wearable devices such as smartwatches

[1473] server

[1474] software:

[1475] Dedicated application (for smartphones)

[1476] Server-side artificial intelligence module

[1477] Database System

[1478] Specific explanation of the system's operation

[1479] Users use a smartphone with a dedicated app installed to take pictures of their daily meals. At this time, they can also enter detailed information about the meal and the types of ingredients. The photographed food images are uploaded to a server via the smartphone. At this time, metadata such as the date and time of the photo and location information are also sent along with the image.

[1480] The server passes the received image file to an artificial intelligence module. The AI ​​analyzes the image, identifies the ingredients in the image, and calculates the protein, fat, carbohydrates, and calories of each ingredient. This data is stored in a database. For example, if you take a photo and upload a meal of bread, eggs, and bananas for breakfast, the AI ​​will automatically generate and record the following: "Bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "Egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "Banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[1481] The devices, such as smartphones and smartwatches, automatically collect users' exercise information (e.g., number of steps, distance traveled, heart rate) and sleep information. This data is periodically sent to a server and stored in a database.

[1482] The server comprehensively analyzes the collected dietary, exercise, and sleep data to evaluate the user's health condition. Based on this evaluation, it suggests appropriate meal and exercise menus to the user. For example, if it determines that the user has not been exercising recently, the server will suggest "go for a 30-minute jog next weekend." If the user's nutrition is unbalanced, it will recommend "eat more fish and vegetables at your next dinner."

[1483] Some examples of specific prompts include:

[1484] "I had bread, eggs, and a banana for breakfast. What are the calories and nutrients in this meal?"

[1485] In the future, the server may use the user's location information to link inventory information from nearby commercial facilities and restaurant menus to support the implementation of suggested menus, making it easier for users to implement suggested health maintenance menus and supporting continuous health management.

[1486] As described above, the system of the present invention enables the user to efficiently manage their health and supports continuous health maintenance activities.

[1487] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1488] Step 1:

[1489] The user launches the dedicated application and takes a photo of the meal with their smartphone. The user also has the option to input ingredients and meal details. The input data is the image of the meal and text information about the ingredients.

[1490] Input: Meal image, ingredient information

[1491] Output: Food image data and ingredient information stored on the smartphone

[1492] Specific behavior:

[1493] For breakfast, users take a photo of bread, eggs, and bananas and upload it to the app.

[1494] Step 2:

[1495] The device (smartphone) transmits the captured food image and the input ingredient information to the server. This transmission includes metadata such as the date and time of the image capture and location information along with the image data.

[1496] Input: Food image data, ingredient information, metadata

[1497] Output: Food image data, ingredient information, and metadata sent to the server

[1498] Specific behavior:

[1499] The smartphone captures a photo of the meal and information about the ingredients, and then sends the photo, information, and metadata all at once to the server.

[1500] Step 3:

[1501] The server then passes the image files to an artificial intelligence module, which uses image analysis technology to identify the ingredients in the image and calculate the protein, fat, carbohydrates, and calories of each ingredient. The analysis results are then stored in a database.

[1502] Input: Submitted food image data, metadata

[1503] Output: Ingredient data, nutrient data, and calorie data for each ingredient stored in the database

[1504] Specific behavior:

[1505] The AI ​​analyzes the image and generates and stores data such as "bread 150kcal, protein 5g, fat 3g, carbohydrates 25g," "egg 80kcal, protein 7g, fat 5g, carbohydrates 1g," and "banana 90kcal, protein 1g, fat 0g, carbohydrates 23g."

[1506] Step 4:

[1507] The device (smartphone or wearable device) automatically collects the user's exercise and sleep information. The collected data includes the number of steps taken, distance traveled, heart rate, and sleep time. This data is sent to a server at regular intervals.

[1508] Input: User's exercise and sleep data

[1509] Output: Exercise and sleep data sent to the server

[1510] Specific behavior:

[1511] The smartwatch monitors your heart rate 24 hours a day and sends your step count and sleep data to a server at the end of the day.

[1512] Step 5:

[1513] The server stores the received exercise and sleep information in a database, making it possible to grasp the user's overall health condition.

[1514] Input: exercise data, sleep data sent

[1515] Output: Exercise data and sleep data stored in a database

[1516] Specific behavior:

[1517] The server records the received exercise data in a database and stores the data by day, week, or month.

[1518] Step 6:

[1519] The server uses AI to perform a comprehensive analysis of the collected dietary, exercise, and sleep data. Based on the results of this analysis, the server evaluates the user's health status and suggests specific diet and exercise menus.

[1520] Input: Food data, exercise data, and sleep data in the database

[1521] Output: Proposal of a health maintenance menu for the user

[1522] Specific behavior:

[1523] The server generates a health maintenance menu such as, "Based on your recent data, you have continued to lack exercise, so we recommend that you go for a 30-minute jog next weekend. Also, your nutritional intake tends to be unbalanced, so we recommend that you eat more fish and vegetables for your next dinner," and notifies the user.

[1524] Step 7:

[1525] In the future, the server will use the user's location information to provide a function that links inventory information from nearby commercial facilities and menus offered by restaurants to support the implementation of suggested menus.

[1526] Input: User's location information, suggested health menu

[1527] Output: Information about commercial facilities and restaurants based on the user's location

[1528] Specific behavior:

[1529] The server obtains the user's location information and provides information on stock availability at nearby supermarkets where the suggested ingredients can be purchased, as well as information on restaurants that offer the suggested menu items.

[1530] (Application example 1)

[1531] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1532] Conventional health management systems require users to manually input information, posing challenges in terms of time and accuracy. Furthermore, there was a lack of systems that could comprehensively grasp the health status of employees in specific work environments, such as factories, and provide optimal health management plans. As a result, there is a demand for ways to maintain employee health and improve work efficiency.

[1533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1534] In this invention, the server includes: a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of ingredients; a means for analyzing previously registered meal contents and trends and proposing a meal menu necessary for maintaining health; a means for acquiring exercise and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and proposing an appropriate exercise menu; a means for automatically collecting health data including the user's physiological data and analyzing it in real time; a means for sending alerts and providing appropriate health advice based on the results of the health data analysis; and a means for evaluating the health status of factory workers and supporting appropriate health management. This eliminates the need for users to manually input information and enables real-time understanding and evaluation of health status, thereby improving the health of factory workers and improving work efficiency.

[1535] "User" refers to a person who uses the system to manage their own health condition.

[1536] "Meal contents" refers to all of the foods and menu items consumed by the user.

[1537] "Image" refers to a photograph of the meal contents taken by the user using a smartphone or other photographic device.

[1538] "Upload" refers to the act of sending an image taken by a user to a server.

[1539] "Food ingredients" refers to the nutritional and chemical constituents of the individual foods in a meal.

[1540] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[1541] A "calorie" is a unit of measurement that indicates the amount of energy contained in food, and refers to the energy provided by food.

[1542] "Exercise information" refers to exercise data during the user's daily activities, including the number of steps, heart rate, distance, and the like.

[1543] "Sleep information" refers to information such as the user's sleep patterns and sleep duration.

[1544] An "exercise menu" refers to an exercise plan proposed for the purpose of maintaining the user's health.

[1545] "Physiological data" refers to data on the user's physical physiological condition, such as heart rate, body temperature, and blood pressure.

[1546] "Health Data" refers to data reflecting a user's overall health, including diet, exercise, sleep, and physiological data.

[1547] "Real-time" refers to data being collected and analyzed immediately, with results being provided almost instantly.

[1548] "Alert" refers to a notification sent to alert you based on the results of an analysis of your health data.

[1549] "Health advice" refers to specific advice for maintaining health provided to a user based on analyzed health data.

[1550] "Factory workers" refers to employees who work in a particular industrial facility or manufacturing site.

[1551] "Health status" refers to the overall physical and mental health of a user.

[1552] "Health management" refers to actions and plans to maintain and improve a user's health.

[1553] "Server" refers to a computer system that collects and analyzes data and provides information to users.

[1554] System Overview

[1555] This system allows users to easily record their diet and exercise information and provides them with advice on health management. By using a smartphone or smartwatch, the system automatically collects diet and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus.

[1556] Hardware and software used

[1557] Smartphone: A device that allows users to take pictures of their meals and upload the images to a server.

[1558] Smartwatch: A device that automatically collects a user's exercise and sleep information.

[1559] Server: A central computer system that performs image analysis, stores data, and performs AI-based analysis and proposals.

[1560] AI model: A generative AI model is used to analyze food images and generate optimal health maintenance menus based on individual health conditions.

[1561] Program processing overview

[1562] Input and analysis of dietary information

[1563] Users take photos of their daily meals with their smartphones and use a dedicated app to upload the images to a server. The server also receives image metadata (such as the date and time the photo was taken, location information, etc.) and passes the image files to an AI model. The AI ​​model analyzes the images and calculates the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The results of this analysis are stored in a database for later analysis.

[1564] Acquiring exercise and sleep information

[1565] Smartwatches automatically collect data on the user's exercise and sleep. For example, they record information such as the number of steps taken, distance traveled, heart rate, and sleep time, and periodically send it to a server. The server stores the received exercise and sleep information in a database, enabling the user to grasp their overall health status.

[1566] Suggestions for maintaining good health

[1567] The server uses an AI model to analyze the collected dietary, exercise, and sleep data. It analyzes past data and trends to assess the user's health status. Based on this assessment, it proposes specific diet and exercise menus to help the user maintain their health.

[1568] Support for implementing proposals (future enhancements)

[1569] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[1570] Specific examples

[1571] Assume an employee is wearing a smartwatch, which collects data such as heart rate, steps, and sleep time, and sends it to a server as follows:

[1572] Heart rate: 80 BPM

[1573] Steps: 6700

[1574] Sleep time: 7.2 hours

[1575] Based on this data, the user's health status can be assessed and prompts such as the following can be fed into a generative AI model:

[1576] The user has a heart rate of 80, steps taken 6700, and 7.2 hours of sleep. What health advice should be provided?

[1577] Based on the analysis results, the generative AI model provides specific advice such as, "Today, we recommend that you maintain moderate exercise and eat a vegetable-based diet."

[1578] As described above, this system makes it easier for users to take daily actions to maintain their health and supports sustainable health management.

[1579] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1580] Step 1:

[1581] Users take photos of their meals and upload the images on the spot.

[1582] Input: A user takes a photo of their meal using their smartphone.

[1583] How it works: The user launches a camera app and takes a picture of their meal. After taking the picture, they upload the image to the server via a dedicated app. The server receives the image along with metadata such as the date and time the photo was taken and its location.

[1584] Output: Image data and metadata uploaded to the server.

[1585] Step 2:

[1586] The server analyzes the received image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[1587] Input: Image data uploaded to the server.

[1588] How it works: The server passes the image to the AI ​​model, which recognizes the food and calculates its ingredients, the three major nutrients (protein, fat, carbohydrates) and calories.

[1589] Output: Ingredients, macronutrients, and calorie information for analyzed ingredients.

[1590] Step 3:

[1591] The server stores the analysis results in a database.

[1592] Input: Ingredients, macronutrients, and calorie information.

[1593] Specific operation: The server writes the analysis results to a database, which is used for future analyses required for health management.

[1594] Output: Analysis results stored in a database.

[1595] Step 4:

[1596] The smartwatch collects the user's exercise and sleep information.

[1597] Input: User's exercise and sleep information.

[1598] How it works: The smartwatch uses sensors to record information such as heart rate, steps, distance traveled, and sleep time in real time, and periodically transmits the recorded data to a server via the smartphone.

[1599] Output: Exercise and sleep data sent to the server.

[1600] Step 5:

[1601] The server stores the collected exercise and sleep information in a database.

[1602] Input: Exercise and sleep information sent from your smartwatch.

[1603] What it does: The server stores the received exercise and sleep information in a database, which allows it to understand the user's overall health.

[1604] Output: Exercise and sleep data stored in a database.

[1605] Step 6:

[1606] Based on the data collected by the server, the system evaluates the user's health condition and suggests an appropriate health maintenance menu.

[1607] Input: Food data, exercise data, sleep data.

[1608] Specific operation: The server uses an AI model to analyze past data and trends stored in the database and evaluate the user's health condition. The AI ​​model uses a generative AI model to generate optimal meal and exercise menus based on the user's health condition. It also generates specific health advice by inputting a prompt: "The user's heart rate is X, number of steps is Y, and sleep time is Z hours. What kind of health advice should be provided?"

[1609] Output: Health maintenance menu and advice suggested to the user.

[1610] Step 7:

[1611] The server sends alerts based on the analysis of health data and provides appropriate health advice.

[1612] Input: Alert information and advice generated based on the user's health status.

[1613] How it works: The server generates alerts when user-defined criteria are exceeded and sends notifications via smartphone, including alerts for excessive heart rate or insufficient sleep, along with health advice generated by the AI ​​model.

[1614] Output: Alert notification and health advice to the user.

[1615] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1616] The present invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice necessary for health management. Furthermore, the present invention also includes a feature that incorporates an emotion engine that recognizes the user's emotions. Specifically, the system utilizes a smartphone or smartwatch to automatically acquire dietary and exercise information, analyzes it using artificial intelligence (AI), and suggests appropriate health maintenance menus. The emotion engine also recognizes the user's emotions and improves the suggestions based on those information.

[1617] System configuration and operation

[1618] Input and analysis of dietary information

[1619] Users take photos of their daily meals with their smartphones and upload the images to a server using a dedicated app. The device then transfers the images taken by the user to the server. At this time, the image metadata (e.g., the date and time of the photo, location information, etc.) is also sent.

[1620] The server then passes the image file to the analysis module, where AI analyzes the image to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the food. The analysis results are stored in a database for later analysis.

[1621] Acquiring exercise and sleep information

[1622] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information, such as the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server.

[1623] The server stores the received exercise and sleep information in a database, allowing the user to understand their overall health condition.

[1624] Acquisition and analysis of emotional information

[1625] Users wear smartphones or smartwatches and record emotional information. In particular, emotions are recognized in real time through facial recognition and voice analysis. For example, facial expressions are read using a smartphone camera, and the emotion engine analyzes the emotional state based on that data.

[1626] The device transmits emotional information to a server, so that the user's emotional state is also used as part of the health analysis.

[1627] Suggestions for maintaining good health

[1628] The server passes the accumulated dietary, exercise, sleep, and emotional data to the AI ​​for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, the AI ​​proposes specific dietary and exercise menus for the user to maintain their health.

[1629] For example, if the server determines that the user has not been exercising recently and is under stress, it can suggest that the user "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in the next dinner." If the user's nutritional intake is unbalanced, the server can also suggest "a balanced dish for the next meal."

[1630] Support for implementing proposals (future enhancements)

[1631] In the future, the server will be able to use the user's location information to link with information on the nearest commercial facilities and restaurants, making it possible to make suggestions in a more actionable format. For example, it could provide the user with inventory information for nearby supermarkets where they can purchase the suggested ingredients, or guide them to restaurants that offer the suggested menu.

[1632] As a result, the system of the present invention makes it easier for users to take daily actions to maintain their health and supports sustainable health management that also takes emotional states into consideration.

[1633] The processing flow will be explained below.

[1634] Step 1:

[1635] Users take photos of their daily meals with their smartphones and then use a dedicated app to upload the images to a server, along with metadata such as the date and time the photo was taken and the location.

[1636] Step 2:

[1637] The device (smartphone) transfers the captured images and metadata to the server. Once the transfer is complete, the app displays a "Upload Complete" notification to the user.

[1638] Step 3:

[1639] The server then passes the image files to an analysis module, which runs an AI-based image recognition algorithm.

[1640] Step 4:

[1641] AI (in the server) analyzes the image, detects and calculates the type of ingredients, the amount, the three major nutrients (protein, fat, carbohydrates) and calories. This data is returned to the server as a temporary analysis result.

[1642] Step 5:

[1643] The server stores the analysis results in a database, along with past meal data, for future use in trend analysis and proposal creation.

[1644] Step 6:

[1645] The device (smartphone or smartwatch) automatically records the user's exercise information (e.g., number of steps, distance, heart rate) and sleep information (e.g., bedtime, wake-up time, sleep quality).

[1646] Step 7:

[1647] The device periodically sends recorded exercise and sleep information to the server. The transmission interval can be changed in the app settings.

[1648] Step 8:

[1649] The server stores the received exercise and sleep information in a database, enabling comprehensive management of the user's health condition.

[1650] Step 9:

[1651] Users wear smartphones or smartwatches that record emotional information, and in particular, recognize emotions in real time through facial recognition and voice analysis.

[1652] Step 10:

[1653] The device sends the emotion information to the server, along with the emotion data processed by the emotion engine.

[1654] Step 11:

[1655] The server integrates dietary, exercise, and sleep data, including emotional information, and passes it on to the AI ​​for analysis. The AI ​​analyzes past data and trends to provide a comprehensive assessment of the user's health and emotional state.

[1656] Step 12:

[1657] The server generates specific health maintenance suggestions based on the AI ​​analysis results, such as "do 30 minutes of yoga to relieve stress" or "include foods with a relaxing effect in your next dinner."

[1658] Step 13:

[1659] The server then sends the generated proposal to the user's smartphone, where the user can check the proposal details through a dedicated app.

[1660] Step 14:

[1661] (as a future extension)

[1662] The server uses the user's location information to connect with information on the nearest commercial facilities and restaurants, thereby providing the user with information on where to purchase ingredients needed to prepare the suggested menu and information on restaurants that serve the suggested dishes.

[1663] This makes it easier for users to take daily actions to maintain their health, and creates a system that supports sustainable health management that also takes emotional state into account.

[1664] Example 2

[1665] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1666] Modern society demands comprehensive health management that takes into account individual lifestyle habits and emotional states. However, conventional approaches only collect and analyze individual data (e.g., diet, exercise, sleep), making it difficult to achieve comprehensive health management that also includes emotional information. Furthermore, there is a lack of implementation support to determine whether health management proposals are feasible.

[1667] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of their meal and upload the image on the spot, a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients, and a means for recording the user's emotional information in real time using a smartphone or smartwatch and analyzing it with an emotion engine. This allows for appropriate health management suggestions that take into account the user's overall health condition and also include emotional information. Furthermore, by providing support for implementing the suggested menu, more effective health management is achieved.

[1668] A "user" is an individual who uses this system to record and manage information about their diet, exercise, sleep, and emotions.

[1669] A "terminal" is a device (e.g., a smartphone or smartwatch) that a user uses to take photos of their meals and record exercise information, sleep information, and emotional information.

[1670] The "server" is a central system that receives data sent from the terminals, analyzes it, stores it in a database, and makes suggestions for health management.

[1671] "Upload" is the operation or process of sending data from a terminal to a server.

[1672] "Image analysis" is a process that uses AI technology to analyze uploaded food images and calculate the ingredients, three major nutrients, and calories of the ingredients.

[1673] The "three macronutrients" are protein, fat, and carbohydrates, which are the main components of food.

[1674] A "health maintenance menu" is a meal and exercise plan proposed to maintain or improve the user's health.

[1675] An "emotion engine" is an algorithm or software for analyzing a user's emotional information.

[1676] "Emotional information" is data that indicates the user's current mood or emotional state, and is collected, for example, by facial expression recognition or voice analysis.

[1677] The "database" is a collection of structured data that allows for efficient storage and management of collected dietary, exercise, sleep, and emotional information.

[1678] "Location information" is data that indicates a user's current geographic location.

[1679] "Health status" is comprehensive information that indicates the physical and mental state of the user.

[1680] "Artificial intelligence" is a technology that learns from a user's past health and emotional state data and generates optimal suggestions.

[1681] "Commercial facilities" are stores that sell food ingredients and health-related products, and facilities that provide food and beverage services.

[1682] This invention relates to a system that allows users to easily record their dietary and exercise information and provides them with advice on health management. This system also includes a function for recording and analyzing emotional information in real time. Using devices such as smartphones and smartwatches, the system automatically acquires dietary and exercise information and analyzes it using artificial intelligence (AI) to suggest appropriate health maintenance menus. It also uses an emotion engine to recognize the user's emotions and improve the suggestions based on those emotions.

[1683] Input and analysis of dietary information

[1684] Users take photos of their daily meals with their smartphone and upload the images to a server using a dedicated app. When the smartphone transfers the captured image to the server, it also sends image metadata (e.g., the date and time the image was taken, location information, etc.). The server then passes the received image file to an AI analysis module. The AI ​​analysis module uses image recognition technology (e.g., YOLO or ResNet) to detect and calculate the ingredients, the three major nutrients (protein, fat, carbohydrates), and calories of the ingredients. The analysis results are stored in a database and can be used for later analysis.

[1685] Example: A user takes a photo of the bread and coffee they had for breakfast with their smartphone and uploads the image. The server receives the image, and an AI analysis module automatically analyzes the type of bread and coffee, their nutritional content, and their calories. The analysis results are stored in a database.

[1686] Example prompt sentence:

[1687] Can you analyze a photo of my breakfast and give me calorie and nutrition information?

[1688] Acquiring exercise and sleep information

[1689] The device (smartphone or smartwatch) automatically collects the user's exercise and sleep information. The collected data includes the number of steps, distance, heart rate, and sleep time. This data is periodically sent to a server. The server stores the received exercise and sleep information in a database and uses it as basic data for understanding the user's overall health condition.

[1690] Example: A user wears a smartwatch and goes for a run. After the run, the smartwatch records the number of steps, distance traveled, and heart rate, and automatically sends this data to a server. The server stores this data in a database for later analysis.

[1691] Example prompt sentence:

[1692] Record your running data today and let me know your total steps and calories burned.

[1693] Acquisition and analysis of emotional information

[1694] Users record emotional information using their smartphones or smartwatches. In particular, emotions are recognized in real time through facial recognition and voice analysis. Using the smartphone's camera and microphone, the emotion engine analyzes facial expressions and vocal states to obtain emotional state data. The emotional information collected by the device is sent to a server, and the user's emotional state is also used as part of health analysis.

[1695] Example: A user uses a smartphone and the emotion engine analyzes facial expressions captured by the camera during a video chat, records the user's emotional state (e.g., joy, sadness, stress), and sends it to a server.

[1696] Example prompt sentence:

[1697] Please analyze my current emotional state and let me know.

[1698] Suggestions for maintaining good health

[1699] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI analysis module for analysis. The AI ​​analyzes past data and trends to assess the user's health and emotional state. Based on this assessment, it suggests specific meal and exercise menus to maintain the user's health. For example, if it determines that the user has not been exercising recently and is under stress, the server might suggest "doing 30 minutes of yoga to relieve stress" or "having some relaxing herbal tea with your next dinner." It can also suggest "a balanced meal for your next meal" if the user's nutrition is unbalanced.

[1700] Example: The server determines that the user is under stress based on dietary, exercise, and emotional data. The AI ​​generates a suggestion, such as "Drink some relaxing herbal tea with your next dinner," and notifies the user.

[1701] Example prompt sentence:

[1702] Based on your recent exercise and emotional data, we'll suggest what you should do to take care of your health.

[1703] Support for implementing proposals (future enhancements)

[1704] The server uses the user's location information to support the execution of the suggestions by providing information on nearby commercial facilities and restaurants. For example, it provides the user with information on inventory at the nearest commercial facility where the suggested ingredients can be purchased, or information on restaurants that offer the suggested menu.

[1705] Example: When a user tries to purchase suggested ingredients, the server provides inventory information from the nearest supermarket, helping the user to shop efficiently.

[1706] Example prompt sentence:

[1707] Could you please tell me the nearest supermarket where I can buy the ingredients you suggested?

[1708] In this way, this system is closely integrated into the user's daily life, collecting and analyzing data on diet, exercise, sleep, and emotions, and supporting comprehensive, personalized health management.

[1709] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1710] Step 1: Photograph and upload your meal

[1711] Users take pictures of their daily meals with their smartphones and upload the images to a server using a dedicated app.

[1712] Input: Photographed food image, metadata (photo date and time, location information)

[1713] Output: Food images and metadata sent to the server

[1714] How it works: The user takes a photo of their breakfast and uploads it through a dedicated app. The image is accompanied by information about the date, time, and location of the photo.

[1715] Step 2: Analyze the images

[1716] The server passes the uploaded meal image to an AI analysis module, which analyzes the image and calculates the ingredients, three major nutrients, and calories of the ingredients.

[1717] Input: Uploaded food image and metadata

[1718] Output: Analysis results of ingredients, three major nutrients, and calories

[1719] Specific operation: The server passes the image to the AI ​​analysis module, which analyzes the type of ingredients and their nutritional content.

[1720] Step 3: Saving to the database

[1721] The server stores the analysis results in a database, which includes information such as the ingredients, the ratio of the three major nutrients, and calories.

[1722] Input: Analysis results of ingredients, three major nutrients, and calories

[1723] Output: Analysis data stored in a database

[1724] Specific operation: The server stores the analysis results in a database, making them available for later analysis.

[1725] Step 4: Record your exercise and sleep information

[1726] The device (smartwatch) periodically collects the user's exercise data (number of steps, distance, heart rate) and sleep data (sleep time, sleep quality).

[1727] Input: User's exercise data, sleep data

[1728] Output: Collected exercise data, sleep data

[1729] Specific operation: The user wears the smartwatch and goes out for exercise. The smartwatch automatically records the number of steps taken and the amount of sleep.

[1730] Step 5: Send your exercise and sleep data

[1731] The terminal periodically transmits the collected exercise and sleep data to the server.

[1732] Input: Collected exercise data, sleep data

[1733] Output: Exercise data and sleep data sent to the server

[1734] Specific operation: The smartwatch periodically sends exercise and sleep data to the server.

[1735] Step 6: Save to database

[1736] The server stores the received exercise and sleep data in a database, allowing the user to understand their overall health status.

[1737] Input: Exercise data and sleep data sent to the server

[1738] Output: Exercise data and sleep data stored in a database

[1739] Specific operation: The server stores the received exercise and sleep data in a database and uses it to comprehensively understand the user's health condition.

[1740] Step 7: Recording Emotional Information

[1741] Users record their emotional information using their smartphones or smartwatches, and emotions are recognized in real time through facial recognition and voice analysis.

[1742] Input: User's facial expression data, voice data

[1743] Output: Recorded emotion information

[1744] Specific operation: The user engages in a video chat, and facial expressions are captured on camera and analyzed by the emotion engine.

[1745] Step 8: Sending Emotional Information

[1746] The terminal transmits the collected emotion information to the server.

[1747] Input: Recorded emotion information

[1748] Output: Emotion information sent to the server

[1749] Specific operation: The smartphone sends the recorded emotional information to the server.

[1750] Step 9: Analyze and store emotional information

[1751] The server passes the received emotional information to an analysis module and stores the emotional state in a database.

[1752] Input: Emotion information sent to the server

[1753] Output: Emotional state data stored in a database

[1754] Specific operation: The emotion engine analyzes the received emotion information and stores the results in a database.

[1755] Step 10: Comprehensive data analysis

[1756] The server passes dietary, exercise, sleep, and emotional data to an AI analysis module to evaluate the user's health and emotional state.

[1757] Input: Various data stored in the database (diet, exercise, sleep, emotions)

[1758] Output: Assessment of the user's health and emotional state

[1759] Specific operation: The server analyzes data on diet, exercise, sleep, and emotions to assess overall health.

[1760] Step 11: Proposal for a health maintenance menu

[1761] Based on the analysis results, the server suggests specific meal and exercise menus to the user to maintain their health.

[1762] Input: User's health and emotional state assessment results

[1763] Output: Generated health maintenance menu

[1764] Specific behavior: Based on the user's health and emotional state, the AI ​​generates suggestions such as "do 30 minutes of yoga to relieve stress" or "have some relaxing herbal tea with dinner."

[1765] Step 12: Submit your proposal

[1766] The server transmits the generated proposal content to the terminal and notifies the user.

[1767] Input: Generated health maintenance menu

[1768] Output: The suggestion sent to the user

[1769] Specific operation: The generated health suggestions are displayed on the smartphone and the user is notified.

[1770] Step 13: Proposal implementation support

[1771] The server provides information on nearby commercial facilities and restaurants based on the user's location information, and supports the implementation of the suggestions.

[1772] Input: User's location information, suggestions

[1773] Output: Information to support the execution of the proposal (e.g., food inventory information and restaurant information)

[1774] Specific operation: The server provides inventory information of the nearest commercial establishment where the suggested ingredients can be purchased, helping the user to efficiently execute the suggested menu.

[1775] (Application example 2)

[1776] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1777] In modern self-driving vehicles, health management of drivers and passengers is not sufficiently considered, making it difficult to maintain good health, especially during long periods of driving or riding. In addition, conventional health management systems do not take emotional states into account, which means they are unable to provide health recommendations that address stress and emotional fluctuations. The present invention aims to provide a comfortable and healthy driving environment by efficiently managing the user's health in a self-driving vehicle and providing health recommendations that address emotional fluctuations.

[1778] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to photograph their meal and upload the image on the spot; a means for analyzing the uploaded image and calculating the ingredients, three major nutrients, and calories of the ingredients; a means for analyzing previously registered meal contents and trends and suggesting a meal menu necessary for maintaining health; a means for acquiring exercise information and sleep information from a smartphone or smartwatch; a means for analyzing the amount of exercise necessary for maintaining health based on the acquired information and suggesting an appropriate exercise menu; a means for integrating food records, exercise information, and emotional information into the internal information system of the autonomous vehicle to monitor the health status in real time and provide personalized health advice; and a means for acquiring the emotional state of passengers using an in-vehicle camera and voice recognition system and adjusting health advice based on the emotions. This not only enables health management of users in the vehicle but also makes it possible to provide advice that takes emotional state into consideration.

[1779] "User" refers to an individual who uses the system, particularly for the purpose of health management within an autonomous vehicle.

[1780] "Meal contents" refers to the overall meal that the user ingests, and specifically includes information such as ingredients, the three major nutrients, and calories.

[1781] "Images" refers to visual data such as photos or videos taken by the user of their meal.

[1782] "Means for uploading" refers to the function for sending images taken by the user to the cloud or server.

[1783] "Means for analyzing" refers to the function of analyzing uploaded images and calculating the ingredients, three major nutrients, and calories of ingredients.

[1784] "Ingredients" refers to the nutrients and components contained in each food, specifically vitamins and minerals.

[1785] The "macronutrients" refer to the three main nutrients: protein, fat, and carbohydrates.

[1786] A "calorie" is a unit that indicates the amount of energy contained in food.

[1787] "Means for suggesting meal menus" refers to a function that presents meal menus suitable for the user based on the analyzed meal contents.

[1788] "Exercise information" refers to information such as the amount of exercise, type of exercise, and time of exercise performed by the user.

[1789] "Sleep information" refers to information related to the user's sleep time, sleep quality, and the like.

[1790] "Means of acquisition" refers to the ability to collect data from devices such as smartphones and smartwatches.

[1791] The "means for suggesting an exercise menu" refers to a function that suggests the type and amount of exercise suitable for the user based on the acquired exercise information.

[1792] "Autonomous vehicle" refers to a vehicle that has the ability to drive itself.

[1793] "Internal information system" refers to a system installed within an autonomous vehicle for managing and analyzing healthcare-related data.

[1794] "Means for monitoring health status" refers to a function that integrates food records, exercise information, and emotional information to monitor the user's health status in real time.

[1795] "Means for providing personalized health advice" refers to a function that makes personalized health management suggestions based on user data.

[1796] An "in-car camera" refers to a camera installed inside a vehicle that captures the user's facial expressions and behavior.

[1797] A "voice recognition system" refers to a system that has the function of analyzing a user's speech and extracting linguistic information.

[1798] "Means for acquiring emotional state" refers to the function of assessing the user's emotions using an in-car camera or voice recognition system.

[1799] "Means for adjusting health suggestions based on emotions" refers to a function that uses the results of emotion analysis to modify the suggestions to suit the user's emotional state.

[1800] The present invention relates to a system for managing the health of users in autonomous vehicles. Specifically, the system allows users to photograph and record their meals, acquire exercise information, sleep information, and emotional state, and provide health advice based on this data. This allows users to maintain their health even during long trips in the car.

[1801] Hardware Configuration

[1802] The main hardware configuration of this system is as follows:

[1803] Autonomous vehicle internal information system: This system comprehensively manages various data necessary for health management.

[1804] Smartphones and smartwatches: Users use these devices to capture photos of their meals and to obtain exercise and sleep information.

[1805] In-car camera: Used to analyze the user's facial expressions and capture their emotional state.

[1806] Speech recognition system: Analyzes the user's voice and complements their emotional state.

[1807] Software Configuration

[1808] The software configuration is as follows:

[1809] AI-based analysis module: The present invention uses machine learning frameworks such as TensorFlow and Keras.

[1810] Emotion recognition engine: A model (e.g. TensorFlow / Keras) for analyzing the user's facial expressions and recognizing their emotional state.

[1811] Nutrition Analysis API: Analyzes uploaded food images and extracts nutritional information.

[1812] How it works

[1813] The operation of this system is as follows.

[1814] 1. Entering and analyzing dietary information

[1815] The user takes a photo of their meal using a smartphone, and the image is automatically uploaded to the autonomous vehicle's internal information system.

[1816] The uploaded image is passed to an AI-based analysis module that calculates the ingredients, macronutrients, and calories of the food, and the results are stored in a database for later analysis.

[1817] 2. Acquisition of exercise and sleep information

[1818] Smartphones and smartwatches automatically collect users' exercise and sleep information, including steps, distance, heart rate, and sleep duration.

[1819] The collected data is periodically transmitted to the autonomous vehicle's internal information system and stored in a database.

[1820] 3. Acquisition and analysis of emotional information

[1821] The user's emotional state is captured in real time using an in-car camera and a voice recognition system: the camera reads the user's facial expressions, and the voice recognition system analyzes the user's voice.

[1822] An emotion recognition engine evaluates the user's emotional state and stores it in a database.

[1823] 4. Suggestions for maintaining good health

[1824] The internal information system of the autonomous vehicle passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis.

[1825] The AI ​​analyzes past data and trends to assess the user's health and emotional state, and based on this assessment, suggests specific diet and exercise regimes to help maintain health.

[1826] Specific examples

[1827] For example, if a user is eating lunch in the car, an image of the meal is taken and a nutritional analysis is performed. If the analysis results indicate that the meal is high in calories, appropriate nutritional supplements and advice about the next meal are provided. At the same time, if facial recognition detects that the user's emotion is "sad," suggestions for breaks and relaxation methods to relieve stress are made.

[1828] Prompt Sentence Examples

[1829] "Take a picture of the user eating in the car and provide health advice based on the analysis results."

[1830] In this way, this system provides comprehensive support for health management in the car and offers advice that takes into account the driver's emotional state.

[1831] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1832] Step 1:

[1833] A user takes a photo of their meal using their smartphone and uploads the image to the server. The input is the image of the meal, and the output is the uploaded image data. Specifically, the user launches the smartphone's camera app, takes a photo of the meal, and then transfers the image file to the cloud server via a dedicated app.

[1834] Step 2:

[1835] The server receives the uploaded images and passes them to the analysis module. The input is the uploaded image data, and the output is the transmission of the image data to obtain the analysis results. The server also sends metadata (e.g., shooting date and time, location information, etc.) along with the image file to the analysis module.

[1836] Step 3:

[1837] The analysis module uses the received image to calculate the ingredients, three major nutrients, and calories of the ingredients. The input is image data, and the output is ingredient information, nutrient information, and calorie calculation data. Specifically, it uses AI frameworks such as TensorFlow and Keras to recognize ingredients from the image and calculate the corresponding ingredients and nutritional information based on a database.

[1838] Step 4:

[1839] The server stores the nutrition information received from the analysis module in a database. The input is the analysis result data, and the output is a new entry in the database. The server organizes this data by user and stores it for later analysis.

[1840] Step 5:

[1841] The device acquires exercise and sleep information from a smartphone or smartwatch and sends it to a server. The input is exercise and sleep information from the device, and the output is data sent to the server. Specifically, the device periodically collects data such as the number of steps taken, heart rate, and sleep status, and sends it to a cloud server.

[1842] Step 6:

[1843] The server stores the exercise and sleep information in a database. The input is the submitted exercise and sleep information, and the output is a new entry in the database. The server manages this data for each user and uses it for comprehensive health assessment.

[1844] Step 7:

[1845] The in-car camera and voice recognition system capture the user's emotional information in real time and send it to the server. The input is the user's facial expression and voice data, and the output is the analysis result of the emotional state. Specifically, the camera captures the facial expression, the voice recognition system analyzes the voice, and the data is sent to the emotion recognition engine.

[1846] Step 8:

[1847] The server receives the emotional state from the emotion recognition engine and stores it in a database. The input is the emotional state data, and the output is a new entry in the database. This allows emotional information to be used as part of health analysis.

[1848] Step 9:

[1849] The server passes the accumulated dietary, exercise, sleep, and emotional data to an AI-based analysis module for analysis. These data are the input, and the output is a comprehensive health assessment and advice suggestion data. The AI ​​module analyzes this data comprehensively and evaluates the user's health condition.

[1850] Step 10:

[1851] Based on the results of the AI ​​analysis, the server proposes specific meal and exercise menus to the user to maintain their health. The input is the AI ​​analysis results, and the output is the proposals. The server displays these proposals on the user's smartphone or in-car display, providing them in an...

Claims

1. A means for users to take photos of their meals and upload the images on the spot; A means of analyzing uploaded images and calculating ingredients, three major nutrients, and calories of ingredients; A method for analyzing previously registered dietary content and trends to suggest dietary menus necessary for maintaining health; A means for acquiring exercise information and sleep information from a smartphone or a smartwatch; Based on the acquired information, we will analyze the amount of exercise required to maintain health and provide a means to suggest an appropriate exercise menu. Including system.

2. Using the user's location information, we will provide a means to link inventory information from nearby commercial facilities and menus offered by restaurants to support the implementation of proposed menus. The system of claim 1 .

3. Using artificial intelligence to learn from a user's past health data and generate optimal recommendations based on their individual health status. The system of claim 1 .

Images