System

Abstract

A system is provided.SOLUTION: A system including means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying a face of the user, means for inputting data including a health condition and a meal history of the user to a generative artificial intelligence model and analyzing the data, means for generating personalized health management advice for the user based on an analysis result, and means for notifying the user of the generated advice by voice and image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current home health management and lifestyle support relies on individual efforts, and there is a lack of tools to efficiently manage the health and development of all family members. As a result, proper nutrition and health management may not be achieved, leading to a decline in quality of life. In addition, it is difficult to provide personalized support based on individual health conditions and habits, and efficient methods are needed. [Means for solving the problem]

[0005] The present invention provides a system that uses a camera equipped with image recognition technology and a generative artificial intelligence model to analyze a user's health condition and lifestyle habits in real time and provide individually optimized advice. Specifically, the system includes a means for capturing image or video data of the user using a camera equipped with image recognition technology, a means for analyzing the captured data and identifying the user's face, a means for analyzing the user's health condition and dietary history, a means for generating personalized health management advice based on the analysis results, and a means for notifying the generated advice via audio and image. This improves the efficiency of health management and lifestyle support at home and promotes the health and growth of each individual user.

[0006] "Image recognition technology" is a technology that analyzes images and videos obtained from cameras and sensors to identify and recognize specific objects and people.

[0007] A "camera" is a device that captures light and records it as an image or video.

[0008] "User" refers to an individual or household member who uses the system.

[0009] "Data" refers to information recorded in the form of images, video, audio, numerical information, etc.

[0010] "Analysis" refers to the process of processing and analyzing data and extracting useful information from it.

[0011] A "generative artificial intelligence model" refers to an algorithm or system that uses artificial intelligence techniques to analyze data and generate new information or actions.

[0012] "Health Status" refers to information and indicators related to a user's physical and mental health.

[0013] "Lifestyle" refers to a user's daily actions and habits, particularly behavioral patterns related to diet and exercise.

[0014] "Personalization" refers to providing information and services that are optimized for each individual user.

[0015] "Advice" refers to specific instructions or suggestions given to a user to improve their health or lifestyle.

[0016] "Notification" refers to the act of informing the user of generated information or advice. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention provides a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The system is configured and operates as follows.

[0039] 1. System Configuration

[0040] 1.1 Terminal (small robot)

[0041] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0042] 1.2 Server

[0043] The server receives the image and video data sent from the device and uses image recognition technology to identify the user's face.The server also contains a generative artificial intelligence model that analyzes the user's health condition and dietary history.

[0044] 1.3 Communication Networks

[0045] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[0046] 2. Program Processing

[0047] 2.1 Data Collection

[0048] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0049] 2.2 Image Recognition and Face Identification

[0050] The server analyzes the received image and video data and identifies the user's face, allowing it to identify each family member and associate their health status and dietary history.

[0051] 2.3 Health Data Analysis

[0052] The server inputs the user's health data and dietary history into a generative AI model for analysis. Based on the analysis results, the user's health condition and nutritional deficiencies are identified.

[0053] 2.4 Generating Personalized Advice

[0054] Based on the analysis results, a generative AI model generates personalized advice for each user, including suggestions for improving their health and diet.

[0055] 2.5 Advice Notice

[0056] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or a display. For example, if a father has a cold, the device might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds this time of year."

[0057] 3. Specific Examples

[0058] Example 1: Dad has a cold

[0059] User: Dad

[0060] Device: Small robot captures dad's daily life

[0061] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0062] Device: Advises, "Chicken and vegetable soup is recommended for those battling a cold this time of year," and displays the recipe on the screen

[0063] Example 2: Nutritional management for growing children

[0064] User: Growing child

[0065] Device: Small robot captures children's daily lives

[0066] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0067] Device: "You've grown 1cm since a month ago. You may be deficient in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, and the specific ingredients and recommended intake are displayed on the screen.

[0068] 4. Response to additional questions

[0069] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0070] User: Child asks robot a question

[0071] Terminal: Sends question to server

[0072] Server: Analyzes using a generative AI model and identifies appropriate fatigue recovery items

[0073] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises, and displays images of specific items to purchase on the screen.

[0074] In this way, the system efficiently manages users' health conditions and lifestyle habits and provides individually optimized advice in real time, improving the health and quality of life of the entire family.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] Subject: Device

[0078] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[0079] Step 2:

[0080] Subject: Server

[0081] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[0082] Step 3:

[0083] Subject: Server

[0084] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[0085] Step 4:

[0086] Subject: Server

[0087] The server provides real-time updates on the user's daily life and health status based on a database, and this information is fed into a generative artificial intelligence model.

[0088] Step 5:

[0089] Subject: Server

[0090] The server uses the generated AI model to analyze the user's health condition and dietary history. The model identifies the user's nutritional deficiencies and health risks and generates analysis results.

[0091] Step 6:

[0092] Subject: Server

[0093] Based on the analysis results, the server generates personalized health management advice, including specific suggestions for actions and dietary improvements.

[0094] Step 7:

[0095] Subject: Server

[0096] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[0097] Step 8:

[0098] Subject: Device

[0099] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[0100] Step 9:

[0101] Subject: User

[0102] The user can ask the robot additional questions, such as, "I have a stop at a convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0103] Step 10:

[0104] Subject: Device

[0105] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[0106] Step 11:

[0107] Subject: Server

[0108] The server uses a generative AI model to analyze the user's question and generate the optimal answer, taking into account the amount of practice and past data.

[0109] Step 12:

[0110] Subject: Server

[0111] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[0112] Step 13:

[0113] Subject: Device

[0114] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] Conventional health management systems have difficulty understanding a user's individual health condition and lifestyle habits in detail and providing optimized advice in real time. Furthermore, if the user's face is not identified or data analysis is not performed accurately, the advice provided may be inappropriate. Furthermore, it is difficult to respond to follow-up questions from users, which hinders the improvement of user satisfaction.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes means for capturing image or video data of a user using a photographing device equipped with image recognition technology, means for transmitting the captured image or video data in real time to an analysis device via a communication device, means for analyzing the received image or video data in the analysis device and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice via an audio output device and a display device. This makes it possible to efficiently and accurately manage the user's health condition and lifestyle habits and provide individually optimized advice in real time.

[0120] "Image recognition technology" is a technology that uses computer vision algorithms to detect and identify specific objects or features within images or video data.

[0121] "Photographing device" refers to a device for taking images or videos, such as a camera or video camera.

[0122] A "communications device" is a device for transmitting and receiving data, and includes devices that use network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[0123] An "analysis device" is a computer system or device for processing and analyzing received data.

[0124] "Means for identifying a user's face" refers to algorithms or software that extract the user's facial features from image or video data and identify that face.

[0125] "Health status" refers to information about the user's physical health in general, including data such as blood pressure, heart rate, and body temperature.

[0126] "Diet history" refers to information about meals the user has had in the past, and includes data such as ingredients, calorie intake, and meal times.

[0127] A "generative artificial intelligence model" is an AI model that uses machine learning and deep learning technologies to generate analysis results and predictions from input data.

[0128] The "means for generating advice based on analysis results" is software or an algorithm for generating optimal health management and lifestyle improvement advice for the user based on the analysis results.

[0129] The "audio output device" is a device such as a speaker that reproduces the generated advice by voice.

[0130] A "display device" is a device such as a display or monitor for visually displaying the generated advice.

[0131] The present invention is a system for supporting family health management and lifestyle optimization, which includes a photographing device equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The configuration and operation of this system are described in detail below.

[0132] System Configuration

[0133] The system consists of three main parts:

[0134] 1. Terminal (a small device with a built-in camera)

[0135] 2. Server (computer system for data analysis)

[0136] 3. Communication network (infrastructure for sending and receiving data)

[0137] Terminal

[0138] The device is a small device with a built-in camera that records the user's daily life. The device has the following features:

[0139] Camera: Equipped with image recognition technology, it takes pictures of the user's face and how they are eating.

[0140] Communication module: Sends data to the server in real time using Wi-Fi or Bluetooth.

[0141] Audio output: The generated advice is notified to the user by voice.

[0142] Display: Visually displays generated advice and notifications.

[0143] server

[0144] The server is a computer system that receives image and video data sent from the device and performs multiple analyses. The server has the following functions:

[0145] Data reception: Receives data sent from the terminal.

[0146] Image Recognition: Preprocess image data and reduce noise using Python's OpenCV library.

[0147] Facial recognition: A deep learning model using TensorFlow extracts facial features and matches them with facial data registered in a database.

[0148] Health data analysis: User health data and dietary history are input into a generative artificial intelligence model (e.g., PyTorch model) and analyzed.

[0149] Personalized advice generation: The analysis results are converted into text using a natural language generation (NLG) API, and the most appropriate advice is generated for the user.

[0150] communication network

[0151] A communications network is an infrastructure that enables real-time data transmission between devices and servers, and primarily uses network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[0152] Specific examples

[0153] Example 1: Dad has a cold

[0154] User:Dad

[0155] Device: A small device captures Dad's daily life and collects data such as changes in his complexion and the frequency of his coughs.

[0156] Server: Analyzes the received data, identifies cold symptoms, and compares them with past data to generate an appropriate soup recipe.

[0157] Device: The generated advice is announced aloud, such as "Chicken and vegetable soup is recommended for colds at this time of year," and the recipe is shown on the screen.

[0158] Example 2: Nutritional management for growing children

[0159] User: Growing child

[0160] Device: A small device that captures a child's daily life and records what they eat and how they exercise.

[0161] Server: Analyzes the received data and identifies iron and zinc deficiencies based on height and weight measurements and dietary history.

[0162] Device: "You've grown 1cm taller compared to a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken a day," the device says, showing specific ingredients and recommended intake amounts on the display and providing a voice notification.

[0163] Example 3: Advice for recovering from fatigue

[0164] User: My child has baseball practice after school, but I still have time to stop by a convenience store.

[0165] Terminal: Sends the question to the server.

[0166] Server: Analyzes the question using a generative artificial intelligence model and identifies appropriate fatigue recovery items.

[0167] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises aloud, and displays images of specific items to purchase on the screen.

[0168] The system efficiently manages users' health conditions and lifestyle habits and provides personalized, optimized advice in real time, improving the health and quality of life of the entire family.

[0169] Prompt Sentence Examples

[0170] "Generate appropriate dietary advice for users who are showing signs of a cold."

[0171] "Please analyze the nutritional status of growing children and advise them on the nutrients they need."

[0172] "Please tell me some foods that are effective in relieving fatigue."

[0173] In this way, the system can provide personalized health care advice based on the user's individual data.

[0174] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0175] Step 1:

[0176] Data collection

[0177] The device captures images and video data of the user's daily life and sends the data to a server in real time.

[0178] Input: Image and video data capturing user activity

[0179] Specific operation: The device's built-in camera captures the user's face and actions, and sends the image and video data to a server using communication methods such as Wi-Fi or Bluetooth.

[0180] Output: Image and video data sent to the server

[0181] Step 2:

[0182] Image Recognition and Facial Identification

[0183] The server analyzes the image and video data it receives and identifies the user's face.

[0184] Input: Image and video data sent from the device

[0185] How it works: The server preprocesses the image data using Python's OpenCV library, extracts and identifies facial features using a deep learning model (e.g., TensorFlow), and matches the facial information in the database for face identification.

[0186] Output: Identified user's face information

[0187] Step 3:

[0188] Health data analysis

[0189] The server inputs the user's health data and dietary history into a generative AI model for analysis, identifying the user's health condition and nutritional deficiencies.

[0190] Input: Identified user's face information, user's health data, diet history

[0191] Specific operation: The server inputs health data (e.g., blood pressure, heart rate, etc.) and dietary history into a generative AI model (e.g., PyTorch model) for analysis. The analysis then estimates nutritional and health status.

[0192] Output: Analysis results (health status, nutritional deficiency, etc.)

[0193] Step 4:

[0194] Generating personalized advice

[0195] The server generates personalized health management advice for the user based on the analysis results.

[0196] Input: Analysis results (health status, nutritional deficiency, etc.)

[0197] Specific operation: The server uses a natural language generation (NLG) API to convert the analysis results into easy-to-understand sentences. For example, if there are signs of a cold, it generates specific advice such as "Chicken and vegetable soup is recommended for colds at this time of year."

[0198] Output: Personalized health advice

[0199] Step 5:

[0200] Advice Notice

[0201] The server sends the generated advice to the device, which notifies the user through voice or display.

[0202] Enter: personalized health advice.

[0203] Specific operation: The server sends the generated advice to the device using a message transmission protocol (e.g., MQTT). The device receives the advice and notifies the user by voice output or display on the screen. For example, the device may notify the user by voice, saying, "Chicken and vegetable soup is recommended for colds this time of year," and show the recipe on the screen.

[0204] Output: Audio and display advice

[0205] In this way, the system effectively manages users' health and lifestyle habits and provides personalized advice in real time.

[0206] (Application example 1)

[0207] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0208] In today's world, autonomous vehicles are becoming increasingly common, but there is a lack of systems that can monitor the health and stress levels of occupants in real time and provide appropriate advice and encourage rest. If this problem is not solved, fatigue and stress will accumulate due to long periods of driving, increasing the risk of accidents and health problems. The present invention aims to solve this problem by effectively monitoring the health and stress levels of occupants in autonomous vehicles and providing appropriate advice.

[0209] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0210] In this invention, the server includes: means for capturing images or video data of a user using a camera equipped with image recognition technology; means for analyzing the captured images or video data and identifying the user's face; means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis; means for generating personalized health management advice for the user based on the analysis results; means for notifying the user of the generated advice by voice and image; means for monitoring the health condition and stress level of occupants in an autonomous vehicle and collecting health data using an on-board camera and sensor; means for generating appropriate rest and refreshment advice for the occupants based on the collected health data; and means for notifying the occupants of the advice using an in-vehicle display or audio output device. This makes it possible to effectively manage the health condition and stress level of occupants in an autonomous vehicle in real time and provide appropriate advice.

[0211] "Image recognition technology" is a technology that automatically identifies specific objects or patterns from images or video data captured by a camera.

[0212] "User" refers to a person who uses the system, and in this case includes the occupants of an autonomous vehicle.

[0213] A "generative artificial intelligence model" is a model that uses machine learning algorithms to learn specific patterns and characteristics from large amounts of data and make judgments such as predictions and classifications.

[0214] "Personalized health management advice" refers to advice that is individually optimized based on the user's analyzed health condition and behavioral history.

[0215] A "camera" is a device that converts light into electrical signals to capture images and videos.

[0216] An "on-board camera" is a camera installed inside an autonomous vehicle to capture images of the occupants and their surroundings.

[0217] A "sensor" is a device that measures a physical or chemical property and is primarily used in this system to collect health data.

[0218] A "display" is a device that visually displays information.

[0219] An "audio output device" is a device that outputs information in the form of audio.

[0220] This invention is a system for monitoring the health status and stress level of occupants in an autonomous vehicle and providing appropriate advice. This system is implemented with the following configuration.

[0221] First, the server captures images or video data of the occupants using the vehicle's onboard camera. The captured images and video data are sent to the server via a communications network. The server then analyzes the received data using image recognition technology to identify the occupants' faces. Image processing software such as OpenCV and TensorFlow is used for facial recognition.

[0222] The server then inputs data, including the occupant's health status and dietary history, into a generative AI model for analysis. This generative AI model is used to estimate the occupant's stress level and fatigue level from their facial color and facial expression. The generative AI model uses Keras / TensorFlow. Additional sensors may also be used as needed to collect health data such as heart rate and respiratory rate.

[0223] Based on the analysis results, the server generates personalized health management advice for the occupant. For example, this advice might be, "If high fatigue level is detected while driving, notify the occupant to take a break." The generated advice is communicated to the occupant via a display or audio output device in the vehicle.

[0224] For example, if a passenger begins to feel tired during a long drive, an on-board camera will capture a change in facial color, which the generative AI model will analyze and detect that fatigue is building up. As a result, the server will generate appropriate advice, such as "Take a break at the next service area," and notify the passenger via display or voice.

[0225] Additionally, if a passenger asks a specific question, the generative AI model will perform further analysis based on the question and generate an appropriate response. For example, if the passenger asks, "I'm feeling tired today. What can I do to refresh myself?", the system will provide advice such as, "I recommend taking a short walk and getting some fresh air."

[0226] An example of a prompt might be "Input a picture of the user's face and estimate their fatigue level." This prompt enables the generative AI model to analyze the user's health condition and generate appropriate advice.

[0227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0228] Step 1:

[0229] The terminal (a system inside the vehicle) captures images or video data of the occupants using an on-board camera. The input is the camera image, and the output is high-resolution image or video data. This data is sent to the server in real time.

[0230] Step 2:

[0231] The server analyzes the received image and video data and identifies the faces of passengers using image recognition technology (OpenCV or TensorFlow). The input is image or video data, and the output is the coordinates of the facial area and identification information. Based on the facial identification results, the data of each passenger is associated.

[0232] Step 3:

[0233] The server collects the health condition data and dietary history data of the identified occupants and inputs them into a generative AI model. The input is facial identification information and past health data, and the output is the analysis results of the health condition. Using the generative AI model (Keras / TensorFlow), stress levels and fatigue levels are estimated from facial color and facial expressions.

[0234] Step 4:

[0235] The server generates personalized health management advice for the occupants based on the analysis results. The input is the health status analysis result, and the output is the advice content. This advice includes encouraging them to take breaks as needed and specific health management suggestions.

[0236] Step 5:

[0237] The generated advice is sent to the terminal, which then notifies the occupant using a display or audio output device. The input is the generated advice, and the output is the notification to the occupant. In this case, the advice is provided to the occupant using speech synthesis (gTTS) or text display.

[0238] Step 6:

[0239] When a user (passenger) asks a specific question to the terminal, the terminal sends the question to the server. The input is the passenger's voice question, and the output is the question data.

[0240] Step 7:

[0241] The server inputs the received question into a generative artificial intelligence model to generate appropriate advice based on the question. The input is the question data and related health data, and the output is the answer advice.

[0242] Step 8:

[0243] The generated answer advice is sent to the terminal, and the terminal notifies the occupant using a display or audio output device. The input is the answer advice, and the output is the answer notification to the occupant. This notification is provided in a form that is easy for the occupant to understand using voice synthesis or text display.

[0244] This series of processing flows makes it possible to manage the health status and stress levels of occupants in self-driving vehicles in real time and provide appropriate advice.

[0245] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0246] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice using an emotion engine. The system is configured and operates as follows.

[0247] 1. System Configuration

[0248] 1.1 Terminal (small robot)

[0249] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0250] 1.2 Server

[0251] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and emotion engine to analyze the user's health condition, dietary history, and emotional state.

[0252] 1.3 Communication Networks

[0253] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[0254] 2. Program Processing

[0255] 2.1 Data Collection

[0256] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0257] 2.2 Image Recognition and Face Identification

[0258] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[0259] 2.3 Health Data Analysis

[0260] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[0261] 2.4 Generating Personalized Advice

[0262] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[0263] 2.5 Advice Notice

[0264] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or display. For example, if a father has a cold, the system might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds at this time of year." The system also makes suggestions that take the user's emotional state into account.

[0265] 3. Specific Examples

[0266] Example 1: Dad has a cold

[0267] User: Dad

[0268] Device: Small robot captures dad's daily life

[0269] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0270] Emotion engine: Analyzes dad's emotional state to identify stress

[0271] Device: "Chicken and vegetable soup is a good option for those with a cold this time of year," the device advises, and displays a recipe on the display. It also suggests relaxation techniques to improve Dad's mood.

[0272] Example 2: Nutritional management for growing children

[0273] User: Growing child

[0274] Device: Small robot captures children's daily lives

[0275] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0276] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[0277] Device: "You've grown 1cm since a month ago. You may be lacking in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, showing specific ingredients and the recommended intake amount on the display. It also provides encouraging messages to motivate the child.

[0278] 4. Response to additional questions

[0279] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0280] User: Child asks robot a question

[0281] Terminal: Sends question to server

[0282] Server: Analyzes using a generative AI model and generates the optimal answer. Also uses an emotion engine.

[0283] Device: The device will announce, "Bananas and yogurt are recommended to help relieve fatigue," and will display images of specific items to purchase. It will also suggest other options based on the child's preferences.

[0284] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[0285] The processing flow will be explained below.

[0286] Step 1:

[0287] Subject: Device

[0288] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[0289] Step 2:

[0290] Subject: Server

[0291] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[0292] Step 3:

[0293] Subject: Server

[0294] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[0295] Step 4:

[0296] Subject: Server

[0297] The server inputs the acquired image and video data into the emotion engine, which analyzes the user's facial expressions and movements to identify their emotional state.

[0298] Step 5:

[0299] Subject: Server

[0300] The server updates the user's emotional state in real time based on the results of the emotion engine, thereby building a database that reflects the user's current emotional state.

[0301] Step 6:

[0302] Subject: Server

[0303] The server inputs health data, dietary history, and emotional state into a generated artificial intelligence model based on this database and performs analysis.

[0304] Step 7:

[0305] Subject: Server

[0306] Based on the analysis results, the generative AI model generates personalized health management advice, including specific suggested actions and dietary improvements.

[0307] Step 8:

[0308] Subject: Server

[0309] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[0310] Step 9:

[0311] Subject: Device

[0312] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[0313] Step 10:

[0314] Subject: User

[0315] The user asks a follow-up question to the device, for example, "I have a chance to stop by a convenience store after school before going to baseball practice. What should I buy to help me recover?"

[0316] Step 11:

[0317] Subject: Device

[0318] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[0319] Step 12:

[0320] Subject: Server

[0321] The server uses a generative artificial intelligence model to analyze the question and generate the best answer, along with an emotion engine that takes into account the user's current emotional state.

[0322] Step 13:

[0323] Subject: Server

[0324] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[0325] Step 14:

[0326] Subject: Device

[0327] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[0328] Example 2

[0329] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] In modern families, it is difficult to manage the health and lifestyles of all family members at once. Providing personalized health management advice for each family member is particularly time-consuming and labor-intensive. Furthermore, there is a lack of means to grasp the health and emotional state of each family member in real time and quickly provide appropriate advice based on that information.

[0331] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0332] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for acquiring the user's health condition, dietary history, and emotional state, means for inputting the acquired data into a generative AI model and an emotion engine for analysis, means for generating personalized health management advice for the user based on the analysis results, means for generating prompt sentences and inputting them into the generative AI model for analysis, means for notifying the user of the generated advice by voice and image, and means for transmitting data from a terminal to the server and receiving the analysis results. This makes it possible to analyze the health conditions and emotional states of all family members in real time and quickly provide individually optimized advice.

[0333] "Image recognition technology" is a technology that analyzes images and videos taken with a camera and identifies the objects and people contained within them.

[0334] A "camera" is a device for taking pictures and videos.

[0335] "User" refers to each member of a family who uses the system.

[0336] "Means for identifying faces" refers to technology for identifying a user's face from captured images or video data.

[0337] "Health status" refers to information about the user's physical health, such as body temperature, appetite, and sleep status.

[0338] "Dietary history" refers to a record of the food and drinks consumed by a user.

[0339] "Emotional state" refers to information that represents the user's psychological state, such as joy, sadness, anger, etc.

[0340] A "generative artificial intelligence model" is an artificial intelligence technology that generates advice suited to the user based on various data.

[0341] An "emotion engine" is a technology that analyzes a user's emotional state and supports appropriate advice.

[0342] A "prompt sentence" refers to a question or instruction sentence that is input into an artificial intelligence model.

[0343] "Analysis results" are information that includes advice and suggestions generated based on your health and emotional state.

[0344] "Audio notification means" refers to a technique or device for conveying the generated advice to the user as audio.

[0345] "Means for notifying by image" refers to a technique or device for displaying the generated advice on a display and conveying it to the user.

[0346] "Terminal" refers to a small robot or device that captures the user's daily life.

[0347] "Means for transmitting data" refers to the technology used to transmit captured images and videos to a server.

[0348] "Means for receiving analysis results" refers to technology for receiving analysis results sent from the server.

[0349] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative AI model, and a means for providing personalized advice using an emotion engine. Specific embodiments of the system are described below.

[0350] First, this system includes a terminal equipped with a built-in camera equipped with image recognition technology for capturing images of the user's daily life. The terminal is in the form of a small robot, and captures images of the user as they go about their daily life. Images and video data captured by this camera are sent to a server in real time. The server and terminal are connected via a communications network, allowing for rapid data transmission and reception.

[0351] The server then analyzes the transmitted image and video data. It uses image recognition technology to identify the user's face, thereby identifying each family member. The server also acquires the user's health status, dietary history, and emotional state. This data is then fed into a generative artificial intelligence model (generative AI model) and emotion engine for analysis.

[0352] Based on the analyzed data, the server generates personalized health management advice optimized for each user. The prompt sentences used to generate this advice are also generated by the server. For example, a prompt sentence such as "Please tell me how I can reduce my recent stress" is input into the generative AI model. The generated advice is sent to the device and communicated to the user via voice and images.

[0353] As a specific operational scenario, consider the case where Dad has caught a cold. The device takes pictures of Dad's daily life and sends them to the server. The server identifies the signs of a cold and generates advice suggesting a soup recipe that is effective against colds. The device notifies Dad by voice, saying, "Chicken and vegetable soup is recommended for colds at this time of year," and shows the recipe on the display.

[0354] In addition, when managing the nutrition of growing children, the device takes pictures of the child's daily life and sends the data to a server. The server analyzes the data and identifies iron and zinc deficiencies. The server advises, "You've grown 1cm taller than you were a month ago. You're deficient in iron and zinc, so you should eat 100 grams of spinach and chicken per day," and the device notifies the child of the specific foods and the amount to consume.

[0355] Examples of prompts include, "What meals and recipes would you recommend if Dad had a cold?" and "Please suggest meals that contain the nutrients a growing child needs."

[0356] In this way, the system can analyze the user's health and emotional state in real time and quickly provide individually optimized advice, which is expected to improve the health and quality of life of the entire family.

[0357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0358] Program processing flow

[0359] Step 1: Data collection

[0360] Operation:

[0361] The device uses a built-in camera to capture images and video data of the user's daily life.

[0362] The device transmits this data to the server in real time.

[0363] Input: Images and video data from the user's daily life.

[0364] Output: Image and video data sent to the server.

[0365] Step 2: Image Recognition and Face Identification

[0366] Operation:

[0367] The server analyzes the image and video data received from the device and identifies the user's face.

[0368] The server associates the identified facial data with individual user profiles.

[0369] Input: Image and video data transmitted in real time.

[0370] Output: User's face identification results and corresponding profile data.

[0371] Step 3: Acquire data and prepare for analysis

[0372] Operation:

[0373] The server retrieves the user's health status, dietary history, and emotional state from a database.

[0374] The server prepares the acquired data for input into the generative artificial intelligence model and emotion engine.

[0375] Input: User profile data, health status, diet history, emotional state.

[0376] Output: Input data for analysis.

[0377] Step 4: Analyze your health and emotional state

[0378] Operation:

[0379] The server uses a generative artificial intelligence model and an emotion engine to analyze the user's health and emotional state.

[0380] The server calculates the analysis results.

[0381] Input: The input data for the analysis.

[0382] Output: Analysis of the user's health and emotional state.

[0383] Step 5: Generating Advice

[0384] Operation:

[0385] Based on the analysis results, the server generates health management advice optimized for each user.

[0386] The server generates a prompt sentence and inputs it into the generative AI model.

[0387] Input: Analysis results of health and emotional state, prompt text.

[0388] Output: Personalized health care advice.

[0389] Step 6: Advice Notification

[0390] Operation:

[0391] The server transmits the generated advice to the terminal.

[0392] The terminal notifies the user of the advice by voice and image.

[0393] Input: Generated personalized health care advice.

[0394] Output: Advice notification to the user.

[0395] Specific examples of each step

[0396] Step 1: Data collection

[0397] Operation: The device takes pictures of the user eating breakfast with its camera and sends the captured images and videos to a server every five minutes.

[0398] Input: Image and video data of a user having breakfast.

[0399] Output: Image and video data received by the server.

[0400] Step 2: Image Recognition and Face Identification

[0401] How it works: The server analyzes the image data sent from breakfast and uses a recognition algorithm to identify Dad's face.

[0402] Input: Image data taken during breakfast.

[0403] Output: Dad's face identification result and corresponding profile data.

[0404] Step 3: Acquire data and prepare for analysis

[0405] How it works: The server retrieves Dad's eating history and emotional state from the database for the past week and prepares it as data for analysis.

[0406] Input: Dad's profile data, food history for the past week, and emotional state.

[0407] Output: Input data for Daddy's analysis.

[0408] Step 4: Analyze your health and emotional state

[0409] How it works: The server uses a generative artificial intelligence model and an emotion engine to identify that Dad has a vitamin C deficiency and has recently had high stress levels.

[0410] Input: Input data for Daddy's analysis.

[0411] Output: Analysis of dad's health and emotional state.

[0412] Step 5: Generating Advice

[0413] How it works: Based on the analysis results, the server generates advice such as "Eat orange juice and vegetables to make up for vitamin C deficiency" and inputs the relevant prompt sentence into the generative AI model.

[0414] Input: Analysis results of dad's health and emotional state, prompt text.

[0415] Output: Personalized health advice for dad.

[0416] Step 6: Advice Notification

[0417] How it works: The server sends the generated advice to the device, which then announces aloud, "To compensate for your vitamin C deficiency, you should drink orange juice," and shows the specific suggestion on the display.

[0418] Enter: personalized health care advice for dads.

[0419] Output: Audio notification and image display of advice to dad.

[0420] (Application example 2)

[0421] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0422] Conventional health management systems monitor users' health status and analyze their dietary history, but lack a means for users to easily obtain recommended ingredients and nutrients. Furthermore, simply providing personalized advice makes it difficult to encourage specific behavioral changes, limiting its effectiveness, especially in today's busy society. Furthermore, systems lack sufficient support for health management that takes into account the user's emotional state. Therefore, more comprehensive and practical health management support is needed.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0424] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice by voice and image and enabling the user to order appropriate meals in cooperation with a food delivery service. This allows the user to easily order specific meals based on the health management advice, enabling more practical and comprehensive health management to be achieved.

[0425] "Image recognition technology" is a technology that analyzes images and videos through cameras and other sensors to recognize specific objects and patterns.

[0426] A "generative artificial intelligence model" is a machine learning algorithm or deep learning model that uses large amounts of data to self-learn and perform a specific task (e.g., health analysis or generating personalized advice).

[0427] An "emotion engine" is an algorithm or system that analyzes a user's emotional state from their voice and facial expressions and identifies changes in their emotions.

[0428] "Personalized health management advice" involves analyzing a user's individual health condition and dietary history to generate advice optimized for the user.

[0429] A "food delivery service" is a service that delivers meals ordered by a user to a location specified by the user.

[0430] A "communications network" is an infrastructure that allows devices or systems in different locations to send and receive data.

[0431] 1. System Configuration

[0432] 1.1 Device (smartphone)

[0433] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0434] 1.2 Server

[0435] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and an emotion engine that analyzes the user's health condition, dietary history, and emotional state. Based on the analysis results, it generates personalized health management advice. It also has the ability to link with food delivery services and order appropriate meals.

[0436] 1.3 Communication Networks

[0437] The terminal and the server are connected via a communication network, typically the Internet, and data is sent and received in real time.

[0438] 2. Program Processing

[0439] 2.1 Data Collection

[0440] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0441] 2.2 Image Recognition and Face Identification

[0442] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[0443] 2.3 Health Data Analysis

[0444] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[0445] 2.4 Generating Personalized Advice

[0446] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[0447] 2.5 Notification of Advice and Collaboration with Food Delivery Services

[0448] The server generates advice and sends it to the device, which then receives it and notifies the user via voice or display. Based on the advice, the user can then order an appropriate meal from a food delivery service with the touch of a button.

[0449] 3. Specific Examples

[0450] Example 1: Dad has a cold

[0451] User: Dad

[0452] Device: Smartphone captures dad's daily life

[0453] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0454] Emotion engine: Analyzes dad's emotional state and identifies whether he is stressed or not

[0455] Terminal: Advises, "Chicken and vegetable soup is recommended for this cold season," and displays the recipe on the screen. It also links with food delivery services, allowing soup to be ordered with the touch of a button.

[0456] Example 2: Nutritional management for growing children

[0457] User: Growing child

[0458] Device: Smartphones capture children's daily lives

[0459] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0460] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[0461] Device: "You've grown 1cm taller than you were a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken per day," the device advises, showing specific ingredients and recommended intake amounts on the display. Furthermore, the device will link with a food delivery service, allowing users to order spinach and chicken.

[0462] Prompt Sentence Examples

[0463] "Please suggest a recommended meal menu for users who have caught a cold."

[0464] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[0465] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0466] Step 1:

[0467] The device camera captures images and video data of the user's daily life, and this data is sent to the server in real time. The input is the image and video data captured by the device camera, and the output is the raw image and video data sent to the server.

[0468] Step 2:

[0469] The server analyzes the received image and video data and uses image recognition technology to identify the user's face. The input is the image and video data sent in step 1, and the output is the identified user's face data. Specifically, the server runs a facial recognition algorithm to extract facial feature points.

[0470] Step 3:

[0471] Based on the identified facial data, the server inputs data including the user's health status and dietary history into a generative artificial intelligence model. The input is the user's facial data, past health status data, and dietary history, and the output is the analysis result regarding the health status. Specifically, the server feeds the health data and dietary history into the AI ​​model to generate a health status.

[0472] Step 4:

[0473] The server uses an emotion engine to analyze the user's emotional state. The input is the user's facial data and voice data, and the output is the analysis result regarding the user's emotional state. Specifically, the server runs a voice analysis algorithm and a facial expression analysis algorithm to generate an emotion tag.

[0474] Step 5:

[0475] The server generates personalized health management advice for the user based on the health data analysis results and emotional state. The input is the health condition analysis results and emotional state analysis results, and the output is health management advice provided to the user. Specifically, the server uses a generative AI model to generate optimal advice in text format from the health analysis results and emotional tags.

[0476] Step 6:

[0477] The server sends the generated advice to the terminal, which then notifies the user of it through voice and display. The input is the advice data sent from the server, and the output is the voice and image data presented to the user. Specifically, the terminal uses voice synthesis technology to convert the advice into voice and displays the text on the display.

[0478] Step 7:

[0479] The server works with food delivery services to provide options for ordering appropriate meals based on the generated advice. The input is the generated advice data and the user's location information, and the output is a meal menu that the user can select and a delivery order screen. Specifically, the server accesses a delivery API and generates an interface that allows the user to order with one click.

[0480] Step 8:

[0481] When the user confirms the order, the server sends the order data to the food delivery service and sends an order confirmation notice to the user. The input is the user's order data, and the output is the order confirmation data sent to the delivery service and the order confirmation notice to the user. Specifically, the server sends the order data to the delivery service's system and sends a confirmation message to the terminal if the order is successful.

[0482] Through this series of processes, users can receive health management advice and easily order delivery meals based on that advice.

[0483] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0484] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0485] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0486] [Second embodiment]

[0487] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0488] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0489] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0490] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0491] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0492] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0493] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0494] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0495] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0496] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0497] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0498] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0499] The present invention provides a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The system is configured and operates as follows.

[0500] 1. System Configuration

[0501] 1.1 Terminal (small robot)

[0502] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0503] 1.2 Server

[0504] The server receives the image and video data sent from the device and uses image recognition technology to identify the user's face.The server also contains a generative artificial intelligence model that analyzes the user's health condition and dietary history.

[0505] 1.3 Communication Networks

[0506] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[0507] 2. Program Processing

[0508] 2.1 Data Collection

[0509] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0510] 2.2 Image Recognition and Face Identification

[0511] The server analyzes the received image and video data and identifies the user's face, allowing it to identify each family member and associate their health status and dietary history.

[0512] 2.3 Health Data Analysis

[0513] The server inputs the user's health data and dietary history into a generative AI model for analysis. Based on the analysis results, the user's health condition and nutritional deficiencies are identified.

[0514] 2.4 Generating Personalized Advice

[0515] Based on the analysis results, a generative AI model generates personalized advice for each user, including suggestions for improving their health and diet.

[0516] 2.5 Advice Notice

[0517] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or a display. For example, if a father has a cold, the device might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds this time of year."

[0518] 3. Specific Examples

[0519] Example 1: Dad has a cold

[0520] User: Dad

[0521] Device: Small robot captures dad's daily life

[0522] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0523] Device: Advises, "Chicken and vegetable soup is recommended for those battling a cold this time of year," and displays the recipe on the screen

[0524] Example 2: Nutritional management for growing children

[0525] User: Growing child

[0526] Device: Small robot captures children's daily lives

[0527] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0528] Device: "You've grown 1cm since a month ago. You may be deficient in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, and the specific ingredients and recommended intake are displayed on the screen.

[0529] 4. Response to additional questions

[0530] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0531] User: Child asks robot a question

[0532] Terminal: Sends question to server

[0533] Server: Analyzes using a generative AI model and identifies appropriate fatigue recovery items

[0534] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises, and displays images of specific items to purchase on the screen.

[0535] In this way, the system efficiently manages users' health conditions and lifestyle habits and provides individually optimized advice in real time, improving the health and quality of life of the entire family.

[0536] The processing flow will be explained below.

[0537] Step 1:

[0538] Subject: Device

[0539] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[0540] Step 2:

[0541] Subject: Server

[0542] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[0543] Step 3:

[0544] Subject: Server

[0545] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[0546] Step 4:

[0547] Subject: Server

[0548] The server provides real-time updates on the user's daily life and health status based on a database, and this information is fed into a generative artificial intelligence model.

[0549] Step 5:

[0550] Subject: Server

[0551] The server uses the generated AI model to analyze the user's health condition and dietary history. The model identifies the user's nutritional deficiencies and health risks and generates analysis results.

[0552] Step 6:

[0553] Subject: Server

[0554] Based on the analysis results, the server generates personalized health management advice, including specific suggestions for actions and dietary improvements.

[0555] Step 7:

[0556] Subject: Server

[0557] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[0558] Step 8:

[0559] Subject: Device

[0560] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[0561] Step 9:

[0562] Subject: User

[0563] The user can ask the robot additional questions, such as, "I have a stop at a convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0564] Step 10:

[0565] Subject: Device

[0566] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[0567] Step 11:

[0568] Subject: Server

[0569] The server uses a generative AI model to analyze the user's question and generate the optimal answer, taking into account the amount of practice and past data.

[0570] Step 12:

[0571] Subject: Server

[0572] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[0573] Step 13:

[0574] Subject: Device

[0575] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[0576] Example 1

[0577] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] Conventional health management systems have difficulty understanding a user's individual health condition and lifestyle habits in detail and providing optimized advice in real time. Furthermore, if the user's face is not identified or data analysis is not performed accurately, the advice provided may be inappropriate. Furthermore, it is difficult to respond to follow-up questions from users, which hinders the improvement of user satisfaction.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0580] In this invention, the server includes means for capturing image or video data of a user using a photographing device equipped with image recognition technology, means for transmitting the captured image or video data in real time to an analysis device via a communication device, means for analyzing the received image or video data in the analysis device and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice via an audio output device and a display device. This makes it possible to efficiently and accurately manage the user's health condition and lifestyle habits and provide individually optimized advice in real time.

[0581] "Image recognition technology" is a technology that uses computer vision algorithms to detect and identify specific objects or features within images or video data.

[0582] "Photographing device" refers to a device for taking images or videos, such as a camera or video camera.

[0583] A "communications device" is a device for transmitting and receiving data, and includes devices that use network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[0584] An "analysis device" is a computer system or device for processing and analyzing received data.

[0585] "Means for identifying a user's face" refers to algorithms or software that extract the user's facial features from image or video data and identify that face.

[0586] "Health status" refers to information about the user's physical health in general, including data such as blood pressure, heart rate, and body temperature.

[0587] "Diet history" refers to information about meals the user has had in the past, and includes data such as ingredients, calorie intake, and meal times.

[0588] A "generative artificial intelligence model" is an AI model that uses machine learning and deep learning technologies to generate analysis results and predictions from input data.

[0589] The "means for generating advice based on analysis results" is software or an algorithm for generating optimal health management and lifestyle improvement advice for the user based on the analysis results.

[0590] The "audio output device" is a device such as a speaker that reproduces the generated advice by voice.

[0591] A "display device" is a device such as a display or monitor for visually displaying the generated advice.

[0592] The present invention is a system for supporting family health management and lifestyle optimization, which includes a photographing device equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The configuration and operation of this system are described in detail below.

[0593] System Configuration

[0594] The system consists of three main parts:

[0595] 1. Terminal (a small device with a built-in camera)

[0596] 2. Server (computer system for data analysis)

[0597] 3. Communication network (infrastructure for sending and receiving data)

[0598] Terminal

[0599] The device is a small device with a built-in camera that records the user's daily life. The device has the following features:

[0600] Camera: Equipped with image recognition technology, it takes pictures of the user's face and how they are eating.

[0601] Communication module: Sends data to the server in real time using Wi-Fi or Bluetooth.

[0602] Audio output: The generated advice is notified to the user by voice.

[0603] Display: Visually displays generated advice and notifications.

[0604] server

[0605] The server is a computer system that receives image and video data sent from the device and performs multiple analyses. The server has the following functions:

[0606] Data reception: Receives data sent from the terminal.

[0607] Image Recognition: Preprocess image data and reduce noise using Python's OpenCV library.

[0608] Facial recognition: A deep learning model using TensorFlow extracts facial features and matches them with facial data registered in a database.

[0609] Health data analysis: User health data and dietary history are input into a generative artificial intelligence model (e.g., PyTorch model) and analyzed.

[0610] Personalized advice generation: The analysis results are converted into text using a natural language generation (NLG) API, and the most appropriate advice is generated for the user.

[0611] communication network

[0612] A communications network is an infrastructure that enables real-time data transmission between devices and servers, and primarily uses network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[0613] Specific examples

[0614] Example 1: Dad has a cold

[0615] User:Dad

[0616] Device: A small device captures Dad's daily life and collects data such as changes in his complexion and the frequency of his coughs.

[0617] Server: Analyzes the received data, identifies cold symptoms, and compares them with past data to generate an appropriate soup recipe.

[0618] Device: The generated advice is announced aloud, such as "Chicken and vegetable soup is recommended for colds at this time of year," and the recipe is shown on the screen.

[0619] Example 2: Nutritional management for growing children

[0620] User: Growing child

[0621] Device: A small device that captures a child's daily life and records what they eat and how they exercise.

[0622] Server: Analyzes the received data and identifies iron and zinc deficiencies based on height and weight measurements and dietary history.

[0623] Device: "You've grown 1cm taller compared to a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken a day," the device says, showing specific ingredients and recommended intake amounts on the display and providing a voice notification.

[0624] Example 3: Advice for recovering from fatigue

[0625] User: My child has baseball practice after school, but I still have time to stop by a convenience store.

[0626] Terminal: Sends the question to the server.

[0627] Server: Analyzes the question using a generative artificial intelligence model and identifies appropriate fatigue recovery items.

[0628] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises aloud, and displays images of specific items to purchase on the screen.

[0629] The system efficiently manages users' health conditions and lifestyle habits and provides personalized, optimized advice in real time, improving the health and quality of life of the entire family.

[0630] Prompt Sentence Examples

[0631] "Generate appropriate dietary advice for users who are showing signs of a cold."

[0632] "Please analyze the nutritional status of growing children and advise them on the nutrients they need."

[0633] "Please tell me some foods that are effective in relieving fatigue."

[0634] In this way, the system can provide personalized health care advice based on the user's individual data.

[0635] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0636] Step 1:

[0637] Data collection

[0638] The device captures images and video data of the user's daily life and sends the data to a server in real time.

[0639] Input: Image and video data capturing user activity

[0640] Specific operation: The device's built-in camera captures the user's face and actions, and sends the image and video data to a server using communication methods such as Wi-Fi or Bluetooth.

[0641] Output: Image and video data sent to the server

[0642] Step 2:

[0643] Image Recognition and Facial Identification

[0644] The server analyzes the image and video data it receives and identifies the user's face.

[0645] Input: Image and video data sent from the device

[0646] How it works: The server preprocesses the image data using Python's OpenCV library, extracts and identifies facial features using a deep learning model (e.g., TensorFlow), and matches the facial information in the database for face identification.

[0647] Output: Identified user's face information

[0648] Step 3:

[0649] Health data analysis

[0650] The server inputs the user's health data and dietary history into a generative AI model for analysis, identifying the user's health condition and nutritional deficiencies.

[0651] Input: Identified user's face information, user's health data, diet history

[0652] Specific operation: The server inputs health data (e.g., blood pressure, heart rate, etc.) and dietary history into a generative AI model (e.g., PyTorch model) for analysis. The analysis then estimates nutritional and health status.

[0653] Output: Analysis results (health status, nutritional deficiency, etc.)

[0654] Step 4:

[0655] Generating personalized advice

[0656] The server generates personalized health management advice for the user based on the analysis results.

[0657] Input: Analysis results (health status, nutritional deficiency, etc.)

[0658] Specific operation: The server uses a natural language generation (NLG) API to convert the analysis results into easy-to-understand sentences. For example, if there are signs of a cold, it generates specific advice such as "Chicken and vegetable soup is recommended for colds at this time of year."

[0659] Output: Personalized health advice

[0660] Step 5:

[0661] Advice Notice

[0662] The server sends the generated advice to the device, which notifies the user through voice or display.

[0663] Enter: personalized health advice.

[0664] Specific operation: The server sends the generated advice to the device using a message transmission protocol (e.g., MQTT). The device receives the advice and notifies the user by voice output or display on the screen. For example, the device may notify the user by voice, saying, "Chicken and vegetable soup is recommended for colds this time of year," and show the recipe on the screen.

[0665] Output: Audio and display advice

[0666] In this way, the system effectively manages users' health and lifestyle habits and provides personalized advice in real time.

[0667] (Application example 1)

[0668] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0669] In today's world, autonomous vehicles are becoming increasingly common, but there is a lack of systems that can monitor the health and stress levels of occupants in real time and provide appropriate advice and encourage rest. If this problem is not solved, fatigue and stress will accumulate due to long periods of driving, increasing the risk of accidents and health problems. The present invention aims to solve this problem by effectively monitoring the health and stress levels of occupants in autonomous vehicles and providing appropriate advice.

[0670] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0671] In this invention, the server includes: means for capturing images or video data of a user using a camera equipped with image recognition technology; means for analyzing the captured images or video data and identifying the user's face; means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis; means for generating personalized health management advice for the user based on the analysis results; means for notifying the user of the generated advice by voice and image; means for monitoring the health condition and stress level of occupants in an autonomous vehicle and collecting health data using an on-board camera and sensor; means for generating appropriate rest and refreshment advice for the occupants based on the collected health data; and means for notifying the occupants of the advice using an in-vehicle display or audio output device. This makes it possible to effectively manage the health condition and stress level of occupants in an autonomous vehicle in real time and provide appropriate advice.

[0672] "Image recognition technology" is a technology that automatically identifies specific objects or patterns from images or video data captured by a camera.

[0673] "User" refers to a person who uses the system, and in this case includes the occupants of an autonomous vehicle.

[0674] A "generative artificial intelligence model" is a model that uses machine learning algorithms to learn specific patterns and characteristics from large amounts of data and make judgments such as predictions and classifications.

[0675] "Personalized health management advice" refers to advice that is individually optimized based on the user's analyzed health condition and behavioral history.

[0676] A "camera" is a device that converts light into electrical signals to capture images and videos.

[0677] An "on-board camera" is a camera installed inside an autonomous vehicle to capture images of the occupants and their surroundings.

[0678] A "sensor" is a device that measures a physical or chemical property and is primarily used in this system to collect health data.

[0679] A "display" is a device that visually displays information.

[0680] An "audio output device" is a device that outputs information in the form of audio.

[0681] This invention is a system for monitoring the health status and stress level of occupants in an autonomous vehicle and providing appropriate advice. This system is implemented with the following configuration.

[0682] First, the server captures images or video data of the occupants using the vehicle's onboard camera. The captured images and video data are sent to the server via a communications network. The server then analyzes the received data using image recognition technology to identify the occupants' faces. Image processing software such as OpenCV and TensorFlow is used for facial recognition.

[0683] The server then inputs data, including the occupant's health status and dietary history, into a generative AI model for analysis. This generative AI model is used to estimate the occupant's stress level and fatigue level from their facial color and facial expression. The generative AI model uses Keras / TensorFlow. Additional sensors may also be used as needed to collect health data such as heart rate and respiratory rate.

[0684] Based on the analysis results, the server generates personalized health management advice for the occupant. For example, this advice might be, "If high fatigue level is detected while driving, notify the occupant to take a break." The generated advice is communicated to the occupant via a display or audio output device in the vehicle.

[0685] For example, if a passenger begins to feel tired during a long drive, an on-board camera will capture a change in facial color, which the generative AI model will analyze and detect that fatigue is building up. As a result, the server will generate appropriate advice, such as "Take a break at the next service area," and notify the passenger via display or voice.

[0686] Additionally, if a passenger asks a specific question, the generative AI model will perform further analysis based on the question and generate an appropriate response. For example, if the passenger asks, "I'm feeling tired today. What can I do to refresh myself?", the system will provide advice such as, "I recommend taking a short walk and getting some fresh air."

[0687] An example of a prompt might be "Input a picture of the user's face and estimate their fatigue level." This prompt enables the generative AI model to analyze the user's health condition and generate appropriate advice.

[0688] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0689] Step 1:

[0690] The terminal (a system inside the vehicle) captures images or video data of the occupants using an on-board camera. The input is the camera image, and the output is high-resolution image or video data. This data is sent to the server in real time.

[0691] Step 2:

[0692] The server analyzes the received image and video data and identifies the faces of passengers using image recognition technology (OpenCV or TensorFlow). The input is image or video data, and the output is the coordinates of the facial area and identification information. Based on the facial identification results, the data of each passenger is associated.

[0693] Step 3:

[0694] The server collects the health condition data and dietary history data of the identified occupants and inputs them into a generative AI model. The input is facial identification information and past health data, and the output is the analysis results of the health condition. Using the generative AI model (Keras / TensorFlow), stress levels and fatigue levels are estimated from facial color and facial expressions.

[0695] Step 4:

[0696] The server generates personalized health management advice for the occupants based on the analysis results. The input is the health status analysis result, and the output is the advice content. This advice includes encouraging them to take breaks as needed and specific health management suggestions.

[0697] Step 5:

[0698] The generated advice is sent to the terminal, which then notifies the occupant using a display or audio output device. The input is the generated advice, and the output is the notification to the occupant. In this case, the advice is provided to the occupant using speech synthesis (gTTS) or text display.

[0699] Step 6:

[0700] When a user (passenger) asks a specific question to the terminal, the terminal sends the question to the server. The input is the passenger's voice question, and the output is the question data.

[0701] Step 7:

[0702] The server inputs the received question into a generative artificial intelligence model to generate appropriate advice based on the question. The input is the question data and related health data, and the output is the answer advice.

[0703] Step 8:

[0704] The generated answer advice is sent to the terminal, and the terminal notifies the occupant using a display or audio output device. The input is the answer advice, and the output is the answer notification to the occupant. This notification is provided in a form that is easy for the occupant to understand using voice synthesis or text display.

[0705] This series of processing flows makes it possible to manage the health status and stress levels of occupants in self-driving vehicles in real time and provide appropriate advice.

[0706] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0707] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice using an emotion engine. The system is configured and operates as follows.

[0708] 1. System Configuration

[0709] 1.1 Terminal (small robot)

[0710] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0711] 1.2 Server

[0712] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and emotion engine to analyze the user's health condition, dietary history, and emotional state.

[0713] 1.3 Communication Networks

[0714] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[0715] 2. Program Processing

[0716] 2.1 Data Collection

[0717] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0718] 2.2 Image Recognition and Face Identification

[0719] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[0720] 2.3 Health Data Analysis

[0721] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[0722] 2.4 Generating Personalized Advice

[0723] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[0724] 2.5 Advice Notice

[0725] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or display. For example, if a father has a cold, the system might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds at this time of year." The system also makes suggestions that take the user's emotional state into account.

[0726] 3. Specific Examples

[0727] Example 1: Dad has a cold

[0728] User: Dad

[0729] Device: Small robot captures dad's daily life

[0730] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0731] Emotion engine: Analyzes dad's emotional state to identify stress

[0732] Device: "Chicken and vegetable soup is a good option for those with a cold this time of year," the device advises, and displays a recipe on the display. It also suggests relaxation techniques to improve Dad's mood.

[0733] Example 2: Nutritional management for growing children

[0734] User: Growing child

[0735] Device: Small robot captures children's daily lives

[0736] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0737] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[0738] Device: "You've grown 1cm since a month ago. You may be lacking in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, showing specific ingredients and the recommended intake amount on the display. It also provides encouraging messages to motivate the child.

[0739] 4. Response to additional questions

[0740] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0741] User: Child asks robot a question

[0742] Terminal: Sends question to server

[0743] Server: Analyzes using a generative AI model and generates the optimal answer. Also uses an emotion engine.

[0744] Device: The device will announce, "Bananas and yogurt are recommended to help relieve fatigue," and will display images of specific items to purchase. It will also suggest other options based on the child's preferences.

[0745] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[0746] The processing flow will be explained below.

[0747] Step 1:

[0748] Subject: Device

[0749] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[0750] Step 2:

[0751] Subject: Server

[0752] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[0753] Step 3:

[0754] Subject: Server

[0755] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[0756] Step 4:

[0757] Subject: Server

[0758] The server inputs the acquired image and video data into the emotion engine, which analyzes the user's facial expressions and movements to identify their emotional state.

[0759] Step 5:

[0760] Subject: Server

[0761] The server updates the user's emotional state in real time based on the results of the emotion engine, thereby building a database that reflects the user's current emotional state.

[0762] Step 6:

[0763] Subject: Server

[0764] The server inputs health data, dietary history, and emotional state into a generated artificial intelligence model based on this database and performs analysis.

[0765] Step 7:

[0766] Subject: Server

[0767] Based on the analysis results, the generative AI model generates personalized health management advice, including specific suggested actions and dietary improvements.

[0768] Step 8:

[0769] Subject: Server

[0770] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[0771] Step 9:

[0772] Subject: Device

[0773] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[0774] Step 10:

[0775] Subject: User

[0776] The user asks a follow-up question to the device, for example, "I have a chance to stop by a convenience store after school before going to baseball practice. What should I buy to help me recover?"

[0777] Step 11:

[0778] Subject: Device

[0779] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[0780] Step 12:

[0781] Subject: Server

[0782] The server uses a generative artificial intelligence model to analyze the question and generate the best answer, along with an emotion engine that takes into account the user's current emotional state.

[0783] Step 13:

[0784] Subject: Server

[0785] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[0786] Step 14:

[0787] Subject: Device

[0788] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[0789] Example 2

[0790] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0791] In modern families, it is difficult to manage the health and lifestyles of all family members at once. Providing personalized health management advice for each family member is particularly time-consuming and labor-intensive. Furthermore, there is a lack of means to grasp the health and emotional state of each family member in real time and quickly provide appropriate advice based on that information.

[0792] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0793] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for acquiring the user's health condition, dietary history, and emotional state, means for inputting the acquired data into a generative AI model and an emotion engine for analysis, means for generating personalized health management advice for the user based on the analysis results, means for generating prompt sentences and inputting them into the generative AI model for analysis, means for notifying the user of the generated advice by voice and image, and means for transmitting data from a terminal to the server and receiving the analysis results. This makes it possible to analyze the health conditions and emotional states of all family members in real time and quickly provide individually optimized advice.

[0794] "Image recognition technology" is a technology that analyzes images and videos taken with a camera and identifies the objects and people contained within them.

[0795] A "camera" is a device for taking pictures and videos.

[0796] "User" refers to each member of a family who uses the system.

[0797] "Means for identifying faces" refers to technology for identifying a user's face from captured images or video data.

[0798] "Health status" refers to information about the user's physical health, such as body temperature, appetite, and sleep status.

[0799] "Dietary history" refers to a record of the food and drinks consumed by a user.

[0800] "Emotional state" refers to information that represents the user's psychological state, such as joy, sadness, anger, etc.

[0801] A "generative artificial intelligence model" is an artificial intelligence technology that generates advice suited to the user based on various data.

[0802] An "emotion engine" is a technology that analyzes a user's emotional state and supports appropriate advice.

[0803] A "prompt sentence" refers to a question or instruction sentence that is input into an artificial intelligence model.

[0804] "Analysis results" are information that includes advice and suggestions generated based on your health and emotional state.

[0805] "Audio notification means" refers to a technique or device for conveying the generated advice to the user as audio.

[0806] "Means for notifying by image" refers to a technique or device for displaying the generated advice on a display and conveying it to the user.

[0807] "Terminal" refers to a small robot or device that captures the user's daily life.

[0808] "Means for transmitting data" refers to the technology used to transmit captured images and videos to a server.

[0809] "Means for receiving analysis results" refers to technology for receiving analysis results sent from the server.

[0810] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative AI model, and a means for providing personalized advice using an emotion engine. Specific embodiments of the system are described below.

[0811] First, this system includes a terminal equipped with a built-in camera equipped with image recognition technology for capturing images of the user's daily life. The terminal is in the form of a small robot, and captures images of the user as they go about their daily life. Images and video data captured by this camera are sent to a server in real time. The server and terminal are connected via a communications network, allowing for rapid data transmission and reception.

[0812] The server then analyzes the transmitted image and video data. It uses image recognition technology to identify the user's face, thereby identifying each family member. The server also acquires the user's health status, dietary history, and emotional state. This data is then fed into a generative artificial intelligence model (generative AI model) and emotion engine for analysis.

[0813] Based on the analyzed data, the server generates personalized health management advice optimized for each user. The prompt sentences used to generate this advice are also generated by the server. For example, a prompt sentence such as "Please tell me how I can reduce my recent stress" is input into the generative AI model. The generated advice is sent to the device and communicated to the user via voice and images.

[0814] As a specific operational scenario, consider the case where Dad has caught a cold. The device takes pictures of Dad's daily life and sends them to the server. The server identifies the signs of a cold and generates advice suggesting a soup recipe that is effective against colds. The device notifies Dad by voice, saying, "Chicken and vegetable soup is recommended for colds at this time of year," and shows the recipe on the display.

[0815] In addition, when managing the nutrition of growing children, the device takes pictures of the child's daily life and sends the data to a server. The server analyzes the data and identifies iron and zinc deficiencies. The server advises, "You've grown 1cm taller than you were a month ago. You're deficient in iron and zinc, so you should eat 100 grams of spinach and chicken per day," and the device notifies the child of the specific foods and the amount to consume.

[0816] Examples of prompts include, "What meals and recipes would you recommend if Dad had a cold?" and "Please suggest meals that contain the nutrients a growing child needs."

[0817] In this way, the system can analyze the user's health and emotional state in real time and quickly provide individually optimized advice, which is expected to improve the health and quality of life of the entire family.

[0818] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0819] Program processing flow

[0820] Step 1: Data collection

[0821] Operation:

[0822] The device uses a built-in camera to capture images and video data of the user's daily life.

[0823] The device transmits this data to the server in real time.

[0824] Input: Images and video data from the user's daily life.

[0825] Output: Image and video data sent to the server.

[0826] Step 2: Image Recognition and Face Identification

[0827] Operation:

[0828] The server analyzes the image and video data received from the device and identifies the user's face.

[0829] The server associates the identified facial data with individual user profiles.

[0830] Input: Image and video data transmitted in real time.

[0831] Output: User's face identification results and corresponding profile data.

[0832] Step 3: Acquire data and prepare for analysis

[0833] Operation:

[0834] The server retrieves the user's health status, dietary history, and emotional state from a database.

[0835] The server prepares the acquired data for input into the generative artificial intelligence model and emotion engine.

[0836] Input: User profile data, health status, diet history, emotional state.

[0837] Output: Input data for analysis.

[0838] Step 4: Analyze your health and emotional state

[0839] Operation:

[0840] The server uses a generative artificial intelligence model and an emotion engine to analyze the user's health and emotional state.

[0841] The server calculates the analysis results.

[0842] Input: The input data for the analysis.

[0843] Output: Analysis of the user's health and emotional state.

[0844] Step 5: Generating Advice

[0845] Operation:

[0846] Based on the analysis results, the server generates health management advice optimized for each user.

[0847] The server generates a prompt sentence and inputs it into the generative AI model.

[0848] Input: Analysis results of health and emotional state, prompt text.

[0849] Output: Personalized health care advice.

[0850] Step 6: Advice Notification

[0851] Operation:

[0852] The server transmits the generated advice to the terminal.

[0853] The terminal notifies the user of the advice by voice and image.

[0854] Input: Generated personalized health care advice.

[0855] Output: Advice notification to the user.

[0856] Specific examples of each step

[0857] Step 1: Data collection

[0858] Operation: The device takes pictures of the user eating breakfast with its camera and sends the captured images and videos to a server every five minutes.

[0859] Input: Image and video data of a user having breakfast.

[0860] Output: Image and video data received by the server.

[0861] Step 2: Image Recognition and Face Identification

[0862] How it works: The server analyzes the image data sent from breakfast and uses a recognition algorithm to identify Dad's face.

[0863] Input: Image data taken during breakfast.

[0864] Output: Dad's face identification result and corresponding profile data.

[0865] Step 3: Acquire data and prepare for analysis

[0866] How it works: The server retrieves Dad's eating history and emotional state from the database for the past week and prepares it as data for analysis.

[0867] Input: Dad's profile data, food history for the past week, and emotional state.

[0868] Output: Input data for Daddy's analysis.

[0869] Step 4: Analyze your health and emotional state

[0870] How it works: The server uses a generative artificial intelligence model and an emotion engine to identify that Dad has a vitamin C deficiency and has recently had high stress levels.

[0871] Input: Input data for Daddy's analysis.

[0872] Output: Analysis of dad's health and emotional state.

[0873] Step 5: Generating Advice

[0874] How it works: Based on the analysis results, the server generates advice such as "Eat orange juice and vegetables to make up for vitamin C deficiency" and inputs the relevant prompt sentence into the generative AI model.

[0875] Input: Analysis results of dad's health and emotional state, prompt text.

[0876] Output: Personalized health advice for dad.

[0877] Step 6: Advice Notification

[0878] How it works: The server sends the generated advice to the device, which then announces aloud, "To compensate for your vitamin C deficiency, you should drink orange juice," and shows the specific suggestion on the display.

[0879] Enter: personalized health care advice for dads.

[0880] Output: Audio notification and image display of advice to dad.

[0881] (Application example 2)

[0882] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0883] Conventional health management systems monitor users' health status and analyze their dietary history, but lack a means for users to easily obtain recommended ingredients and nutrients. Furthermore, simply providing personalized advice makes it difficult to encourage specific behavioral changes, limiting its effectiveness, especially in today's busy society. Furthermore, systems lack sufficient support for health management that takes into account the user's emotional state. Therefore, more comprehensive and practical health management support is needed.

[0884] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0885] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice by voice and image and enabling the user to order appropriate meals in cooperation with a food delivery service. This allows the user to easily order specific meals based on the health management advice, enabling more practical and comprehensive health management to be achieved.

[0886] "Image recognition technology" is a technology that analyzes images and videos through cameras and other sensors to recognize specific objects and patterns.

[0887] A "generative artificial intelligence model" is a machine learning algorithm or deep learning model that uses large amounts of data to self-learn and perform a specific task (e.g., health analysis or generating personalized advice).

[0888] An "emotion engine" is an algorithm or system that analyzes a user's emotional state from their voice and facial expressions and identifies changes in their emotions.

[0889] "Personalized health management advice" involves analyzing a user's individual health condition and dietary history to generate advice optimized for the user.

[0890] A "food delivery service" is a service that delivers meals ordered by a user to a location specified by the user.

[0891] A "communications network" is an infrastructure that allows devices or systems in different locations to send and receive data.

[0892] 1. System Configuration

[0893] 1.1 Device (smartphone)

[0894] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0895] 1.2 Server

[0896] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and an emotion engine that analyzes the user's health condition, dietary history, and emotional state. Based on the analysis results, it generates personalized health management advice. It also has the ability to link with food delivery services and order appropriate meals.

[0897] 1.3 Communication Networks

[0898] The terminal and the server are connected via a communication network, typically the Internet, and data is sent and received in real time.

[0899] 2. Program Processing

[0900] 2.1 Data Collection

[0901] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0902] 2.2 Image Recognition and Face Identification

[0903] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[0904] 2.3 Health Data Analysis

[0905] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[0906] 2.4 Generating Personalized Advice

[0907] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[0908] 2.5 Notification of Advice and Collaboration with Food Delivery Services

[0909] The server generates advice and sends it to the device, which then receives it and notifies the user via voice or display. Based on the advice, the user can then order an appropriate meal from a food delivery service with the touch of a button.

[0910] 3. Specific Examples

[0911] Example 1: Dad has a cold

[0912] User: Dad

[0913] Device: Smartphone captures dad's daily life

[0914] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0915] Emotion engine: Analyzes dad's emotional state and identifies whether he is stressed or not

[0916] Terminal: Advises, "Chicken and vegetable soup is recommended for this cold season," and displays the recipe on the screen. It also links with food delivery services, allowing soup to be ordered with the touch of a button.

[0917] Example 2: Nutritional management for growing children

[0918] User: Growing child

[0919] Device: Smartphones capture children's daily lives

[0920] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0921] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[0922] Device: "You've grown 1cm taller than you were a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken per day," the device advises, showing specific ingredients and recommended intake amounts on the display. Furthermore, the device will link with a food delivery service, allowing users to order spinach and chicken.

[0923] Prompt Sentence Examples

[0924] "Please suggest a recommended meal menu for users who have caught a cold."

[0925] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[0926] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0927] Step 1:

[0928] The device camera captures images and video data of the user's daily life, and this data is sent to the server in real time. The input is the image and video data captured by the device camera, and the output is the raw image and video data sent to the server.

[0929] Step 2:

[0930] The server analyzes the received image and video data and uses image recognition technology to identify the user's face. The input is the image and video data sent in step 1, and the output is the identified user's face data. Specifically, the server runs a facial recognition algorithm to extract facial feature points.

[0931] Step 3:

[0932] Based on the identified facial data, the server inputs data including the user's health status and dietary history into a generative artificial intelligence model. The input is the user's facial data, past health status data, and dietary history, and the output is the analysis result regarding the health status. Specifically, the server feeds the health data and dietary history into the AI ​​model to generate a health status.

[0933] Step 4:

[0934] The server uses an emotion engine to analyze the user's emotional state. The input is the user's facial data and voice data, and the output is the analysis result regarding the user's emotional state. Specifically, the server runs a voice analysis algorithm and a facial expression analysis algorithm to generate an emotion tag.

[0935] Step 5:

[0936] The server generates personalized health management advice for the user based on the health data analysis results and emotional state. The input is the health condition analysis results and emotional state analysis results, and the output is health management advice provided to the user. Specifically, the server uses a generative AI model to generate optimal advice in text format from the health analysis results and emotional tags.

[0937] Step 6:

[0938] The server sends the generated advice to the terminal, which then notifies the user of it through voice and display. The input is the advice data sent from the server, and the output is the voice and image data presented to the user. Specifically, the terminal uses voice synthesis technology to convert the advice into voice and displays the text on the display.

[0939] Step 7:

[0940] The server works with food delivery services to provide options for ordering appropriate meals based on the generated advice. The input is the generated advice data and the user's location information, and the output is a meal menu that the user can select and a delivery order screen. Specifically, the server accesses a delivery API and generates an interface that allows the user to order with one click.

[0941] Step 8:

[0942] When the user confirms the order, the server sends the order data to the food delivery service and sends an order confirmation notice to the user. The input is the user's order data, and the output is the order confirmation data sent to the delivery service and the order confirmation notice to the user. Specifically, the server sends the order data to the delivery service's system and sends a confirmation message to the terminal if the order is successful.

[0943] Through this series of processes, users can receive health management advice and easily order delivery meals based on that advice.

[0944] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0945] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0946] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0947] [Third embodiment]

[0948] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0949] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0950] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0951] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0952] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0953] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0954] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0955] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0956] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0957] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0958] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0959] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0960] The present invention provides a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The system is configured and operates as follows.

[0961] 1. System Configuration

[0962] 1.1 Terminal (small robot)

[0963] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[0964] 1.2 Server

[0965] The server receives the image and video data sent from the device and uses image recognition technology to identify the user's face.The server also contains a generative artificial intelligence model that analyzes the user's health condition and dietary history.

[0966] 1.3 Communication Networks

[0967] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[0968] 2. Program Processing

[0969] 2.1 Data Collection

[0970] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[0971] 2.2 Image Recognition and Face Identification

[0972] The server analyzes the received image and video data and identifies the user's face, allowing it to identify each family member and associate their health status and dietary history.

[0973] 2.3 Health Data Analysis

[0974] The server inputs the user's health data and dietary history into a generative AI model for analysis. Based on the analysis results, the user's health condition and nutritional deficiencies are identified.

[0975] 2.4 Generating Personalized Advice

[0976] Based on the analysis results, a generative AI model generates personalized advice for each user, including suggestions for improving their health and diet.

[0977] 2.5 Advice Notice

[0978] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or a display. For example, if a father has a cold, the device might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds this time of year."

[0979] 3. Specific Examples

[0980] Example 1: Dad has a cold

[0981] User: Dad

[0982] Device: Small robot captures dad's daily life

[0983] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[0984] Device: Advises, "Chicken and vegetable soup is recommended for those battling a cold this time of year," and displays the recipe on the screen

[0985] Example 2: Nutritional management for growing children

[0986] User: Growing child

[0987] Device: Small robot captures children's daily lives

[0988] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[0989] Device: "You've grown 1cm since a month ago. You may be deficient in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, and the specific ingredients and recommended intake are displayed on the screen.

[0990] 4. Response to additional questions

[0991] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[0992] User: Child asks robot a question

[0993] Terminal: Sends question to server

[0994] Server: Analyzes using a generative AI model and identifies appropriate fatigue recovery items

[0995] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises, and displays images of specific items to purchase on the screen.

[0996] In this way, the system efficiently manages users' health conditions and lifestyle habits and provides individually optimized advice in real time, improving the health and quality of life of the entire family.

[0997] The processing flow will be explained below.

[0998] Step 1:

[0999] Subject: Device

[1000] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[1001] Step 2:

[1002] Subject: Server

[1003] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[1004] Step 3:

[1005] Subject: Server

[1006] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[1007] Step 4:

[1008] Subject: Server

[1009] The server provides real-time updates on the user's daily life and health status based on a database, and this information is fed into a generative artificial intelligence model.

[1010] Step 5:

[1011] Subject: Server

[1012] The server uses the generated AI model to analyze the user's health condition and dietary history. The model identifies the user's nutritional deficiencies and health risks and generates analysis results.

[1013] Step 6:

[1014] Subject: Server

[1015] Based on the analysis results, the server generates personalized health management advice, including specific suggestions for actions and dietary improvements.

[1016] Step 7:

[1017] Subject: Server

[1018] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[1019] Step 8:

[1020] Subject: Device

[1021] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[1022] Step 9:

[1023] Subject: User

[1024] The user can ask the robot additional questions, such as, "I have a stop at a convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[1025] Step 10:

[1026] Subject: Device

[1027] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[1028] Step 11:

[1029] Subject: Server

[1030] The server uses a generative AI model to analyze the user's question and generate the optimal answer, taking into account the amount of practice and past data.

[1031] Step 12:

[1032] Subject: Server

[1033] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[1034] Step 13:

[1035] Subject: Device

[1036] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[1037] Example 1

[1038] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1039] Conventional health management systems have difficulty understanding a user's individual health condition and lifestyle habits in detail and providing optimized advice in real time. Furthermore, if the user's face is not identified or data analysis is not performed accurately, the advice provided may be inappropriate. Furthermore, it is difficult to respond to follow-up questions from users, which hinders the improvement of user satisfaction.

[1040] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1041] In this invention, the server includes means for capturing image or video data of a user using a photographing device equipped with image recognition technology, means for transmitting the captured image or video data in real time to an analysis device via a communication device, means for analyzing the received image or video data in the analysis device and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice via an audio output device and a display device. This makes it possible to efficiently and accurately manage the user's health condition and lifestyle habits and provide individually optimized advice in real time.

[1042] "Image recognition technology" is a technology that uses computer vision algorithms to detect and identify specific objects or features within images or video data.

[1043] "Photographing device" refers to a device for taking images or videos, such as a camera or video camera.

[1044] A "communications device" is a device for transmitting and receiving data, and includes devices that use network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[1045] An "analysis device" is a computer system or device for processing and analyzing received data.

[1046] "Means for identifying a user's face" refers to algorithms or software that extract the user's facial features from image or video data and identify that face.

[1047] "Health status" refers to information about the user's physical health in general, including data such as blood pressure, heart rate, and body temperature.

[1048] "Diet history" refers to information about meals the user has had in the past, and includes data such as ingredients, calorie intake, and meal times.

[1049] A "generative artificial intelligence model" is an AI model that uses machine learning and deep learning technologies to generate analysis results and predictions from input data.

[1050] The "means for generating advice based on analysis results" is software or an algorithm for generating optimal health management and lifestyle improvement advice for the user based on the analysis results.

[1051] The "audio output device" is a device such as a speaker that reproduces the generated advice by voice.

[1052] A "display device" is a device such as a display or monitor for visually displaying the generated advice.

[1053] The present invention is a system for supporting family health management and lifestyle optimization, which includes a photographing device equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The configuration and operation of this system are described in detail below.

[1054] System Configuration

[1055] The system consists of three main parts:

[1056] 1. Terminal (a small device with a built-in camera)

[1057] 2. Server (computer system for data analysis)

[1058] 3. Communication network (infrastructure for sending and receiving data)

[1059] Terminal

[1060] The device is a small device with a built-in camera that records the user's daily life. The device has the following features:

[1061] Camera: Equipped with image recognition technology, it takes pictures of the user's face and how they are eating.

[1062] Communication module: Sends data to the server in real time using Wi-Fi or Bluetooth.

[1063] Audio output: The generated advice is notified to the user by voice.

[1064] Display: Visually displays generated advice and notifications.

[1065] server

[1066] The server is a computer system that receives image and video data sent from the device and performs multiple analyses. The server has the following functions:

[1067] Data reception: Receives data sent from the terminal.

[1068] Image Recognition: Preprocess image data and reduce noise using Python's OpenCV library.

[1069] Facial recognition: A deep learning model using TensorFlow extracts facial features and matches them with facial data registered in a database.

[1070] Health data analysis: User health data and dietary history are input into a generative artificial intelligence model (e.g., PyTorch model) and analyzed.

[1071] Personalized advice generation: The analysis results are converted into text using a natural language generation (NLG) API, and the most appropriate advice is generated for the user.

[1072] communication network

[1073] A communications network is an infrastructure that enables real-time data transmission between devices and servers, and primarily uses network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[1074] Specific examples

[1075] Example 1: Dad has a cold

[1076] User:Dad

[1077] Device: A small device captures Dad's daily life and collects data such as changes in his complexion and the frequency of his coughs.

[1078] Server: Analyzes the received data, identifies cold symptoms, and compares them with past data to generate an appropriate soup recipe.

[1079] Device: The generated advice is announced aloud, such as "Chicken and vegetable soup is recommended for colds at this time of year," and the recipe is shown on the screen.

[1080] Example 2: Nutritional management for growing children

[1081] User: Growing child

[1082] Device: A small device that captures a child's daily life and records what they eat and how they exercise.

[1083] Server: Analyzes the received data and identifies iron and zinc deficiencies based on height and weight measurements and dietary history.

[1084] Device: "You've grown 1cm taller compared to a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken a day," the device says, showing specific ingredients and recommended intake amounts on the display and providing a voice notification.

[1085] Example 3: Advice for recovering from fatigue

[1086] User: My child has baseball practice after school, but I still have time to stop by a convenience store.

[1087] Terminal: Sends the question to the server.

[1088] Server: Analyzes the question using a generative artificial intelligence model and identifies appropriate fatigue recovery items.

[1089] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises aloud, and displays images of specific items to purchase on the screen.

[1090] The system efficiently manages users' health conditions and lifestyle habits and provides personalized, optimized advice in real time, improving the health and quality of life of the entire family.

[1091] Prompt Sentence Examples

[1092] "Generate appropriate dietary advice for users who are showing signs of a cold."

[1093] "Please analyze the nutritional status of growing children and advise them on the nutrients they need."

[1094] "Please tell me some foods that are effective in relieving fatigue."

[1095] In this way, the system can provide personalized health care advice based on the user's individual data.

[1096] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1097] Step 1:

[1098] Data collection

[1099] The device captures images and video data of the user's daily life and sends the data to a server in real time.

[1100] Input: Image and video data capturing user activity

[1101] Specific operation: The device's built-in camera captures the user's face and actions, and sends the image and video data to a server using communication methods such as Wi-Fi or Bluetooth.

[1102] Output: Image and video data sent to the server

[1103] Step 2:

[1104] Image Recognition and Facial Identification

[1105] The server analyzes the image and video data it receives and identifies the user's face.

[1106] Input: Image and video data sent from the device

[1107] How it works: The server preprocesses the image data using Python's OpenCV library, extracts and identifies facial features using a deep learning model (e.g., TensorFlow), and matches the facial information in the database for face identification.

[1108] Output: Identified user's face information

[1109] Step 3:

[1110] Health data analysis

[1111] The server inputs the user's health data and dietary history into a generative AI model for analysis, identifying the user's health condition and nutritional deficiencies.

[1112] Input: Identified user's face information, user's health data, diet history

[1113] Specific operation: The server inputs health data (e.g., blood pressure, heart rate, etc.) and dietary history into a generative AI model (e.g., PyTorch model) for analysis. The analysis then estimates nutritional and health status.

[1114] Output: Analysis results (health status, nutritional deficiency, etc.)

[1115] Step 4:

[1116] Generating personalized advice

[1117] The server generates personalized health management advice for the user based on the analysis results.

[1118] Input: Analysis results (health status, nutritional deficiency, etc.)

[1119] Specific operation: The server uses a natural language generation (NLG) API to convert the analysis results into easy-to-understand sentences. For example, if there are signs of a cold, it generates specific advice such as "Chicken and vegetable soup is recommended for colds at this time of year."

[1120] Output: Personalized health advice

[1121] Step 5:

[1122] Advice Notice

[1123] The server sends the generated advice to the device, which notifies the user through voice or display.

[1124] Enter: personalized health advice.

[1125] Specific operation: The server sends the generated advice to the device using a message transmission protocol (e.g., MQTT). The device receives the advice and notifies the user by voice output or display on the screen. For example, the device may notify the user by voice, saying, "Chicken and vegetable soup is recommended for colds this time of year," and show the recipe on the screen.

[1126] Output: Audio and display advice

[1127] In this way, the system effectively manages users' health and lifestyle habits and provides personalized advice in real time.

[1128] (Application example 1)

[1129] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1130] In today's world, autonomous vehicles are becoming increasingly common, but there is a lack of systems that can monitor the health and stress levels of occupants in real time and provide appropriate advice and encourage rest. If this problem is not solved, fatigue and stress will accumulate due to long periods of driving, increasing the risk of accidents and health problems. The present invention aims to solve this problem by effectively monitoring the health and stress levels of occupants in autonomous vehicles and providing appropriate advice.

[1131] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1132] In this invention, the server includes: means for capturing images or video data of a user using a camera equipped with image recognition technology; means for analyzing the captured images or video data and identifying the user's face; means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis; means for generating personalized health management advice for the user based on the analysis results; means for notifying the user of the generated advice by voice and image; means for monitoring the health condition and stress level of occupants in an autonomous vehicle and collecting health data using an on-board camera and sensor; means for generating appropriate rest and refreshment advice for the occupants based on the collected health data; and means for notifying the occupants of the advice using an in-vehicle display or audio output device. This makes it possible to effectively manage the health condition and stress level of occupants in an autonomous vehicle in real time and provide appropriate advice.

[1133] "Image recognition technology" is a technology that automatically identifies specific objects or patterns from images or video data captured by a camera.

[1134] "User" refers to a person who uses the system, and in this case includes the occupants of an autonomous vehicle.

[1135] A "generative artificial intelligence model" is a model that uses machine learning algorithms to learn specific patterns and characteristics from large amounts of data and make judgments such as predictions and classifications.

[1136] "Personalized health management advice" refers to advice that is individually optimized based on the user's analyzed health condition and behavioral history.

[1137] A "camera" is a device that converts light into electrical signals to capture images and videos.

[1138] An "on-board camera" is a camera installed inside an autonomous vehicle to capture images of the occupants and their surroundings.

[1139] A "sensor" is a device that measures a physical or chemical property and is primarily used in this system to collect health data.

[1140] A "display" is a device that visually displays information.

[1141] An "audio output device" is a device that outputs information in the form of audio.

[1142] This invention is a system for monitoring the health status and stress level of occupants in an autonomous vehicle and providing appropriate advice. This system is implemented with the following configuration.

[1143] First, the server captures images or video data of the occupants using the vehicle's onboard camera. The captured images and video data are sent to the server via a communications network. The server then analyzes the received data using image recognition technology to identify the occupants' faces. Image processing software such as OpenCV and TensorFlow is used for facial recognition.

[1144] The server then inputs data, including the occupant's health status and dietary history, into a generative AI model for analysis. This generative AI model is used to estimate the occupant's stress level and fatigue level from their facial color and facial expression. The generative AI model uses Keras / TensorFlow. Additional sensors may also be used as needed to collect health data such as heart rate and respiratory rate.

[1145] Based on the analysis results, the server generates personalized health management advice for the occupant. For example, this advice might be, "If high fatigue level is detected while driving, notify the occupant to take a break." The generated advice is communicated to the occupant via a display or audio output device in the vehicle.

[1146] For example, if a passenger begins to feel tired during a long drive, an on-board camera will capture a change in facial color, which the generative AI model will analyze and detect that fatigue is building up. As a result, the server will generate appropriate advice, such as "Take a break at the next service area," and notify the passenger via display or voice.

[1147] Additionally, if a passenger asks a specific question, the generative AI model will perform further analysis based on the question and generate an appropriate response. For example, if the passenger asks, "I'm feeling tired today. What can I do to refresh myself?", the system will provide advice such as, "I recommend taking a short walk and getting some fresh air."

[1148] An example of a prompt might be "Input a picture of the user's face and estimate their fatigue level." This prompt enables the generative AI model to analyze the user's health condition and generate appropriate advice.

[1149] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1150] Step 1:

[1151] The terminal (a system inside the vehicle) captures images or video data of the occupants using an on-board camera. The input is the camera image, and the output is high-resolution image or video data. This data is sent to the server in real time.

[1152] Step 2:

[1153] The server analyzes the received image and video data and identifies the faces of passengers using image recognition technology (OpenCV or TensorFlow). The input is image or video data, and the output is the coordinates of the facial area and identification information. Based on the facial identification results, the data of each passenger is associated.

[1154] Step 3:

[1155] The server collects the health condition data and dietary history data of the identified occupants and inputs them into a generative AI model. The input is facial identification information and past health data, and the output is the analysis results of the health condition. Using the generative AI model (Keras / TensorFlow), stress levels and fatigue levels are estimated from facial color and facial expressions.

[1156] Step 4:

[1157] The server generates personalized health management advice for the occupants based on the analysis results. The input is the health status analysis result, and the output is the advice content. This advice includes encouraging them to take breaks as needed and specific health management suggestions.

[1158] Step 5:

[1159] The generated advice is sent to the terminal, which then notifies the occupant using a display or audio output device. The input is the generated advice, and the output is the notification to the occupant. In this case, the advice is provided to the occupant using speech synthesis (gTTS) or text display.

[1160] Step 6:

[1161] When a user (passenger) asks a specific question to the terminal, the terminal sends the question to the server. The input is the passenger's voice question, and the output is the question data.

[1162] Step 7:

[1163] The server inputs the received question into a generative artificial intelligence model to generate appropriate advice based on the question. The input is the question data and related health data, and the output is the answer advice.

[1164] Step 8:

[1165] The generated answer advice is sent to the terminal, and the terminal notifies the occupant using a display or audio output device. The input is the answer advice, and the output is the answer notification to the occupant. This notification is provided in a form that is easy for the occupant to understand using voice synthesis or text display.

[1166] This series of processing flows makes it possible to manage the health status and stress levels of occupants in self-driving vehicles in real time and provide appropriate advice.

[1167] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1168] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice using an emotion engine. The system is configured and operates as follows.

[1169] 1. System Configuration

[1170] 1.1 Terminal (small robot)

[1171] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[1172] 1.2 Server

[1173] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and emotion engine to analyze the user's health condition, dietary history, and emotional state.

[1174] 1.3 Communication Networks

[1175] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[1176] 2. Program Processing

[1177] 2.1 Data Collection

[1178] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[1179] 2.2 Image Recognition and Face Identification

[1180] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[1181] 2.3 Health Data Analysis

[1182] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[1183] 2.4 Generating Personalized Advice

[1184] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[1185] 2.5 Advice Notice

[1186] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or display. For example, if a father has a cold, the system might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds at this time of year." The system also makes suggestions that take the user's emotional state into account.

[1187] 3. Specific Examples

[1188] Example 1: Dad has a cold

[1189] User: Dad

[1190] Device: Small robot captures dad's daily life

[1191] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[1192] Emotion engine: Analyzes dad's emotional state to identify stress

[1193] Device: "Chicken and vegetable soup is a good option for those with a cold this time of year," the device advises, and displays a recipe on the display. It also suggests relaxation techniques to improve Dad's mood.

[1194] Example 2: Nutritional management for growing children

[1195] User: Growing child

[1196] Device: Small robot captures children's daily lives

[1197] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[1198] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[1199] Device: "You've grown 1cm since a month ago. You may be lacking in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, showing specific ingredients and the recommended intake amount on the display. It also provides encouraging messages to motivate the child.

[1200] 4. Response to additional questions

[1201] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[1202] User: Child asks robot a question

[1203] Terminal: Sends question to server

[1204] Server: Analyzes using a generative AI model and generates the optimal answer. Also uses an emotion engine.

[1205] Device: The device will announce, "Bananas and yogurt are recommended to help relieve fatigue," and will display images of specific items to purchase. It will also suggest other options based on the child's preferences.

[1206] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[1207] The processing flow will be explained below.

[1208] Step 1:

[1209] Subject: Device

[1210] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[1211] Step 2:

[1212] Subject: Server

[1213] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[1214] Step 3:

[1215] Subject: Server

[1216] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[1217] Step 4:

[1218] Subject: Server

[1219] The server inputs the acquired image and video data into the emotion engine, which analyzes the user's facial expressions and movements to identify their emotional state.

[1220] Step 5:

[1221] Subject: Server

[1222] The server updates the user's emotional state in real time based on the results of the emotion engine, thereby building a database that reflects the user's current emotional state.

[1223] Step 6:

[1224] Subject: Server

[1225] The server inputs health data, dietary history, and emotional state into a generated artificial intelligence model based on this database and performs analysis.

[1226] Step 7:

[1227] Subject: Server

[1228] Based on the analysis results, the generative AI model generates personalized health management advice, including specific suggested actions and dietary improvements.

[1229] Step 8:

[1230] Subject: Server

[1231] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[1232] Step 9:

[1233] Subject: Device

[1234] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[1235] Step 10:

[1236] Subject: User

[1237] The user asks a follow-up question to the device, for example, "I have a chance to stop by a convenience store after school before going to baseball practice. What should I buy to help me recover?"

[1238] Step 11:

[1239] Subject: Device

[1240] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[1241] Step 12:

[1242] Subject: Server

[1243] The server uses a generative artificial intelligence model to analyze the question and generate the best answer, along with an emotion engine that takes into account the user's current emotional state.

[1244] Step 13:

[1245] Subject: Server

[1246] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[1247] Step 14:

[1248] Subject: Device

[1249] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[1250] Example 2

[1251] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] In modern families, it is difficult to manage the health and lifestyles of all family members at once. Providing personalized health management advice for each family member is particularly time-consuming and labor-intensive. Furthermore, there is a lack of means to grasp the health and emotional state of each family member in real time and quickly provide appropriate advice based on that information.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1254] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for acquiring the user's health condition, dietary history, and emotional state, means for inputting the acquired data into a generative AI model and an emotion engine for analysis, means for generating personalized health management advice for the user based on the analysis results, means for generating prompt sentences and inputting them into the generative AI model for analysis, means for notifying the user of the generated advice by voice and image, and means for transmitting data from a terminal to the server and receiving the analysis results. This makes it possible to analyze the health conditions and emotional states of all family members in real time and quickly provide individually optimized advice.

[1255] "Image recognition technology" is a technology that analyzes images and videos taken with a camera and identifies the objects and people contained within them.

[1256] A "camera" is a device for taking pictures and videos.

[1257] "User" refers to each member of a family who uses the system.

[1258] "Means for identifying faces" refers to technology for identifying a user's face from captured images or video data.

[1259] "Health status" refers to information about the user's physical health, such as body temperature, appetite, and sleep status.

[1260] "Dietary history" refers to a record of the food and drinks consumed by a user.

[1261] "Emotional state" refers to information that represents the user's psychological state, such as joy, sadness, anger, etc.

[1262] A "generative artificial intelligence model" is an artificial intelligence technology that generates advice suited to the user based on various data.

[1263] An "emotion engine" is a technology that analyzes a user's emotional state and supports appropriate advice.

[1264] A "prompt sentence" refers to a question or instruction sentence that is input into an artificial intelligence model.

[1265] "Analysis results" are information that includes advice and suggestions generated based on your health and emotional state.

[1266] "Audio notification means" refers to a technique or device for conveying the generated advice to the user as audio.

[1267] "Means for notifying by image" refers to a technique or device for displaying the generated advice on a display and conveying it to the user.

[1268] "Terminal" refers to a small robot or device that captures the user's daily life.

[1269] "Means for transmitting data" refers to the technology used to transmit captured images and videos to a server.

[1270] "Means for receiving analysis results" refers to technology for receiving analysis results sent from the server.

[1271] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative AI model, and a means for providing personalized advice using an emotion engine. Specific embodiments of the system are described below.

[1272] First, this system includes a terminal equipped with a built-in camera equipped with image recognition technology for capturing images of the user's daily life. The terminal is in the form of a small robot, and captures images of the user as they go about their daily life. Images and video data captured by this camera are sent to a server in real time. The server and terminal are connected via a communications network, allowing for rapid data transmission and reception.

[1273] The server then analyzes the transmitted image and video data. It uses image recognition technology to identify the user's face, thereby identifying each family member. The server also acquires the user's health status, dietary history, and emotional state. This data is then fed into a generative artificial intelligence model (generative AI model) and emotion engine for analysis.

[1274] Based on the analyzed data, the server generates personalized health management advice optimized for each user. The prompt sentences used to generate this advice are also generated by the server. For example, a prompt sentence such as "Please tell me how I can reduce my recent stress" is input into the generative AI model. The generated advice is sent to the device and communicated to the user via voice and images.

[1275] As a specific operational scenario, consider the case where Dad has caught a cold. The device takes pictures of Dad's daily life and sends them to the server. The server identifies the signs of a cold and generates advice suggesting a soup recipe that is effective against colds. The device notifies Dad by voice, saying, "Chicken and vegetable soup is recommended for colds at this time of year," and shows the recipe on the display.

[1276] In addition, when managing the nutrition of growing children, the device takes pictures of the child's daily life and sends the data to a server. The server analyzes the data and identifies iron and zinc deficiencies. The server advises, "You've grown 1cm taller than you were a month ago. You're deficient in iron and zinc, so you should eat 100 grams of spinach and chicken per day," and the device notifies the child of the specific foods and the amount to consume.

[1277] Examples of prompts include, "What meals and recipes would you recommend if Dad had a cold?" and "Please suggest meals that contain the nutrients a growing child needs."

[1278] In this way, the system can analyze the user's health and emotional state in real time and quickly provide individually optimized advice, which is expected to improve the health and quality of life of the entire family.

[1279] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1280] Program processing flow

[1281] Step 1: Data collection

[1282] Operation:

[1283] The device uses a built-in camera to capture images and video data of the user's daily life.

[1284] The device transmits this data to the server in real time.

[1285] Input: Images and video data from the user's daily life.

[1286] Output: Image and video data sent to the server.

[1287] Step 2: Image Recognition and Face Identification

[1288] Operation:

[1289] The server analyzes the image and video data received from the device and identifies the user's face.

[1290] The server associates the identified facial data with individual user profiles.

[1291] Input: Image and video data transmitted in real time.

[1292] Output: User's face identification results and corresponding profile data.

[1293] Step 3: Acquire data and prepare for analysis

[1294] Operation:

[1295] The server retrieves the user's health status, dietary history, and emotional state from a database.

[1296] The server prepares the acquired data for input into the generative artificial intelligence model and emotion engine.

[1297] Input: User profile data, health status, diet history, emotional state.

[1298] Output: Input data for analysis.

[1299] Step 4: Analyze your health and emotional state

[1300] Operation:

[1301] The server uses a generative artificial intelligence model and an emotion engine to analyze the user's health and emotional state.

[1302] The server calculates the analysis results.

[1303] Input: The input data for the analysis.

[1304] Output: Analysis of the user's health and emotional state.

[1305] Step 5: Generating Advice

[1306] Operation:

[1307] Based on the analysis results, the server generates health management advice optimized for each user.

[1308] The server generates a prompt sentence and inputs it into the generative AI model.

[1309] Input: Analysis results of health and emotional state, prompt text.

[1310] Output: Personalized health care advice.

[1311] Step 6: Advice Notification

[1312] Operation:

[1313] The server transmits the generated advice to the terminal.

[1314] The terminal notifies the user of the advice by voice and image.

[1315] Input: Generated personalized health care advice.

[1316] Output: Advice notification to the user.

[1317] Specific examples of each step

[1318] Step 1: Data collection

[1319] Operation: The device takes pictures of the user eating breakfast with its camera and sends the captured images and videos to a server every five minutes.

[1320] Input: Image and video data of a user having breakfast.

[1321] Output: Image and video data received by the server.

[1322] Step 2: Image Recognition and Face Identification

[1323] How it works: The server analyzes the image data sent from breakfast and uses a recognition algorithm to identify Dad's face.

[1324] Input: Image data taken during breakfast.

[1325] Output: Dad's face identification result and corresponding profile data.

[1326] Step 3: Acquire data and prepare for analysis

[1327] How it works: The server retrieves Dad's eating history and emotional state from the database for the past week and prepares it as data for analysis.

[1328] Input: Dad's profile data, food history for the past week, and emotional state.

[1329] Output: Input data for Daddy's analysis.

[1330] Step 4: Analyze your health and emotional state

[1331] How it works: The server uses a generative artificial intelligence model and an emotion engine to identify that Dad has a vitamin C deficiency and has recently had high stress levels.

[1332] Input: Input data for Daddy's analysis.

[1333] Output: Analysis of dad's health and emotional state.

[1334] Step 5: Generating Advice

[1335] How it works: Based on the analysis results, the server generates advice such as "Eat orange juice and vegetables to make up for vitamin C deficiency" and inputs the relevant prompt sentence into the generative AI model.

[1336] Input: Analysis results of dad's health and emotional state, prompt text.

[1337] Output: Personalized health advice for dad.

[1338] Step 6: Advice Notification

[1339] How it works: The server sends the generated advice to the device, which then announces aloud, "To compensate for your vitamin C deficiency, you should drink orange juice," and shows the specific suggestion on the display.

[1340] Enter: personalized health care advice for dads.

[1341] Output: Audio notification and image display of advice to dad.

[1342] (Application example 2)

[1343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1344] Conventional health management systems monitor users' health status and analyze their dietary history, but lack a means for users to easily obtain recommended ingredients and nutrients. Furthermore, simply providing personalized advice makes it difficult to encourage specific behavioral changes, limiting its effectiveness, especially in today's busy society. Furthermore, systems lack sufficient support for health management that takes into account the user's emotional state. Therefore, more comprehensive and practical health management support is needed.

[1345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1346] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice by voice and image and enabling the user to order appropriate meals in cooperation with a food delivery service. This allows the user to easily order specific meals based on the health management advice, enabling more practical and comprehensive health management to be achieved.

[1347] "Image recognition technology" is a technology that analyzes images and videos through cameras and other sensors to recognize specific objects and patterns.

[1348] A "generative artificial intelligence model" is a machine learning algorithm or deep learning model that uses large amounts of data to self-learn and perform a specific task (e.g., health analysis or generating personalized advice).

[1349] An "emotion engine" is an algorithm or system that analyzes a user's emotional state from their voice and facial expressions and identifies changes in their emotions.

[1350] "Personalized health management advice" involves analyzing a user's individual health condition and dietary history to generate advice optimized for the user.

[1351] A "food delivery service" is a service that delivers meals ordered by a user to a location specified by the user.

[1352] A "communications network" is an infrastructure that allows devices or systems in different locations to send and receive data.

[1353] 1. System Configuration

[1354] 1.1 Device (smartphone)

[1355] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[1356] 1.2 Server

[1357] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and an emotion engine that analyzes the user's health condition, dietary history, and emotional state. Based on the analysis results, it generates personalized health management advice. It also has the ability to link with food delivery services and order appropriate meals.

[1358] 1.3 Communication Networks

[1359] The terminal and the server are connected via a communication network, typically the Internet, and data is sent and received in real time.

[1360] 2. Program Processing

[1361] 2.1 Data Collection

[1362] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[1363] 2.2 Image Recognition and Face Identification

[1364] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[1365] 2.3 Health Data Analysis

[1366] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[1367] 2.4 Generating Personalized Advice

[1368] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[1369] 2.5 Notification of Advice and Collaboration with Food Delivery Services

[1370] The server generates advice and sends it to the device, which then receives it and notifies the user via voice or display. Based on the advice, the user can then order an appropriate meal from a food delivery service with the touch of a button.

[1371] 3. Specific Examples

[1372] Example 1: Dad has a cold

[1373] User: Dad

[1374] Device: Smartphone captures dad's daily life

[1375] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[1376] Emotion engine: Analyzes dad's emotional state and identifies whether he is stressed or not

[1377] Terminal: Advises, "Chicken and vegetable soup is recommended for this cold season," and displays the recipe on the screen. It also links with food delivery services, allowing soup to be ordered with the touch of a button.

[1378] Example 2: Nutritional management for growing children

[1379] User: Growing child

[1380] Device: Smartphones capture children's daily lives

[1381] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[1382] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[1383] Device: "You've grown 1cm taller than you were a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken per day," the device advises, showing specific ingredients and recommended intake amounts on the display. Furthermore, the device will link with a food delivery service, allowing users to order spinach and chicken.

[1384] Prompt Sentence Examples

[1385] "Please suggest a recommended meal menu for users who have caught a cold."

[1386] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[1387] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1388] Step 1:

[1389] The device camera captures images and video data of the user's daily life, and this data is sent to the server in real time. The input is the image and video data captured by the device camera, and the output is the raw image and video data sent to the server.

[1390] Step 2:

[1391] The server analyzes the received image and video data and uses image recognition technology to identify the user's face. The input is the image and video data sent in step 1, and the output is the identified user's face data. Specifically, the server runs a facial recognition algorithm to extract facial feature points.

[1392] Step 3:

[1393] Based on the identified facial data, the server inputs data including the user's health status and dietary history into a generative artificial intelligence model. The input is the user's facial data, past health status data, and dietary history, and the output is the analysis result regarding the health status. Specifically, the server feeds the health data and dietary history into the AI ​​model to generate a health status.

[1394] Step 4:

[1395] The server uses an emotion engine to analyze the user's emotional state. The input is the user's facial data and voice data, and the output is the analysis result regarding the user's emotional state. Specifically, the server runs a voice analysis algorithm and a facial expression analysis algorithm to generate an emotion tag.

[1396] Step 5:

[1397] The server generates personalized health management advice for the user based on the health data analysis results and emotional state. The input is the health condition analysis results and emotional state analysis results, and the output is health management advice provided to the user. Specifically, the server uses a generative AI model to generate optimal advice in text format from the health analysis results and emotional tags.

[1398] Step 6:

[1399] The server sends the generated advice to the terminal, which then notifies the user of it through voice and display. The input is the advice data sent from the server, and the output is the voice and image data presented to the user. Specifically, the terminal uses voice synthesis technology to convert the advice into voice and displays the text on the display.

[1400] Step 7:

[1401] The server works with food delivery services to provide options for ordering appropriate meals based on the generated advice. The input is the generated advice data and the user's location information, and the output is a meal menu that the user can select and a delivery order screen. Specifically, the server accesses a delivery API and generates an interface that allows the user to order with one click.

[1402] Step 8:

[1403] When the user confirms the order, the server sends the order data to the food delivery service and sends an order confirmation notice to the user. The input is the user's order data, and the output is the order confirmation data sent to the delivery service and the order confirmation notice to the user. Specifically, the server sends the order data to the delivery service's system and sends a confirmation message to the terminal if the order is successful.

[1404] Through this series of processes, users can receive health management advice and easily order delivery meals based on that advice.

[1405] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1407] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1408] [Fourth embodiment]

[1409] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1410] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1411] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1412] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1413] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1415] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1416] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1417] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1418] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1419] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1420] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1421] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1422] The present invention provides a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The system is configured and operates as follows.

[1423] 1. System Configuration

[1424] 1.1 Terminal (small robot)

[1425] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[1426] 1.2 Server

[1427] The server receives the image and video data sent from the device and uses image recognition technology to identify the user's face.The server also contains a generative artificial intelligence model that analyzes the user's health condition and dietary history.

[1428] 1.3 Communication Networks

[1429] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[1430] 2. Program Processing

[1431] 2.1 Data Collection

[1432] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[1433] 2.2 Image Recognition and Face Identification

[1434] The server analyzes the received image and video data and identifies the user's face, allowing it to identify each family member and associate their health status and dietary history.

[1435] 2.3 Health Data Analysis

[1436] The server inputs the user's health data and dietary history into a generative AI model for analysis. Based on the analysis results, the user's health condition and nutritional deficiencies are identified.

[1437] 2.4 Generating Personalized Advice

[1438] Based on the analysis results, a generative AI model generates personalized advice for each user, including suggestions for improving their health and diet.

[1439] 2.5 Advice Notice

[1440] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or a display. For example, if a father has a cold, the device might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds this time of year."

[1441] 3. Specific Examples

[1442] Example 1: Dad has a cold

[1443] User: Dad

[1444] Device: Small robot captures dad's daily life

[1445] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[1446] Device: Advises, "Chicken and vegetable soup is recommended for those battling a cold this time of year," and displays the recipe on the screen

[1447] Example 2: Nutritional management for growing children

[1448] User: Growing child

[1449] Device: Small robot captures children's daily lives

[1450] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[1451] Device: "You've grown 1cm since a month ago. You may be deficient in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, and the specific ingredients and recommended intake are displayed on the screen.

[1452] 4. Response to additional questions

[1453] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[1454] User: Child asks robot a question

[1455] Terminal: Sends question to server

[1456] Server: Analyzes using a generative AI model and identifies appropriate fatigue recovery items

[1457] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises, and displays images of specific items to purchase on the screen.

[1458] In this way, the system efficiently manages users' health conditions and lifestyle habits and provides individually optimized advice in real time, improving the health and quality of life of the entire family.

[1459] The processing flow will be explained below.

[1460] Step 1:

[1461] Subject: Device

[1462] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[1463] Step 2:

[1464] Subject: Server

[1465] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[1466] Step 3:

[1467] Subject: Server

[1468] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[1469] Step 4:

[1470] Subject: Server

[1471] The server provides real-time updates on the user's daily life and health status based on a database, and this information is fed into a generative artificial intelligence model.

[1472] Step 5:

[1473] Subject: Server

[1474] The server uses the generated AI model to analyze the user's health condition and dietary history. The model identifies the user's nutritional deficiencies and health risks and generates analysis results.

[1475] Step 6:

[1476] Subject: Server

[1477] Based on the analysis results, the server generates personalized health management advice, including specific suggestions for actions and dietary improvements.

[1478] Step 7:

[1479] Subject: Server

[1480] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[1481] Step 8:

[1482] Subject: Device

[1483] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[1484] Step 9:

[1485] Subject: User

[1486] The user can ask the robot additional questions, such as, "I have a stop at a convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[1487] Step 10:

[1488] Subject: Device

[1489] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[1490] Step 11:

[1491] Subject: Server

[1492] The server uses a generative AI model to analyze the user's question and generate the optimal answer, taking into account the amount of practice and past data.

[1493] Step 12:

[1494] Subject: Server

[1495] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[1496] Step 13:

[1497] Subject: Device

[1498] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[1499] Example 1

[1500] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1501] Conventional health management systems have difficulty understanding a user's individual health condition and lifestyle habits in detail and providing optimized advice in real time. Furthermore, if the user's face is not identified or data analysis is not performed accurately, the advice provided may be inappropriate. Furthermore, it is difficult to respond to follow-up questions from users, which hinders the improvement of user satisfaction.

[1502] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1503] In this invention, the server includes means for capturing image or video data of a user using a photographing device equipped with image recognition technology, means for transmitting the captured image or video data in real time to an analysis device via a communication device, means for analyzing the received image or video data in the analysis device and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice via an audio output device and a display device. This makes it possible to efficiently and accurately manage the user's health condition and lifestyle habits and provide individually optimized advice in real time.

[1504] "Image recognition technology" is a technology that uses computer vision algorithms to detect and identify specific objects or features within images or video data.

[1505] "Photographing device" refers to a device for taking images or videos, such as a camera or video camera.

[1506] A "communications device" is a device for transmitting and receiving data, and includes devices that use network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[1507] An "analysis device" is a computer system or device for processing and analyzing received data.

[1508] "Means for identifying a user's face" refers to algorithms or software that extract the user's facial features from image or video data and identify that face.

[1509] "Health status" refers to information about the user's physical health in general, including data such as blood pressure, heart rate, and body temperature.

[1510] "Diet history" refers to information about meals the user has had in the past, and includes data such as ingredients, calorie intake, and meal times.

[1511] A "generative artificial intelligence model" is an AI model that uses machine learning and deep learning technologies to generate analysis results and predictions from input data.

[1512] The "means for generating advice based on analysis results" is software or an algorithm for generating optimal health management and lifestyle improvement advice for the user based on the analysis results.

[1513] The "audio output device" is a device such as a speaker that reproduces the generated advice by voice.

[1514] A "display device" is a device such as a display or monitor for visually displaying the generated advice.

[1515] The present invention is a system for supporting family health management and lifestyle optimization, which includes a photographing device equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice. The configuration and operation of this system are described in detail below.

[1516] System Configuration

[1517] The system consists of three main parts:

[1518] 1. Terminal (a small device with a built-in camera)

[1519] 2. Server (computer system for data analysis)

[1520] 3. Communication network (infrastructure for sending and receiving data)

[1521] Terminal

[1522] The device is a small device with a built-in camera that records the user's daily life. The device has the following features:

[1523] Camera: Equipped with image recognition technology, it takes pictures of the user's face and how they are eating.

[1524] Communication module: Sends data to the server in real time using Wi-Fi or Bluetooth.

[1525] Audio output: The generated advice is notified to the user by voice.

[1526] Display: Visually displays generated advice and notifications.

[1527] server

[1528] The server is a computer system that receives image and video data sent from the device and performs multiple analyses. The server has the following functions:

[1529] Data reception: Receives data sent from the terminal.

[1530] Image Recognition: Preprocess image data and reduce noise using Python's OpenCV library.

[1531] Facial recognition: A deep learning model using TensorFlow extracts facial features and matches them with facial data registered in a database.

[1532] Health data analysis: User health data and dietary history are input into a generative artificial intelligence model (e.g., PyTorch model) and analyzed.

[1533] Personalized advice generation: The analysis results are converted into text using a natural language generation (NLG) API, and the most appropriate advice is generated for the user.

[1534] communication network

[1535] A communications network is an infrastructure that enables real-time data transmission between devices and servers, and primarily uses network technologies such as Wi-Fi, Bluetooth, 4G, and 5G.

[1536] Specific examples

[1537] Example 1: Dad has a cold

[1538] User:Dad

[1539] Device: A small device captures Dad's daily life and collects data such as changes in his complexion and the frequency of his coughs.

[1540] Server: Analyzes the received data, identifies cold symptoms, and compares them with past data to generate an appropriate soup recipe.

[1541] Device: The generated advice is announced aloud, such as "Chicken and vegetable soup is recommended for colds at this time of year," and the recipe is shown on the screen.

[1542] Example 2: Nutritional management for growing children

[1543] User: Growing child

[1544] Device: A small device that captures a child's daily life and records what they eat and how they exercise.

[1545] Server: Analyzes the received data and identifies iron and zinc deficiencies based on height and weight measurements and dietary history.

[1546] Device: "You've grown 1cm taller compared to a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken a day," the device says, showing specific ingredients and recommended intake amounts on the display and providing a voice notification.

[1547] Example 3: Advice for recovering from fatigue

[1548] User: My child has baseball practice after school, but I still have time to stop by a convenience store.

[1549] Terminal: Sends the question to the server.

[1550] Server: Analyzes the question using a generative artificial intelligence model and identifies appropriate fatigue recovery items.

[1551] Device: "Bananas and yogurt are recommended for relieving fatigue," the device advises aloud, and displays images of specific items to purchase on the screen.

[1552] The system efficiently manages users' health conditions and lifestyle habits and provides personalized, optimized advice in real time, improving the health and quality of life of the entire family.

[1553] Prompt Sentence Examples

[1554] "Generate appropriate dietary advice for users who are showing signs of a cold."

[1555] "Please analyze the nutritional status of growing children and advise them on the nutrients they need."

[1556] "Please tell me some foods that are effective in relieving fatigue."

[1557] In this way, the system can provide personalized health care advice based on the user's individual data.

[1558] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1559] Step 1:

[1560] Data collection

[1561] The device captures images and video data of the user's daily life and sends the data to a server in real time.

[1562] Input: Image and video data capturing user activity

[1563] Specific operation: The device's built-in camera captures the user's face and actions, and sends the image and video data to a server using communication methods such as Wi-Fi or Bluetooth.

[1564] Output: Image and video data sent to the server

[1565] Step 2:

[1566] Image Recognition and Facial Identification

[1567] The server analyzes the image and video data it receives and identifies the user's face.

[1568] Input: Image and video data sent from the device

[1569] How it works: The server preprocesses the image data using Python's OpenCV library, extracts and identifies facial features using a deep learning model (e.g., TensorFlow), and matches the facial information in the database for face identification.

[1570] Output: Identified user's face information

[1571] Step 3:

[1572] Health data analysis

[1573] The server inputs the user's health data and dietary history into a generative AI model for analysis, identifying the user's health condition and nutritional deficiencies.

[1574] Input: Identified user's face information, user's health data, diet history

[1575] Specific operation: The server inputs health data (e.g., blood pressure, heart rate, etc.) and dietary history into a generative AI model (e.g., PyTorch model) for analysis. The analysis then estimates nutritional and health status.

[1576] Output: Analysis results (health status, nutritional deficiency, etc.)

[1577] Step 4:

[1578] Generating personalized advice

[1579] The server generates personalized health management advice for the user based on the analysis results.

[1580] Input: Analysis results (health status, nutritional deficiency, etc.)

[1581] Specific operation: The server uses a natural language generation (NLG) API to convert the analysis results into easy-to-understand sentences. For example, if there are signs of a cold, it generates specific advice such as "Chicken and vegetable soup is recommended for colds at this time of year."

[1582] Output: Personalized health advice

[1583] Step 5:

[1584] Advice Notice

[1585] The server sends the generated advice to the device, which notifies the user through voice or display.

[1586] Enter: personalized health advice.

[1587] Specific operation: The server sends the generated advice to the device using a message transmission protocol (e.g., MQTT). The device receives the advice and notifies the user by voice output or display on the screen. For example, the device may notify the user by voice, saying, "Chicken and vegetable soup is recommended for colds this time of year," and show the recipe on the screen.

[1588] Output: Audio and display advice

[1589] In this way, the system effectively manages users' health and lifestyle habits and provides personalized advice in real time.

[1590] (Application example 1)

[1591] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1592] In today's world, autonomous vehicles are becoming increasingly common, but there is a lack of systems that can monitor the health and stress levels of occupants in real time and provide appropriate advice and encourage rest. If this problem is not solved, fatigue and stress will accumulate due to long periods of driving, increasing the risk of accidents and health problems. The present invention aims to solve this problem by effectively monitoring the health and stress levels of occupants in autonomous vehicles and providing appropriate advice.

[1593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1594] In this invention, the server includes: means for capturing images or video data of a user using a camera equipped with image recognition technology; means for analyzing the captured images or video data and identifying the user's face; means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis; means for generating personalized health management advice for the user based on the analysis results; means for notifying the user of the generated advice by voice and image; means for monitoring the health condition and stress level of occupants in an autonomous vehicle and collecting health data using an on-board camera and sensor; means for generating appropriate rest and refreshment advice for the occupants based on the collected health data; and means for notifying the occupants of the advice using an in-vehicle display or audio output device. This makes it possible to effectively manage the health condition and stress level of occupants in an autonomous vehicle in real time and provide appropriate advice.

[1595] "Image recognition technology" is a technology that automatically identifies specific objects or patterns from images or video data captured by a camera.

[1596] "User" refers to a person who uses the system, and in this case includes the occupants of an autonomous vehicle.

[1597] A "generative artificial intelligence model" is a model that uses machine learning algorithms to learn specific patterns and characteristics from large amounts of data and make judgments such as predictions and classifications.

[1598] "Personalized health management advice" refers to advice that is individually optimized based on the user's analyzed health condition and behavioral history.

[1599] A "camera" is a device that converts light into electrical signals to capture images and videos.

[1600] An "on-board camera" is a camera installed inside an autonomous vehicle to capture images of the occupants and their surroundings.

[1601] A "sensor" is a device that measures a physical or chemical property and is primarily used in this system to collect health data.

[1602] A "display" is a device that visually displays information.

[1603] An "audio output device" is a device that outputs information in the form of audio.

[1604] This invention is a system for monitoring the health status and stress level of occupants in an autonomous vehicle and providing appropriate advice. This system is implemented with the following configuration.

[1605] First, the server captures images or video data of the occupants using the vehicle's onboard camera. The captured images and video data are sent to the server via a communications network. The server then analyzes the received data using image recognition technology to identify the occupants' faces. Image processing software such as OpenCV and TensorFlow is used for facial recognition.

[1606] The server then inputs data, including the occupant's health status and dietary history, into a generative AI model for analysis. This generative AI model is used to estimate the occupant's stress level and fatigue level from their facial color and facial expression. The generative AI model uses Keras / TensorFlow. Additional sensors may also be used as needed to collect health data such as heart rate and respiratory rate.

[1607] Based on the analysis results, the server generates personalized health management advice for the occupant. For example, this advice might be, "If high fatigue level is detected while driving, notify the occupant to take a break." The generated advice is communicated to the occupant via a display or audio output device in the vehicle.

[1608] For example, if a passenger begins to feel tired during a long drive, an on-board camera will capture a change in facial color, which the generative AI model will analyze and detect that fatigue is building up. As a result, the server will generate appropriate advice, such as "Take a break at the next service area," and notify the passenger via display or voice.

[1609] Additionally, if a passenger asks a specific question, the generative AI model will perform further analysis based on the question and generate an appropriate response. For example, if the passenger asks, "I'm feeling tired today. What can I do to refresh myself?", the system will provide advice such as, "I recommend taking a short walk and getting some fresh air."

[1610] An example of a prompt might be "Input a picture of the user's face and estimate their fatigue level." This prompt enables the generative AI model to analyze the user's health condition and generate appropriate advice.

[1611] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1612] Step 1:

[1613] The terminal (a system inside the vehicle) captures images or video data of the occupants using an on-board camera. The input is the camera image, and the output is high-resolution image or video data. This data is sent to the server in real time.

[1614] Step 2:

[1615] The server analyzes the received image and video data and identifies the faces of passengers using image recognition technology (OpenCV or TensorFlow). The input is image or video data, and the output is the coordinates of the facial area and identification information. Based on the facial identification results, the data of each passenger is associated.

[1616] Step 3:

[1617] The server collects the health condition data and dietary history data of the identified occupants and inputs them into a generative AI model. The input is facial identification information and past health data, and the output is the analysis results of the health condition. Using the generative AI model (Keras / TensorFlow), stress levels and fatigue levels are estimated from facial color and facial expressions.

[1618] Step 4:

[1619] The server generates personalized health management advice for the occupants based on the analysis results. The input is the health status analysis result, and the output is the advice content. This advice includes encouraging them to take breaks as needed and specific health management suggestions.

[1620] Step 5:

[1621] The generated advice is sent to the terminal, which then notifies the occupant using a display or audio output device. The input is the generated advice, and the output is the notification to the occupant. In this case, the advice is provided to the occupant using speech synthesis (gTTS) or text display.

[1622] Step 6:

[1623] When a user (passenger) asks a specific question to the terminal, the terminal sends the question to the server. The input is the passenger's voice question, and the output is the question data.

[1624] Step 7:

[1625] The server inputs the received question into a generative artificial intelligence model to generate appropriate advice based on the question. The input is the question data and related health data, and the output is the answer advice.

[1626] Step 8:

[1627] The generated answer advice is sent to the terminal, and the terminal notifies the occupant using a display or audio output device. The input is the answer advice, and the output is the answer notification to the occupant. This notification is provided in a form that is easy for the occupant to understand using voice synthesis or text display.

[1628] This series of processing flows makes it possible to manage the health status and stress levels of occupants in self-driving vehicles in real time and provide appropriate advice.

[1629] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1630] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative artificial intelligence model, and a means for providing personalized advice using an emotion engine. The system is configured and operates as follows.

[1631] 1. System Configuration

[1632] 1.1 Terminal (small robot)

[1633] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures images of the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[1634] 1.2 Server

[1635] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and emotion engine to analyze the user's health condition, dietary history, and emotional state.

[1636] 1.3 Communication Networks

[1637] The terminal and the server are connected via a communication network, and data is sent and received in real time.

[1638] 2. Program Processing

[1639] 2.1 Data Collection

[1640] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[1641] 2.2 Image Recognition and Face Identification

[1642] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[1643] 2.3 Health Data Analysis

[1644] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[1645] 2.4 Generating Personalized Advice

[1646] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[1647] 2.5 Advice Notice

[1648] The server generates advice and sends it to the device. The device receives it and notifies the user through voice or display. For example, if a father has a cold, the system might suggest a recipe along with advice like, "Your voice sounds different than usual. Chicken and vegetable soup is recommended for colds at this time of year." The system also makes suggestions that take the user's emotional state into account.

[1649] 3. Specific Examples

[1650] Example 1: Dad has a cold

[1651] User: Dad

[1652] Device: Small robot captures dad's daily life

[1653] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[1654] Emotion engine: Analyzes dad's emotional state to identify stress

[1655] Device: "Chicken and vegetable soup is a good option for those with a cold this time of year," the device advises, and displays a recipe on the display. It also suggests relaxation techniques to improve Dad's mood.

[1656] Example 2: Nutritional management for growing children

[1657] User: Growing child

[1658] Device: Small robot captures children's daily lives

[1659] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[1660] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[1661] Device: "You've grown 1cm since a month ago. You may be lacking in iron and zinc, so try eating 100g of spinach and chicken per day," the device advises, showing specific ingredients and the recommended intake amount on the display. It also provides encouraging messages to motivate the child.

[1662] 4. Response to additional questions

[1663] If the user asks a follow-up question, for example, a child asks, "I can stop by the convenience store after school on my way to baseball practice. What should I buy to help me recover?"

[1664] User: Child asks robot a question

[1665] Terminal: Sends question to server

[1666] Server: Analyzes using a generative AI model and generates the optimal answer. Also uses an emotion engine.

[1667] Device: The device will announce, "Bananas and yogurt are recommended to help relieve fatigue," and will display images of specific items to purchase. It will also suggest other options based on the child's preferences.

[1668] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[1669] The processing flow will be explained below.

[1670] Step 1:

[1671] Subject: Device

[1672] The device's built-in camera captures the user's daily life, periodically capturing images and video data and sending them to a server in real time.

[1673] Step 2:

[1674] Subject: Server

[1675] The server receives the image and video data sent from the device, analyzes the received data, and uses image recognition technology to identify the user's face.

[1676] Step 3:

[1677] Subject: Server

[1678] The server then compares the identified user's facial information with a historical database to identify the individual user, thereby extracting the user's health history and behavioral patterns.

[1679] Step 4:

[1680] Subject: Server

[1681] The server inputs the acquired image and video data into the emotion engine, which analyzes the user's facial expressions and movements to identify their emotional state.

[1682] Step 5:

[1683] Subject: Server

[1684] The server updates the user's emotional state in real time based on the results of the emotion engine, thereby building a database that reflects the user's current emotional state.

[1685] Step 6:

[1686] Subject: Server

[1687] The server inputs health data, dietary history, and emotional state into a generated artificial intelligence model based on this database and performs analysis.

[1688] Step 7:

[1689] Subject: Server

[1690] Based on the analysis results, the generative AI model generates personalized health management advice, including specific suggested actions and dietary improvements.

[1691] Step 8:

[1692] Subject: Server

[1693] The server generates advice and sends related images to the device, including specific recipes and nutritional information.

[1694] Step 9:

[1695] Subject: Device

[1696] The device notifies the user of the advice received from the server. Notifications are made by voice and on the display. For example, the device may say, "Chicken and vegetable soup is recommended for colds this time of year," and display an image of the recipe.

[1697] Step 10:

[1698] Subject: User

[1699] The user asks a follow-up question to the device, for example, "I have a chance to stop by a convenience store after school before going to baseball practice. What should I buy to help me recover?"

[1700] Step 11:

[1701] Subject: Device

[1702] The device sends the user's question to the server, which converts the spoken question into digital data and sends it to the server for analysis.

[1703] Step 12:

[1704] Subject: Server

[1705] The server uses a generative artificial intelligence model to analyze the question and generate the best answer, along with an emotion engine that takes into account the user's current emotional state.

[1706] Step 13:

[1707] Subject: Server

[1708] The server generates a response and sends related information to the device, including specific fatigue recovery items and where to purchase them.

[1709] Step 14:

[1710] Subject: Device

[1711] The device notifies the user of the information received from the server. For example, it may notify the user by voice, saying, "Bananas and yogurt are recommended for relieving fatigue," and show an image of the specific item to purchase on the display.

[1712] Example 2

[1713] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1714] In modern families, it is difficult to manage the health and lifestyles of all family members at once. Providing personalized health management advice for each family member is particularly time-consuming and labor-intensive. Furthermore, there is a lack of means to grasp the health and emotional state of each family member in real time and quickly provide appropriate advice based on that information.

[1715] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1716] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for acquiring the user's health condition, dietary history, and emotional state, means for inputting the acquired data into a generative AI model and an emotion engine for analysis, means for generating personalized health management advice for the user based on the analysis results, means for generating prompt sentences and inputting them into the generative AI model for analysis, means for notifying the user of the generated advice by voice and image, and means for transmitting data from a terminal to the server and receiving the analysis results. This makes it possible to analyze the health conditions and emotional states of all family members in real time and quickly provide individually optimized advice.

[1717] "Image recognition technology" is a technology that analyzes images and videos taken with a camera and identifies the objects and people contained within them.

[1718] A "camera" is a device for taking pictures and videos.

[1719] "User" refers to each member of a family who uses the system.

[1720] "Means for identifying faces" refers to technology for identifying a user's face from captured images or video data.

[1721] "Health status" refers to information about the user's physical health, such as body temperature, appetite, and sleep status.

[1722] "Dietary history" refers to a record of the food and drinks consumed by a user.

[1723] "Emotional state" refers to information that represents the user's psychological state, such as joy, sadness, anger, etc.

[1724] A "generative artificial intelligence model" is an artificial intelligence technology that generates advice suited to the user based on various data.

[1725] An "emotion engine" is a technology that analyzes a user's emotional state and supports appropriate advice.

[1726] A "prompt sentence" refers to a question or instruction sentence that is input into an artificial intelligence model.

[1727] "Analysis results" are information that includes advice and suggestions generated based on your health and emotional state.

[1728] "Audio notification means" refers to a technique or device for conveying the generated advice to the user as audio.

[1729] "Means for notifying by image" refers to a technique or device for displaying the generated advice on a display and conveying it to the user.

[1730] "Terminal" refers to a small robot or device that captures the user's daily life.

[1731] "Means for transmitting data" refers to the technology used to transmit captured images and videos to a server.

[1732] "Means for receiving analysis results" refers to technology for receiving analysis results sent from the server.

[1733] The present invention is a system for supporting family health management and lifestyle optimization, which includes a camera equipped with image recognition technology, a generative AI model, and a means for providing personalized advice using an emotion engine. Specific embodiments of the system are described below.

[1734] First, this system includes a terminal equipped with a built-in camera equipped with image recognition technology for capturing images of the user's daily life. The terminal is in the form of a small robot, and captures images of the user as they go about their daily life. Images and video data captured by this camera are sent to a server in real time. The server and terminal are connected via a communications network, allowing for rapid data transmission and reception.

[1735] The server then analyzes the transmitted image and video data. It uses image recognition technology to identify the user's face, thereby identifying each family member. The server also acquires the user's health status, dietary history, and emotional state. This data is then fed into a generative artificial intelligence model (generative AI model) and emotion engine for analysis.

[1736] Based on the analyzed data, the server generates personalized health management advice optimized for each user. The prompt sentences used to generate this advice are also generated by the server. For example, a prompt sentence such as "Please tell me how I can reduce my recent stress" is input into the generative AI model. The generated advice is sent to the device and communicated to the user via voice and images.

[1737] As a specific operational scenario, consider the case where Dad has caught a cold. The device takes pictures of Dad's daily life and sends them to the server. The server identifies the signs of a cold and generates advice suggesting a soup recipe that is effective against colds. The device notifies Dad by voice, saying, "Chicken and vegetable soup is recommended for colds at this time of year," and shows the recipe on the display.

[1738] In addition, when managing the nutrition of growing children, the device takes pictures of the child's daily life and sends the data to a server. The server analyzes the data and identifies iron and zinc deficiencies. The server advises, "You've grown 1cm taller than you were a month ago. You're deficient in iron and zinc, so you should eat 100 grams of spinach and chicken per day," and the device notifies the child of the specific foods and the amount to consume.

[1739] Examples of prompts include, "What meals and recipes would you recommend if Dad had a cold?" and "Please suggest meals that contain the nutrients a growing child needs."

[1740] In this way, the system can analyze the user's health and emotional state in real time and quickly provide individually optimized advice, which is expected to improve the health and quality of life of the entire family.

[1741] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1742] Program processing flow

[1743] Step 1: Data collection

[1744] Operation:

[1745] The device uses a built-in camera to capture images and video data of the user's daily life.

[1746] The device transmits this data to the server in real time.

[1747] Input: Images and video data from the user's daily life.

[1748] Output: Image and video data sent to the server.

[1749] Step 2: Image Recognition and Face Identification

[1750] Operation:

[1751] The server analyzes the image and video data received from the device and identifies the user's face.

[1752] The server associates the identified facial data with individual user profiles.

[1753] Input: Image and video data transmitted in real time.

[1754] Output: User's face identification results and corresponding profile data.

[1755] Step 3: Acquire data and prepare for analysis

[1756] Operation:

[1757] The server retrieves the user's health status, dietary history, and emotional state from a database.

[1758] The server prepares the acquired data for input into the generative artificial intelligence model and emotion engine.

[1759] Input: User profile data, health status, diet history, emotional state.

[1760] Output: Input data for analysis.

[1761] Step 4: Analyze your health and emotional state

[1762] Operation:

[1763] The server uses a generative artificial intelligence model and an emotion engine to analyze the user's health and emotional state.

[1764] The server calculates the analysis results.

[1765] Input: The input data for the analysis.

[1766] Output: Analysis of the user's health and emotional state.

[1767] Step 5: Generating Advice

[1768] Operation:

[1769] Based on the analysis results, the server generates health management advice optimized for each user.

[1770] The server generates a prompt sentence and inputs it into the generative AI model.

[1771] Input: Analysis results of health and emotional state, prompt text.

[1772] Output: Personalized health care advice.

[1773] Step 6: Advice Notification

[1774] Operation:

[1775] The server transmits the generated advice to the terminal.

[1776] The terminal notifies the user of the advice by voice and image.

[1777] Input: Generated personalized health care advice.

[1778] Output: Advice notification to the user.

[1779] Specific examples of each step

[1780] Step 1: Data collection

[1781] Operation: The device takes pictures of the user eating breakfast with its camera and sends the captured images and videos to a server every five minutes.

[1782] Input: Image and video data of a user having breakfast.

[1783] Output: Image and video data received by the server.

[1784] Step 2: Image Recognition and Face Identification

[1785] How it works: The server analyzes the image data sent from breakfast and uses a recognition algorithm to identify Dad's face.

[1786] Input: Image data taken during breakfast.

[1787] Output: Dad's face identification result and corresponding profile data.

[1788] Step 3: Acquire data and prepare for analysis

[1789] How it works: The server retrieves Dad's eating history and emotional state from the database for the past week and prepares it as data for analysis.

[1790] Input: Dad's profile data, food history for the past week, and emotional state.

[1791] Output: Input data for Daddy's analysis.

[1792] Step 4: Analyze your health and emotional state

[1793] How it works: The server uses a generative artificial intelligence model and an emotion engine to identify that Dad has a vitamin C deficiency and has recently had high stress levels.

[1794] Input: Input data for Daddy's analysis.

[1795] Output: Analysis of dad's health and emotional state.

[1796] Step 5: Generating Advice

[1797] How it works: Based on the analysis results, the server generates advice such as "Eat orange juice and vegetables to make up for vitamin C deficiency" and inputs the relevant prompt sentence into the generative AI model.

[1798] Input: Analysis results of dad's health and emotional state, prompt text.

[1799] Output: Personalized health advice for dad.

[1800] Step 6: Advice Notification

[1801] How it works: The server sends the generated advice to the device, which then announces aloud, "To compensate for your vitamin C deficiency, you should drink orange juice," and shows the specific suggestion on the display.

[1802] Enter: personalized health care advice for dads.

[1803] Output: Audio notification and image display of advice to dad.

[1804] (Application example 2)

[1805] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1806] Conventional health management systems monitor users' health status and analyze their dietary history, but lack a means for users to easily obtain recommended ingredients and nutrients. Furthermore, simply providing personalized advice makes it difficult to encourage specific behavioral changes, limiting its effectiveness, especially in today's busy society. Furthermore, systems lack sufficient support for health management that takes into account the user's emotional state. Therefore, more comprehensive and practical health management support is needed.

[1807] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1808] In this invention, the server includes means for capturing image or video data of a user using a camera equipped with image recognition technology, means for analyzing the captured image or video data and identifying the user's face, means for inputting data including the user's health condition and dietary history into a generative artificial intelligence model for analysis, means for generating personalized health management advice for the user based on the analysis results, and means for notifying the user of the generated advice by voice and image and enabling the user to order appropriate meals in cooperation with a food delivery service. This allows the user to easily order specific meals based on the health management advice, enabling more practical and comprehensive health management to be achieved.

[1809] "Image recognition technology" is a technology that analyzes images and videos through cameras and other sensors to recognize specific objects and patterns.

[1810] A "generative artificial intelligence model" is a machine learning algorithm or deep learning model that uses large amounts of data to self-learn and perform a specific task (e.g., health analysis or generating personalized advice).

[1811] An "emotion engine" is an algorithm or system that analyzes a user's emotional state from their voice and facial expressions and identifies changes in their emotions.

[1812] "Personalized health management advice" involves analyzing a user's individual health condition and dietary history to generate advice optimized for the user.

[1813] A "food delivery service" is a service that delivers meals ordered by a user to a location specified by the user.

[1814] A "communications network" is an infrastructure that allows devices or systems in different locations to send and receive data.

[1815] 1. System Configuration

[1816] 1.1 Device (smartphone)

[1817] The device is equipped with a built-in camera equipped with image recognition technology. This camera captures the user's daily life and transmits the captured images and video data to a server in real time. The device also has audio output capabilities and a display, which are used to provide advice and notifications to the user.

[1818] 1.2 Server

[1819] The server receives image and video data sent from the device and identifies the user's face using image recognition technology. Furthermore, the server is equipped with a generative artificial intelligence model and an emotion engine that analyzes the user's health condition, dietary history, and emotional state. Based on the analysis results, it generates personalized health management advice. It also has the ability to link with food delivery services and order appropriate meals.

[1820] 1.3 Communication Networks

[1821] The terminal and the server are connected via a communication network, typically the Internet, and data is sent and received in real time.

[1822] 2. Program Processing

[1823] 2.1 Data Collection

[1824] The device's camera captures images and video data of the user's daily life, which is then periodically sent to a server for subsequent analysis.

[1825] 2.2 Image Recognition and Face Identification

[1826] The server analyzes the received image and video data and identifies the user's face, which allows it to identify each family member and associate their health condition, dietary history, and emotional state.

[1827] 2.3 Health Data Analysis

[1828] The server inputs the user's health data, dietary history, and emotional state into a generative AI model and emotion engine for analysis. Based on the analysis results, the server identifies the user's health condition, nutritional deficiencies, and emotional changes.

[1829] 2.4 Generating Personalized Advice

[1830] Based on the analysis results, a generative AI model and emotion engine generate personalized health management advice for each user, including health management and dietary improvements, as well as emotion-based behavioral suggestions.

[1831] 2.5 Notification of Advice and Collaboration with Food Delivery Services

[1832] The server generates advice and sends it to the device, which then receives it and notifies the user via voice or display. Based on the advice, the user can then order an appropriate meal from a food delivery service with the touch of a button.

[1833] 3. Specific Examples

[1834] Example 1: Dad has a cold

[1835] User: Dad

[1836] Device: Smartphone captures dad's daily life

[1837] Server: Analyzes dad's image and video data to identify signs of a cold. Compares this with past data and generates an appropriate soup recipe.

[1838] Emotion engine: Analyzes dad's emotional state and identifies whether he is stressed or not

[1839] Terminal: Advises, "Chicken and vegetable soup is recommended for this cold season," and displays the recipe on the screen. It also links with food delivery services, allowing soup to be ordered with the touch of a button.

[1840] Example 2: Nutritional management for growing children

[1841] User: Growing child

[1842] Device: Smartphones capture children's daily lives

[1843] Server: Analyzes children's image and video data, and analyzes their height, weight, and dietary history to identify iron and zinc deficiencies.

[1844] Emotion Engine: Analyzes children's emotional state and identifies fluctuations in motivation

[1845] Device: "You've grown 1cm taller than you were a month ago. You may be deficient in iron and zinc, so try eating 100 grams of spinach and chicken per day," the device advises, showing specific ingredients and recommended intake amounts on the display. Furthermore, the device will link with a food delivery service, allowing users to order spinach and chicken.

[1846] Prompt Sentence Examples

[1847] "Please suggest a recommended meal menu for users who have caught a cold."

[1848] In this way, the system analyzes the user's health and emotional state in real time and provides individually optimized advice, improving the health and quality of life of the entire family.

[1849] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1850] Step 1:

[1851] The device camera captures images and video data of the user's daily life, and this data is sent to the server in real time. The input is the image and video data captured by the device camera, and the output is the raw image and video data sent to the server.

[1852] Step 2:

[1853] The server analyzes the received image and video data and uses image recognition technology to identify the user's face. The input is the image and video data sent in step 1, and the output is the identified user's face data. Specifically, the server runs a facial recognition algorithm to extract facial feature points.

[1854] Step 3:

[1855] Based on the identified facial data, the server inputs data including the user's health status and dietary history into a generative artificial intelligence model. The input is the user's facial data, past health status data, and dietary history, and the output is the analysis result regarding the health status. Specifically, the server feeds the health data and dietary history into the AI ​​model to generate a health status.

[1856] Step 4:

[1857] The server uses an emotion engine to analyze the user's emotional state. The input is the user's facial data and voice data, and the output is the analysis result regarding the user's emotional state. Specifically, the server runs a voice analysis algorithm and a facial expression analysis algorithm to generate an emotion tag.

[1858] Step 5:

[1859] The server generates personalized health management advice for the user based on the health data analysis results and emotional state. The input is the health condition analysis results and emotional state analysis results, and the output is health management advice provided to the user. Specifically, the server uses a generative AI model to generate optimal advice in text format from the health analysis results and emotional tags.

[1860] Step 6:

[1861] The server sends the generated advice to the terminal, which then notifies the user of it through voice and display. The input is the advice data sent from the server, and the output is the voice and image data presented to the user. Specifically, the terminal uses voice synthesis technology to convert the advice into voice and displays the text on the display.

[1862] Step 7:

[1863] The server works with food delivery services to provide options for ordering appropriate meals based on the generated advice. The input is the generated advice data and the user's location information, and the output is a meal menu that the user can select and a delivery order screen. Specifically, the server accesses a delivery API and generates an interface that allows the user to order with one click.

[1864] Step 8:

[1865] When the user confirms the order, the server sends the order data to the food delivery service and sends an order confirmation notice to the user. The input is the user's order data, and the output is the order confirmation data sent to the delivery service and the order confirmation notice to the user. Specifically, the server sends the order data to the delivery service's system and sends a confirmation message to the terminal if the order is successful.

[1866] Through this series of processes, users can receive health management advice and easily order delivery meals based on that advice.

[1867] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1868] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1869] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1870] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1871] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1872] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1873] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1874] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1875] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1876] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1877] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1878] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1879] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1880] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1881] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1882] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1883] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1884] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1885] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1886] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1887] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1888] The following is further disclosed regarding the above embodiment.

[1889] (Claim 1)

[1890] A means for capturing an image or video data of a user using a camera equipped with image recognition technology;

[1891] A means for analyzing the captured image or video data and identifying the user's face;

[1892] A means for inputting data including the user's health condition and dietary history into a generating artificial intelligence model for analysis;

[1893] means for generating personalized health management advice for the user based on the analysis results;

[1894] means for notifying the user of the generated advice by voice and image;

[1895] A system including:

[1896] (Claim 2)

[1897] 2. The system according to claim 1, further comprising means for predicting the user's future behavior and health condition based on the user's past behavioral patterns and health history, and generating advice based thereon.

[1898] (Claim 3)

[1899] The system according to claim 1, further comprising means for responding to additional questions through voice dialogue with the user, performing analysis using a generative artificial intelligence model based on the questions, and generating and notifying response advice.

[1900] "Example 1"

[1901] (Claim 1)

[1902] A means for capturing images or video data of a user using a photographing device equipped with image recognition technology;

[1903] a means for transmitting the captured image or video data to an analysis device via a communication device in real time;

[1904] means for analyzing the received image or video data in an analysis device and identifying a user's face;

[1905] A means for inputting data including the user's health condition and dietary history into a generating artificial intelligence model for analysis;

[1906] means for generating personalized health management advice for the user based on the analysis results;

[1907] means for notifying the user of the generated advice by an audio output device and a display device;

[1908] A system including:

[1909] (Claim 2)

[1910] 2. The system according to claim 1, further comprising means for predicting the user's future behavior and health condition based on the user's past behavioral patterns and health history, and generating advice based thereon.

[1911] (Claim 3)

[1912] The system according to claim 1, further comprising means for responding to additional questions through voice dialogue with the user, performing analysis using a generative artificial intelligence model based on the questions, and generating and notifying response advice.

[1913] "Application Example 1"

[1914] (Claim 1)

[1915] A means for capturing an image or video data of a user using a camera equipped with image recognition technology;

[1916] A means for analyzing the captured image or video data and identifying the user's face;

[1917] A means for inputting data including the user's health condition and dietary history into a generating artificial intelligence model for analysis;

[1918] means for generating personalized health management advice for the user based on the analysis results;

[1919] means for notifying the user of the generated advice by voice and image;

[1920] A means of monitoring the health and stress levels of occupants in autonomous vehicles and collecting health data using on-board cameras and sensors; and

[1921] A means for generating appropriate rest and refreshment advice for occupants based on the collected health data; and

[1922] a means for notifying an occupant of the advice using a display or an audio output device in the vehicle;

[1923] A system including:

[1924] (Claim 2)

[1925] 2. The system according to claim 1, further comprising means for predicting the user's future behavior and health condition based on the user's past behavioral patterns and health history, and generating advice based thereon.

[1926] (Claim 3)

[1927] The system according to claim 1, further comprising means for responding to additional questions through voice dialogue with the user, performing analysis using a generative artificial intelligence model based on the questions, and generating and notifying response advice.

[1928] "Example 2: Combining Emotion Engines"

[1929] (Claim 1)

[1930] A means for capturing an image or video data of a user using a camera equipped with image recognition technology;

[1931] A means for analyzing the captured image or video data and identifying the user's face;

[1932] means for acquiring a user's health status, dietary history, and emotional state;

[1933] A means for inputting the acquired data into a generative artificial intelligence model and an emotion engine for analysis;

[1934] means for generating personalized health management advice for the user based on the analysis results;

[1935] means for generating prompt sentences and inputting them into a generative artificial intelligence model for analysis;

[1936] means for notifying the user of the generated advice by voice and image;

[1937] means for transmitting data from the terminal to a server and receiving analysis results;

[1938] A system including:

[1939] (Claim 2)

[1940] 2. The system according to claim 1, further comprising means for predicting the user's future behavior and health condition based on the user's past behavioral patterns and health history, and generating advice based thereon.

[1941] (Claim 3)

[1942] The system according to claim 1, further comprising means for responding to additional questions through voice dialogue with the user, performing analysis using a generative artificial intelligence model based on the questions, and generating and notifying response advice.

[1943] "Application example 2 when combining emotion engines"

[1944] (Claim 1)

[1945] A means for capturing an image or video data of a user using a camera equipped with image recognition technology;

[1946] A means for analyzing the captured image or video data and identifying the user's face;

[1947] A means for inputting data including the user's health condition and dietary history into a generating artificial intelligence model for analysis;

[1948] means for generating personalized health management advice for the user based on the analysis results;

[1949] A means for notifying the user of the generated advice by voice and image, and linking with a food delivery service to order appropriate meals;

[1950] A system including:

[1951] (Claim 2)

[1952] 2. The system according to claim 1, further comprising means for predicting the user's future behavior and health condition based on the user's past behavioral patterns and health history, and generating advice based thereon.

[1953] (Claim 3)

[1954] The system according to claim 1, further comprising means for responding to additional questions through voice dialogue with the user, performing analysis using a generative artificial intelligence model based on the questions, and generating and notifying response advice. [Explanation of symbols]

[1955] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for capturing an image or video data of a user using a camera equipped with image recognition technology; A means for analyzing the captured image or video data and identifying the user's face; A means for inputting data including the user's health condition and dietary history into a generating artificial intelligence model for analysis; means for generating personalized health management advice for the user based on the analysis results; means for notifying the user of the generated advice by voice and image; A system including:

2. 2. The system according to claim 1, further comprising means for predicting the user's future behavior and health condition based on the user's past behavioral patterns and health history, and generating advice based thereon.

3. 2. The system according to claim 1, further comprising means for responding to additional questions through voice dialogue with the user, performing analysis using a generative artificial intelligence model based on the questions, and generating and notifying response advice.

Images