System

Abstract

A system is provided.SOLUTION: The system includes a terminal for recording an exercise amount and a meal content of an individual, a server for receiving data of the recorded exercise amount and the meal content and performing preprocessing and storage, a server for extracting a feature from the stored data and generating a risk and a countermeasure plan for each disease, a means for notifying the terminal of the generated countermeasure plan, and a server for receiving feedback data from the terminal and updating the risk evaluation and the countermeasure plan.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, it is important for individuals to manage their lifestyle habits and take appropriate health measures. However, it is not easy to accurately record daily exercise and dietary habits and correlate them with disease risk. Furthermore, there is a lack of systems that provide specific measures tailored to individuals based on this recorded data. This makes health management difficult for users, and may delay disease prevention. The present invention aims to solve these problems by providing a system that uses individual lifestyle habit data to assess disease risk and provide appropriate measures. [Means for solving the problem]

[0005] The present invention is a system that includes a device that records an individual's exercise amount and dietary details, a server that receives the recorded exercise amount and dietary details data and performs preprocessing and storage, a server that extracts features from the stored data and generates disease-specific risk and countermeasures, a means for notifying the device of the generated countermeasures, and a server that receives feedback data from the device and updates the risk assessment and countermeasures. Specifically, the system uses a device equipped with an acceleration sensor and GPS to record exercise amount, and a device equipped with a camera and image processing algorithms to record dietary details. The system collects detailed lifestyle data on an individual, evaluates disease risk based on this data, and provides specific countermeasures tailored to the individual. This allows users to easily manage their health in their daily lives.

[0006] "Amount of exercise" refers to the total amount of physical activity, such as the amount of energy consumed by an individual in daily life and physical activities, distance traveled, number of steps taken, etc.

[0007] "Dietary content" refers to the type, amount, and composition of the food and beverages consumed by an individual, and specifically includes information such as calories, nutrients, and ingredients.

[0008] "Terminal" refers to a device for collecting, processing, and communicating data, and in the present invention includes glasses with recording capabilities.

[0009] A "server" refers to a computer system that stores and processes data on a network, and is responsible for receiving and preprocessing data from multiple terminals.

[0010] "Preprocessing" refers to processes such as data cleaning, noise removal, and missing value completion to prepare collected data in a form that can be analyzed.

[0011] "Feature extraction" is the process of identifying important patterns or numerical features from stored data and using them in analysis.

[0012] "Countermeasures" refers to specific guidelines for action and lifestyle improvement proposals provided based on the results of disease risk assessment.

[0013] "Risk assessment" is the calculation and analysis of the probability of occurrence of a specific disease and risk factors based on collected data.

[0014] "Feedback data" refers to data provided by users such as results, impressions, and observed changes after execution, and is used to make new risk assessments and update countermeasure proposals. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention provides a system for recording an individual's exercise amount and dietary content, analyzing the data to assess the risk of illness, and providing appropriate countermeasures. As an embodiment of the present invention, a program is generated as follows, and its processing will be described in detail.

[0037] Recording exercise volume

[0038] A user wears the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity in real time. For example, when a user is jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[0039] Recording dietary information

[0040] When a user eats a meal, the user takes an image of the meal using the camera attached to the device. The image taken by the camera is temporarily saved in the device. This meal image data becomes important information for later analysis of the meal contents. For example, if a user eats a sandwich for lunch, the user takes an image of the sandwich with the camera and saves it on the device.

[0041] Data transmission and storage practices

[0042] The device periodically transmits the collected exercise data and meal content data to a server. The transmitted data is received by the server and stored in a database. The server then preprocesses the received data, removing noise and filling in missing values.

[0043] Performing feature extraction and analysis

[0044] The server extracts characteristics of exercise volume and dietary content from the stored data. Specifically, it calculates average steps, distance traveled, and exercise intensity from exercise data, and identifies calories and types of nutrients from dietary image data. This feature extraction is performed using machine learning algorithms and analyzed on the server.

[0045] Conducting risk assessments

[0046] The server then uses the extracted features to assess disease risk. For example, to assess the risk of diabetes, calculations are made based on a lack of physical activity and the frequency of high-calorie food intake. This risk assessment is performed objectively using statistical methods and machine learning algorithms.

[0047] Generate countermeasures and notify

[0048] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%." These countermeasures are then sent to the device, where the user can confirm them.

[0049] Collecting feedback and updating countermeasures

[0050] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends this feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user.

[0051] Specific examples

[0052] 1. Jogging and Lunch Log:

[0053] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[0054] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[0055] 2. Data transmission and storage:

[0056] The device sends the collected jogging data and meal images to a server.

[0057] The server receives the data and stores it in a database.

[0058] 3. Feature extraction and risk assessment:

[0059] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[0060] The server uses this data to assess diabetes risk.

[0061] 4. Countermeasure generation and notification:

[0062] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[0063] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0064] 5. Feedback and solution update:

[0065] The user implements the measures and records the results on the device.

[0066] The terminal transmits the feedback data to the server again.

[0067] The server updates the risk assessment and countermeasures based on the received data.

[0068] As described above, this invention supports personal health management through the process of data exchange and analysis between the terminal and the server. This system allows users to easily manage their own health in their daily lives and can be useful for disease prevention.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] The user starts exercising, for example, jogging or walking.

[0072] Step 2:

[0073] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[0074] Step 3:

[0075] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[0076] Step 4:

[0077] The device periodically compiles exercise data and meal image data into packets.

[0078] Step 5:

[0079] The device uses Wi-Fi or a mobile communication network to send exercise data and dietary image data to a server.

[0080] Step 6:

[0081] The server stores the received exercise data and meal image data in a database.

[0082] Step 7:

[0083] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[0084] Step 8:

[0085] The server extracts features from the preprocessed data, calculating the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifying calories and types of nutrients from the dietary image data.

[0086] Step 9:

[0087] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise and frequency of high-calorie meals to determine diabetes risk.

[0088] Step 10:

[0089] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week" or "reducing carbohydrate intake by 50%."

[0090] Step 11:

[0091] The server sends the generated countermeasures to the terminal.

[0092] Step 12:

[0093] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[0094] Step 13:

[0095] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[0096] Step 14:

[0097] After implementing the proposed measures, the user records the results and impressions on the device.

[0098] Step 15:

[0099] The terminal transmits the feedback data to the server.

[0100] Step 16:

[0101] The server receives and stores the feedback data.

[0102] Step 17:

[0103] The server updates the risk assessment and countermeasures based on the new data, and if necessary, adjusts the countermeasures and sends them again to the device.

[0104] This allows the system to continuously support users in managing their health and provide actions that help prevent disease.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] In personal health management, there is a need for a system that can properly record exercise volume and dietary content, analyze this data to assess disease risk, and provide accurate countermeasures. However, conventional systems have had problems in efficiently collecting, storing, and analyzing this data, and lacked a means to effectively provide feedback to users to help them manage their health sustainably.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes a device that records an individual's amount of exercise and dietary details, means for receiving the recorded information on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored information and generating a disease risk and countermeasures, means for notifying the device of the generated countermeasures, and means for receiving feedback information from the device and updating the risk assessment and countermeasures. This makes it possible to effectively record an individual's amount of exercise and dietary details, accurately assess disease risk, and provide appropriate countermeasures.

[0110] "Amount of exercise" refers to data related to the physical activity of an individual, and specifically includes the number of steps taken, distance traveled, exercise intensity, and the like.

[0111] "Dietary details" refers to data including the type, amount, calorie, and nutrient information of the food consumed by an individual.

[0112] The "device" is a device for recording exercise and dietary content, and may include an acceleration sensor, a location information system, a camera, an image processing algorithm, and the like.

[0113] An "accelerometer" is a device that detects body movements and acquires acceleration information.

[0114] A "location information system" is a system that uses technology such as GPS to measure an individual's travel distance and location.

[0115] A "camera" is a device for taking pictures of the food.

[0116] An "image processing algorithm" is a computational method for extracting characteristics of food content from captured images and estimating calories and nutrients.

[0117] A "server" is a network computer that receives, preprocesses, and stores data sent from devices, extracts and analyzes features from the data, and generates risk assessments and countermeasures.

[0118] "Preprocessing" refers to processing such as removing noise from received data, filling in missing values, and correcting outliers.

[0119] "Feature extraction" is the process of extracting useful information (e.g., average number of steps, total distance traveled, calories, types of nutrients, etc.) from the stored data.

[0120] "Risk assessment" is the process of analyzing and assessing the risk of developing a disease based on the extracted features.

[0121] "Countermeasures" are specific proposals aimed at improving an individual's health that are generated based on the results of a risk assessment.

[0122] "Notification" refers to the act of transmitting the generated countermeasure plan to the device and informing the user of the information.

[0123] "Feedback" refers to data such as the results, impressions, and implementation status of the user's actions based on the proposed measures.

[0124] "Update" is the process of reassessing risk assessments and countermeasures based on newly acquired data and revising them as necessary.

[0125] The present invention is a system that records the amount of exercise and dietary content of an individual, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments are described below.

[0126] System configuration

[0127] The system consists of the following main components:

[0128] 1. Device worn by the user

[0129] Acceleration sensor: Used to measure movement volume, obtaining step count and acceleration data in real time.

[0130] GPS: Used to measure distance traveled and location.

[0131] Camera: Captures images of food to record eating habits.

[0132] Image processing algorithms are used to extract food content features from the captured images.

[0133] 2. Server

[0134] Data reception and preprocessing unit: Receives data sent from the terminal and performs noise removal and missing value completion.

[0135] Feature extraction module: Extracts features of exercise and dietary content from the stored data.

[0136] Risk assessment module: assesses disease risk based on extracted features.

[0137] Countermeasure proposal generation module: Generates appropriate countermeasure proposals based on the results of risk assessment.

[0138] Notification module: Sends the generated countermeasures to the device.

[0139] Feedback receiving and updating module: receives feedback data from users and updates risk assessment and countermeasure proposals.

[0140] Feature details

[0141] 1. Collecting exercise data

[0142] Users wear a device with a built-in accelerometer and GPS on a daily basis. This device collects exercise data (number of steps, distance traveled, etc.) in real time. For example, when you start jogging, the device immediately starts recording this data.

[0143] 2. Collection of dietary data

[0144] When a user eats, they take a photo of their meal with the device's camera. This image data is analyzed by the device to identify the meal contents (e.g., type of sandwich, amount, calories, etc.).

[0145] 3. Data Transmission

[0146] The device sends the collected data to the server at regular intervals, using Wi-Fi or mobile data communication.

[0147] 4. Data Preprocessing

[0148] The server preprocesses the received data, specifically removing noise, imputing missing values, and detecting and correcting outliers.

[0149] 5. Data feature extraction

[0150] The server's feature extraction module extracts useful information from the pre-processed data, such as average steps, total distance traveled, calories, types of nutrients, etc. Image recognition algorithms and machine learning models are used in this process.

[0151] 6. Risk Assessment

[0152] The server's risk assessment module assesses disease risk based on the extracted features. For example, diabetes risk is assessed based on factors such as lack of exercise and frequency of high-calorie food intake.

[0153] 7. Generation and notification of countermeasures

[0154] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include exercise plans and dietary restriction suggestions. For example, specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%" are made. These countermeasures are notified to the device.

[0155] 8. Collecting and Updating Feedback

[0156] The user records the results and impressions of actions taken based on the proposed countermeasures on the device. This feedback data is then sent back to the server and used to evaluate risks and update the proposed countermeasures.

[0157] Specific examples

[0158] 1. Jogging and Lunch Log

[0159] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[0160] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[0161] 2. Data transmission and storage

[0162] The device sends the collected jogging data and meal images to a server.

[0163] The server receives the data and stores it in a database.

[0164] 3. Feature Extraction and Risk Assessment

[0165] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[0166] The server uses this data to assess diabetes risk.

[0167] 4. Countermeasures generation and notification

[0168] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[0169] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0170] 5. Feedback and solution updates

[0171] The user implements the measures and records the results on the device.

[0172] The terminal transmits the feedback data to the server again.

[0173] The server updates the risk assessment and countermeasures based on the received data.

[0174] Prompt Sentence Examples

[0175] Below are some example prompts to input to a generative AI model:

[0176] Please explain the system flow for recording an individual's exercise volume and dietary habits, analyzing that data to assess disease risk, and generating appropriate countermeasures. Please provide a detailed explanation, including the names of the specific hardware and software used and how the data is processed. Please also provide examples.

[0177] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0178] Step 1:

[0179] The user wears the device.

[0180] Input: The device's accelerometer and GPS are initialized and ready to go.

[0181] Operation: The device detects the user's movements (steps and vibrations) using an accelerometer and acquires location information using GPS. Each measurement data is recorded internally along with the time.

[0182] Output: Real-time updated exercise data (number of steps, distance traveled, exercise intensity).

[0183] Step 2:

[0184] The user takes a photo of the meal.

[0185] Input: The user activates the device's camera and captures an image of the food.

[0186] How it works: Image data captured by the camera is temporarily stored on the device. Optionally, users have the option to manually enter meal details.

[0187] Output: Saved meal image data or manually entered meal content data.

[0188] Step 3:

[0189] The device sends the data to the server.

[0190] Input: Exercise data and dietary data stored in the device at regular intervals (for example, once a day).

[0191] How it works: Your device uses Wi-Fi or mobile data to upload data to a server, which may compress the data before sending it.

[0192] Output: Compressed exercise data and compressed dietary data sent to the server.

[0193] Step 4:

[0194] The server pre-processes the received data.

[0195] Input: Compressed exercise data and compressed dietary data sent from the terminal.

[0196] How it works: The server decompresses the data, removes noise, imputes missing values, and detects and corrects outliers.

[0197] Output: Clean, pre-processed exercise and diet data.

[0198] Step 5:

[0199] The server performs feature extraction.

[0200] Input: Preprocessed physical activity data and dietary data.

[0201] How it works: The server's machine learning algorithm extracts average steps, total distance traveled, and exercise intensity from exercise data, and calorie and nutrient information from dietary image data.

[0202] Output: Extracted feature data of exercise amount and feature data of dietary content.

[0203] Step 6:

[0204] The server assesses the disease risk.

[0205] Input: Extracted feature data of exercise amount and feature data of dietary content.

[0206] How it works: Risk assessment algorithms on the server calculate your risk of certain diseases, such as diabetes or heart disease, using statistical methods and machine learning models.

[0207] Output: Assessed disease risk data.

[0208] Step 7:

[0209] The server generates a countermeasure plan.

[0210] Input: Assessed disease risk data.

[0211] How it works: The server generates countermeasures based on the risk assessment results, such as suggesting exercise plans or dietary restrictions.

[0212] Output: Generated countermeasure proposal data.

[0213] Step 8:

[0214] The server will notify you of the proposed solution.

[0215] Input: Generated countermeasure proposal data.

[0216] Operation: The server sends the countermeasure data to the device, which then notifies the user via push notification or email.

[0217] Output: Notification of proposed measures displayed on the user's device.

[0218] Step 9:

[0219] The user records feedback.

[0220] Input: Results and impressions of users who have implemented measures.

[0221] How it works: The user enters their status after implementing the measures into the device, which also sends periodic reminders to record compliance.

[0222] Output: Recorded feedback data.

[0223] Step 10:

[0224] The terminal transmits the feedback data to the server.

[0225] Input: Feedback data entered by the user.

[0226] Operation: The terminal sends the collected feedback data to the server at regular intervals.

[0227] Output: Feedback data sent to the server.

[0228] Step 11:

[0229] The server updates the risk assessment and countermeasures.

[0230] Input: Feedback data, past exercise data, and dietary data.

[0231] Actions: The server re-performs risk assessment based on new data and updates its countermeasure recommendations, which may include retraining machine learning models.

[0232] Output: Updated risk assessment and proposed countermeasures.

[0233] As described above, the program of this system records the amount of exercise and diet of an individual in detail, evaluates the risk of disease based on this information, and provides appropriate countermeasures, allowing users to easily manage their health in their daily lives.

[0234] (Application example 1)

[0235] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] While managing personal health is important, there is a lack of systems in brick-and-mortar stores that can properly record customers' exercise and dietary habits and manage their health based on that information. Furthermore, there are insufficient means to provide risk assessments and health management recommendations based on this data. Therefore, there is a need for specific methods and systems to effectively support health management in brick-and-mortar stores.

[0237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0238] In this invention, the server includes a terminal that records an individual's amount of exercise and dietary details, means for receiving the recorded data on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored data and generating risks and countermeasures for each disease, means for notifying the terminal of the generated countermeasures, means for receiving feedback data from the terminal and updating the risk assessment and countermeasures, and means for providing menu suggestions and health management advice based on the customer's health condition in a physical store. This enables health management based on the customer's amount of exercise and dietary details in a physical store.

[0239] "Individual exercise amount" is data that indicates the level of physical activity that an individual engages in on a daily basis, such as the number of steps taken, distance traveled, and exercise intensity.

[0240] "Dietary details" refers to data including the type, amount, nutrients, calories, etc. of food and beverages consumed.

[0241] A "terminal" is a device that records an individual's amount of exercise and dietary details and transmits the collected data to a server using a communication means.

[0242] The "server" is a computer system that receives the recorded data and performs preprocessing, storage, feature extraction, risk assessment, and countermeasure generation.

[0243] "Preprocessing" refers to data processing performed before analysis, such as removing noise from data and filling in missing values.

[0244] "Storage" means recording the preprocessed data in a database in the server.

[0245] "Feature extraction" is the process of extracting necessary information from exercise and dietary data and putting it into an analyzable form.

[0246] "Risk assessment" is the process of calculating and assessing the risk of developing a particular disease based on the extracted features.

[0247] "Countermeasures" are specific action plans or proposals based on the results of risk assessment, aimed at improving health and preventing disease.

[0248] "Notification" refers to the act of sending the generated countermeasure plan to the terminal and notifying the user.

[0249] "Feedback data" refers to data that includes the results and impressions of users who have taken actions based on the proposed measures.

[0250] A "physical store" is a physical business establishment, such as a cafe or restaurant, where consumers visit in person to receive services.

[0251] "Menu suggestions" refers to recommending appropriate meals and drinks based on the customer's health condition.

[0252] "Health management advice" refers to specific action plans and lifestyle guidance to improve a customer's health.

[0253] This paper describes a system that records an individual's exercise and dietary habits, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specifically, this system is designed to provide menu suggestions and health management advice based on the customer's health status in a brick-and-mortar store.

[0254] Recording exercise volume

[0255] A user uses the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity. For example, when a user goes jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[0256] Recording dietary information

[0257] When a user eats at a physical restaurant, they take pictures of their meal using the camera attached to their device. The images taken by the camera are temporarily stored on the device, and this data becomes important information for later analysis of the meal contents. For example, if a user orders a salad at a cafe, they take a picture of the salad with the camera and save it on their device.

[0258] Data transmission and storage practices

[0259] The device periodically transmits the collected exercise data and meal content data to the server. The server receives this data and stores it in a database. Preprocessing of the data involves noise removal and missing value completion.

[0260] Performing feature extraction and analysis

[0261] The server extracts characteristics of exercise volume and dietary content from the stored data. Average steps, distance traveled, and exercise intensity are calculated from the exercise data, and calories and types of nutrients are identified from dietary image data. Machine learning algorithms are used to extract these characteristics.

[0262] Conducting risk assessments

[0263] The server then assesses disease risk based on the extracted features. For example, to assess diabetes risk, it considers factors such as lack of exercise and frequency of high-calorie food intake. This risk assessment is performed using statistical methods and machine learning algorithms.

[0264] Generate countermeasures and notify

[0265] Based on the results of the risk assessment, the server generates appropriate measures for the user. The generated measures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" and "reducing carbohydrate intake by 50%." These measures are then notified to the device.

[0266] Gathering feedback and updating countermeasures

[0267] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends the feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user again.

[0268] Specific examples

[0269] Jogging and cafe meal record:

[0270] A user goes jogging in the morning, and the device records the number of steps and distance traveled using the accelerometer and GPS.

[0271] A user orders a salad at a cafe and takes a picture of the salad with the camera on the device.

[0272] Data transmission and storage:

[0273] The device sends the collected jogging data and meal images to a server.

[0274] The server receives the data and stores it in a database.

[0275] Feature extraction and risk assessment:

[0276] The server extracts information on the amount of exercise (number of steps, distance traveled) and calories in the salad.

[0277] The server uses this data to assess diabetes risk.

[0278] Countermeasure generation and notification:

[0279] The server generates recommendations such as "jogging at least three times a week" and "low-carb menus."

[0280] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0281] Feedback and solution update:

[0282] The user implements the measures and records the results on the device.

[0283] The terminal transmits the feedback data to the server again.

[0284] The server updates the risk assessment and countermeasures based on the received data.

[0285] Prompt Sentence Examples

[0286] "The customer entered the number of steps and distance they jogged, as well as a photo of their meal (salad) at the cafe, into the app. Based on this data, the app assesses their health risks and generates optimal health management advice."

[0287] The present invention makes it possible to easily provide health management based on the amount of exercise and dietary content of users even in brick-and-mortar stores, thereby integrating the provision of health services and customer health management in brick-and-mortar stores.

[0288] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0289] Step 1:

[0290] When a user exercises, the device uses an acceleration sensor and GPS to collect exercise data. The input is the user's exercise status (number of steps, distance traveled, exercise intensity), and the device measures data in real time based on this. The output is the exercise data obtained as a result of the measurement.

[0291] Step 2:

[0292] When a user eats at a physical restaurant, they take a photo of their meal using the camera on their device. The input is the photo of the meal, and the device acquires the data by temporarily saving this image. The output is the saved meal image data.

[0293] Step 3:

[0294] The device periodically transmits the collected exercise data and dietary image data to the server. The input is the collected exercise data and dietary data, and the device uploads this data to the server. The output is the data transmitted to the server.

[0295] Step 4:

[0296] The server stores the received exercise data and dietary image data. The input is the exercise data and dietary data sent from the device, which the server records in a database and performs data preprocessing such as noise removal and missing value completion. The output is the preprocessed data.

[0297] Step 5:

[0298] The server extracts features from the preprocessed data. The input is the preprocessed exercise data and dietary data, and the server extracts feature information such as the number of steps, distance traveled, exercise intensity, and dietary calories and nutrients from these data. The output is the extracted feature data.

[0299] Step 6:

[0300] The server evaluates disease risk based on the extracted feature data. The input is the feature data, and the server performs risk assessment using machine learning algorithms and statistical methods. For example, this includes assessing diabetes risk. The output is the risk assessment result.

[0301] Step 7:

[0302] The server generates countermeasures based on the risk assessment results. The input is the risk assessment results, and the server creates countermeasures such as specific exercise plans and dietary restrictions. The output is the generated countermeasures.

[0303] Step 8:

[0304] The generated countermeasures are notified to the terminal by the server. The input is the generated countermeasures, which the server sends to the terminal. The output is the countermeasures notified to the terminal.

[0305] Step 9:

[0306] The user implements the proposed measures and records the results and their impressions on the device. The input is the user's action results and feedback, which the device records as data. The output is the recorded feedback data.

[0307] Step 10:

[0308] The terminal sends the collected feedback data back to the server. The input is the recorded feedback data that the terminal uploads to the server. The output is the feedback data sent to the server.

[0309] Step 11:

[0310] The server updates the risk assessment and countermeasures based on the feedback data. The input is feedback data from the user, and the server reassessssments and updates the countermeasures based on this. The output is the updated risk assessment and countermeasures.

[0311] The above processing steps enable health management based on the user's exercise volume and dietary content even in brick-and-mortar stores, making it possible to integrate health service provision in brick-and-mortar stores with customer health management.

[0312] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0313] The present invention combines a system that records an individual's amount of exercise and dietary details, assesses disease risk based on this data, and provides appropriate countermeasures with an emotion engine that recognizes the user's emotions. This makes it possible to provide more personalized countermeasures that take into account the user's subjective state. As an embodiment of the present invention, a program is generated as follows, and its specific processing is described below.

[0314] Emotional data recording

[0315] A user wears the device of the present invention in their daily life. This device is equipped with a camera, microphone, and sensors, which are used by an emotion engine to analyze the user's facial expressions, tone of voice, and even biosignal data (heart rate, electrodermal activity, etc.) to recognize the user's emotions and generate emotion data. For example, when the user smiles or feels stressed, this emotion data is recorded in real time.

[0316] Data transmission and storage practices

[0317] The device periodically compiles exercise data, food image data, and emotion data into packets, which are then sent to a server via Wi-Fi or a mobile network. The server then stores the received data in a database.

[0318] Performing feature extraction and analysis

[0319] The server extracts various features from the stored data. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from the dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from the emotion data.

[0320] Conducting risk assessments

[0321] The server then assesses disease risk based on the extracted features, taking into account emotional data to assess, for example, the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[0322] Generate countermeasures and notify

[0323] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. For example, if stress levels are high, the server can generate countermeasures such as "jogging three times a week" and "yoga for relaxation." These countermeasures are then sent to the device, where the user can confirm them.

[0324] Collecting feedback and updating countermeasures

[0325] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[0326] Specific examples

[0327] 1. Jogging and Lunch Log:

[0328] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[0329] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[0330] If a user smiles or feels stressed while eating, the emotion engine will recognize this and record the emotion data.

[0331] 2. Data transmission and storage:

[0332] The device sends the collected jogging data, meal images, and emotional data to a server.

[0333] The server receives this data and stores it in a database.

[0334] 3. Feature extraction and risk assessment:

[0335] The server extracts the amount of exercise (number of steps, distance traveled), calorie information of the sandwich, and emotional data (stress level, etc.).

[0336] The server uses this data to assess diabetes risk.

[0337] 4. Countermeasure generation and notification:

[0338] Based on the risk assessment, the server generates suggested countermeasures such as "jogging at least three times a week," "reducing carbohydrate intake by 50%," and "recommending yoga for relaxation."

[0339] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0340] 5. Feedback and solution update:

[0341] The user implements the measures and records the results and impressions on the device.

[0342] The terminal transmits the feedback data to the server again.

[0343] The server updates the risk assessment and countermeasures based on new data, and also takes emotional data into account to adjust the countermeasures.

[0344] This allows the system to more personalize the user's health management and take emotional state into account to help with comprehensive disease prevention and health maintenance.

[0345] The processing flow will be explained below.

[0346] Step 1:

[0347] The user begins exercising, for example, jogging or walking.

[0348] Step 2:

[0349] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[0350] Step 3:

[0351] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[0352] Step 4:

[0353] The device uses a built-in emotion engine to recognize the user's emotions. Emotional data is generated based on facial expression recognition, voice analysis, and biometric data (heart rate, electrodermal activity, etc.). For example, if the user smiles or feels stressed, that emotion is recorded.

[0354] Step 5:

[0355] The terminal periodically assembles the exercise data, meal image data, and emotion data into packets.

[0356] Step 6:

[0357] The device uses Wi-Fi or a mobile communication network to send exercise data, food image data, and emotional data to a server.

[0358] Step 7:

[0359] The server stores the received exercise data, meal image data, and emotion data in a database.

[0360] Step 8:

[0361] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[0362] Step 9:

[0363] The server extracts features from the preprocessed data. From the exercise data, it calculates the average number of steps, distance traveled, and exercise intensity. From the food image data, it identifies calories and types of nutrients. From the emotion data, it extracts emotional states such as joy, sadness, and stress.

[0364] Step 10:

[0365] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise, frequency of high-calorie meals, and stress levels to determine diabetes risk.

[0366] Step 11:

[0367] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week is recommended," "reducing carbohydrate intake by 50%," or "yoga is recommended for relaxation."

[0368] Step 12:

[0369] The server sends the generated countermeasures to the terminal.

[0370] Step 13:

[0371] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[0372] Step 14:

[0373] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[0374] Step 15:

[0375] After implementing the proposed measures, the user records the results, impressions, and emotional state on the device.

[0376] Step 16:

[0377] The device sends feedback data (amount of exercise, dietary content, emotional state) to a server.

[0378] Step 17:

[0379] The server receives and stores the feedback data.

[0380] Step 18:

[0381] The server updates the risk assessment and countermeasures based on the new data. If necessary, it adjusts the countermeasures and sends them back to the device. It also takes emotional data into account to make the countermeasures more personalized.

[0382] This allows the system to continuously support the user's health management and achieve comprehensive disease prevention that also takes emotional state into account.

[0383] Example 2

[0384] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0385] In modern health management systems, providing more personalized health measures by taking into account not only the amount of exercise and dietary content but also the user's emotional data is a challenge. In particular, there is a need to properly evaluate the impact of stress and emotional state on health risks and to develop appropriate measures. However, conventional systems have difficulty in comprehensively handling these factors, making it difficult to provide users with appropriate measures.

[0386] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0387] In this invention, the server includes: a means for recording the amount of exercise, dietary content, and emotion data of an individual;

[0388] A means for receiving, preprocessing and storing the recorded exercise amount, dietary content and emotion data;

[0389] A means for extracting features from the stored data, assessing the risk of each disease, and generating countermeasures that take into account the emotional data;

[0390] a means for notifying the terminal of the generated countermeasures;

[0391] means for receiving feedback data from the terminal and updating the risk assessment and countermeasures;

[0392] This allows for more personalized health recommendations that take into account the user's emotional state.

[0393] "Amount of exercise" refers to the amount of physical activity an individual engages in over a certain period of time, and includes the number of steps taken, distance traveled, exercise intensity, etc.

[0394] "Dietary content" refers to the types, amounts, and nutrients of the foods and beverages consumed by an individual, including information on calories and nutritional components.

[0395] "Emotional data" refers to information about a user's emotional state obtained by analyzing their facial expressions, tone of voice, biometric signals, etc.

[0396] "Device" refers to an electronic device worn by an individual to record exercise volume, dietary content, emotional data, etc.

[0397] "Server" refers to a computer system that receives data from terminals, stores it, analyzes it, evaluates risks, and generates countermeasures.

[0398] "Feature extraction" refers to the process of extracting important information or patterns from stored data, such as the average number of steps taken or calorie information for meals.

[0399] "Risk assessment" refers to the process of calculating individual health risks based on collected data and planning appropriate countermeasures.

[0400] "Countermeasures" refers to plans or proposals that recommend individual health management methods or actions, generated based on the results of a risk assessment.

[0401] "Feedback data" refers to data that records the results and impressions of users when they perform an action.

[0402] "Preprocessing" refers to the process of converting received data into a suitable format for analysis and storage.

[0403] The present invention is a system that comprehensively monitors and analyzes an individual's exercise amount, dietary content, and emotional data, and provides the user with appropriate health measures. Specific embodiments for carrying out the present invention will be described in detail below.

[0404] System Overview

[0405] This system consists of a device worn by the user and a server that collects, analyzes, and stores data, and generates and notifies users of countermeasures. The device has built-in hardware for data collection, and the server has hardware and software for data processing.

[0406] Hardware and Software Configuration

[0407] Terminal

[0408] Camera: Takes pictures of the user's meal and collects image data.

[0409] Microphone: Collects the user's tone of voice to help with sentiment data.

[0410] Acceleration sensor: Measures the user's exercise volume (number of steps, distance traveled, and exercise intensity).

[0411] Location information acquisition device (GPS): Measures the user's travel distance.

[0412] Heart rate sensor: Measures the user's heart rate and collects biometric data.

[0413] Electrodermal activity sensor: Measures the user's electrodermal activity and estimates stress levels.

[0414] server

[0415] Database (e.g., Amazon Web Services RDS): Stores received exercise, dietary, and emotion data.

[0416] Machine learning algorithms (e.g., TENSORFLOW®): Analyze collected food image data and identify calories and types of nutrients.

[0417] Python libraries (e.g., Pandas, scikit-learn): Extract features from stored data and perform risk assessment.

[0418] Emotion Engine: Analyzes collected data and recognizes the user's emotional state.

[0419] System Operation

[0420] Data collection

[0421] The user wears the device in their daily life, and the device's built-in sensors collect real-time data on the user's exercise, diet, and emotions. Specifically, the camera captures images of meals, the microphone collects voice tone, the accelerometer and GPS measure exercise, and the heart rate sensor and electrodermal activity sensor collect biometric data.

[0422] Data transmission and storage

[0423] The device collects data at regular intervals, compiles it into packets, and sends them to a server via Wi-Fi or a mobile network. The server stores the received data in a database for subsequent analysis.

[0424] Data analysis and feature extraction

[0425] The server analyzes the stored data and extracts various features, using machine learning algorithms to identify calories and nutrients from food images, and Python libraries to analyze exercise volume and emotional data.

[0426] Risk assessment and countermeasure generation

[0427] The server performs a risk assessment based on the extracted feature data. It also includes emotional data in the assessment and calculates the overall health risk. It then generates optimal countermeasures based on the risk and notifies the user. These countermeasures include specific recommendations such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[0428] Gathering feedback and updating countermeasures

[0429] The user acts according to the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and countermeasures based on the new data. This makes it possible to provide more personalized countermeasures that take the user's emotional state into account.

[0430] Examples and prompts

[0431] Examples:

[0432] When a user goes for a morning jog, the device uses the accelerometer and GPS to record the number of steps and distance traveled. When the user eats a sandwich for lunch, the device's camera takes a photo of the meal. If the user smiles while eating, the emotion engine recognizes this and records emotional data.

[0433] Example prompt sentence:

[0434] How is data transmitted and stored?

[0435] "Give me an example of user emotion data."

[0436] "How do I generate personalized countermeasures?"

[0437] In this way, the present invention comprehensively supports the user's health management and, by taking into account the user's emotional state, can provide more accurate health measures.

[0438] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0439] Step 1:

[0440] Emotional data recording

[0441] The user wears the device. The device uses a camera, microphone, heart rate sensor, and electrodermal activity sensor to collect the user's facial expressions, tone of voice, and biosignal data. This data is analyzed in real time by the emotion engine to generate emotion data. Specifically, when the user smiles, the emotion engine generates emotion data for "happiness."

[0442] Input: User's facial expressions, audio, and biometric signals

[0443] Output: Emotion data (e.g., happiness, stress)

[0444] Step 2:

[0445] Recording exercise and food data

[0446] When users exercise in their daily lives, the device's accelerometer and GPS record the number of steps taken and the distance traveled. When users eat, the device's camera captures photos of their meals and collects dietary data.

[0447] Input: Steps, distance traveled, food images

[0448] Output: Exercise data (e.g., steps, distance traveled), food data (e.g., food images)

[0449] Step 3:

[0450] Data transmission and storage

[0451] At regular intervals, the device collects exercise data, dietary data, and emotional data and sends them in packets to a server via Wi-Fi or a mobile network. The server analyzes the received data and stores it in a database.

[0452] Input: Packetized exercise data, food data, and emotion data

[0453] Output: Various data stored in the database

[0454] Step 4:

[0455] Feature Extraction and Analysis

[0456] The server analyzes the stored data and extracts features. First, it uses Python's Pandas to calculate the average number of steps, distance traveled, and exercise intensity from the exercise data. Next, it uses machine learning algorithms (e.g., TensorFlow) to identify calories and types of nutrients from food images. Finally, it uses an emotion engine to analyze the emotion data.

[0457] Input: Stored exercise data, diet data, and emotional data

[0458] Output: Extracted feature data (e.g., average steps, calories, emotional state)

[0459] Step 5:

[0460] Risk Assessment

[0461] The server applies a risk assessment model based on the feature data to objectively calculate the risk of disease. For example, when assessing the risk of diabetes, it uses a machine learning model (using scikit-learn) to determine that lack of exercise and high stress are high-risk factors.

[0462] Input: Extracted feature data

[0463] Output: Risk assessment results (e.g., diabetes risk score)

[0464] Step 6:

[0465] Generation and notification of countermeasures

[0466] Based on the results of the risk assessment, the server generates optimal countermeasures for the user. For example, if the risk assessment diagnoses high stress, the server will generate specific countermeasures such as "jogging three times a week" and "recommended yoga for relaxation." The server notifies the device of the countermeasures and informs the user.

[0467] Input: Risk assessment results

[0468] Output: suggested action (e.g. jogging, yoga)

[0469] Step 7:

[0470] Collect feedback and update countermeasures

[0471] The user acts on the proposed measures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and proposed measures based on the newly received data. This allows for more personalized proposals that take the user's emotional state into account.

[0472] Input: User feedback data

[0473] Output: Updated risk assessment and proposed countermeasures

[0474] (Application example 2)

[0475] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0476] In today's busy lifestyles, personal health management is extremely important, but conventional systems only record the amount of exercise and dietary content and evaluate disease risk based on this, so they are unable to take measures that take into account the user's emotions and stress level. As a result, individualized and effective health management is difficult, making it difficult to maintain employee performance and reduce security risks, especially in high-stress environments.

[0477] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0478] In this invention, the server includes a means for generating emotion data using a camera, microphone, and sensor, a means for performing risk assessment including the emotion data and generating countermeasures, and a means for notifying the terminal of the generated countermeasures, thereby enabling personalized health management and reducing security risks in high-stress environments.

[0479] "Amount of exercise" is an index that indicates the amount of energy and intensity that an individual expends in daily life and physical activity.

[0480] "Dietary content" is data that indicates the types and amounts of food and beverages consumed by an individual, as well as information on the nutrients contained therein.

[0481] A "terminal" is a device that can be worn or carried by an individual and records exercise, dietary habits, and even emotional data.

[0482] "Server" refers to the central processing unit that receives the recorded data, performs pre-processing and storage, extracts features, and generates risk assessments and countermeasures.

[0483] "Preprocessing" refers to the initial processing of data, such as organizing, standardizing, and filtering, to make the data sent to the server easier to analyze.

[0484] "Storage" refers to keeping data for an extended period of time and making it accessible when needed.

[0485] "Feature extraction" is the process of extracting important information from stored data that is necessary for risk assessment and countermeasure generation.

[0486] "Risk assessment" refers to the objective evaluation of the risk of developing a specific health condition or disease based on extracted feature data.

[0487] "Countermeasures" are proposals that specifically outline preventative and improvement measures that users should take based on the results of risk assessment.

[0488] "Emotion data" is data that indicates the user's emotional state, and is generated based on facial expressions, tone of voice, heart rate, and electrodermal activity.

[0489] An "emotion engine" is software or hardware that analyzes data collected by cameras, microphones, and sensors, recognizes the user's emotions, and generates emotional data.

[0490] "Notification" refers to the act of sending the generated countermeasure plan to the user's terminal so that the user can check it.

[0491] "Feedback data" refers to data that records user actions based on proposed countermeasures, their results, and their impressions.

[0492] This invention is a system that comprehensively records an individual's exercise amount, dietary content, and emotional data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments of this system are described below.

[0493] System configuration

[0494] 1. Terminal

[0495] Hardware:

[0496] Acceleration sensors and position measuring devices

[0497] camera

[0498] microphone

[0499] Sensors (heart rate monitor, electrodermal activity sensor)

[0500] software:

[0501] Emotion recognition engines (e.g., generic image and voice analysis software)

[0502] Data collection and transmission program

[0503] 2. Server

[0504] Hardware:

[0505] Cloud-based central processing units (e.g., general-purpose server equipment from cloud providers)

[0506] software:

[0507] Machine learning algorithms (e.g., general-purpose machine learning frameworks)

[0508] Databases (e.g. cloud provider databases)

[0509] Data analysis platform (e.g., general-purpose data analysis tool)

[0510] Notification systems (e.g., general-purpose notification services)

[0511] Program processing

[0512] The device is worn by individuals and collects data on the amount of exercise, dietary habits, and emotions in daily life. Data is recorded in real time using a camera, microphone, and sensors, and emotion data is generated using an emotion engine. This allows data to be obtained that takes into account the user's subjective state.

[0513] The device periodically collects data and sends it to a server via Wi-Fi or a mobile network. The server receives the data and stores it in a database.

[0514] Once the data is saved, the server extracts features based on it. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from emotion data.

[0515] The server then assesses disease risk based on the extracted features, taking into account emotional data, for example, to assess the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[0516] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. The generated countermeasures are then sent to the device, where the user can confirm them.

[0517] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[0518] Specific examples

[0519] 1. Monitoring Employee A:

[0520] While Employee A is working, smart glasses monitor his facial expressions and record his tone of voice.

[0521] Heart rate and electrodermal activity are measured in real time to generate emotional data.

[0522] Emotion data is sent to the server in real time.

[0523] 2. Data transmission and storage:

[0524] The data received by the server is stored in a general-purpose database.

[0525] 3. Risk Assessment:

[0526] A machine learning algorithm on the server assesses security risks when employee A's stress level increases.

[0527] Generates countermeasures when high stress levels are detected.

[0528] 4. Countermeasure generation and notification:

[0529] The server generates suggested countermeasures such as "take a short break" or "take a deep breath."

[0530] Audio and visual notifications displayed on smart glasses.

[0531] 5. Feedback and solution update:

[0532] Employee A takes action based on the measures and records the results and impressions on the smart glasses.

[0533] The server analyzes the feedback and updates the countermeasures.

[0534] Prompt Sentence Examples

[0535] "How can I design a system that monitors employees' stress levels and notifies them in real time of appropriate relaxation measures when high stress levels are detected?"

[0536] This allows companies to effectively manage employee stress and reduce security risks.

[0537] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0538] Step 1:

[0539] The user puts on the device and begins their daily or work activities.

[0540] Hardware: Accelerometer, positioning device, camera, microphone, heart rate monitor, electrodermal activity sensor.

[0541] Specific operation: The device records the user's exercise (number of steps, distance traveled, etc.), dietary content (camera images), facial expressions (camera), tone of voice (microphone), heart rate, and skin electrodermal activity in real time.

[0542] Input: User's exercise, diet, facial expressions, voice, heart rate, and electrodermal activity.

[0543] Output: Raw recorded data.

[0544] Step 2:

[0545] The terminal collects the recorded data and sends it to the server.

[0546] Specific operation: Data is packetized at regular intervals and sent via Wi-Fi or mobile networks.

[0547] Input: Recorded exercise data, dietary data, facial expression data, voice data, and biometric data.

[0548] Output: The data packets sent.

[0549] Step 3:

[0550] The server stores the received data in a database and preprocesses it for further processing.

[0551] Hardware: A cloud-based central processing unit.

[0552] Software: Database (general-purpose database from the cloud provider), data preprocessing program.

[0553] Specific actions: Organize, standardize, filter, and store incoming data in a database.

[0554] Input: The data packet sent.

[0555] Output: Saved clean data.

[0556] Step 4:

[0557] The server extracts features from the data stored in the database.

[0558] Software: Machine learning algorithms (general-purpose machine learning frameworks).

[0559] Specific operations: Calculates the number of steps, distance traveled, and exercise intensity from exercise data, identifies calories and types of nutrients from dietary image data, and analyzes emotional state from facial expressions, voice, and biometric data.

[0560] Input: Saved clean data.

[0561] Output: Extracted feature data.

[0562] Step 5:

[0563] The server evaluates the risk of disease based on the extracted feature data.

[0564] Software: Machine learning algorithms, statistical models.

[0565] Specific operation: Conduct a comprehensive risk assessment that includes emotional data. For example, evaluate the impact of stress on diabetes risk.

[0566] Input: Extracted feature data.

[0567] Output: The results of the risk assessment.

[0568] Step 6:

[0569] The server generates countermeasures based on the results of the risk assessment and notifies the terminal.

[0570] Software: Notification System (general-purpose notification service).

[0571] Specific actions: Suggested measures such as "jog at least three times a week," "reduce carbohydrate intake by 50%," and "recommend yoga for relaxation" are generated and notified to the device.

[0572] Input: The results of the risk assessment.

[0573] Output: Generated countermeasures and notifications.

[0574] Step 7:

[0575] The user begins to take action based on the proposed measures and records the results and impressions on the device.

[0576] Specific actions: The user records their actions based on the measures as feedback data via a smart device.

[0577] Input: User actions based on proposed measures.

[0578] Output: Feedback data.

[0579] Step 8:

[0580] The device sends feedback data to the server, which updates the risk assessment and proposed countermeasures.

[0581] Specific operation: The feedback data is packetized and sent, and the server receives and stores it. Then, the risk assessment model and countermeasures are updated based on the feedback.

[0582] Input: The submitted feedback data.

[0583] Output: Updated risk assessment and new countermeasures.

[0584] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0585] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0586] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0587] [Second embodiment]

[0588] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0589] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0590] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0591] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0592] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0593] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0594] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0595] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0596] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0597] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0598] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0599] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0600] The present invention provides a system for recording an individual's exercise amount and dietary content, analyzing the data to assess the risk of illness, and providing appropriate countermeasures. As an embodiment of the present invention, a program is generated as follows, and its processing will be described in detail.

[0601] Recording exercise volume

[0602] A user wears the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity in real time. For example, when a user is jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[0603] Recording dietary information

[0604] When a user eats a meal, the user takes an image of the meal using the camera attached to the device. The image taken by the camera is temporarily saved in the device. This meal image data becomes important information for later analysis of the meal contents. For example, if a user eats a sandwich for lunch, the user takes an image of the sandwich with the camera and saves it on the device.

[0605] Data transmission and storage practices

[0606] The device periodically transmits the collected exercise data and meal content data to a server. The transmitted data is received by the server and stored in a database. The server then preprocesses the received data, removing noise and filling in missing values.

[0607] Performing feature extraction and analysis

[0608] The server extracts characteristics of exercise volume and dietary content from the stored data. Specifically, it calculates average steps, distance traveled, and exercise intensity from exercise data, and identifies calories and types of nutrients from dietary image data. This feature extraction is performed using machine learning algorithms and analyzed on the server.

[0609] Conducting risk assessments

[0610] The server then uses the extracted features to assess disease risk. For example, to assess the risk of diabetes, calculations are made based on a lack of physical activity and the frequency of high-calorie food intake. This risk assessment is performed objectively using statistical methods and machine learning algorithms.

[0611] Generate countermeasures and notify

[0612] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%." These countermeasures are then sent to the device, where the user can confirm them.

[0613] Collecting feedback and updating countermeasures

[0614] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends this feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user.

[0615] Specific examples

[0616] 1. Jogging and Lunch Log:

[0617] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[0618] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[0619] 2. Data transmission and storage:

[0620] The device sends the collected jogging data and meal images to a server.

[0621] The server receives the data and stores it in a database.

[0622] 3. Feature extraction and risk assessment:

[0623] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[0624] The server uses this data to assess diabetes risk.

[0625] 4. Countermeasure generation and notification:

[0626] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[0627] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0628] 5. Feedback and solution update:

[0629] The user implements the measures and records the results on the device.

[0630] The terminal transmits the feedback data to the server again.

[0631] The server updates the risk assessment and countermeasures based on the received data.

[0632] As described above, this invention supports personal health management through the process of data exchange and analysis between the terminal and the server. This system allows users to easily manage their own health in their daily lives and can be useful for disease prevention.

[0633] The processing flow will be explained below.

[0634] Step 1:

[0635] The user starts exercising, for example, jogging or walking.

[0636] Step 2:

[0637] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[0638] Step 3:

[0639] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[0640] Step 4:

[0641] The device periodically compiles exercise data and meal image data into packets.

[0642] Step 5:

[0643] The device uses Wi-Fi or a mobile communication network to send exercise data and dietary image data to a server.

[0644] Step 6:

[0645] The server stores the received exercise data and meal image data in a database.

[0646] Step 7:

[0647] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[0648] Step 8:

[0649] The server extracts features from the preprocessed data, calculating the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifying calories and types of nutrients from the dietary image data.

[0650] Step 9:

[0651] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise and frequency of high-calorie meals to determine diabetes risk.

[0652] Step 10:

[0653] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week" or "reducing carbohydrate intake by 50%."

[0654] Step 11:

[0655] The server sends the generated countermeasures to the terminal.

[0656] Step 12:

[0657] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[0658] Step 13:

[0659] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[0660] Step 14:

[0661] After implementing the proposed measures, the user records the results and impressions on the device.

[0662] Step 15:

[0663] The terminal transmits the feedback data to the server.

[0664] Step 16:

[0665] The server receives and stores the feedback data.

[0666] Step 17:

[0667] The server updates the risk assessment and countermeasures based on the new data, and if necessary, adjusts the countermeasures and sends them again to the device.

[0668] This allows the system to continuously support users in managing their health and provide actions that help prevent disease.

[0669] Example 1

[0670] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0671] In personal health management, there is a need for a system that can properly record exercise volume and dietary content, analyze this data to assess disease risk, and provide accurate countermeasures. However, conventional systems have had problems in efficiently collecting, storing, and analyzing this data, and lacked a means to effectively provide feedback to users to help them manage their health sustainably.

[0672] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0673] In this invention, the server includes a device that records an individual's amount of exercise and dietary details, means for receiving the recorded information on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored information and generating a disease risk and countermeasures, means for notifying the device of the generated countermeasures, and means for receiving feedback information from the device and updating the risk assessment and countermeasures. This makes it possible to effectively record an individual's amount of exercise and dietary details, accurately assess disease risk, and provide appropriate countermeasures.

[0674] "Amount of exercise" refers to data related to the physical activity of an individual, and specifically includes the number of steps taken, distance traveled, exercise intensity, and the like.

[0675] "Dietary details" refers to data including the type, amount, calorie, and nutrient information of the food consumed by an individual.

[0676] The "device" is a device for recording exercise and dietary content, and may include an acceleration sensor, a location information system, a camera, an image processing algorithm, and the like.

[0677] An "accelerometer" is a device that detects body movements and acquires acceleration information.

[0678] A "location information system" is a system that uses technology such as GPS to measure an individual's travel distance and location.

[0679] A "camera" is a device for taking pictures of the food.

[0680] An "image processing algorithm" is a computational method for extracting characteristics of food content from captured images and estimating calories and nutrients.

[0681] A "server" is a network computer that receives, preprocesses, and stores data sent from devices, extracts and analyzes features from the data, and generates risk assessments and countermeasures.

[0682] "Preprocessing" refers to processing such as removing noise from received data, filling in missing values, and correcting outliers.

[0683] "Feature extraction" is the process of extracting useful information (e.g., average number of steps, total distance traveled, calories, types of nutrients, etc.) from the stored data.

[0684] "Risk assessment" is the process of analyzing and assessing the risk of developing a disease based on the extracted features.

[0685] "Countermeasures" are specific proposals aimed at improving an individual's health that are generated based on the results of a risk assessment.

[0686] "Notification" refers to the act of transmitting the generated countermeasure plan to the device and informing the user of the information.

[0687] "Feedback" refers to data such as the results, impressions, and implementation status of the user's actions based on the proposed measures.

[0688] "Update" is the process of reassessing risk assessments and countermeasures based on newly acquired data and revising them as necessary.

[0689] The present invention is a system that records the amount of exercise and dietary content of an individual, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments are described below.

[0690] System configuration

[0691] The system consists of the following main components:

[0692] 1. Device worn by the user

[0693] Acceleration sensor: Used to measure movement volume, obtaining step count and acceleration data in real time.

[0694] GPS: Used to measure distance traveled and location.

[0695] Camera: Captures images of food to record eating habits.

[0696] Image processing algorithms are used to extract food content features from the captured images.

[0697] 2. Server

[0698] Data reception and preprocessing unit: Receives data sent from the terminal and performs noise removal and missing value completion.

[0699] Feature extraction module: Extracts features of exercise and dietary content from the stored data.

[0700] Risk assessment module: assesses disease risk based on extracted features.

[0701] Countermeasure proposal generation module: Generates appropriate countermeasure proposals based on the results of risk assessment.

[0702] Notification module: Sends the generated countermeasures to the device.

[0703] Feedback receiving and updating module: receives feedback data from users and updates risk assessment and countermeasure proposals.

[0704] Feature details

[0705] 1. Collecting exercise data

[0706] Users wear a device with a built-in accelerometer and GPS on a daily basis. This device collects exercise data (number of steps, distance traveled, etc.) in real time. For example, when you start jogging, the device immediately starts recording this data.

[0707] 2. Collection of dietary data

[0708] When a user eats, they take a photo of their meal with the device's camera. This image data is analyzed by the device to identify the meal contents (e.g., type of sandwich, amount, calories, etc.).

[0709] 3. Data Transmission

[0710] The device sends the collected data to the server at regular intervals, using Wi-Fi or mobile data communication.

[0711] 4. Data Preprocessing

[0712] The server preprocesses the received data, specifically removing noise, imputing missing values, and detecting and correcting outliers.

[0713] 5. Data feature extraction

[0714] The server's feature extraction module extracts useful information from the pre-processed data, such as average steps, total distance traveled, calories, types of nutrients, etc. Image recognition algorithms and machine learning models are used in this process.

[0715] 6. Risk Assessment

[0716] The server's risk assessment module assesses disease risk based on the extracted features. For example, diabetes risk is assessed based on factors such as lack of exercise and frequency of high-calorie food intake.

[0717] 7. Generation and notification of countermeasures

[0718] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include exercise plans and dietary restriction suggestions. For example, specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%" are made. These countermeasures are notified to the device.

[0719] 8. Collecting and Updating Feedback

[0720] The user records the results and impressions of actions taken based on the proposed countermeasures on the device. This feedback data is then sent back to the server and used to evaluate risks and update the proposed countermeasures.

[0721] Specific examples

[0722] 1. Jogging and Lunch Log

[0723] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[0724] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[0725] 2. Data transmission and storage

[0726] The device sends the collected jogging data and meal images to a server.

[0727] The server receives the data and stores it in a database.

[0728] 3. Feature Extraction and Risk Assessment

[0729] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[0730] The server uses this data to assess diabetes risk.

[0731] 4. Countermeasures generation and notification

[0732] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[0733] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0734] 5. Feedback and solution updates

[0735] The user implements the measures and records the results on the device.

[0736] The terminal transmits the feedback data to the server again.

[0737] The server updates the risk assessment and countermeasures based on the received data.

[0738] Prompt Sentence Examples

[0739] Below are some example prompts to input to a generative AI model:

[0740] Please explain the system flow for recording an individual's exercise volume and dietary habits, analyzing that data to assess disease risk, and generating appropriate countermeasures. Please provide a detailed explanation, including the names of the specific hardware and software used and how the data is processed. Please also provide examples.

[0741] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0742] Step 1:

[0743] The user wears the device.

[0744] Input: The device's accelerometer and GPS are initialized and ready to go.

[0745] Operation: The device detects the user's movements (steps and vibrations) using an accelerometer and acquires location information using GPS. Each measurement data is recorded internally along with the time.

[0746] Output: Real-time updated exercise data (number of steps, distance traveled, exercise intensity).

[0747] Step 2:

[0748] The user takes a photo of the meal.

[0749] Input: The user activates the device's camera and captures an image of the food.

[0750] How it works: Image data captured by the camera is temporarily stored on the device. Optionally, users have the option to manually enter meal details.

[0751] Output: Saved meal image data or manually entered meal content data.

[0752] Step 3:

[0753] The device sends the data to the server.

[0754] Input: Exercise data and dietary data stored in the device at regular intervals (for example, once a day).

[0755] How it works: Your device uses Wi-Fi or mobile data to upload data to a server, which may compress the data before sending it.

[0756] Output: Compressed exercise data and compressed dietary data sent to the server.

[0757] Step 4:

[0758] The server pre-processes the received data.

[0759] Input: Compressed exercise data and compressed dietary data sent from the terminal.

[0760] How it works: The server decompresses the data, removes noise, imputes missing values, and detects and corrects outliers.

[0761] Output: Clean, pre-processed exercise and diet data.

[0762] Step 5:

[0763] The server performs feature extraction.

[0764] Input: Preprocessed physical activity data and dietary data.

[0765] How it works: The server's machine learning algorithm extracts average steps, total distance traveled, and exercise intensity from exercise data, and calorie and nutrient information from dietary image data.

[0766] Output: Extracted feature data of exercise amount and feature data of dietary content.

[0767] Step 6:

[0768] The server assesses the disease risk.

[0769] Input: Extracted feature data of exercise amount and feature data of dietary content.

[0770] How it works: Risk assessment algorithms on the server calculate your risk of certain diseases, such as diabetes or heart disease, using statistical methods and machine learning models.

[0771] Output: Assessed disease risk data.

[0772] Step 7:

[0773] The server generates a countermeasure plan.

[0774] Input: Assessed disease risk data.

[0775] How it works: The server generates countermeasures based on the risk assessment results, such as suggesting exercise plans or dietary restrictions.

[0776] Output: Generated countermeasure proposal data.

[0777] Step 8:

[0778] The server will notify you of the proposed solution.

[0779] Input: Generated countermeasure proposal data.

[0780] Operation: The server sends the countermeasure data to the device, which then notifies the user via push notification or email.

[0781] Output: Notification of proposed measures displayed on the user's device.

[0782] Step 9:

[0783] The user records feedback.

[0784] Input: Results and impressions of users who have implemented measures.

[0785] How it works: The user enters their status after implementing the measures into the device, which also sends periodic reminders to record compliance.

[0786] Output: Recorded feedback data.

[0787] Step 10:

[0788] The terminal transmits the feedback data to the server.

[0789] Input: Feedback data entered by the user.

[0790] Operation: The terminal sends the collected feedback data to the server at regular intervals.

[0791] Output: Feedback data sent to the server.

[0792] Step 11:

[0793] The server updates the risk assessment and countermeasures.

[0794] Input: Feedback data, past exercise data, and dietary data.

[0795] Actions: The server re-performs risk assessment based on new data and updates its countermeasure recommendations, which may include retraining machine learning models.

[0796] Output: Updated risk assessment and proposed countermeasures.

[0797] As described above, the program of this system records the amount of exercise and diet of an individual in detail, evaluates the risk of disease based on this information, and provides appropriate countermeasures, allowing users to easily manage their health in their daily lives.

[0798] (Application example 1)

[0799] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] While managing personal health is important, there is a lack of systems in brick-and-mortar stores that can properly record customers' exercise and dietary habits and manage their health based on that information. Furthermore, there are insufficient means to provide risk assessments and health management recommendations based on this data. Therefore, there is a need for specific methods and systems to effectively support health management in brick-and-mortar stores.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0802] In this invention, the server includes a terminal that records an individual's amount of exercise and dietary details, means for receiving the recorded data on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored data and generating risks and countermeasures for each disease, means for notifying the terminal of the generated countermeasures, means for receiving feedback data from the terminal and updating the risk assessment and countermeasures, and means for providing menu suggestions and health management advice based on the customer's health condition in a physical store. This enables health management based on the customer's amount of exercise and dietary details in a physical store.

[0803] "Individual exercise amount" is data that indicates the level of physical activity that an individual engages in on a daily basis, such as the number of steps taken, distance traveled, and exercise intensity.

[0804] "Dietary details" refers to data including the type, amount, nutrients, calories, etc. of food and beverages consumed.

[0805] A "terminal" is a device that records an individual's amount of exercise and dietary details and transmits the collected data to a server using a communication means.

[0806] The "server" is a computer system that receives the recorded data and performs preprocessing, storage, feature extraction, risk assessment, and countermeasure generation.

[0807] "Preprocessing" refers to data processing performed before analysis, such as removing noise from data and filling in missing values.

[0808] "Storage" means recording the preprocessed data in a database in the server.

[0809] "Feature extraction" is the process of extracting necessary information from exercise and dietary data and putting it into an analyzable form.

[0810] "Risk assessment" is the process of calculating and assessing the risk of developing a particular disease based on the extracted features.

[0811] "Countermeasures" are specific action plans or proposals based on the results of risk assessment, aimed at improving health and preventing disease.

[0812] "Notification" refers to the act of sending the generated countermeasure plan to the terminal and notifying the user.

[0813] "Feedback data" refers to data that includes the results and impressions of users who have taken actions based on the proposed measures.

[0814] A "physical store" is a physical business establishment, such as a cafe or restaurant, where consumers visit in person to receive services.

[0815] "Menu suggestions" refers to recommending appropriate meals and drinks based on the customer's health condition.

[0816] "Health management advice" refers to specific action plans and lifestyle guidance to improve a customer's health.

[0817] This paper describes a system that records an individual's exercise and dietary habits, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specifically, this system is designed to provide menu suggestions and health management advice based on the customer's health status in a brick-and-mortar store.

[0818] Recording exercise volume

[0819] A user uses the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity. For example, when a user goes jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[0820] Recording dietary information

[0821] When a user eats at a physical restaurant, they take pictures of their meal using the camera attached to their device. The images taken by the camera are temporarily stored on the device, and this data becomes important information for later analysis of the meal contents. For example, if a user orders a salad at a cafe, they take a picture of the salad with the camera and save it on their device.

[0822] Data transmission and storage practices

[0823] The device periodically transmits the collected exercise data and meal content data to the server. The server receives this data and stores it in a database. Preprocessing of the data involves noise removal and missing value completion.

[0824] Performing feature extraction and analysis

[0825] The server extracts characteristics of exercise volume and dietary content from the stored data. Average steps, distance traveled, and exercise intensity are calculated from the exercise data, and calories and types of nutrients are identified from dietary image data. Machine learning algorithms are used to extract these characteristics.

[0826] Conducting risk assessments

[0827] The server then assesses disease risk based on the extracted features. For example, to assess diabetes risk, it considers factors such as lack of exercise and frequency of high-calorie food intake. This risk assessment is performed using statistical methods and machine learning algorithms.

[0828] Generate countermeasures and notify

[0829] Based on the results of the risk assessment, the server generates appropriate measures for the user. The generated measures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" and "reducing carbohydrate intake by 50%." These measures are then notified to the device.

[0830] Gathering feedback and updating countermeasures

[0831] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends the feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user again.

[0832] Specific examples

[0833] Jogging and cafe meal record:

[0834] A user goes jogging in the morning, and the device records the number of steps and distance traveled using the accelerometer and GPS.

[0835] A user orders a salad at a cafe and takes a picture of the salad with the camera on the device.

[0836] Data transmission and storage:

[0837] The device sends the collected jogging data and meal images to a server.

[0838] The server receives the data and stores it in a database.

[0839] Feature extraction and risk assessment:

[0840] The server extracts information on the amount of exercise (number of steps, distance traveled) and calories in the salad.

[0841] The server uses this data to assess diabetes risk.

[0842] Countermeasure generation and notification:

[0843] The server generates recommendations such as "jogging at least three times a week" and "low-carb menus."

[0844] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0845] Feedback and solution update:

[0846] The user implements the measures and records the results on the device.

[0847] The terminal transmits the feedback data to the server again.

[0848] The server updates the risk assessment and countermeasures based on the received data.

[0849] Prompt Sentence Examples

[0850] "The customer entered the number of steps and distance they jogged, as well as a photo of their meal (salad) at the cafe, into the app. Based on this data, the app assesses their health risks and generates optimal health management advice."

[0851] The present invention makes it possible to easily provide health management based on the amount of exercise and dietary content of users even in brick-and-mortar stores, thereby integrating the provision of health services and customer health management in brick-and-mortar stores.

[0852] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0853] Step 1:

[0854] When a user exercises, the device uses an acceleration sensor and GPS to collect exercise data. The input is the user's exercise status (number of steps, distance traveled, exercise intensity), and the device measures data in real time based on this. The output is the exercise data obtained as a result of the measurement.

[0855] Step 2:

[0856] When a user eats at a physical restaurant, they take a photo of their meal using the camera on their device. The input is the photo of the meal, and the device acquires the data by temporarily saving this image. The output is the saved meal image data.

[0857] Step 3:

[0858] The device periodically transmits the collected exercise data and dietary image data to the server. The input is the collected exercise data and dietary data, and the device uploads this data to the server. The output is the data transmitted to the server.

[0859] Step 4:

[0860] The server stores the received exercise data and dietary image data. The input is the exercise data and dietary data sent from the device, which the server records in a database and performs data preprocessing such as noise removal and missing value completion. The output is the preprocessed data.

[0861] Step 5:

[0862] The server extracts features from the preprocessed data. The input is the preprocessed exercise data and dietary data, and the server extracts feature information such as the number of steps, distance traveled, exercise intensity, and dietary calories and nutrients from these data. The output is the extracted feature data.

[0863] Step 6:

[0864] The server evaluates disease risk based on the extracted feature data. The input is the feature data, and the server performs risk assessment using machine learning algorithms and statistical methods. For example, this includes assessing diabetes risk. The output is the risk assessment result.

[0865] Step 7:

[0866] The server generates countermeasures based on the risk assessment results. The input is the risk assessment results, and the server creates countermeasures such as specific exercise plans and dietary restrictions. The output is the generated countermeasures.

[0867] Step 8:

[0868] The generated countermeasures are notified to the terminal by the server. The input is the generated countermeasures, which the server sends to the terminal. The output is the countermeasures notified to the terminal.

[0869] Step 9:

[0870] The user implements the proposed measures and records the results and their impressions on the device. The input is the user's action results and feedback, which the device records as data. The output is the recorded feedback data.

[0871] Step 10:

[0872] The terminal sends the collected feedback data back to the server. The input is the recorded feedback data that the terminal uploads to the server. The output is the feedback data sent to the server.

[0873] Step 11:

[0874] The server updates the risk assessment and countermeasures based on the feedback data. The input is feedback data from the user, and the server reassessssments and updates the countermeasures based on this. The output is the updated risk assessment and countermeasures.

[0875] The above processing steps enable health management based on the user's exercise volume and dietary content even in brick-and-mortar stores, making it possible to integrate health service provision in brick-and-mortar stores with customer health management.

[0876] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0877] The present invention combines a system that records an individual's amount of exercise and dietary details, assesses disease risk based on this data, and provides appropriate countermeasures with an emotion engine that recognizes the user's emotions. This makes it possible to provide more personalized countermeasures that take into account the user's subjective state. As an embodiment of the present invention, a program is generated as follows, and its specific processing is described below.

[0878] Emotional data recording

[0879] A user wears the device of the present invention in their daily life. This device is equipped with a camera, microphone, and sensors, which are used by an emotion engine to analyze the user's facial expressions, tone of voice, and even biosignal data (heart rate, electrodermal activity, etc.) to recognize the user's emotions and generate emotion data. For example, when the user smiles or feels stressed, this emotion data is recorded in real time.

[0880] Data transmission and storage practices

[0881] The device periodically compiles exercise data, food image data, and emotion data into packets, which are then sent to a server via Wi-Fi or a mobile network. The server then stores the received data in a database.

[0882] Performing feature extraction and analysis

[0883] The server extracts various features from the stored data. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from the dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from the emotion data.

[0884] Conducting risk assessments

[0885] The server then assesses disease risk based on the extracted features, taking into account emotional data to assess, for example, the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[0886] Generate countermeasures and notify

[0887] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. For example, if stress levels are high, the server can generate countermeasures such as "jogging three times a week" and "yoga for relaxation." These countermeasures are then sent to the device, where the user can confirm them.

[0888] Collecting feedback and updating countermeasures

[0889] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[0890] Specific examples

[0891] 1. Jogging and Lunch Log:

[0892] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[0893] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[0894] If a user smiles or feels stressed while eating, the emotion engine will recognize this and record the emotion data.

[0895] 2. Data transmission and storage:

[0896] The device sends the collected jogging data, meal images, and emotional data to a server.

[0897] The server receives this data and stores it in a database.

[0898] 3. Feature extraction and risk assessment:

[0899] The server extracts the amount of exercise (number of steps, distance traveled), calorie information of the sandwich, and emotional data (stress level, etc.).

[0900] The server uses this data to assess diabetes risk.

[0901] 4. Countermeasure generation and notification:

[0902] Based on the risk assessment, the server generates suggested countermeasures such as "jogging at least three times a week," "reducing carbohydrate intake by 50%," and "recommending yoga for relaxation."

[0903] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[0904] 5. Feedback and solution update:

[0905] The user implements the measures and records the results and impressions on the device.

[0906] The terminal transmits the feedback data to the server again.

[0907] The server updates the risk assessment and countermeasures based on new data, and also takes emotional data into account to adjust the countermeasures.

[0908] This allows the system to more personalize the user's health management and take emotional state into account to help with comprehensive disease prevention and health maintenance.

[0909] The processing flow will be explained below.

[0910] Step 1:

[0911] The user begins exercising, for example, jogging or walking.

[0912] Step 2:

[0913] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[0914] Step 3:

[0915] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[0916] Step 4:

[0917] The device uses a built-in emotion engine to recognize the user's emotions. Emotional data is generated based on facial expression recognition, voice analysis, and biometric data (heart rate, electrodermal activity, etc.). For example, if the user smiles or feels stressed, that emotion is recorded.

[0918] Step 5:

[0919] The terminal periodically assembles the exercise data, meal image data, and emotion data into packets.

[0920] Step 6:

[0921] The device uses Wi-Fi or a mobile communication network to send exercise data, food image data, and emotional data to a server.

[0922] Step 7:

[0923] The server stores the received exercise data, meal image data, and emotion data in a database.

[0924] Step 8:

[0925] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[0926] Step 9:

[0927] The server extracts features from the preprocessed data. From the exercise data, it calculates the average number of steps, distance traveled, and exercise intensity. From the food image data, it identifies calories and types of nutrients. From the emotion data, it extracts emotional states such as joy, sadness, and stress.

[0928] Step 10:

[0929] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise, frequency of high-calorie meals, and stress levels to determine diabetes risk.

[0930] Step 11:

[0931] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week is recommended," "reducing carbohydrate intake by 50%," or "yoga is recommended for relaxation."

[0932] Step 12:

[0933] The server sends the generated countermeasures to the terminal.

[0934] Step 13:

[0935] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[0936] Step 14:

[0937] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[0938] Step 15:

[0939] After implementing the proposed measures, the user records the results, impressions, and emotional state on the device.

[0940] Step 16:

[0941] The device sends feedback data (amount of exercise, dietary content, emotional state) to a server.

[0942] Step 17:

[0943] The server receives and stores the feedback data.

[0944] Step 18:

[0945] The server updates the risk assessment and countermeasures based on the new data. If necessary, it adjusts the countermeasures and sends them back to the device. It also takes emotional data into account to make the countermeasures more personalized.

[0946] This allows the system to continuously support the user's health management and achieve comprehensive disease prevention that also takes emotional state into account.

[0947] Example 2

[0948] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0949] In modern health management systems, providing more personalized health measures by taking into account not only the amount of exercise and dietary content but also the user's emotional data is a challenge. In particular, there is a need to properly evaluate the impact of stress and emotional state on health risks and to develop appropriate measures. However, conventional systems have difficulty in comprehensively handling these factors, making it difficult to provide users with appropriate measures.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0951] In this invention, the server includes: a means for recording the amount of exercise, dietary content, and emotion data of an individual;

[0952] A means for receiving, preprocessing and storing the recorded exercise amount, dietary content and emotion data;

[0953] A means for extracting features from the stored data, assessing the risk of each disease, and generating countermeasures that take into account the emotional data;

[0954] a means for notifying the terminal of the generated countermeasures;

[0955] means for receiving feedback data from the terminal and updating the risk assessment and countermeasures;

[0956] This allows for more personalized health recommendations that take into account the user's emotional state.

[0957] "Amount of exercise" refers to the amount of physical activity an individual engages in over a certain period of time, and includes the number of steps taken, distance traveled, exercise intensity, etc.

[0958] "Dietary content" refers to the types, amounts, and nutrients of the foods and beverages consumed by an individual, including information on calories and nutritional components.

[0959] "Emotional data" refers to information about a user's emotional state obtained by analyzing their facial expressions, tone of voice, biometric signals, etc.

[0960] "Device" refers to an electronic device worn by an individual to record exercise volume, dietary content, emotional data, etc.

[0961] "Server" refers to a computer system that receives data from terminals, stores it, analyzes it, evaluates risks, and generates countermeasures.

[0962] "Feature extraction" refers to the process of extracting important information or patterns from stored data, such as the average number of steps taken or calorie information for meals.

[0963] "Risk assessment" refers to the process of calculating individual health risks based on collected data and planning appropriate countermeasures.

[0964] "Countermeasures" refers to plans or proposals that recommend individual health management methods or actions, generated based on the results of a risk assessment.

[0965] "Feedback data" refers to data that records the results and impressions of users when they perform an action.

[0966] "Preprocessing" refers to the process of converting received data into a suitable format for analysis and storage.

[0967] The present invention is a system that comprehensively monitors and analyzes an individual's exercise amount, dietary content, and emotional data, and provides the user with appropriate health measures. Specific embodiments for carrying out the present invention will be described in detail below.

[0968] System Overview

[0969] This system consists of a device worn by the user and a server that collects, analyzes, and stores data, and generates and notifies users of countermeasures. The device has built-in hardware for data collection, and the server has hardware and software for data processing.

[0970] Hardware and Software Configuration

[0971] Terminal

[0972] Camera: Takes pictures of the user's meal and collects image data.

[0973] Microphone: Collects the user's tone of voice to help with sentiment data.

[0974] Acceleration sensor: Measures the user's exercise volume (number of steps, distance traveled, and exercise intensity).

[0975] Location information acquisition device (GPS): Measures the user's travel distance.

[0976] Heart rate sensor: Measures the user's heart rate and collects biometric data.

[0977] Electrodermal activity sensor: Measures the user's electrodermal activity and estimates stress levels.

[0978] server

[0979] Database (e.g., Amazon Web Services RDS): Stores received exercise, dietary, and emotion data.

[0980] Machine learning algorithms (e.g., TensorFlow): Analyze collected food image data and identify calories and types of nutrients.

[0981] Python libraries (e.g., Pandas, scikit-learn): Extract features from stored data and perform risk assessment.

[0982] Emotion Engine: Analyzes collected data and recognizes the user's emotional state.

[0983] System Operation

[0984] Data collection

[0985] The user wears the device in their daily life, and the device's built-in sensors collect real-time data on the user's exercise, diet, and emotions. Specifically, the camera captures images of meals, the microphone collects voice tone, the accelerometer and GPS measure exercise, and the heart rate sensor and electrodermal activity sensor collect biometric data.

[0986] Data transmission and storage

[0987] The device collects data at regular intervals, compiles it into packets, and sends them to a server via Wi-Fi or a mobile network. The server stores the received data in a database for subsequent analysis.

[0988] Data analysis and feature extraction

[0989] The server analyzes the stored data and extracts various features, using machine learning algorithms to identify calories and nutrients from food images, and Python libraries to analyze exercise volume and emotional data.

[0990] Risk assessment and countermeasure generation

[0991] The server performs a risk assessment based on the extracted feature data. It also includes emotional data in the assessment and calculates the overall health risk. It then generates optimal countermeasures based on the risk and notifies the user. These countermeasures include specific recommendations such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[0992] Gathering feedback and updating countermeasures

[0993] The user acts according to the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and countermeasures based on the new data. This makes it possible to provide more personalized countermeasures that take the user's emotional state into account.

[0994] Examples and prompts

[0995] Examples:

[0996] When a user goes for a morning jog, the device uses the accelerometer and GPS to record the number of steps and distance traveled. When the user eats a sandwich for lunch, the device's camera takes a photo of the meal. If the user smiles while eating, the emotion engine recognizes this and records emotional data.

[0997] Example prompt sentence:

[0998] How is data transmitted and stored?

[0999] "Give me an example of user emotion data."

[1000] "How do I generate personalized countermeasures?"

[1001] In this way, the present invention comprehensively supports the user's health management and, by taking into account the user's emotional state, can provide more accurate health measures.

[1002] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1003] Step 1:

[1004] Emotional data recording

[1005] The user wears the device. The device uses a camera, microphone, heart rate sensor, and electrodermal activity sensor to collect the user's facial expressions, tone of voice, and biosignal data. This data is analyzed in real time by the emotion engine to generate emotion data. Specifically, when the user smiles, the emotion engine generates emotion data for "happiness."

[1006] Input: User's facial expressions, audio, and biometric signals

[1007] Output: Emotion data (e.g., happiness, stress)

[1008] Step 2:

[1009] Recording exercise and food data

[1010] When users exercise in their daily lives, the device's accelerometer and GPS record the number of steps taken and the distance traveled. When users eat, the device's camera captures photos of their meals and collects dietary data.

[1011] Input: Steps, distance traveled, food images

[1012] Output: Exercise data (e.g., steps, distance traveled), food data (e.g., food images)

[1013] Step 3:

[1014] Data transmission and storage

[1015] At regular intervals, the device collects exercise data, dietary data, and emotional data and sends them in packets to a server via Wi-Fi or a mobile network. The server analyzes the received data and stores it in a database.

[1016] Input: Packetized exercise data, food data, and emotion data

[1017] Output: Various data stored in the database

[1018] Step 4:

[1019] Feature Extraction and Analysis

[1020] The server analyzes the stored data and extracts features. First, it uses Python's Pandas to calculate the average number of steps, distance traveled, and exercise intensity from the exercise data. Next, it uses machine learning algorithms (e.g., TensorFlow) to identify calories and types of nutrients from food images. Finally, it uses an emotion engine to analyze the emotion data.

[1021] Input: Stored exercise data, diet data, and emotional data

[1022] Output: Extracted feature data (e.g., average steps, calories, emotional state)

[1023] Step 5:

[1024] Risk Assessment

[1025] The server applies a risk assessment model based on the feature data to objectively calculate the risk of disease. For example, when assessing the risk of diabetes, it uses a machine learning model (using scikit-learn) to determine that lack of exercise and high stress are high-risk factors.

[1026] Input: Extracted feature data

[1027] Output: Risk assessment results (e.g., diabetes risk score)

[1028] Step 6:

[1029] Generation and notification of countermeasures

[1030] Based on the results of the risk assessment, the server generates optimal countermeasures for the user. For example, if the risk assessment diagnoses high stress, the server will generate specific countermeasures such as "jogging three times a week" and "recommended yoga for relaxation." The server notifies the device of the countermeasures and informs the user.

[1031] Input: Risk assessment results

[1032] Output: suggested action (e.g. jogging, yoga)

[1033] Step 7:

[1034] Collect feedback and update countermeasures

[1035] The user acts on the proposed measures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and proposed measures based on the newly received data. This allows for more personalized proposals that take the user's emotional state into account.

[1036] Input: User feedback data

[1037] Output: Updated risk assessment and proposed countermeasures

[1038] (Application example 2)

[1039] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1040] In today's busy lifestyles, personal health management is extremely important, but conventional systems only record the amount of exercise and dietary content and evaluate disease risk based on this, so they are unable to take measures that take into account the user's emotions and stress level. As a result, individualized and effective health management is difficult, making it difficult to maintain employee performance and reduce security risks, especially in high-stress environments.

[1041] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1042] In this invention, the server includes a means for generating emotion data using a camera, microphone, and sensor, a means for performing risk assessment including the emotion data and generating countermeasures, and a means for notifying the terminal of the generated countermeasures, thereby enabling personalized health management and reducing security risks in high-stress environments.

[1043] "Amount of exercise" is an index that indicates the amount of energy and intensity that an individual expends in daily life and physical activity.

[1044] "Dietary content" is data that indicates the types and amounts of food and beverages consumed by an individual, as well as information on the nutrients contained therein.

[1045] A "terminal" is a device that can be worn or carried by an individual and records exercise, dietary habits, and even emotional data.

[1046] "Server" refers to the central processing unit that receives the recorded data, performs pre-processing and storage, extracts features, and generates risk assessments and countermeasures.

[1047] "Preprocessing" refers to the initial processing of data, such as organizing, standardizing, and filtering, to make the data sent to the server easier to analyze.

[1048] "Storage" refers to keeping data for an extended period of time and making it accessible when needed.

[1049] "Feature extraction" is the process of extracting important information from stored data that is necessary for risk assessment and countermeasure generation.

[1050] "Risk assessment" refers to the objective evaluation of the risk of developing a specific health condition or disease based on extracted feature data.

[1051] "Countermeasures" are proposals that specifically outline preventative and improvement measures that users should take based on the results of risk assessment.

[1052] "Emotion data" is data that indicates the user's emotional state, and is generated based on facial expressions, tone of voice, heart rate, and electrodermal activity.

[1053] An "emotion engine" is software or hardware that analyzes data collected by cameras, microphones, and sensors, recognizes the user's emotions, and generates emotional data.

[1054] "Notification" refers to the act of sending the generated countermeasure plan to the user's terminal so that the user can check it.

[1055] "Feedback data" refers to data that records user actions based on proposed countermeasures, their results, and their impressions.

[1056] This invention is a system that comprehensively records an individual's exercise amount, dietary content, and emotional data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments of this system are described below.

[1057] System configuration

[1058] 1. Terminal

[1059] Hardware:

[1060] Acceleration sensors and position measuring devices

[1061] camera

[1062] microphone

[1063] Sensors (heart rate monitor, electrodermal activity sensor)

[1064] software:

[1065] Emotion recognition engines (e.g., generic image and voice analysis software)

[1066] Data collection and transmission program

[1067] 2. Server

[1068] Hardware:

[1069] Cloud-based central processing units (e.g., general-purpose server equipment from cloud providers)

[1070] software:

[1071] Machine learning algorithms (e.g., general-purpose machine learning frameworks)

[1072] Databases (e.g. cloud provider databases)

[1073] Data analysis platform (e.g., general-purpose data analysis tool)

[1074] Notification systems (e.g., general-purpose notification services)

[1075] Program processing

[1076] The device is worn by individuals and collects data on the amount of exercise, dietary habits, and emotions in daily life. Data is recorded in real time using a camera, microphone, and sensors, and emotion data is generated using an emotion engine. This allows data to be obtained that takes into account the user's subjective state.

[1077] The device periodically collects data and sends it to a server via Wi-Fi or a mobile network. The server receives the data and stores it in a database.

[1078] Once the data is saved, the server extracts features based on it. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from emotion data.

[1079] The server then assesses disease risk based on the extracted features, taking into account emotional data, for example, to assess the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[1080] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. The generated countermeasures are then sent to the device, where the user can confirm them.

[1081] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[1082] Specific examples

[1083] 1. Monitoring Employee A:

[1084] While Employee A is working, smart glasses monitor his facial expressions and record his tone of voice.

[1085] Heart rate and electrodermal activity are measured in real time to generate emotional data.

[1086] Emotion data is sent to the server in real time.

[1087] 2. Data transmission and storage:

[1088] The data received by the server is stored in a general-purpose database.

[1089] 3. Risk Assessment:

[1090] A machine learning algorithm on the server assesses security risks when employee A's stress level increases.

[1091] Generates countermeasures when high stress levels are detected.

[1092] 4. Countermeasure generation and notification:

[1093] The server generates suggested countermeasures such as "take a short break" or "take a deep breath."

[1094] Audio and visual notifications displayed on smart glasses.

[1095] 5. Feedback and solution update:

[1096] Employee A takes action based on the measures and records the results and impressions on the smart glasses.

[1097] The server analyzes the feedback and updates the countermeasures.

[1098] Prompt Sentence Examples

[1099] "How can I design a system that monitors employees' stress levels and notifies them in real time of appropriate relaxation measures when high stress levels are detected?"

[1100] This allows companies to effectively manage employee stress and reduce security risks.

[1101] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1102] Step 1:

[1103] The user puts on the device and begins their daily or work activities.

[1104] Hardware: Accelerometer, positioning device, camera, microphone, heart rate monitor, electrodermal activity sensor.

[1105] Specific operation: The device records the user's exercise (number of steps, distance traveled, etc.), dietary content (camera images), facial expressions (camera), tone of voice (microphone), heart rate, and skin electrodermal activity in real time.

[1106] Input: User's exercise, diet, facial expressions, voice, heart rate, and electrodermal activity.

[1107] Output: Raw recorded data.

[1108] Step 2:

[1109] The terminal collects the recorded data and sends it to the server.

[1110] Specific operation: Data is packetized at regular intervals and sent via Wi-Fi or mobile networks.

[1111] Input: Recorded exercise data, dietary data, facial expression data, voice data, and biometric data.

[1112] Output: The data packets sent.

[1113] Step 3:

[1114] The server stores the received data in a database and preprocesses it for further processing.

[1115] Hardware: A cloud-based central processing unit.

[1116] Software: Database (general-purpose database from the cloud provider), data preprocessing program.

[1117] Specific actions: Organize, standardize, filter, and store incoming data in a database.

[1118] Input: The data packet sent.

[1119] Output: Saved clean data.

[1120] Step 4:

[1121] The server extracts features from the data stored in the database.

[1122] Software: Machine learning algorithms (general-purpose machine learning frameworks).

[1123] Specific operations: Calculates the number of steps, distance traveled, and exercise intensity from exercise data, identifies calories and types of nutrients from dietary image data, and analyzes emotional state from facial expressions, voice, and biometric data.

[1124] Input: Saved clean data.

[1125] Output: Extracted feature data.

[1126] Step 5:

[1127] The server evaluates the risk of disease based on the extracted feature data.

[1128] Software: Machine learning algorithms, statistical models.

[1129] Specific operation: Conduct a comprehensive risk assessment that includes emotional data. For example, evaluate the impact of stress on diabetes risk.

[1130] Input: Extracted feature data.

[1131] Output: The results of the risk assessment.

[1132] Step 6:

[1133] The server generates countermeasures based on the results of the risk assessment and notifies the terminal.

[1134] Software: Notification System (general-purpose notification service).

[1135] Specific actions: Suggested measures such as "jog at least three times a week," "reduce carbohydrate intake by 50%," and "recommend yoga for relaxation" are generated and notified to the device.

[1136] Input: The results of the risk assessment.

[1137] Output: Generated countermeasures and notifications.

[1138] Step 7:

[1139] The user begins to take action based on the proposed measures and records the results and impressions on the device.

[1140] Specific actions: The user records their actions based on the measures as feedback data via a smart device.

[1141] Input: User actions based on proposed measures.

[1142] Output: Feedback data.

[1143] Step 8:

[1144] The device sends feedback data to the server, which updates the risk assessment and proposed countermeasures.

[1145] Specific operation: The feedback data is packetized and sent, and the server receives and stores it. Then, the risk assessment model and countermeasures are updated based on the feedback.

[1146] Input: The submitted feedback data.

[1147] Output: Updated risk assessment and new countermeasures.

[1148] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1149] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1150] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1151] [Third embodiment]

[1152] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1153] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1154] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1155] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1156] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1157] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1158] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1159] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1160] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1161] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1162] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1163] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1164] The present invention provides a system for recording an individual's exercise amount and dietary content, analyzing the data to assess the risk of illness, and providing appropriate countermeasures. As an embodiment of the present invention, a program is generated as follows, and its processing will be described in detail.

[1165] Recording exercise volume

[1166] A user wears the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity in real time. For example, when a user is jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[1167] Recording dietary information

[1168] When a user eats a meal, the user takes an image of the meal using the camera attached to the device. The image taken by the camera is temporarily saved in the device. This meal image data becomes important information for later analysis of the meal contents. For example, if a user eats a sandwich for lunch, the user takes an image of the sandwich with the camera and saves it on the device.

[1169] Data transmission and storage practices

[1170] The device periodically transmits the collected exercise data and meal content data to a server. The transmitted data is received by the server and stored in a database. The server then preprocesses the received data, removing noise and filling in missing values.

[1171] Performing feature extraction and analysis

[1172] The server extracts characteristics of exercise volume and dietary content from the stored data. Specifically, it calculates average steps, distance traveled, and exercise intensity from exercise data, and identifies calories and types of nutrients from dietary image data. This feature extraction is performed using machine learning algorithms and analyzed on the server.

[1173] Conducting risk assessments

[1174] The server then uses the extracted features to assess disease risk. For example, to assess the risk of diabetes, calculations are made based on a lack of physical activity and the frequency of high-calorie food intake. This risk assessment is performed objectively using statistical methods and machine learning algorithms.

[1175] Generate countermeasures and notify

[1176] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%." These countermeasures are then sent to the device, where the user can confirm them.

[1177] Collecting feedback and updating countermeasures

[1178] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends this feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user.

[1179] Specific examples

[1180] 1. Jogging and Lunch Log:

[1181] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[1182] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[1183] 2. Data transmission and storage:

[1184] The device sends the collected jogging data and meal images to a server.

[1185] The server receives the data and stores it in a database.

[1186] 3. Feature extraction and risk assessment:

[1187] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[1188] The server uses this data to assess diabetes risk.

[1189] 4. Countermeasure generation and notification:

[1190] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[1191] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1192] 5. Feedback and solution update:

[1193] The user implements the measures and records the results on the device.

[1194] The terminal transmits the feedback data to the server again.

[1195] The server updates the risk assessment and countermeasures based on the received data.

[1196] As described above, this invention supports personal health management through the process of data exchange and analysis between the terminal and the server. This system allows users to easily manage their own health in their daily lives and can be useful for disease prevention.

[1197] The processing flow will be explained below.

[1198] Step 1:

[1199] The user starts exercising, for example, jogging or walking.

[1200] Step 2:

[1201] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[1202] Step 3:

[1203] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[1204] Step 4:

[1205] The device periodically compiles exercise data and meal image data into packets.

[1206] Step 5:

[1207] The device uses Wi-Fi or a mobile communication network to send exercise data and dietary image data to a server.

[1208] Step 6:

[1209] The server stores the received exercise data and meal image data in a database.

[1210] Step 7:

[1211] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[1212] Step 8:

[1213] The server extracts features from the preprocessed data, calculating the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifying calories and types of nutrients from the dietary image data.

[1214] Step 9:

[1215] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise and frequency of high-calorie meals to determine diabetes risk.

[1216] Step 10:

[1217] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week" or "reducing carbohydrate intake by 50%."

[1218] Step 11:

[1219] The server sends the generated countermeasures to the terminal.

[1220] Step 12:

[1221] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[1222] Step 13:

[1223] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[1224] Step 14:

[1225] After implementing the proposed measures, the user records the results and impressions on the device.

[1226] Step 15:

[1227] The terminal transmits the feedback data to the server.

[1228] Step 16:

[1229] The server receives and stores the feedback data.

[1230] Step 17:

[1231] The server updates the risk assessment and countermeasures based on the new data, and if necessary, adjusts the countermeasures and sends them again to the device.

[1232] This allows the system to continuously support users in managing their health and provide actions that help prevent disease.

[1233] Example 1

[1234] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1235] In personal health management, there is a need for a system that can properly record exercise volume and dietary content, analyze this data to assess disease risk, and provide accurate countermeasures. However, conventional systems have had problems in efficiently collecting, storing, and analyzing this data, and lacked a means to effectively provide feedback to users to help them manage their health sustainably.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1237] In this invention, the server includes a device that records an individual's amount of exercise and dietary details, means for receiving the recorded information on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored information and generating a disease risk and countermeasures, means for notifying the device of the generated countermeasures, and means for receiving feedback information from the device and updating the risk assessment and countermeasures. This makes it possible to effectively record an individual's amount of exercise and dietary details, accurately assess disease risk, and provide appropriate countermeasures.

[1238] "Amount of exercise" refers to data related to the physical activity of an individual, and specifically includes the number of steps taken, distance traveled, exercise intensity, and the like.

[1239] "Dietary details" refers to data including the type, amount, calorie, and nutrient information of the food consumed by an individual.

[1240] The "device" is a device for recording exercise and dietary content, and may include an acceleration sensor, a location information system, a camera, an image processing algorithm, and the like.

[1241] An "accelerometer" is a device that detects body movements and acquires acceleration information.

[1242] A "location information system" is a system that uses technology such as GPS to measure an individual's travel distance and location.

[1243] A "camera" is a device for taking pictures of the food.

[1244] An "image processing algorithm" is a computational method for extracting characteristics of food content from captured images and estimating calories and nutrients.

[1245] A "server" is a network computer that receives, preprocesses, and stores data sent from devices, extracts and analyzes features from the data, and generates risk assessments and countermeasures.

[1246] "Preprocessing" refers to processing such as removing noise from received data, filling in missing values, and correcting outliers.

[1247] "Feature extraction" is the process of extracting useful information (e.g., average number of steps, total distance traveled, calories, types of nutrients, etc.) from the stored data.

[1248] "Risk assessment" is the process of analyzing and assessing the risk of developing a disease based on the extracted features.

[1249] "Countermeasures" are specific proposals aimed at improving an individual's health that are generated based on the results of a risk assessment.

[1250] "Notification" refers to the act of transmitting the generated countermeasure plan to the device and informing the user of the information.

[1251] "Feedback" refers to data such as the results, impressions, and implementation status of the user's actions based on the proposed measures.

[1252] "Update" is the process of reassessing risk assessments and countermeasures based on newly acquired data and revising them as necessary.

[1253] The present invention is a system that records the amount of exercise and dietary content of an individual, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments are described below.

[1254] System configuration

[1255] The system consists of the following main components:

[1256] 1. Device worn by the user

[1257] Acceleration sensor: Used to measure movement volume, obtaining step count and acceleration data in real time.

[1258] GPS: Used to measure distance traveled and location.

[1259] Camera: Captures images of food to record eating habits.

[1260] Image processing algorithms are used to extract food content features from the captured images.

[1261] 2. Server

[1262] Data reception and preprocessing unit: Receives data sent from the terminal and performs noise removal and missing value completion.

[1263] Feature extraction module: Extracts features of exercise and dietary content from the stored data.

[1264] Risk assessment module: assesses disease risk based on extracted features.

[1265] Countermeasure proposal generation module: Generates appropriate countermeasure proposals based on the results of risk assessment.

[1266] Notification module: Sends the generated countermeasures to the device.

[1267] Feedback receiving and updating module: receives feedback data from users and updates risk assessment and countermeasure proposals.

[1268] Feature details

[1269] 1. Collecting exercise data

[1270] Users wear a device with a built-in accelerometer and GPS on a daily basis. This device collects exercise data (number of steps, distance traveled, etc.) in real time. For example, when you start jogging, the device immediately starts recording this data.

[1271] 2. Collection of dietary data

[1272] When a user eats, they take a photo of their meal with the device's camera. This image data is analyzed by the device to identify the meal contents (e.g., type of sandwich, amount, calories, etc.).

[1273] 3. Data Transmission

[1274] The device sends the collected data to the server at regular intervals, using Wi-Fi or mobile data communication.

[1275] 4. Data Preprocessing

[1276] The server preprocesses the received data, specifically removing noise, imputing missing values, and detecting and correcting outliers.

[1277] 5. Data feature extraction

[1278] The server's feature extraction module extracts useful information from the pre-processed data, such as average steps, total distance traveled, calories, types of nutrients, etc. Image recognition algorithms and machine learning models are used in this process.

[1279] 6. Risk Assessment

[1280] The server's risk assessment module assesses disease risk based on the extracted features. For example, diabetes risk is assessed based on factors such as lack of exercise and frequency of high-calorie food intake.

[1281] 7. Generation and notification of countermeasures

[1282] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include exercise plans and dietary restriction suggestions. For example, specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%" are made. These countermeasures are notified to the device.

[1283] 8. Collecting and Updating Feedback

[1284] The user records the results and impressions of actions taken based on the proposed countermeasures on the device. This feedback data is then sent back to the server and used to evaluate risks and update the proposed countermeasures.

[1285] Specific examples

[1286] 1. Jogging and Lunch Log

[1287] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[1288] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[1289] 2. Data transmission and storage

[1290] The device sends the collected jogging data and meal images to a server.

[1291] The server receives the data and stores it in a database.

[1292] 3. Feature Extraction and Risk Assessment

[1293] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[1294] The server uses this data to assess diabetes risk.

[1295] 4. Countermeasures generation and notification

[1296] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[1297] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1298] 5. Feedback and solution updates

[1299] The user implements the measures and records the results on the device.

[1300] The terminal transmits the feedback data to the server again.

[1301] The server updates the risk assessment and countermeasures based on the received data.

[1302] Prompt Sentence Examples

[1303] Below are some example prompts to input to a generative AI model:

[1304] Please explain the system flow for recording an individual's exercise volume and dietary habits, analyzing that data to assess disease risk, and generating appropriate countermeasures. Please provide a detailed explanation, including the names of the specific hardware and software used and how the data is processed. Please also provide examples.

[1305] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1306] Step 1:

[1307] The user wears the device.

[1308] Input: The device's accelerometer and GPS are initialized and ready to go.

[1309] Operation: The device detects the user's movements (steps and vibrations) using an accelerometer and acquires location information using GPS. Each measurement data is recorded internally along with the time.

[1310] Output: Real-time updated exercise data (number of steps, distance traveled, exercise intensity).

[1311] Step 2:

[1312] The user takes a photo of the meal.

[1313] Input: The user activates the device's camera and captures an image of the food.

[1314] How it works: Image data captured by the camera is temporarily stored on the device. Optionally, users have the option to manually enter meal details.

[1315] Output: Saved meal image data or manually entered meal content data.

[1316] Step 3:

[1317] The device sends the data to the server.

[1318] Input: Exercise data and dietary data stored in the device at regular intervals (for example, once a day).

[1319] How it works: Your device uses Wi-Fi or mobile data to upload data to a server, which may compress the data before sending it.

[1320] Output: Compressed exercise data and compressed dietary data sent to the server.

[1321] Step 4:

[1322] The server pre-processes the received data.

[1323] Input: Compressed exercise data and compressed dietary data sent from the terminal.

[1324] How it works: The server decompresses the data, removes noise, imputes missing values, and detects and corrects outliers.

[1325] Output: Clean, pre-processed exercise and diet data.

[1326] Step 5:

[1327] The server performs feature extraction.

[1328] Input: Preprocessed physical activity data and dietary data.

[1329] How it works: The server's machine learning algorithm extracts average steps, total distance traveled, and exercise intensity from exercise data, and calorie and nutrient information from dietary image data.

[1330] Output: Extracted feature data of exercise amount and feature data of dietary content.

[1331] Step 6:

[1332] The server assesses the disease risk.

[1333] Input: Extracted feature data of exercise amount and feature data of dietary content.

[1334] How it works: Risk assessment algorithms on the server calculate your risk of certain diseases, such as diabetes or heart disease, using statistical methods and machine learning models.

[1335] Output: Assessed disease risk data.

[1336] Step 7:

[1337] The server generates a countermeasure plan.

[1338] Input: Assessed disease risk data.

[1339] How it works: The server generates countermeasures based on the risk assessment results, such as suggesting exercise plans or dietary restrictions.

[1340] Output: Generated countermeasure proposal data.

[1341] Step 8:

[1342] The server will notify you of the proposed solution.

[1343] Input: Generated countermeasure proposal data.

[1344] Operation: The server sends the countermeasure data to the device, which then notifies the user via push notification or email.

[1345] Output: Notification of proposed measures displayed on the user's device.

[1346] Step 9:

[1347] The user records feedback.

[1348] Input: Results and impressions of users who have implemented measures.

[1349] How it works: The user enters their status after implementing the measures into the device, which also sends periodic reminders to record compliance.

[1350] Output: Recorded feedback data.

[1351] Step 10:

[1352] The terminal transmits the feedback data to the server.

[1353] Input: Feedback data entered by the user.

[1354] Operation: The terminal sends the collected feedback data to the server at regular intervals.

[1355] Output: Feedback data sent to the server.

[1356] Step 11:

[1357] The server updates the risk assessment and countermeasures.

[1358] Input: Feedback data, past exercise data, and dietary data.

[1359] Actions: The server re-performs risk assessment based on new data and updates its countermeasure recommendations, which may include retraining machine learning models.

[1360] Output: Updated risk assessment and proposed countermeasures.

[1361] As described above, the program of this system records the amount of exercise and diet of an individual in detail, evaluates the risk of disease based on this information, and provides appropriate countermeasures, allowing users to easily manage their health in their daily lives.

[1362] (Application example 1)

[1363] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1364] While managing personal health is important, there is a lack of systems in brick-and-mortar stores that can properly record customers' exercise and dietary habits and manage their health based on that information. Furthermore, there are insufficient means to provide risk assessments and health management recommendations based on this data. Therefore, there is a need for specific methods and systems to effectively support health management in brick-and-mortar stores.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1366] In this invention, the server includes a terminal that records an individual's amount of exercise and dietary details, means for receiving the recorded data on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored data and generating risks and countermeasures for each disease, means for notifying the terminal of the generated countermeasures, means for receiving feedback data from the terminal and updating the risk assessment and countermeasures, and means for providing menu suggestions and health management advice based on the customer's health condition in a physical store. This enables health management based on the customer's amount of exercise and dietary details in a physical store.

[1367] "Individual exercise amount" is data that indicates the level of physical activity that an individual engages in on a daily basis, such as the number of steps taken, distance traveled, and exercise intensity.

[1368] "Dietary details" refers to data including the type, amount, nutrients, calories, etc. of food and beverages consumed.

[1369] A "terminal" is a device that records an individual's amount of exercise and dietary details and transmits the collected data to a server using a communication means.

[1370] The "server" is a computer system that receives the recorded data and performs preprocessing, storage, feature extraction, risk assessment, and countermeasure generation.

[1371] "Preprocessing" refers to data processing performed before analysis, such as removing noise from data and filling in missing values.

[1372] "Storage" means recording the preprocessed data in a database in the server.

[1373] "Feature extraction" is the process of extracting necessary information from exercise and dietary data and putting it into an analyzable form.

[1374] "Risk assessment" is the process of calculating and assessing the risk of developing a particular disease based on the extracted features.

[1375] "Countermeasures" are specific action plans or proposals based on the results of risk assessment, aimed at improving health and preventing disease.

[1376] "Notification" refers to the act of sending the generated countermeasure plan to the terminal and notifying the user.

[1377] "Feedback data" refers to data that includes the results and impressions of users who have taken actions based on the proposed measures.

[1378] A "physical store" is a physical business establishment, such as a cafe or restaurant, where consumers visit in person to receive services.

[1379] "Menu suggestions" refers to recommending appropriate meals and drinks based on the customer's health condition.

[1380] "Health management advice" refers to specific action plans and lifestyle guidance to improve a customer's health.

[1381] This paper describes a system that records an individual's exercise and dietary habits, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specifically, this system is designed to provide menu suggestions and health management advice based on the customer's health status in a brick-and-mortar store.

[1382] Recording exercise volume

[1383] A user uses the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity. For example, when a user goes jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[1384] Recording dietary information

[1385] When a user eats at a physical restaurant, they take pictures of their meal using the camera attached to their device. The images taken by the camera are temporarily stored on the device, and this data becomes important information for later analysis of the meal contents. For example, if a user orders a salad at a cafe, they take a picture of the salad with the camera and save it on their device.

[1386] Data transmission and storage practices

[1387] The device periodically transmits the collected exercise data and meal content data to the server. The server receives this data and stores it in a database. Preprocessing of the data involves noise removal and missing value completion.

[1388] Performing feature extraction and analysis

[1389] The server extracts characteristics of exercise volume and dietary content from the stored data. Average steps, distance traveled, and exercise intensity are calculated from the exercise data, and calories and types of nutrients are identified from dietary image data. Machine learning algorithms are used to extract these characteristics.

[1390] Conducting risk assessments

[1391] The server then assesses disease risk based on the extracted features. For example, to assess diabetes risk, it considers factors such as lack of exercise and frequency of high-calorie food intake. This risk assessment is performed using statistical methods and machine learning algorithms.

[1392] Generate countermeasures and notify

[1393] Based on the results of the risk assessment, the server generates appropriate measures for the user. The generated measures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" and "reducing carbohydrate intake by 50%." These measures are then notified to the device.

[1394] Gathering feedback and updating countermeasures

[1395] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends the feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user again.

[1396] Specific examples

[1397] Jogging and cafe meal record:

[1398] A user goes jogging in the morning, and the device records the number of steps and distance traveled using the accelerometer and GPS.

[1399] A user orders a salad at a cafe and takes a picture of the salad with the camera on the device.

[1400] Data transmission and storage:

[1401] The device sends the collected jogging data and meal images to a server.

[1402] The server receives the data and stores it in a database.

[1403] Feature extraction and risk assessment:

[1404] The server extracts information on the amount of exercise (number of steps, distance traveled) and calories in the salad.

[1405] The server uses this data to assess diabetes risk.

[1406] Countermeasure generation and notification:

[1407] The server generates recommendations such as "jogging at least three times a week" and "low-carb menus."

[1408] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1409] Feedback and solution update:

[1410] The user implements the measures and records the results on the device.

[1411] The terminal transmits the feedback data to the server again.

[1412] The server updates the risk assessment and countermeasures based on the received data.

[1413] Prompt Sentence Examples

[1414] "The customer entered the number of steps and distance they jogged, as well as a photo of their meal (salad) at the cafe, into the app. Based on this data, the app assesses their health risks and generates optimal health management advice."

[1415] The present invention makes it possible to easily provide health management based on the amount of exercise and dietary content of users even in brick-and-mortar stores, thereby integrating the provision of health services and customer health management in brick-and-mortar stores.

[1416] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1417] Step 1:

[1418] When a user exercises, the device uses an acceleration sensor and GPS to collect exercise data. The input is the user's exercise status (number of steps, distance traveled, exercise intensity), and the device measures data in real time based on this. The output is the exercise data obtained as a result of the measurement.

[1419] Step 2:

[1420] When a user eats at a physical restaurant, they take a photo of their meal using the camera on their device. The input is the photo of the meal, and the device acquires the data by temporarily saving this image. The output is the saved meal image data.

[1421] Step 3:

[1422] The device periodically transmits the collected exercise data and dietary image data to the server. The input is the collected exercise data and dietary data, and the device uploads this data to the server. The output is the data transmitted to the server.

[1423] Step 4:

[1424] The server stores the received exercise data and dietary image data. The input is the exercise data and dietary data sent from the device, which the server records in a database and performs data preprocessing such as noise removal and missing value completion. The output is the preprocessed data.

[1425] Step 5:

[1426] The server extracts features from the preprocessed data. The input is the preprocessed exercise data and dietary data, and the server extracts feature information such as the number of steps, distance traveled, exercise intensity, and dietary calories and nutrients from these data. The output is the extracted feature data.

[1427] Step 6:

[1428] The server evaluates disease risk based on the extracted feature data. The input is the feature data, and the server performs risk assessment using machine learning algorithms and statistical methods. For example, this includes assessing diabetes risk. The output is the risk assessment result.

[1429] Step 7:

[1430] The server generates countermeasures based on the risk assessment results. The input is the risk assessment results, and the server creates countermeasures such as specific exercise plans and dietary restrictions. The output is the generated countermeasures.

[1431] Step 8:

[1432] The generated countermeasures are notified to the terminal by the server. The input is the generated countermeasures, which the server sends to the terminal. The output is the countermeasures notified to the terminal.

[1433] Step 9:

[1434] The user implements the proposed measures and records the results and their impressions on the device. The input is the user's action results and feedback, which the device records as data. The output is the recorded feedback data.

[1435] Step 10:

[1436] The terminal sends the collected feedback data back to the server. The input is the recorded feedback data that the terminal uploads to the server. The output is the feedback data sent to the server.

[1437] Step 11:

[1438] The server updates the risk assessment and countermeasures based on the feedback data. The input is feedback data from the user, and the server reassessssments and updates the countermeasures based on this. The output is the updated risk assessment and countermeasures.

[1439] The above processing steps enable health management based on the user's exercise volume and dietary content even in brick-and-mortar stores, making it possible to integrate health service provision in brick-and-mortar stores with customer health management.

[1440] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1441] The present invention combines a system that records an individual's amount of exercise and dietary details, assesses disease risk based on this data, and provides appropriate countermeasures with an emotion engine that recognizes the user's emotions. This makes it possible to provide more personalized countermeasures that take into account the user's subjective state. As an embodiment of the present invention, a program is generated as follows, and its specific processing is described below.

[1442] Emotional data recording

[1443] A user wears the device of the present invention in their daily life. This device is equipped with a camera, microphone, and sensors, which are used by an emotion engine to analyze the user's facial expressions, tone of voice, and even biosignal data (heart rate, electrodermal activity, etc.) to recognize the user's emotions and generate emotion data. For example, when the user smiles or feels stressed, this emotion data is recorded in real time.

[1444] Data transmission and storage practices

[1445] The device periodically compiles exercise data, food image data, and emotion data into packets, which are then sent to a server via Wi-Fi or a mobile network. The server then stores the received data in a database.

[1446] Performing feature extraction and analysis

[1447] The server extracts various features from the stored data. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from the dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from the emotion data.

[1448] Conducting risk assessments

[1449] The server then assesses disease risk based on the extracted features, taking into account emotional data to assess, for example, the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[1450] Generate countermeasures and notify

[1451] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. For example, if stress levels are high, the server can generate countermeasures such as "jogging three times a week" and "yoga for relaxation." These countermeasures are then sent to the device, where the user can confirm them.

[1452] Collecting feedback and updating countermeasures

[1453] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[1454] Specific examples

[1455] 1. Jogging and Lunch Log:

[1456] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[1457] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[1458] If a user smiles or feels stressed while eating, the emotion engine will recognize this and record the emotion data.

[1459] 2. Data transmission and storage:

[1460] The device sends the collected jogging data, meal images, and emotional data to a server.

[1461] The server receives this data and stores it in a database.

[1462] 3. Feature extraction and risk assessment:

[1463] The server extracts the amount of exercise (number of steps, distance traveled), calorie information of the sandwich, and emotional data (stress level, etc.).

[1464] The server uses this data to assess diabetes risk.

[1465] 4. Countermeasure generation and notification:

[1466] Based on the risk assessment, the server generates suggested countermeasures such as "jogging at least three times a week," "reducing carbohydrate intake by 50%," and "recommending yoga for relaxation."

[1467] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1468] 5. Feedback and solution update:

[1469] The user implements the measures and records the results and impressions on the device.

[1470] The terminal transmits the feedback data to the server again.

[1471] The server updates the risk assessment and countermeasures based on new data, and also takes emotional data into account to adjust the countermeasures.

[1472] This allows the system to more personalize the user's health management and take emotional state into account to help with comprehensive disease prevention and health maintenance.

[1473] The processing flow will be explained below.

[1474] Step 1:

[1475] The user begins exercising, for example, jogging or walking.

[1476] Step 2:

[1477] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[1478] Step 3:

[1479] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[1480] Step 4:

[1481] The device uses a built-in emotion engine to recognize the user's emotions. Emotional data is generated based on facial expression recognition, voice analysis, and biometric data (heart rate, electrodermal activity, etc.). For example, if the user smiles or feels stressed, that emotion is recorded.

[1482] Step 5:

[1483] The terminal periodically assembles the exercise data, meal image data, and emotion data into packets.

[1484] Step 6:

[1485] The device uses Wi-Fi or a mobile communication network to send exercise data, food image data, and emotional data to a server.

[1486] Step 7:

[1487] The server stores the received exercise data, meal image data, and emotion data in a database.

[1488] Step 8:

[1489] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[1490] Step 9:

[1491] The server extracts features from the preprocessed data. From the exercise data, it calculates the average number of steps, distance traveled, and exercise intensity. From the food image data, it identifies calories and types of nutrients. From the emotion data, it extracts emotional states such as joy, sadness, and stress.

[1492] Step 10:

[1493] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise, frequency of high-calorie meals, and stress levels to determine diabetes risk.

[1494] Step 11:

[1495] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week is recommended," "reducing carbohydrate intake by 50%," or "yoga is recommended for relaxation."

[1496] Step 12:

[1497] The server sends the generated countermeasures to the terminal.

[1498] Step 13:

[1499] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[1500] Step 14:

[1501] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[1502] Step 15:

[1503] After implementing the proposed measures, the user records the results, impressions, and emotional state on the device.

[1504] Step 16:

[1505] The device sends feedback data (amount of exercise, dietary content, emotional state) to a server.

[1506] Step 17:

[1507] The server receives and stores the feedback data.

[1508] Step 18:

[1509] The server updates the risk assessment and countermeasures based on the new data. If necessary, it adjusts the countermeasures and sends them back to the device. It also takes emotional data into account to make the countermeasures more personalized.

[1510] This allows the system to continuously support the user's health management and achieve comprehensive disease prevention that also takes emotional state into account.

[1511] Example 2

[1512] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1513] In modern health management systems, providing more personalized health measures by taking into account not only the amount of exercise and dietary content but also the user's emotional data is a challenge. In particular, there is a need to properly evaluate the impact of stress and emotional state on health risks and to develop appropriate measures. However, conventional systems have difficulty in comprehensively handling these factors, making it difficult to provide users with appropriate measures.

[1514] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1515] In this invention, the server includes: a means for recording the amount of exercise, dietary content, and emotion data of an individual;

[1516] A means for receiving, preprocessing and storing the recorded exercise amount, dietary content and emotion data;

[1517] A means for extracting features from the stored data, assessing the risk of each disease, and generating countermeasures that take into account the emotional data;

[1518] a means for notifying the terminal of the generated countermeasures;

[1519] means for receiving feedback data from the terminal and updating the risk assessment and countermeasures;

[1520] This allows for more personalized health recommendations that take into account the user's emotional state.

[1521] "Amount of exercise" refers to the amount of physical activity an individual engages in over a certain period of time, and includes the number of steps taken, distance traveled, exercise intensity, etc.

[1522] "Dietary content" refers to the types, amounts, and nutrients of the foods and beverages consumed by an individual, including information on calories and nutritional components.

[1523] "Emotional data" refers to information about a user's emotional state obtained by analyzing their facial expressions, tone of voice, biometric signals, etc.

[1524] "Device" refers to an electronic device worn by an individual to record exercise volume, dietary content, emotional data, etc.

[1525] "Server" refers to a computer system that receives data from terminals, stores it, analyzes it, evaluates risks, and generates countermeasures.

[1526] "Feature extraction" refers to the process of extracting important information or patterns from stored data, such as the average number of steps taken or calorie information for meals.

[1527] "Risk assessment" refers to the process of calculating individual health risks based on collected data and planning appropriate countermeasures.

[1528] "Countermeasures" refers to plans or proposals that recommend individual health management methods or actions, generated based on the results of a risk assessment.

[1529] "Feedback data" refers to data that records the results and impressions of users when they perform an action.

[1530] "Preprocessing" refers to the process of converting received data into a suitable format for analysis and storage.

[1531] The present invention is a system that comprehensively monitors and analyzes an individual's exercise amount, dietary content, and emotional data, and provides the user with appropriate health measures. Specific embodiments for carrying out the present invention will be described in detail below.

[1532] System Overview

[1533] This system consists of a device worn by the user and a server that collects, analyzes, and stores data, and generates and notifies users of countermeasures. The device has built-in hardware for data collection, and the server has hardware and software for data processing.

[1534] Hardware and Software Configuration

[1535] Terminal

[1536] Camera: Takes pictures of the user's meal and collects image data.

[1537] Microphone: Collects the user's tone of voice to help with sentiment data.

[1538] Acceleration sensor: Measures the user's exercise volume (number of steps, distance traveled, and exercise intensity).

[1539] Location information acquisition device (GPS): Measures the user's travel distance.

[1540] Heart rate sensor: Measures the user's heart rate and collects biometric data.

[1541] Electrodermal activity sensor: Measures the user's electrodermal activity and estimates stress levels.

[1542] server

[1543] Database (e.g., Amazon Web Services RDS): Stores received exercise, dietary, and emotion data.

[1544] Machine learning algorithms (e.g., TensorFlow): Analyze collected food image data and identify calories and types of nutrients.

[1545] Python libraries (e.g., Pandas, scikit-learn): Extract features from stored data and perform risk assessment.

[1546] Emotion Engine: Analyzes collected data and recognizes the user's emotional state.

[1547] System Operation

[1548] Data collection

[1549] The user wears the device in their daily life, and the device's built-in sensors collect real-time data on the user's exercise, diet, and emotions. Specifically, the camera captures images of meals, the microphone collects voice tone, the accelerometer and GPS measure exercise, and the heart rate sensor and electrodermal activity sensor collect biometric data.

[1550] Data transmission and storage

[1551] The device collects data at regular intervals, compiles it into packets, and sends them to a server via Wi-Fi or a mobile network. The server stores the received data in a database for subsequent analysis.

[1552] Data analysis and feature extraction

[1553] The server analyzes the stored data and extracts various features, using machine learning algorithms to identify calories and nutrients from food images, and Python libraries to analyze exercise volume and emotional data.

[1554] Risk assessment and countermeasure generation

[1555] The server performs a risk assessment based on the extracted feature data. It also includes emotional data in the assessment and calculates the overall health risk. It then generates optimal countermeasures based on the risk and notifies the user. These countermeasures include specific recommendations such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[1556] Gathering feedback and updating countermeasures

[1557] The user acts according to the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and countermeasures based on the new data. This makes it possible to provide more personalized countermeasures that take the user's emotional state into account.

[1558] Examples and prompts

[1559] Examples:

[1560] When a user goes for a morning jog, the device uses the accelerometer and GPS to record the number of steps and distance traveled. When the user eats a sandwich for lunch, the device's camera takes a photo of the meal. If the user smiles while eating, the emotion engine recognizes this and records emotional data.

[1561] Example prompt sentence:

[1562] How is data transmitted and stored?

[1563] "Give me an example of user emotion data."

[1564] "How do I generate personalized countermeasures?"

[1565] In this way, the present invention comprehensively supports the user's health management and, by taking into account the user's emotional state, can provide more accurate health measures.

[1566] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1567] Step 1:

[1568] Emotional data recording

[1569] The user wears the device. The device uses a camera, microphone, heart rate sensor, and electrodermal activity sensor to collect the user's facial expressions, tone of voice, and biosignal data. This data is analyzed in real time by the emotion engine to generate emotion data. Specifically, when the user smiles, the emotion engine generates emotion data for "happiness."

[1570] Input: User's facial expressions, audio, and biometric signals

[1571] Output: Emotion data (e.g., happiness, stress)

[1572] Step 2:

[1573] Recording exercise and food data

[1574] When users exercise in their daily lives, the device's accelerometer and GPS record the number of steps taken and the distance traveled. When users eat, the device's camera captures photos of their meals and collects dietary data.

[1575] Input: Steps, distance traveled, food images

[1576] Output: Exercise data (e.g., steps, distance traveled), food data (e.g., food images)

[1577] Step 3:

[1578] Data transmission and storage

[1579] At regular intervals, the device collects exercise data, dietary data, and emotional data and sends them in packets to a server via Wi-Fi or a mobile network. The server analyzes the received data and stores it in a database.

[1580] Input: Packetized exercise data, food data, and emotion data

[1581] Output: Various data stored in the database

[1582] Step 4:

[1583] Feature Extraction and Analysis

[1584] The server analyzes the stored data and extracts features. First, it uses Python's Pandas to calculate the average number of steps, distance traveled, and exercise intensity from the exercise data. Next, it uses machine learning algorithms (e.g., TensorFlow) to identify calories and types of nutrients from food images. Finally, it uses an emotion engine to analyze the emotion data.

[1585] Input: Stored exercise data, diet data, and emotional data

[1586] Output: Extracted feature data (e.g., average steps, calories, emotional state)

[1587] Step 5:

[1588] Risk Assessment

[1589] The server applies a risk assessment model based on the feature data to objectively calculate the risk of disease. For example, when assessing the risk of diabetes, it uses a machine learning model (using scikit-learn) to determine that lack of exercise and high stress are high-risk factors.

[1590] Input: Extracted feature data

[1591] Output: Risk assessment results (e.g., diabetes risk score)

[1592] Step 6:

[1593] Generation and notification of countermeasures

[1594] Based on the results of the risk assessment, the server generates optimal countermeasures for the user. For example, if the risk assessment diagnoses high stress, the server will generate specific countermeasures such as "jogging three times a week" and "recommended yoga for relaxation." The server notifies the device of the countermeasures and informs the user.

[1595] Input: Risk assessment results

[1596] Output: suggested action (e.g. jogging, yoga)

[1597] Step 7:

[1598] Collect feedback and update countermeasures

[1599] The user acts on the proposed measures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and proposed measures based on the newly received data. This allows for more personalized proposals that take the user's emotional state into account.

[1600] Input: User feedback data

[1601] Output: Updated risk assessment and proposed countermeasures

[1602] (Application example 2)

[1603] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1604] In today's busy lifestyles, personal health management is extremely important, but conventional systems only record the amount of exercise and dietary content and evaluate disease risk based on this, so they are unable to take measures that take into account the user's emotions and stress level. As a result, individualized and effective health management is difficult, making it difficult to maintain employee performance and reduce security risks, especially in high-stress environments.

[1605] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1606] In this invention, the server includes a means for generating emotion data using a camera, microphone, and sensor, a means for performing risk assessment including the emotion data and generating countermeasures, and a means for notifying the terminal of the generated countermeasures, thereby enabling personalized health management and reducing security risks in high-stress environments.

[1607] "Amount of exercise" is an index that indicates the amount of energy and intensity that an individual expends in daily life and physical activity.

[1608] "Dietary content" is data that indicates the types and amounts of food and beverages consumed by an individual, as well as information on the nutrients contained therein.

[1609] A "terminal" is a device that can be worn or carried by an individual and records exercise, dietary habits, and even emotional data.

[1610] "Server" refers to the central processing unit that receives the recorded data, performs pre-processing and storage, extracts features, and generates risk assessments and countermeasures.

[1611] "Preprocessing" refers to the initial processing of data, such as organizing, standardizing, and filtering, to make the data sent to the server easier to analyze.

[1612] "Storage" refers to keeping data for an extended period of time and making it accessible when needed.

[1613] "Feature extraction" is the process of extracting important information from stored data that is necessary for risk assessment and countermeasure generation.

[1614] "Risk assessment" refers to the objective evaluation of the risk of developing a specific health condition or disease based on extracted feature data.

[1615] "Countermeasures" are proposals that specifically outline preventative and improvement measures that users should take based on the results of risk assessment.

[1616] "Emotion data" is data that indicates the user's emotional state, and is generated based on facial expressions, tone of voice, heart rate, and electrodermal activity.

[1617] An "emotion engine" is software or hardware that analyzes data collected by cameras, microphones, and sensors, recognizes the user's emotions, and generates emotional data.

[1618] "Notification" refers to the act of sending the generated countermeasure plan to the user's terminal so that the user can check it.

[1619] "Feedback data" refers to data that records user actions based on proposed countermeasures, their results, and their impressions.

[1620] This invention is a system that comprehensively records an individual's exercise amount, dietary content, and emotional data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments of this system are described below.

[1621] System configuration

[1622] 1. Terminal

[1623] Hardware:

[1624] Acceleration sensors and position measuring devices

[1625] camera

[1626] microphone

[1627] Sensors (heart rate monitor, electrodermal activity sensor)

[1628] software:

[1629] Emotion recognition engines (e.g., generic image and voice analysis software)

[1630] Data collection and transmission program

[1631] 2. Server

[1632] Hardware:

[1633] Cloud-based central processing units (e.g., general-purpose server equipment from cloud providers)

[1634] software:

[1635] Machine learning algorithms (e.g., general-purpose machine learning frameworks)

[1636] Databases (e.g. cloud provider databases)

[1637] Data analysis platform (e.g., general-purpose data analysis tool)

[1638] Notification systems (e.g., general-purpose notification services)

[1639] Program processing

[1640] The device is worn by individuals and collects data on the amount of exercise, dietary habits, and emotions in daily life. Data is recorded in real time using a camera, microphone, and sensors, and emotion data is generated using an emotion engine. This allows data to be obtained that takes into account the user's subjective state.

[1641] The device periodically collects data and sends it to a server via Wi-Fi or a mobile network. The server receives the data and stores it in a database.

[1642] Once the data is saved, the server extracts features based on it. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from emotion data.

[1643] The server then assesses disease risk based on the extracted features, taking into account emotional data, for example, to assess the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[1644] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. The generated countermeasures are then sent to the device, where the user can confirm them.

[1645] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[1646] Specific examples

[1647] 1. Monitoring Employee A:

[1648] While Employee A is working, smart glasses monitor his facial expressions and record his tone of voice.

[1649] Heart rate and electrodermal activity are measured in real time to generate emotional data.

[1650] Emotion data is sent to the server in real time.

[1651] 2. Data transmission and storage:

[1652] The data received by the server is stored in a general-purpose database.

[1653] 3. Risk Assessment:

[1654] A machine learning algorithm on the server assesses security risks when employee A's stress level increases.

[1655] Generates countermeasures when high stress levels are detected.

[1656] 4. Countermeasure generation and notification:

[1657] The server generates suggested countermeasures such as "take a short break" or "take a deep breath."

[1658] Audio and visual notifications displayed on smart glasses.

[1659] 5. Feedback and solution update:

[1660] Employee A takes action based on the measures and records the results and impressions on the smart glasses.

[1661] The server analyzes the feedback and updates the countermeasures.

[1662] Prompt Sentence Examples

[1663] "How can I design a system that monitors employees' stress levels and notifies them in real time of appropriate relaxation measures when high stress levels are detected?"

[1664] This allows companies to effectively manage employee stress and reduce security risks.

[1665] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1666] Step 1:

[1667] The user puts on the device and begins their daily or work activities.

[1668] Hardware: Accelerometer, positioning device, camera, microphone, heart rate monitor, electrodermal activity sensor.

[1669] Specific operation: The device records the user's exercise (number of steps, distance traveled, etc.), dietary content (camera images), facial expressions (camera), tone of voice (microphone), heart rate, and skin electrodermal activity in real time.

[1670] Input: User's exercise, diet, facial expressions, voice, heart rate, and electrodermal activity.

[1671] Output: Raw recorded data.

[1672] Step 2:

[1673] The terminal collects the recorded data and sends it to the server.

[1674] Specific operation: Data is packetized at regular intervals and sent via Wi-Fi or mobile networks.

[1675] Input: Recorded exercise data, dietary data, facial expression data, voice data, and biometric data.

[1676] Output: The data packets sent.

[1677] Step 3:

[1678] The server stores the received data in a database and preprocesses it for further processing.

[1679] Hardware: A cloud-based central processing unit.

[1680] Software: Database (general-purpose database from the cloud provider), data preprocessing program.

[1681] Specific actions: Organize, standardize, filter, and store incoming data in a database.

[1682] Input: The data packet sent.

[1683] Output: Saved clean data.

[1684] Step 4:

[1685] The server extracts features from the data stored in the database.

[1686] Software: Machine learning algorithms (general-purpose machine learning frameworks).

[1687] Specific operations: Calculates the number of steps, distance traveled, and exercise intensity from exercise data, identifies calories and types of nutrients from dietary image data, and analyzes emotional state from facial expressions, voice, and biometric data.

[1688] Input: Saved clean data.

[1689] Output: Extracted feature data.

[1690] Step 5:

[1691] The server evaluates the risk of disease based on the extracted feature data.

[1692] Software: Machine learning algorithms, statistical models.

[1693] Specific operation: Conduct a comprehensive risk assessment that includes emotional data. For example, evaluate the impact of stress on diabetes risk.

[1694] Input: Extracted feature data.

[1695] Output: The results of the risk assessment.

[1696] Step 6:

[1697] The server generates countermeasures based on the results of the risk assessment and notifies the terminal.

[1698] Software: Notification System (general-purpose notification service).

[1699] Specific actions: Suggested measures such as "jog at least three times a week," "reduce carbohydrate intake by 50%," and "recommend yoga for relaxation" are generated and notified to the device.

[1700] Input: The results of the risk assessment.

[1701] Output: Generated countermeasures and notifications.

[1702] Step 7:

[1703] The user begins to take action based on the proposed measures and records the results and impressions on the device.

[1704] Specific actions: The user records their actions based on the measures as feedback data via a smart device.

[1705] Input: User actions based on proposed measures.

[1706] Output: Feedback data.

[1707] Step 8:

[1708] The device sends feedback data to the server, which updates the risk assessment and proposed countermeasures.

[1709] Specific operation: The feedback data is packetized and sent, and the server receives and stores it. Then, the risk assessment model and countermeasures are updated based on the feedback.

[1710] Input: The submitted feedback data.

[1711] Output: Updated risk assessment and new countermeasures.

[1712] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1713] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1714] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1715] [Fourth embodiment]

[1716] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1717] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1718] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1719] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1720] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1721] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1722] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1723] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1724] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1725] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1726] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1727] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1728] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1729] The present invention provides a system for recording an individual's exercise amount and dietary content, analyzing the data to assess the risk of illness, and providing appropriate countermeasures. As an embodiment of the present invention, a program is generated as follows, and its processing will be described in detail.

[1730] Recording exercise volume

[1731] A user wears the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity in real time. For example, when a user is jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[1732] Recording dietary information

[1733] When a user eats a meal, the user takes an image of the meal using the camera attached to the device. The image taken by the camera is temporarily saved in the device. This meal image data becomes important information for later analysis of the meal contents. For example, if a user eats a sandwich for lunch, the user takes an image of the sandwich with the camera and saves it on the device.

[1734] Data transmission and storage practices

[1735] The device periodically transmits the collected exercise data and meal content data to a server. The transmitted data is received by the server and stored in a database. The server then preprocesses the received data, removing noise and filling in missing values.

[1736] Performing feature extraction and analysis

[1737] The server extracts characteristics of exercise volume and dietary content from the stored data. Specifically, it calculates average steps, distance traveled, and exercise intensity from exercise data, and identifies calories and types of nutrients from dietary image data. This feature extraction is performed using machine learning algorithms and analyzed on the server.

[1738] Conducting risk assessments

[1739] The server then uses the extracted features to assess disease risk. For example, to assess the risk of diabetes, calculations are made based on a lack of physical activity and the frequency of high-calorie food intake. This risk assessment is performed objectively using statistical methods and machine learning algorithms.

[1740] Generate countermeasures and notify

[1741] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%." These countermeasures are then sent to the device, where the user can confirm them.

[1742] Collecting feedback and updating countermeasures

[1743] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends this feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user.

[1744] Specific examples

[1745] 1. Jogging and Lunch Log:

[1746] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[1747] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[1748] 2. Data transmission and storage:

[1749] The device sends the collected jogging data and meal images to a server.

[1750] The server receives the data and stores it in a database.

[1751] 3. Feature extraction and risk assessment:

[1752] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[1753] The server uses this data to assess diabetes risk.

[1754] 4. Countermeasure generation and notification:

[1755] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[1756] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1757] 5. Feedback and solution update:

[1758] The user implements the measures and records the results on the device.

[1759] The terminal transmits the feedback data to the server again.

[1760] The server updates the risk assessment and countermeasures based on the received data.

[1761] As described above, this invention supports personal health management through the process of data exchange and analysis between the terminal and the server. This system allows users to easily manage their own health in their daily lives and can be useful for disease prevention.

[1762] The processing flow will be explained below.

[1763] Step 1:

[1764] The user starts exercising, for example, jogging or walking.

[1765] Step 2:

[1766] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[1767] Step 3:

[1768] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[1769] Step 4:

[1770] The device periodically compiles exercise data and meal image data into packets.

[1771] Step 5:

[1772] The device uses Wi-Fi or a mobile communication network to send exercise data and dietary image data to a server.

[1773] Step 6:

[1774] The server stores the received exercise data and meal image data in a database.

[1775] Step 7:

[1776] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[1777] Step 8:

[1778] The server extracts features from the preprocessed data, calculating the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifying calories and types of nutrients from the dietary image data.

[1779] Step 9:

[1780] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise and frequency of high-calorie meals to determine diabetes risk.

[1781] Step 10:

[1782] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week" or "reducing carbohydrate intake by 50%."

[1783] Step 11:

[1784] The server sends the generated countermeasures to the terminal.

[1785] Step 12:

[1786] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[1787] Step 13:

[1788] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[1789] Step 14:

[1790] After implementing the proposed measures, the user records the results and impressions on the device.

[1791] Step 15:

[1792] The terminal transmits the feedback data to the server.

[1793] Step 16:

[1794] The server receives and stores the feedback data.

[1795] Step 17:

[1796] The server updates the risk assessment and countermeasures based on the new data, and if necessary, adjusts the countermeasures and sends them again to the device.

[1797] This allows the system to continuously support users in managing their health and provide actions that help prevent disease.

[1798] Example 1

[1799] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1800] In personal health management, there is a need for a system that can properly record exercise volume and dietary content, analyze this data to assess disease risk, and provide accurate countermeasures. However, conventional systems have had problems in efficiently collecting, storing, and analyzing this data, and lacked a means to effectively provide feedback to users to help them manage their health sustainably.

[1801] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1802] In this invention, the server includes a device that records an individual's amount of exercise and dietary details, means for receiving the recorded information on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored information and generating a disease risk and countermeasures, means for notifying the device of the generated countermeasures, and means for receiving feedback information from the device and updating the risk assessment and countermeasures. This makes it possible to effectively record an individual's amount of exercise and dietary details, accurately assess disease risk, and provide appropriate countermeasures.

[1803] "Amount of exercise" refers to data related to the physical activity of an individual, and specifically includes the number of steps taken, distance traveled, exercise intensity, and the like.

[1804] "Dietary details" refers to data including the type, amount, calorie, and nutrient information of the food consumed by an individual.

[1805] The "device" is a device for recording exercise and dietary content, and may include an acceleration sensor, a location information system, a camera, an image processing algorithm, and the like.

[1806] An "accelerometer" is a device that detects body movements and acquires acceleration information.

[1807] A "location information system" is a system that uses technology such as GPS to measure an individual's travel distance and location.

[1808] A "camera" is a device for taking pictures of the food.

[1809] An "image processing algorithm" is a computational method for extracting characteristics of food content from captured images and estimating calories and nutrients.

[1810] A "server" is a network computer that receives, preprocesses, and stores data sent from devices, extracts and analyzes features from the data, and generates risk assessments and countermeasures.

[1811] "Preprocessing" refers to processing such as removing noise from received data, filling in missing values, and correcting outliers.

[1812] "Feature extraction" is the process of extracting useful information (e.g., average number of steps, total distance traveled, calories, types of nutrients, etc.) from the stored data.

[1813] "Risk assessment" is the process of analyzing and assessing the risk of developing a disease based on the extracted features.

[1814] "Countermeasures" are specific proposals aimed at improving an individual's health that are generated based on the results of a risk assessment.

[1815] "Notification" refers to the act of transmitting the generated countermeasure plan to the device and informing the user of the information.

[1816] "Feedback" refers to data such as the results, impressions, and implementation status of the user's actions based on the proposed measures.

[1817] "Update" is the process of reassessing risk assessments and countermeasures based on newly acquired data and revising them as necessary.

[1818] The present invention is a system that records the amount of exercise and dietary content of an individual, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments are described below.

[1819] System configuration

[1820] The system consists of the following main components:

[1821] 1. Device worn by the user

[1822] Acceleration sensor: Used to measure movement volume, obtaining step count and acceleration data in real time.

[1823] GPS: Used to measure distance traveled and location.

[1824] Camera: Captures images of food to record eating habits.

[1825] Image processing algorithms are used to extract food content features from the captured images.

[1826] 2. Server

[1827] Data reception and preprocessing unit: Receives data sent from the terminal and performs noise removal and missing value completion.

[1828] Feature extraction module: Extracts features of exercise and dietary content from the stored data.

[1829] Risk assessment module: assesses disease risk based on extracted features.

[1830] Countermeasure proposal generation module: Generates appropriate countermeasure proposals based on the results of risk assessment.

[1831] Notification module: Sends the generated countermeasures to the device.

[1832] Feedback receiving and updating module: receives feedback data from users and updates risk assessment and countermeasure proposals.

[1833] Feature details

[1834] 1. Collecting exercise data

[1835] Users wear a device with a built-in accelerometer and GPS on a daily basis. This device collects exercise data (number of steps, distance traveled, etc.) in real time. For example, when you start jogging, the device immediately starts recording this data.

[1836] 2. Collection of dietary data

[1837] When a user eats, they take a photo of their meal with the device's camera. This image data is analyzed by the device to identify the meal contents (e.g., type of sandwich, amount, calories, etc.).

[1838] 3. Data Transmission

[1839] The device sends the collected data to the server at regular intervals, using Wi-Fi or mobile data communication.

[1840] 4. Data Preprocessing

[1841] The server preprocesses the received data, specifically removing noise, imputing missing values, and detecting and correcting outliers.

[1842] 5. Data feature extraction

[1843] The server's feature extraction module extracts useful information from the pre-processed data, such as average steps, total distance traveled, calories, types of nutrients, etc. Image recognition algorithms and machine learning models are used in this process.

[1844] 6. Risk Assessment

[1845] The server's risk assessment module assesses disease risk based on the extracted features. For example, diabetes risk is assessed based on factors such as lack of exercise and frequency of high-calorie food intake.

[1846] 7. Generation and notification of countermeasures

[1847] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. The generated countermeasures include exercise plans and dietary restriction suggestions. For example, specific suggestions such as "jogging three times a week" or "reducing carbohydrate intake by 50%" are made. These countermeasures are notified to the device.

[1848] 8. Collecting and Updating Feedback

[1849] The user records the results and impressions of actions taken based on the proposed countermeasures on the device. This feedback data is then sent back to the server and used to evaluate risks and update the proposed countermeasures.

[1850] Specific examples

[1851] 1. Jogging and Lunch Log

[1852] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[1853] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[1854] 2. Data transmission and storage

[1855] The device sends the collected jogging data and meal images to a server.

[1856] The server receives the data and stores it in a database.

[1857] 3. Feature Extraction and Risk Assessment

[1858] The server extracts information about the amount of exercise (number of steps, distance traveled) and calories in the sandwich.

[1859] The server uses this data to assess diabetes risk.

[1860] 4. Countermeasures generation and notification

[1861] The server generates suggested measures such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[1862] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1863] 5. Feedback and solution updates

[1864] The user implements the measures and records the results on the device.

[1865] The terminal transmits the feedback data to the server again.

[1866] The server updates the risk assessment and countermeasures based on the received data.

[1867] Prompt Sentence Examples

[1868] Below are some example prompts to input to a generative AI model:

[1869] Please explain the system flow for recording an individual's exercise volume and dietary habits, analyzing that data to assess disease risk, and generating appropriate countermeasures. Please provide a detailed explanation, including the names of the specific hardware and software used and how the data is processed. Please also provide examples.

[1870] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1871] Step 1:

[1872] The user wears the device.

[1873] Input: The device's accelerometer and GPS are initialized and ready to go.

[1874] Operation: The device detects the user's movements (steps and vibrations) using an accelerometer and acquires location information using GPS. Each measurement data is recorded internally along with the time.

[1875] Output: Real-time updated exercise data (number of steps, distance traveled, exercise intensity).

[1876] Step 2:

[1877] The user takes a photo of the meal.

[1878] Input: The user activates the device's camera and captures an image of the food.

[1879] How it works: Image data captured by the camera is temporarily stored on the device. Optionally, users have the option to manually enter meal details.

[1880] Output: Saved meal image data or manually entered meal content data.

[1881] Step 3:

[1882] The device sends the data to the server.

[1883] Input: Exercise data and dietary data stored in the device at regular intervals (for example, once a day).

[1884] How it works: Your device uses Wi-Fi or mobile data to upload data to a server, which may compress the data before sending it.

[1885] Output: Compressed exercise data and compressed dietary data sent to the server.

[1886] Step 4:

[1887] The server pre-processes the received data.

[1888] Input: Compressed exercise data and compressed dietary data sent from the terminal.

[1889] How it works: The server decompresses the data, removes noise, imputes missing values, and detects and corrects outliers.

[1890] Output: Clean, pre-processed exercise and diet data.

[1891] Step 5:

[1892] The server performs feature extraction.

[1893] Input: Preprocessed physical activity data and dietary data.

[1894] How it works: The server's machine learning algorithm extracts average steps, total distance traveled, and exercise intensity from exercise data, and calorie and nutrient information from dietary image data.

[1895] Output: Extracted feature data of exercise amount and feature data of dietary content.

[1896] Step 6:

[1897] The server assesses the disease risk.

[1898] Input: Extracted feature data of exercise amount and feature data of dietary content.

[1899] How it works: Risk assessment algorithms on the server calculate your risk of certain diseases, such as diabetes or heart disease, using statistical methods and machine learning models.

[1900] Output: Assessed disease risk data.

[1901] Step 7:

[1902] The server generates a countermeasure plan.

[1903] Input: Assessed disease risk data.

[1904] How it works: The server generates countermeasures based on the risk assessment results, such as suggesting exercise plans or dietary restrictions.

[1905] Output: Generated countermeasure proposal data.

[1906] Step 8:

[1907] The server will notify you of the proposed solution.

[1908] Input: Generated countermeasure proposal data.

[1909] Operation: The server sends the countermeasure data to the device, which then notifies the user via push notification or email.

[1910] Output: Notification of proposed measures displayed on the user's device.

[1911] Step 9:

[1912] The user records feedback.

[1913] Input: Results and impressions of users who have implemented measures.

[1914] How it works: The user enters their status after implementing the measures into the device, which also sends periodic reminders to record compliance.

[1915] Output: Recorded feedback data.

[1916] Step 10:

[1917] The terminal transmits the feedback data to the server.

[1918] Input: Feedback data entered by the user.

[1919] Operation: The terminal sends the collected feedback data to the server at regular intervals.

[1920] Output: Feedback data sent to the server.

[1921] Step 11:

[1922] The server updates the risk assessment and countermeasures.

[1923] Input: Feedback data, past exercise data, and dietary data.

[1924] Actions: The server re-performs risk assessment based on new data and updates its countermeasure recommendations, which may include retraining machine learning models.

[1925] Output: Updated risk assessment and proposed countermeasures.

[1926] As described above, the program of this system records the amount of exercise and diet of an individual in detail, evaluates the risk of disease based on this information, and provides appropriate countermeasures, allowing users to easily manage their health in their daily lives.

[1927] (Application example 1)

[1928] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1929] While managing personal health is important, there is a lack of systems in brick-and-mortar stores that can properly record customers' exercise and dietary habits and manage their health based on that information. Furthermore, there are insufficient means to provide risk assessments and health management recommendations based on this data. Therefore, there is a need for specific methods and systems to effectively support health management in brick-and-mortar stores.

[1930] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1931] In this invention, the server includes a terminal that records an individual's amount of exercise and dietary details, means for receiving the recorded data on the amount of exercise and dietary details and performing preprocessing and storage, means for extracting features from the stored data and generating risks and countermeasures for each disease, means for notifying the terminal of the generated countermeasures, means for receiving feedback data from the terminal and updating the risk assessment and countermeasures, and means for providing menu suggestions and health management advice based on the customer's health condition in a physical store. This enables health management based on the customer's amount of exercise and dietary details in a physical store.

[1932] "Individual exercise amount" is data that indicates the level of physical activity that an individual engages in on a daily basis, such as the number of steps taken, distance traveled, and exercise intensity.

[1933] "Dietary details" refers to data including the type, amount, nutrients, calories, etc. of food and beverages consumed.

[1934] A "terminal" is a device that records an individual's amount of exercise and dietary details and transmits the collected data to a server using a communication means.

[1935] The "server" is a computer system that receives the recorded data and performs preprocessing, storage, feature extraction, risk assessment, and countermeasure generation.

[1936] "Preprocessing" refers to data processing performed before analysis, such as removing noise from data and filling in missing values.

[1937] "Storage" means recording the preprocessed data in a database in the server.

[1938] "Feature extraction" is the process of extracting necessary information from exercise and dietary data and putting it into an analyzable form.

[1939] "Risk assessment" is the process of calculating and assessing the risk of developing a particular disease based on the extracted features.

[1940] "Countermeasures" are specific action plans or proposals based on the results of risk assessment, aimed at improving health and preventing disease.

[1941] "Notification" refers to the act of sending the generated countermeasure plan to the terminal and notifying the user.

[1942] "Feedback data" refers to data that includes the results and impressions of users who have taken actions based on the proposed measures.

[1943] A "physical store" is a physical business establishment, such as a cafe or restaurant, where consumers visit in person to receive services.

[1944] "Menu suggestions" refers to recommending appropriate meals and drinks based on the customer's health condition.

[1945] "Health management advice" refers to specific action plans and lifestyle guidance to improve a customer's health.

[1946] This paper describes a system that records an individual's exercise and dietary habits, analyzes the data, evaluates the risk of illness, and provides appropriate countermeasures. Specifically, this system is designed to provide menu suggestions and health management advice based on the customer's health status in a brick-and-mortar store.

[1947] Recording exercise volume

[1948] A user uses the device of the present invention on a daily basis to record the amount of exercise. This device has a built-in acceleration sensor and GPS, which collect data such as the user's number of steps, distance traveled, and exercise intensity. For example, when a user goes jogging, the device uses the GPS to measure the distance traveled and the acceleration sensor to record the number of steps and exercise intensity.

[1949] Recording dietary information

[1950] When a user eats at a physical restaurant, they take pictures of their meal using the camera attached to their device. The images taken by the camera are temporarily stored on the device, and this data becomes important information for later analysis of the meal contents. For example, if a user orders a salad at a cafe, they take a picture of the salad with the camera and save it on their device.

[1951] Data transmission and storage practices

[1952] The device periodically transmits the collected exercise data and meal content data to the server. The server receives this data and stores it in a database. Preprocessing of the data involves noise removal and missing value completion.

[1953] Performing feature extraction and analysis

[1954] The server extracts characteristics of exercise volume and dietary content from the stored data. Average steps, distance traveled, and exercise intensity are calculated from the exercise data, and calories and types of nutrients are identified from dietary image data. Machine learning algorithms are used to extract these characteristics.

[1955] Conducting risk assessments

[1956] The server then assesses disease risk based on the extracted features. For example, to assess diabetes risk, it considers factors such as lack of exercise and frequency of high-calorie food intake. This risk assessment is performed using statistical methods and machine learning algorithms.

[1957] Generate countermeasures and notify

[1958] Based on the results of the risk assessment, the server generates appropriate measures for the user. The generated measures include suggested exercise plans and restrictions on the intake of certain foods. For example, a user at high risk of diabetes may receive specific suggestions such as "jogging three times a week" and "reducing carbohydrate intake by 50%." These measures are then notified to the device.

[1959] Gathering feedback and updating countermeasures

[1960] The user's actions based on the proposed countermeasures and their impressions are recorded on the device. The device then sends the feedback data back to the server, which receives and stores it. The server then updates the risk assessment and proposed countermeasures based on the new data and notifies the user again.

[1961] Specific examples

[1962] Jogging and cafe meal record:

[1963] A user goes jogging in the morning, and the device records the number of steps and distance traveled using the accelerometer and GPS.

[1964] A user orders a salad at a cafe and takes a picture of the salad with the camera on the device.

[1965] Data transmission and storage:

[1966] The device sends the collected jogging data and meal images to a server.

[1967] The server receives the data and stores it in a database.

[1968] Feature extraction and risk assessment:

[1969] The server extracts information on the amount of exercise (number of steps, distance traveled) and calories in the salad.

[1970] The server uses this data to assess diabetes risk.

[1971] Countermeasure generation and notification:

[1972] The server generates recommendations such as "jogging at least three times a week" and "low-carb menus."

[1973] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[1974] Feedback and solution update:

[1975] The user implements the measures and records the results on the device.

[1976] The terminal transmits the feedback data to the server again.

[1977] The server updates the risk assessment and countermeasures based on the received data.

[1978] Prompt Sentence Examples

[1979] "The customer entered the number of steps and distance they jogged, as well as a photo of their meal (salad) at the cafe, into the app. Based on this data, the app assesses their health risks and generates optimal health management advice."

[1980] The present invention makes it possible to easily provide health management based on the amount of exercise and dietary content of users even in brick-and-mortar stores, thereby integrating the provision of health services and customer health management in brick-and-mortar stores.

[1981] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1982] Step 1:

[1983] When a user exercises, the device uses an acceleration sensor and GPS to collect exercise data. The input is the user's exercise status (number of steps, distance traveled, exercise intensity), and the device measures data in real time based on this. The output is the exercise data obtained as a result of the measurement.

[1984] Step 2:

[1985] When a user eats at a physical restaurant, they take a photo of their meal using the camera on their device. The input is the photo of the meal, and the device acquires the data by temporarily saving this image. The output is the saved meal image data.

[1986] Step 3:

[1987] The device periodically transmits the collected exercise data and dietary image data to the server. The input is the collected exercise data and dietary data, and the device uploads this data to the server. The output is the data transmitted to the server.

[1988] Step 4:

[1989] The server stores the received exercise data and dietary image data. The input is the exercise data and dietary data sent from the device, which the server records in a database and performs data preprocessing such as noise removal and missing value completion. The output is the preprocessed data.

[1990] Step 5:

[1991] The server extracts features from the preprocessed data. The input is the preprocessed exercise data and dietary data, and the server extracts feature information such as the number of steps, distance traveled, exercise intensity, and dietary calories and nutrients from these data. The output is the extracted feature data.

[1992] Step 6:

[1993] The server evaluates disease risk based on the extracted feature data. The input is the feature data, and the server performs risk assessment using machine learning algorithms and statistical methods. For example, this includes assessing diabetes risk. The output is the risk assessment result.

[1994] Step 7:

[1995] The server generates countermeasures based on the risk assessment results. The input is the risk assessment results, and the server creates countermeasures such as specific exercise plans and dietary restrictions. The output is the generated countermeasures.

[1996] Step 8:

[1997] The generated countermeasures are notified to the terminal by the server. The input is the generated countermeasures, which the server sends to the terminal. The output is the countermeasures notified to the terminal.

[1998] Step 9:

[1999] The user implements the proposed measures and records the results and their impressions on the device. The input is the user's action results and feedback, which the device records as data. The output is the recorded feedback data.

[2000] Step 10:

[2001] The terminal sends the collected feedback data back to the server. The input is the recorded feedback data that the terminal uploads to the server. The output is the feedback data sent to the server.

[2002] Step 11:

[2003] The server updates the risk assessment and countermeasures based on the feedback data. The input is feedback data from the user, and the server reassessssments and updates the countermeasures based on this. The output is the updated risk assessment and countermeasures.

[2004] The above processing steps enable health management based on the user's exercise volume and dietary content even in brick-and-mortar stores, making it possible to integrate health service provision in brick-and-mortar stores with customer health management.

[2005] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2006] The present invention combines a system that records an individual's amount of exercise and dietary details, assesses disease risk based on this data, and provides appropriate countermeasures with an emotion engine that recognizes the user's emotions. This makes it possible to provide more personalized countermeasures that take into account the user's subjective state. As an embodiment of the present invention, a program is generated as follows, and its specific processing is described below.

[2007] Emotional data recording

[2008] A user wears the device of the present invention in their daily life. This device is equipped with a camera, microphone, and sensors, which are used by an emotion engine to analyze the user's facial expressions, tone of voice, and even biosignal data (heart rate, electrodermal activity, etc.) to recognize the user's emotions and generate emotion data. For example, when the user smiles or feels stressed, this emotion data is recorded in real time.

[2009] Data transmission and storage practices

[2010] The device periodically compiles exercise data, food image data, and emotion data into packets, which are then sent to a server via Wi-Fi or a mobile network. The server then stores the received data in a database.

[2011] Performing feature extraction and analysis

[2012] The server extracts various features from the stored data. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from the dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from the emotion data.

[2013] Conducting risk assessments

[2014] The server then assesses disease risk based on the extracted features, taking into account emotional data to assess, for example, the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[2015] Generate countermeasures and notify

[2016] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server can add stress management and relaxation methods to the traditional suggestions of exercise and diet. For example, if stress levels are high, the server can generate countermeasures such as "jogging three times a week" and "yoga for relaxation." These countermeasures are then sent to the device, where the user can confirm them.

[2017] Collecting feedback and updating countermeasures

[2018] The user begins taking action based on the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server. The server then updates the risk assessment and countermeasures based on the newly received data. By continuously monitoring the emotional data, the system can provide the user with even more personalized countermeasures.

[2019] Specific examples

[2020] 1. Jogging and Lunch Log:

[2021] The user goes for a morning jog, and the device uses the accelerometer and GPS to record the number of steps and distance traveled.

[2022] A user eats a sandwich for lunch and takes a picture of the sandwich with the device's camera.

[2023] If a user smiles or feels stressed while eating, the emotion engine will recognize this and record the emotion data.

[2024] 2. Data transmission and storage:

[2025] The device sends the collected jogging data, meal images, and emotional data to a server.

[2026] The server receives this data and stores it in a database.

[2027] 3. Feature extraction and risk assessment:

[2028] The server extracts the amount of exercise (number of steps, distance traveled), calorie information of the sandwich, and emotional data (stress level, etc.).

[2029] The server uses this data to assess diabetes risk.

[2030] 4. Countermeasure generation and notification:

[2031] Based on the risk assessment, the server generates suggested countermeasures such as "jogging at least three times a week," "reducing carbohydrate intake by 50%," and "recommending yoga for relaxation."

[2032] The server sends the proposed measures to the terminal, and the terminal notifies the user.

[2033] 5. Feedback and solution update:

[2034] The user implements the measures and records the results and impressions on the device.

[2035] The terminal transmits the feedback data to the server again.

[2036] The server updates the risk assessment and countermeasures based on new data, and also takes emotional data into account to adjust the countermeasures.

[2037] This allows the system to more personalize the user's health management and take emotional state into account to help with comprehensive disease prevention and health maintenance.

[2038] The processing flow will be explained below.

[2039] Step 1:

[2040] The user begins exercising, for example, jogging or walking.

[2041] Step 2:

[2042] The device uses a built-in accelerometer and GPS to record the user's exercise data in real time, specifically measuring the number of steps taken, distance traveled, exercise time, and calories burned.

[2043] Step 3:

[2044] When a user eats a meal, the user takes an image of the meal using the device's camera. For example, if the user eats a sandwich for lunch, the user takes a picture of the sandwich.

[2045] Step 4:

[2046] The device uses a built-in emotion engine to recognize the user's emotions. Emotional data is generated based on facial expression recognition, voice analysis, and biometric data (heart rate, electrodermal activity, etc.). For example, if the user smiles or feels stressed, that emotion is recorded.

[2047] Step 5:

[2048] The terminal periodically assembles the exercise data, meal image data, and emotion data into packets.

[2049] Step 6:

[2050] The device uses Wi-Fi or a mobile communication network to send exercise data, food image data, and emotional data to a server.

[2051] Step 7:

[2052] The server stores the received exercise data, meal image data, and emotion data in a database.

[2053] Step 8:

[2054] The server preprocesses the stored data, removing noise and filling in missing values, including normalizing exercise data and adjusting the resolution of food images.

[2055] Step 9:

[2056] The server extracts features from the preprocessed data. From the exercise data, it calculates the average number of steps, distance traveled, and exercise intensity. From the food image data, it identifies calories and types of nutrients. From the emotion data, it extracts emotional states such as joy, sadness, and stress.

[2057] Step 10:

[2058] Based on the extracted features, the server uses machine learning algorithms and statistical models to assess disease risk, taking into account, for example, lack of exercise, frequency of high-calorie meals, and stress levels to determine diabetes risk.

[2059] Step 11:

[2060] The server generates countermeasures based on the results of the risk assessment, such as specific suggestions like "jogging three times a week is recommended," "reducing carbohydrate intake by 50%," or "yoga is recommended for relaxation."

[2061] Step 12:

[2062] The server sends the generated countermeasures to the terminal.

[2063] Step 13:

[2064] The device receives the proposed measures and notifies the user using the screen display, alerts, and reminder functions.

[2065] Step 14:

[2066] The user begins to take action based on the proposed measures, for example, by jogging according to the recommended exercise plan.

[2067] Step 15:

[2068] After implementing the proposed measures, the user records the results, impressions, and emotional state on the device.

[2069] Step 16:

[2070] The device sends feedback data (amount of exercise, dietary content, emotional state) to a server.

[2071] Step 17:

[2072] The server receives and stores the feedback data.

[2073] Step 18:

[2074] The server updates the risk assessment and countermeasures based on the new data. If necessary, it adjusts the countermeasures and sends them back to the device. It also takes emotional data into account to make the countermeasures more personalized.

[2075] This allows the system to continuously support the user's health management and achieve comprehensive disease prevention that also takes emotional state into account.

[2076] Example 2

[2077] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2078] In modern health management systems, providing more personalized health measures by taking into account not only the amount of exercise and dietary content but also the user's emotional data is a challenge. In particular, there is a need to properly evaluate the impact of stress and emotional state on health risks and to develop appropriate measures. However, conventional systems have difficulty in comprehensively handling these factors, making it difficult to provide users with appropriate measures.

[2079] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2080] In this invention, the server includes: a means for recording the amount of exercise, dietary content, and emotion data of an individual;

[2081] A means for receiving, preprocessing and storing the recorded exercise amount, dietary content and emotion data;

[2082] A means for extracting features from the stored data, assessing the risk of each disease, and generating countermeasures that take into account the emotional data;

[2083] a means for notifying the terminal of the generated countermeasures;

[2084] means for receiving feedback data from the terminal and updating the risk assessment and countermeasures;

[2085] This allows for more personalized health recommendations that take into account the user's emotional state.

[2086] "Amount of exercise" refers to the amount of physical activity an individual engages in over a certain period of time, and includes the number of steps taken, distance traveled, exercise intensity, etc.

[2087] "Dietary content" refers to the types, amounts, and nutrients of the foods and beverages consumed by an individual, including information on calories and nutritional components.

[2088] "Emotional data" refers to information about a user's emotional state obtained by analyzing their facial expressions, tone of voice, biometric signals, etc.

[2089] "Device" refers to an electronic device worn by an individual to record exercise volume, dietary content, emotional data, etc.

[2090] "Server" refers to a computer system that receives data from terminals, stores it, analyzes it, evaluates risks, and generates countermeasures.

[2091] "Feature extraction" refers to the process of extracting important information or patterns from stored data, such as the average number of steps taken or calorie information for meals.

[2092] "Risk assessment" refers to the process of calculating individual health risks based on collected data and planning appropriate countermeasures.

[2093] "Countermeasures" refers to plans or proposals that recommend individual health management methods or actions, generated based on the results of a risk assessment.

[2094] "Feedback data" refers to data that records the results and impressions of users when they perform an action.

[2095] "Preprocessing" refers to the process of converting received data into a suitable format for analysis and storage.

[2096] The present invention is a system that comprehensively monitors and analyzes an individual's exercise amount, dietary content, and emotional data, and provides the user with appropriate health measures. Specific embodiments for carrying out the present invention will be described in detail below.

[2097] System Overview

[2098] This system consists of a device worn by the user and a server that collects, analyzes, and stores data, and generates and notifies users of countermeasures. The device has built-in hardware for data collection, and the server has hardware and software for data processing.

[2099] Hardware and Software Configuration

[2100] Terminal

[2101] Camera: Takes pictures of the user's meal and collects image data.

[2102] Microphone: Collects the user's tone of voice to help with sentiment data.

[2103] Acceleration sensor: Measures the user's exercise volume (number of steps, distance traveled, and exercise intensity).

[2104] Location information acquisition device (GPS): Measures the user's travel distance.

[2105] Heart rate sensor: Measures the user's heart rate and collects biometric data.

[2106] Electrodermal activity sensor: Measures the user's electrodermal activity and estimates stress levels.

[2107] server

[2108] Database (e.g., Amazon Web Services RDS): Stores received exercise, dietary, and emotion data.

[2109] Machine learning algorithms (e.g., TensorFlow): Analyze collected food image data and identify calories and types of nutrients.

[2110] Python libraries (e.g., Pandas, scikit-learn): Extract features from stored data and perform risk assessment.

[2111] Emotion Engine: Analyzes collected data and recognizes the user's emotional state.

[2112] System Operation

[2113] Data collection

[2114] The user wears the device in their daily life, and the device's built-in sensors collect real-time data on the user's exercise, diet, and emotions. Specifically, the camera captures images of meals, the microphone collects voice tone, the accelerometer and GPS measure exercise, and the heart rate sensor and electrodermal activity sensor collect biometric data.

[2115] Data transmission and storage

[2116] The device collects data at regular intervals, compiles it into packets, and sends them to a server via Wi-Fi or a mobile network. The server stores the received data in a database for subsequent analysis.

[2117] Data analysis and feature extraction

[2118] The server analyzes the stored data and extracts various features, using machine learning algorithms to identify calories and nutrients from food images, and Python libraries to analyze exercise volume and emotional data.

[2119] Risk assessment and countermeasure generation

[2120] The server performs a risk assessment based on the extracted feature data. It also includes emotional data in the assessment and calculates the overall health risk. It then generates optimal countermeasures based on the risk and notifies the user. These countermeasures include specific recommendations such as "jogging at least three times a week" and "reducing carbohydrate intake by 50%."

[2121] Gathering feedback and updating countermeasures

[2122] The user acts according to the proposed countermeasures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and countermeasures based on the new data. This makes it possible to provide more personalized countermeasures that take the user's emotional state into account.

[2123] Examples and prompts

[2124] Examples:

[2125] When a user goes for a morning jog, the device uses the accelerometer and GPS to record the number of steps and distance traveled. When the user eats a sandwich for lunch, the device's camera takes a photo of the meal. If the user smiles while eating, the emotion engine recognizes this and records emotional data.

[2126] Example prompt sentence:

[2127] How is data transmitted and stored?

[2128] "Give me an example of user emotion data."

[2129] "How do I generate personalized countermeasures?"

[2130] In this way, the present invention comprehensively supports the user's health management and, by taking into account the user's emotional state, can provide more accurate health measures.

[2131] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2132] Step 1:

[2133] Emotional data recording

[2134] The user wears the device. The device uses a camera, microphone, heart rate sensor, and electrodermal activity sensor to collect the user's facial expressions, tone of voice, and biosignal data. This data is analyzed in real time by the emotion engine to generate emotion data. Specifically, when the user smiles, the emotion engine generates emotion data for "happiness."

[2135] Input: User's facial expressions, audio, and biometric signals

[2136] Output: Emotion data (e.g., happiness, stress)

[2137] Step 2:

[2138] Recording exercise and food data

[2139] When users exercise in their daily lives, the device's accelerometer and GPS record the number of steps taken and the distance traveled. When users eat, the device's camera captures photos of their meals and collects dietary data.

[2140] Input: Steps, distance traveled, food images

[2141] Output: Exercise data (e.g., steps, distance traveled), food data (e.g., food images)

[2142] Step 3:

[2143] Data transmission and storage

[2144] At regular intervals, the device collects exercise data, dietary data, and emotional data and sends them in packets to a server via Wi-Fi or a mobile network. The server analyzes the received data and stores it in a database.

[2145] Input: Packetized exercise data, food data, and emotion data

[2146] Output: Various data stored in the database

[2147] Step 4:

[2148] Feature Extraction and Analysis

[2149] The server analyzes the stored data and extracts features. First, it uses Python's Pandas to calculate the average number of steps, distance traveled, and exercise intensity from the exercise data. Next, it uses machine learning algorithms (e.g., TensorFlow) to identify calories and types of nutrients from food images. Finally, it uses an emotion engine to analyze the emotion data.

[2150] Input: Stored exercise data, diet data, and emotional data

[2151] Output: Extracted feature data (e.g., average steps, calories, emotional state)

[2152] Step 5:

[2153] Risk Assessment

[2154] The server applies a risk assessment model based on the feature data to objectively calculate the risk of disease. For example, when assessing the risk of diabetes, it uses a machine learning model (using scikit-learn) to determine that lack of exercise and high stress are high-risk factors.

[2155] Input: Extracted feature data

[2156] Output: Risk assessment results (e.g., diabetes risk score)

[2157] Step 6:

[2158] Generation and notification of countermeasures

[2159] Based on the results of the risk assessment, the server generates optimal countermeasures for the user. For example, if the risk assessment diagnoses high stress, the server will generate specific countermeasures such as "jogging three times a week" and "recommended yoga for relaxation." The server notifies the device of the countermeasures and informs the user.

[2160] Input: Risk assessment results

[2161] Output: suggested action (e.g. jogging, yoga)

[2162] Step 7:

[2163] Collect feedback and update countermeasures

[2164] The user acts on the proposed measures and records their results and impressions on the device. The device then sends this feedback data back to the server, which then updates the risk assessment and proposed measures based on the newly received data. This allows for more personalized proposals that take the user's emotional state into account.

[2165] Input: User feedback data

[2166] Output: Updated risk assessment and proposed countermeasures

[2167] (Application example 2)

[2168] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2169] In today's busy lifestyles, personal health management is extremely important, but conventional systems only record the amount of exercise and dietary content and evaluate disease risk based on this, so they are unable to take measures that take into account the user's emotions and stress level. As a result, individualized and effective health management is difficult, making it difficult to maintain employee performance and reduce security risks, especially in high-stress environments.

[2170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2171] In this invention, the server includes a means for generating emotion data using a camera, microphone, and sensor, a means for performing risk assessment including the emotion data and generating countermeasures, and a means for notifying the terminal of the generated countermeasures, thereby enabling personalized health management and reducing security risks in high-stress environments.

[2172] "Amount of exercise" is an index that indicates the amount of energy and intensity that an individual expends in daily life and physical activity.

[2173] "Dietary content" is data that indicates the types and amounts of food and beverages consumed by an individual, as well as information on the nutrients contained therein.

[2174] A "terminal" is a device that can be worn or carried by an individual and records exercise, dietary habits, and even emotional data.

[2175] "Server" refers to the central processing unit that receives the recorded data, performs pre-processing and storage, extracts features, and generates risk assessments and countermeasures.

[2176] "Preprocessing" refers to the initial processing of data, such as organizing, standardizing, and filtering, to make the data sent to the server easier to analyze.

[2177] "Storage" refers to keeping data for an extended period of time and making it accessible when needed.

[2178] "Feature extraction" is the process of extracting important information from stored data that is necessary for risk assessment and countermeasure generation.

[2179] "Risk assessment" refers to the objective evaluation of the risk of developing a specific health condition or disease based on extracted feature data.

[2180] "Countermeasures" are proposals that specifically outline preventative and improvement measures that users should take based on the results of risk assessment.

[2181] "Emotion data" is data that indicates the user's emotional state, and is generated based on facial expressions, tone of voice, heart rate, and electrodermal activity.

[2182] An "emotion engine" is software or hardware that analyzes data collected by cameras, microphones, and sensors, recognizes the user's emotions, and generates emotional data.

[2183] "Notification" refers to the act of sending the generated countermeasure plan to the user's terminal so that the user can check it.

[2184] "Feedback data" refers to data that records user actions based on proposed countermeasures, their results, and their impressions.

[2185] This invention is a system that comprehensively records an individual's exercise amount, dietary content, and emotional data, evaluates the risk of illness, and provides appropriate countermeasures. Specific embodiments of this system are described below.

[2186] System configuration

[2187] 1. Terminal

[2188] Hardware:

[2189] Acceleration sensors and position measuring devices

[2190] camera

[2191] microphone

[2192] Sensors (heart rate monitor, electrodermal activity sensor)

[2193] software:

[2194] Emotion recognition engines (e.g., generic image and voice analysis software)

[2195] Data collection and transmission program

[2196] 2. Server

[2197] Hardware:

[2198] Cloud-based central processing units (e.g., general-purpose server equipment from cloud providers)

[2199] software:

[2200] Machine learning algorithms (e.g., general-purpose machine learning frameworks)

[2201] Databases (e.g. cloud provider databases)

[2202] Data analysis platform (e.g., general-purpose data analysis tool)

[2203] Notification systems (e.g., general-purpose notification services)

[2204] Program processing

[2205] The device is worn by individuals and collects data on the amount of exercise, dietary habits, and emotions in daily life. Data is recorded in real time using a camera, microphone, and sensors, and emotion data is generated using an emotion engine. This allows data to be obtained that takes into account the user's subjective state.

[2206] The device periodically collects data and sends it to a server via Wi-Fi or a mobile network. The server receives the data and stores it in a database.

[2207] Once the data is saved, the server extracts features based on it. Specifically, it calculates the average number of steps, distance traveled, and exercise intensity from the exercise data, and identifies calories and types of nutrients from dietary image data. It also extracts the user's emotional state (e.g., joy, sadness, stress, etc.) from emotion data.

[2208] The server then assesses disease risk based on the extracted features, taking into account emotional data, for example, to assess the impact of stress on diabetes risk. This assessment is performed objectively using machine learning algorithms and statistical models.

[2209] Based on the results of the risk assessment, the server generates appropriate countermeasures for the user. By taking emotional data into consideration, the server ca...

Claims

1. a device that records an individual's exercise amount and dietary details; a server that receives, pre-processes, and stores the recorded exercise and diet data; A server that extracts features from the stored data and generates risks and countermeasures for each disease; a means for notifying the terminal of the generated countermeasures; a server that receives feedback data from the terminal and updates risk assessments and countermeasures; A system including:

2. The system according to claim 1, further comprising a terminal equipped with an acceleration sensor and a GPS for recording the amount of exercise.

3. 10. The system of claim 1, further comprising a terminal equipped with a camera and image processing algorithms for recording meal contents.

Images