System

Abstract

A system is provided.SOLUTION: A system comprising means for collecting sleep data of a user, means for collecting daytime behavior data, means for transmitting the collected sleep data and the daytime behavior data to a cloud server, means for preprocessing the received data in the cloud server, means for executing an AI algorithm for evaluating a health condition based on the preprocessed data, means for generating a physical condition evaluation and a recommended behavior based on a result of the AI algorithm, means for transmitting the generated physical condition evaluation and the recommended behavior to a user terminal, and means for notifying the user of the transmitted physical condition evaluation and the recommended behavior.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people suffer from insomnia and poor sleep quality, which can lead to poor daytime health and the risk of lifestyle-related diseases. Insufficient sleep and an unhealthy lifestyle can lead to economic losses and health problems. Against this backdrop, there is a growing need for a system that can effectively collect and analyze individual users' sleep and daytime behavior data, and provide appropriate health assessments and recommended actions.

[0005] However, current methods have difficulty in comprehensively assessing a user's health status in real time and providing specific recommendations based on that assessment, which makes it difficult for users to manage their health themselves and prevents them from maintaining their health in the long term. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides the following means: The present invention includes means for collecting sleep data of a user, means for collecting daytime behavior data, and means for transmitting the data to a cloud server.

[0007] Furthermore, the system includes a means for preprocessing the received data in the cloud server and evaluating the health condition using an AI algorithm, a means for generating a health assessment and recommended actions based on the assessment results, and a means for transmitting the generated health assessment and recommended actions to the user's terminal and notifying the user.

[0008] Specifically, sleep data such as the user's sleep duration, sleep depth, and frequency of turning over in bed are collected, while daytime behavior data such as the number of steps taken, heart rate, activity level, and GPS location information are collected. This makes it possible to comprehensively evaluate each user's health status and provide specific, individually customized recommendations based on that evaluation. Such a system can help reduce the risk of insomnia and lifestyle-related diseases and support users in maintaining their health.

[0009] "User" refers to an individual who uses the System.

[0010] "Sleep data" refers to various physiological data of the user while they are asleep, such as data including the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0011] "Daytime activity data" refers to data related to various activities a user engages in during the day, including, for example, steps taken, heart rate, activity level, and GPS location information.

[0012] "Cloud server" refers to a remote computer system for storing, processing, and analyzing data over the Internet.

[0013] "Preprocessing" refers to the initial processing of collected data, such as cleaning, normalizing, and filtering, that is carried out to improve the quality of the data.

[0014] "AI algorithm" refers to a calculation procedure or model that uses artificial intelligence technology to analyze data.

[0015] "Assessing health status" refers to analyzing the user's current physical and mental health level based on the collected data.

[0016] "Health Assessment" refers to information about a user's health status provided based on pre-processed and analyzed data.

[0017] "Recommended actions" refers to specific actions and lifestyle improvement measures that users should take based on the analysis results.

[0018] "User terminal" refers to a device with communication capabilities, such as a smartphone or tablet, used by a user.

[0019] "Notify" refers to an action taken by the system to inform the user of information, such as a pop-up message, an alert, or a screen display. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention relates to a system that uses a user's sleep data and daytime behavior data to provide an individualized physical condition assessment and recommended actions. This system is mainly composed of a smartphone-type device and a cloud server.

[0042] System Configuration

[0043] 1. Smartphone devices

[0044] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0045] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[0046] (Terminal) Sends collected data to the cloud server at specified intervals.

[0047] 2. Cloud Server

[0048] (Server) Receives data sent from the smartphone device and stores it in a database.

[0049] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[0050] (Server) Runs AI algorithms using preprocessed data to assess the user's health status.

[0051] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[0052] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[0053] Program processing

[0054] Data collection

[0055] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[0056] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[0057] (Device) Collected data is sent to the cloud server at regular intervals.

[0058] Data analysis

[0059] (Server) The cloud server stores the received sleep data and daytime behavior data in a database.

[0060] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[0061] (Server) The pre-processed data is input into an AI algorithm to comprehensively assess the user's health status, including the user's sleep quality, fatigue level, and lifestyle risks.

[0062] (Server) Based on the evaluation results, a physical condition assessment and recommended actions for the next day (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[0063] Result notification

[0064] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[0065] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[0066] (User) The user can check the health assessment and recommended actions and adjust their actions based on the advice.

[0067] For example, if a user has slept less than usual, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a short break during the day." As a result, users can receive advice optimized for their individual health condition and improve their lifestyle habits.

[0068] The processing flow will be explained below.

[0069] Step 1: Start collecting data

[0070] (Device) When the user starts going to bed, the device begins collecting sleep data using its built-in sensors (accelerometer, heart rate sensor, etc.).

[0071] (Device) The user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting user inactivity.

[0072] Step 2: Collecting daytime behavioral data

[0073] (Device) When you wake up in the morning, sleep data collection will stop based on the user's manual input or a designed pattern.

[0074] (Device) When the user begins their activities, data on their daily activities such as number of steps, heart rate, amount of exercise, and GPS location information is collected.

[0075] Step 3: Sending data

[0076] (Device) The collected sleep data and daytime behavior data are sent to the cloud server in real time or at specified intervals.

[0077] (Terminal) After data transmission is complete, the terminal notifies the server of this fact.

[0078] Step 4: Receiving and storing data

[0079] (Server) Receives data sent from the device and stores it securely in a database, where it is identified and classified for each user.

[0080] Step 5: Preprocessing the data

[0081] (Server) Performs preprocessing such as noise removal, missing value imputation, and normalization on the received data, enabling accurate and consistent analysis.

[0082] Step 6: AI analysis

[0083] (Server) The preprocessed data is input into an AI model for analysis, which evaluates the user's sleep quality, daytime behavior patterns, risk of lifestyle-related diseases, etc.

[0084] (Server) As a result of the analysis, a detailed assessment of the user's health status is generated.

[0085] Step 7: Generate recommended actions

[0086] (Server) Based on the analysis results, specific recommended actions are generated, such as "go to bed early," "get moderate exercise," and "eat a specific diet."

[0087] (Server) Create a package that summarizes recommended actions and health assessments.

[0088] Step 8: Sending the results

[0089] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[0090] (Terminal) Properly receives the transmitted data and prepares it for presentation to the user.

[0091] Step 9: Notify users

[0092] The device application notifies the user of the received health assessment and recommended actions via pop-ups, alerts, and in-app dashboard displays.

[0093] (User) The user checks the notification and follows the recommended actions presented to improve their health.

[0094] Through this series of steps, users can receive an individually customized health assessment and recommended actions, which they can then incorporate into their daily lives to effectively manage their health.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] In recent years, with growing interest in health management, highly accurate data collection and analysis are required to provide specific recommended actions based on individual health conditions. However, conventional systems have had issues in providing appropriate advice to users due to limited types of collected data and insufficient analysis accuracy. The object of the present invention is to provide a system that collects a variety of biometric and activity data from users and analyzes them using advanced artificial intelligence to provide specific health assessments and recommended actions based on individual health conditions.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for collecting a user's biometric data, means for collecting daytime activity data, means for transmitting the collected biometric data and activity data to a network server, means for preprocessing the data received by the network server, means for executing artificial intelligence for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the artificial intelligence, means for transmitting the generated health condition assessment and recommended actions to a user terminal, and means for notifying the user of the transmitted health condition assessment and recommended actions. This makes it possible to collect and analyze a variety of user data with high accuracy and provide specific advice based on an individual's health condition.

[0100] "Biometric data" refers to information about the user's physical condition, specifically including the amount of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0101] "Activity data" refers to information about a user's daily activities, and specifically includes the number of steps taken, heart rate, amount of exercise, location information, etc.

[0102] "Network server" refers to a server that receives, stores, and processes data sent from a user terminal.

[0103] "Preprocessing" refers to processing of received data, such as removing noise, filling in missing values, and normalizing data.

[0104] "Artificial intelligence" refers to algorithms that assess a user's health status based on pre-processed data.

[0105] "Health assessment" refers to the results of an assessment of the user's health condition based on the analysis results of artificial intelligence.

[0106] "Recommended actions" refer to actions that are specifically recommended to the user based on the health assessment.

[0107] "User terminal" refers to a device that collects biometric data and activity data and transmits it to a network server, and specifically includes a smartphone.

[0108] "Notification" refers to a means for informing a user of the generated health assessment and recommended actions, including pop-up messages, alerts, dashboard displays, etc.

[0109] The present invention relates to a system that uses a user's biometric data and daily activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone-type device and a cloud server.

[0110] The system configuration includes the following elements:

[0111] Data collection by terminal

[0112] (Device) When the user goes to bed, the smartphone device collects sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, and the frequency of turning over in bed.

[0113] (Device) During daytime activities, data on daily activities such as steps taken, heart rate, exercise volume, and GPS location information is collected using the device's activity tracker and location services.

[0114] (Device) Collected data is sent to the cloud server at regular intervals. The sending interval can be changed in the system settings, but is generally set to every hour.

[0115] Data analysis using a cloud server

[0116] (Server) The cloud server receives biometric and activity data sent from the user's smartphone and stores it in a database. Each data item is time-stamped for later analysis.

[0117] (Server) Performs preprocessing on the received data. Specifically, it performs filtering to remove noise, fills in missing values, and normalizes the data. For example, it filters out abnormal heart rate data and fills in missing data using the historical average.

[0118] (Server) The pre-processed data is used to run an artificial intelligence (AI) algorithm, which then compares the data with previously collected data and general health indicators to comprehensively assess the user's health, including the user's sleep quality, fatigue level, and lifestyle risks.

[0119] (Server) Based on the AI's evaluation results, a physical condition assessment and specific recommended actions for the next day are generated. Specific recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients."

[0120] Result notification

[0121] (Server) The generated health assessment and recommended action plan are sent to the user's smartphone, where the user can then receive push notifications and alerts.

[0122] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The user can check the notification content and follow the recommended actions.

[0123] Specific examples

[0124] If a user sleeps less than usual, the sensor records this information and sends the data to the cloud. The cloud server analyzes the data and generates specific advice, such as "go to bed earlier tonight" or "take a short break during the day." This advice is immediately sent to the user's device and notified to the user's application. The user can then adjust their daily schedule accordingly.

[0125] Prompt Sentence Examples

[0126] You can ask the generative AI model for a health assessment and recommended actions using prompts like the following:

[0127] Use the following sleep and daytime activity data from your users to assess their health and provide specific recommendations.

[0128] (Sleep data)

[0129] Sleep time: 6 hours

[0130] Deep sleep percentage: 20%

[0131] Turning frequency: 15 times

[0132] (Daytime behavior data)

[0133] Steps: 8,000

[0134] Heart rate: 80 BPM average

[0135] Activity level: High

[0136] GPS location: Commute from home to work

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1:

[0139] (Device) When the user goes to bed, the smartphone device begins collecting sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, the frequency of turning over in bed, etc. The input is biosignal data obtained from the sensors, and the output is structured sleep data.

[0140] Step 2:

[0141] When the user wakes up, the device automatically switches modes and starts collecting daily activity data. The collected data includes the number of steps taken by the pedometer, heart rate data from the heart rate sensor, activity level data from the activity tracker, and location data from GPS. The input is signal data obtained from various sensors during the day, and the output is structured activity data.

[0142] Step 3:

[0143] (Device) The collected biometric data and activity data are sent to the cloud server at regular intervals (e.g., every hour). The data is encoded and transmitted securely over the network. The input is data obtained from various data collection modules, and the output is the transmitted data arriving at the cloud server.

[0144] Step 4:

[0145] (Server) The cloud server receives the biometric data and activity data sent from the user device and stores them in a database. The input is structured data sent via the network, and the output is the biometric data and activity data stored in the database.

[0146] Step 5:

[0147] (Server) Performs preprocessing on the received data. Preprocessing includes filtering to remove noise, imputing missing values, and normalizing the data. For example, filtering out abnormal heart rate data and imputing missing values ​​using the historical average. The input is raw data, and after preprocessing, clean, normalized data is output.

[0148] Step 6:

[0149] (Server) Using the preprocessed data, an AI algorithm is run to evaluate the user's health status. The AI ​​evaluates the user's health status by comparing it with past data and health indicators. The evaluation includes sleep quality, fatigue level, lifestyle risks, etc. The input is the preprocessed data, and the output is a health status evaluation result.

[0150] Step 7:

[0151] (Server) Based on the AI ​​evaluation results, a health assessment and specific recommended actions for the next day are generated. Recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients." The input is the health assessment results, and the output is a health assessment and a recommended action plan.

[0152] Step 8:

[0153] (Server) The generated health assessment and recommended actions are sent to the user's smartphone. The input is the AI ​​analysis results, and the output is the recommended actions and assessment sent to the user's device.

[0154] Step 9:

[0155] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The input is the recommended action and evaluation data sent from the server, and the output is a visual or audio notification to the user. The user checks the notification and follows the recommended action.

[0156] (Application example 1)

[0157] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] In recent years, there has been growing interest in managing users' health status. However, there are only a limited number of systems that provide specific recommendations based on individual users' health status. Especially for in-store use, there is a need for systems that utilize users' location information to make appropriate recommendations. Furthermore, a mechanism for improving recommendations using user feedback is also needed, but this is similarly lacking. Given this background, there is a need for the development of a system that can provide appropriate in-store recommendations for products and facility use based on the user's health status and collect feedback for further improvement.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0160] In this invention, the server includes means for collecting a user's sleep data, means for collecting daytime behavior data, means for transmitting the collected sleep data and daytime behavior data to a cloud server, means for preprocessing the received data in the cloud server, means for executing an AI algorithm for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the AI ​​algorithm, means for transmitting the generated health condition assessment and recommended actions to a user terminal, means for notifying the user of the transmitted health condition assessment and recommended actions, means for presenting recommended products and facilities suitable for the user using in-store location information, means for encouraging the user to take action based on the recommended products and facilities, and means for collecting feedback and improving the health assessment and recommendations based on the usage history of the recommended products and facilities. This makes it possible to make appropriate recommendations based on the user's health condition, improve the user experience in the physical store, and continuously improve the accuracy and effectiveness of the system using the feedback function.

[0161] "User" refers to an individual who uses the system to manage their own health condition.

[0162] "Sleep data" refers to information collected while a user is asleep, and specifically includes the amount of sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[0163] "Daytime activity data" refers to information about the activities a user engages in during the day, including, for example, the number of steps taken, heart rate, activity level, and GPS location information.

[0164] "Cloud server" refers to a remote server that stores and processes data over the Internet.

[0165] "Preprocessing" refers to the processing that the cloud server performs on the raw data it receives, and includes data cleaning, missing value imputation, normalization, etc.

[0166] "Health Status" refers to the results of an assessment of the user's physical and mental state.

[0167] "AI Algorithm" refers to the artificial intelligence algorithm used to assess a user's health status based on collected data.

[0168] "Recommended actions" refers to specific actions or options suggested to users based on the evaluation results of the AI ​​algorithm.

[0169] "Notification" refers to the act of sending a message or alert to a user's device to convey information to the user.

[0170] "In-store location information" refers to information used to identify a user's current location within a physical store.

[0171] "Recommended Products" refers to products and services suggested to you based on your health status.

[0172] "Facilities" refers to the locations of facilities and services that users are encouraged to use within the physical store.

[0173] "Means to encourage behavior" refers to mechanisms that encourage users to use recommended products or facilities.

[0174] "Feedback" refers to the opinions and ratings provided by users after using the system.

[0175] "Usage history" refers to the history of products and services a user has used in the past and recommended actions.

[0176] This invention relates to a system that uses a user's sleep data and daytime activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone device and a cloud server.

[0177] 1. Smartphone devices

[0178] The smartphone device has the following features:

[0179] Sleep data collection: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0180] Collection of daytime activity data: During daytime activities, we collect daytime activity data such as steps, heart rate, activity level, and GPS location information.

[0181] Data transmission: The collected data is sent to the cloud server at the specified interval.

[0182] 2. Cloud Server

[0183] The cloud server has the following functions:

[0184] Data reception and storage: Receives data sent from the smartphone device and stores it in a database.

[0185] Data preprocessing: Check the received data for noise and missing data, and preprocess it if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[0186] Data analysis: Using the pre-processed data, AI algorithms are run to assess the user's health status, including sleep quality, fatigue level, and lifestyle risks.

[0187] Generation of recommended actions: Based on the assessment results, a physical condition assessment and specific recommended actions based on that assessment (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[0188] Notification of results: The generated physical condition assessment and recommended action plan are sent to the user's device.

[0189] Gathering feedback: Gathering user feedback to improve our ratings and recommendations.

[0190] 3. Use in physical stores

[0191] This system can also be used in physical stores, where it can utilize the user's location information to provide the following services:

[0192] Location information identification: The user's current location is identified using in-store Wi-Fi and Bluetooth beacons.

[0193] Recommended products: Based on the user's health data, the app will suggest suitable products and facilities to use. For example, if you need to relax, it will display recommended menu items at a cafe.

[0194] Action promotion: Send notifications to your smartphone encouraging you to use recommended products or facilities.

[0195] Feedback collection: Collect feedback from users after using the recommended products to help improve the system.

[0196] Hardware and software used

[0197] Smartphone device: Collecting user data and running applications

[0198] Cloud Server: Data analysis and generation of recommended actions

[0199] Data storage and processing: Using AWS (Amazon Web Services) or GCP (Google Cloud Platform)

[0200] Running AI algorithms: using TensorFlow

[0201] Specific examples

[0202] If a user has slept less than usual, the system can analyze the data and provide specific recommendations such as "go to bed earlier tonight" or "take a short break during the day." If the user is in a physical store, the system can determine that they need to relax and recommend a relaxation menu for the cafe.

[0203] Prompt Sentence Examples

[0204] "Please propose a health recommendation application to be provided in physical stores based on the user's sleep data and daytime behavior data. For example, an application that makes recommendations on what items to purchase, which facilities to use, etc. Please also provide detailed information on specific features and how to implement them."

[0205] The above are details of a specific mode for carrying out the invention. The system allows users to receive personalized health advice and improve their lifestyle habits while enhancing their in-store experience.

[0206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0207] Step 1: Data collection

[0208] The device collects the user's sleep data while they sleep. When the user goes to bed, the device's built-in sensors and external sensor devices are activated to collect data such as sleep time, percentage of deep sleep, and frequency of turning over in bed. The collected data is temporarily stored in a format that can be used immediately.

[0209] Step 2: Collecting daytime behavioral data

[0210] The device collects user behavior data during daytime activities. After the user wakes up, the device automatically switches to daytime activity data collection mode. Step count, heart rate, activity level, GPS location information, etc. are collected and temporarily stored.

[0211] Step 3: Send data

[0212] The device sends the collected sleep data and daytime behavior data to a cloud server. At specified intervals, the data is sent in batches to the cloud server for processing on the server side. The data is encrypted before transmission and decrypted upon arrival.

[0213] Step 4: Receiving and storing data

[0214] The server receives the data sent from the device and stores it in a database. To securely store the received data, it uses cloud storage services such as AWS and GCP. The data is then classified by user and prepared for analysis.

[0215] Step 5: Data Preprocessing

[0216] The server preprocesses the data it receives. Preprocessing includes data cleaning, missing value imputation, normalization, etc. This removes noise from the data and organizes it into a consistent format. Libraries such as Pandas are sometimes used for data cleaning.

[0217] Step 6: Data analysis

[0218] The server uses the preprocessed data to run AI algorithms to assess the user's health. Specifically, collected sleep data and daytime behavior data are input into a generative AI model such as TensorFlow to evaluate sleep quality, fatigue level, lifestyle risks, etc.

[0219] Step 7: Generate recommended actions

[0220] Based on the results of the AI ​​algorithm's evaluation, the server generates a physical condition assessment and specific recommended actions, such as performing relaxation exercises, going to bed earlier, and specific dietary advice.

[0221] Step 8: Notification of results

[0222] The server then sends the generated health assessment and recommended actions to the user's device, and notifications are sent in real time as pop-up messages or alerts on the user's smartphone.

[0223] Step 9: Identify your location

[0224] The device identifies the user's location within the physical store, detects the user's current location using Wi-Fi or Bluetooth beacons, and sends the location information to a cloud server.

[0225] Step 10: Recommendations

[0226] The server generates recommended products and facilities based on the user's health data and location information and sends them to the device. For example, a user who needs to relax will be shown recommended menu items at a cafe.

[0227] Step 11: Drive action

[0228] The device will send notifications to the user encouraging them to use the recommended products and facilities. These notifications will be displayed on the user's device and will serve as a guide for actions in the physical store.

[0229] Step 12: Gather feedback

[0230] The server collects user feedback to improve the system's ratings and recommendations. Feedback is entered by users through the application and is reflected in future recommendations.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[0233] System Configuration

[0234] 1. Smartphone devices

[0235] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0236] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[0237] (Device) Equipped with an emotion engine that analyzes the user's voice and facial expressions, it collects emotional data from the user. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[0238] (Terminal) Sends collected data to the cloud server at specified intervals.

[0239] 2. Cloud Server

[0240] (Server) Receives data sent from the smartphone device and stores it in a database.

[0241] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[0242] (Server) Runs AI algorithms using the preprocessed data to assess the user's health, taking into account emotional data.

[0243] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[0244] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[0245] Program processing

[0246] Data collection

[0247] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[0248] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[0249] (Device) During the day, the emotion engine analyzes voice and facial expressions, recognizes the user's emotions in real time, and collects them as data.

[0250] (Device) Collected data is sent to the cloud server at regular intervals.

[0251] Data analysis

[0252] (Server) The cloud server stores the received sleep data, daytime behavior data, and emotion data in a database.

[0253] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[0254] (Server) The pre-processed data is fed into an AI algorithm to provide a comprehensive assessment of the user's health status, including the user's sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[0255] (Server) Adjusts health assessment based on emotional data, for example, if the user is feeling stressed, prioritizes recommended actions to reduce that stress.

[0256] Result notification

[0257] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[0258] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[0259] (User) The user checks the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[0260] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[0261] The processing flow will be explained below.

[0262] Step 1: Start collecting sleep data

[0263] (Device) When the user goes to bed, the device starts collecting sleep data using built-in sensors (accelerometer, heart rate sensor, etc.).

[0264] (Device) A user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting inactivity.

[0265] Step 2: Collecting daytime behavioral data

[0266] (Device) When you wake up in the morning, stop collecting sleep data and switch to daytime activity data collection mode.

[0267] (Device) Measures the user's daytime activity and records steps, heart rate, activity level, and GPS location.

[0268] Step 3: Collecting emotion data

[0269] (Device) The user's voice data and facial expression data are collected during the day using a microphone and camera, and the emotion engine analyzes this data to recognize the user's emotions.

[0270] (Device) Records emotional data at regular intervals during the day and transmits it to the cloud server as needed.

[0271] Step 4: Sending data

[0272] (Device) The collected sleep data, daytime behavior data, and emotional data are sent to a cloud server periodically or in real time.

[0273] Step 5: Receiving and storing data

[0274] (Server) Receives data sent from the device and stores it in a secure database. Data is identified for each user.

[0275] Step 6: Preprocessing the data

[0276] (Server) Preprocessing is performed on the received data, including noise removal, missing value completion, and normalization. This allows for highly accurate analysis.

[0277] Step 7: AI analysis

[0278] (Server) The preprocessed data is input into an AI model to comprehensively evaluate the user's health condition, including the risk of lifestyle-related diseases based on sleep quality, daytime behavior patterns, and emotional state.

[0279] (Server) Adjusts the physical condition assessment on a case-by-case basis based on emotional data. For example, if the user is in a high stress state, prioritizing stress relief.

[0280] Step 8: Generate recommended actions

[0281] (Server) Based on the analysis results, specific recommended actions are generated for the user (e.g., "go to bed early," "do some light exercise," "take time to relax," etc.).

[0282] (Server) The generated health assessment and recommended actions are compiled into a single package.

[0283] Step 9: Sending the results

[0284] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[0285] (Terminal) Receives transmitted data and prepares analysis results.

[0286] Step 10: Notify users

[0287] The device will notify the user of their health status assessment and recommended actions via the application, which may include pop-up messages, alerts, or in-app dashboard displays.

[0288] (User) The user reviews the notification and adjusts their behavior according to the recommended actions and advice provided.

[0289] This step allows users to receive a detailed health assessment including emotional data and a personalized improvement plan, helping them take greater control of their health in their daily lives.

[0290] Example 2

[0291] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0292] Current health management systems can collect a user's sleep data and daytime behavior data individually, but it is difficult to analyze them comprehensively and provide a comprehensive health assessment that includes emotional data. They also lack the ability to provide specific recommended actions that take into account the user's emotional state. This can lead to issues such as users being unable to take appropriate actions and making it difficult to maintain their health.

[0293] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0294] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for evaluating the health status based on the preprocessed data, thereby enabling a comprehensive health assessment including the user's emotional state and providing specific and individual recommended actions based on the assessment.

[0295] "Sleep data" refers to data related to the user's sleep, such as the user's sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[0296] "Daytime activity data" refers to data about a user's daytime activities, such as the user's steps, heart rate, activity level, and GPS location information.

[0297] "Emotional Data" is data about a user's emotional state collected through analysis of the user's voice and facial expressions.

[0298] A "cloud server" is a remote server that can receive collected data, store it, analyze it as needed, and send the results to a user terminal.

[0299] "Preprocessing" refers to performing processes such as data cleaning, missing value imputation, and normalization on collected data.

[0300] "AI algorithm" is an artificial intelligence technology that uses pre-processed data to assess a user's health status and generate a physical condition assessment and recommended actions.

[0301] "Health Assessment" is the result of an assessment of the user's health condition, generated based on an AI algorithm.

[0302] "Recommended actions" are specific actions that users should take based on their physical condition assessment.

[0303] "User terminal" means a device that can be operated by a user, which transmits collected data and receives and notifies health assessments and recommended actions.

[0304] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[0305] System Configuration

[0306] 1. Smartphone devices

[0307] Device: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0308] Device: During daytime activities, data on daily activities such as steps, heart rate, activity level, and GPS location is collected.

[0309] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions to collect emotional data. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[0310] Terminal: Sends collected data to the cloud server at specified intervals.

[0311] 2. Cloud Server

[0312] Server: Receives data sent from the smartphone device and stores it in a database.

[0313] Server: Checks the received data for noise and missing data, and performs preprocessing if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[0314] Server: Runs AI algorithms using pre-processed data to assess the user's health, taking emotional data into account.

[0315] Server: Based on the evaluation results, a physical condition evaluation and specific recommended actions are generated.

[0316] Server: Sends the generated physical condition assessment and recommended action plan to the user's terminal.

[0317] Specific examples

[0318] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[0319] Prompt Sentence Examples

[0320] An example of a prompt might be something like the following, given to a generative AI model:

[0321] If a user has had less sleep than usual and the emotion engine detects high stress during the day, what recommended actions should the system provide?

[0322] In this way, a system can be provided that enables individual health management based on the user's multifaceted data.

[0323] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0324] Step 1:

[0325] Data Collection - Sleep Data

[0326] Device: When a user goes to bed, sleep mode is initiated using the smartphone's built-in sensors (e.g., accelerometer, microphone) or an external sensor device (e.g., smartwatch).

[0327] Input: User's bedtime input.

[0328] Processing: Detects when sleep begins from bedtime and records sleep time, percentage of deep sleep, and frequency of turning over.

[0329] Output: Recorded sleep data. Specifically, data is collected every minute after falling asleep.

[0330] Step 2:

[0331] Data collection - daytime behavior data

[0332] Device: When the user wakes up in the morning, the smartphone switches to daytime activity data collection mode. The smartphone acquires data using built-in sensors (e.g., accelerometer, GPS, heart rate monitor).

[0333] Input: Enter the user's wake-up time.

[0334] Processing: Collection of daytime activity data begins from the time you wake up, and records steps, heart rate, activity level, and GPS location.

[0335] Output: Recorded daytime behavior data. Specifically, data is updated and recorded every 5 minutes.

[0336] Step 3:

[0337] Data Collection - Emotional Data

[0338] Device: During daytime activities, the emotion engine analyzes voice and facial expressions in real time, using the microphone and camera to detect changes in voice tone and facial expressions.

[0339] Input: Voice and facial expression data from the user during daytime activities.

[0340] Processing: Analyzes voice and facial expressions in real time to determine emotional states such as stress, joy, and sadness.

[0341] Output: Recorded emotional data. Specifically, the data is collected the moment the user starts talking, and the emotional state is analyzed every minute.

[0342] Step 4:

[0343] Data transmission

[0344] Device: The device transmits collected sleep data, daytime behavior data, and emotional data to a cloud server at regular intervals.

[0345] Input: All recorded data (sleep data, daytime behavior data, emotional data).

[0346] Processing: Data is compressed, encrypted and sent to the cloud server.

[0347] Output: Data sent to the cloud server. Specifically, data is sent once every hour.

[0348] Step 5:

[0349] Receiving and storing data

[0350] Server: The cloud server receives the data sent from the device and stores it in a database.

[0351] Input: Data sent from the terminal.

[0352] Processing: Receiving data and saving it to a database.

[0353] Output: Stored data. The specific operation is to execute the procedure to store the received data in real time.

[0354] Step 6:

[0355] Data Preprocessing

[0356] Server: Performs preprocessing on the received data.

[0357] Input: Saved data.

[0358] Processing: Data cleaning, missing value imputation, and normalization. Specifically, outliers are removed, missing data is imputed with the mean, and all data is normalized to the range 0 to 1.

[0359] Output: Preprocessed data. The specific operation is to run a process to preprocess the data every hour.

[0360] Step 7:

[0361] Health assessment

[0362] Server: Inputs preprocessed data into the AI ​​algorithm to assess the user's health status.

[0363] Input: Preprocessed data.

[0364] Processing: AI algorithms assess sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[0365] Output: Health assessment results. Specific actions include assessing health status once a day at night.

[0366] Step 8:

[0367] Tailoring recommended actions with emotional data

[0368] Server: Considers emotional data and adjusts health assessment based on the user's emotional state.

[0369] Input: Health assessment results, emotion data.

[0370] Processing: Adjusting health assessment based on emotional data and generating specific recommended actions. For example, if stress levels are high, recommending relaxation activities.

[0371] Output: Adjusted health assessment and recommended actions. The specific behavior is to adjust the recommended actions every 30 minutes based on the assessment results.

[0372] Step 9:

[0373] Generate and send recommended actions

[0374] Server: Sends the health assessment and recommended actions generated using an AI algorithm to the user's device.

[0375] Input: Tailored health assessment and recommended actions.

[0376] Processing: Generate recommended actions and send them to the device.

[0377] Output: Data sent to the user terminal. Specifically, it is sent to the user terminal once at night.

[0378] Step 10:

[0379] User Notification

[0380] Device: The smartphone uses notifications to inform the user of their health assessment and recommended actions.

[0381] Input: Received health assessment and recommended action.

[0382] Actions: Notifications via pop-up messages, alerts, and dashboard displays.

[0383] Output: A notification message to the user. The specific behavior is to notify the user immediately to provide timely follow-up.

[0384] Step 11:

[0385] Implementing the recommended actions

[0386] User: The user reviews the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[0387] Input: Health assessment and recommended actions provided by the device.

[0388] Treatment: Adjust your daily schedule and activities according to the recommended actions.

[0389] Output: The recommended actions that were taken. Specific actions include adjusting your bedtime or practicing relaxation techniques.

[0390] (Application example 2)

[0391] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0392] Conventional health assessment systems could only assess a user's health based on their sleep data and daytime behavior data, and did not take emotional data into account. As a result, the health assessment was sometimes inaccurate. Furthermore, because no assessment related to security risks was performed, appropriate security recommendations were not made, especially for users in a physically or mentally unstable state.

[0393] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0394] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for assessing health status and security risks based on the preprocessed data, thereby enabling a comprehensive assessment of the user's physical condition and security risks, including their emotional state.

[0395] "Sleep Data" refers to information collected while a user is asleep, including, specifically, the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0396] "Daytime Activity Data" refers to information about a user's activities during the day, including steps taken, heart rate, activity level, and GPS location information.

[0397] "Emotional data" refers to information that indicates a user's emotional state, including data collected through voice analysis and facial expression analysis.

[0398] A "cloud server" refers to a remote server for storing, managing, and processing data via the Internet.

[0399] "Preprocessing" refers to processes such as cleaning, missing value imputation, and normalization performed on data, and is a basic process for improving the accuracy of analysis.

[0400] "AI algorithm" refers to a computational method that uses artificial intelligence technology to analyze data and make specific evaluations.

[0401] "Health assessment" refers to the process of comprehensively assessing a user's health status based on collected data.

[0402] "Security risk assessment" refers to the process of comprehensively assessing security risks, taking into account the user's emotional state and physical condition.

[0403] "Recommended actions" refers to advice on actions that a user should take based on the results of the health assessment and security risk assessment.

[0404] "Notification" refers to the process of sending alerts or messages to users to provide information.

[0405] This invention is a system that collects and analyzes a user's sleep data, daytime behavior data, and emotional data to provide an individualized physical condition assessment and recommended actions. The system aims to evaluate the user's health status and security risks and recommend appropriate actions.

[0406] The system mainly consists of a user terminal, a cloud server, and an AI algorithm.

[0407] (user device)

[0408] A smartphone is used as the user terminal. The smartphone collects the following data using built-in sensors and external sensor devices (e.g., wearable devices).

[0409] 1. Sleep data:

[0410] This data is collected when the user goes to bed, specifically including the duration of sleep, the percentage of deep sleep, the frequency of turning over in bed, etc. This data is recorded using the smartphone's accelerometer and proximity sensor.

[0411] 2. Daytime behavior data:

[0412] This includes steps, heart rate, activity level, GPS location, etc. This data is recorded using the smartphone's built-in GPS and pedometer sensors.

[0413] 3. Emotional Data:

[0414] This data is collected through voice and facial expression analysis, and is analyzed in real time by an emotion engine (e.g., voice recognition API or facial recognition software) using the smartphone's microphone and camera.

[0415] (Cloud server)

[0416] The cloud server is the main hardware that receives and stores data sent from smartphone devices and analyzes that data. The cloud server performs the following processes:

[0417] 1. Pretreatment:

[0418] The received data is cleaned, filled in with missing values, normalized, etc.

[0419] 2. AI algorithms:

[0420] The preprocessed data is used to run AI algorithms that assess the user's health and security risks, using generative AI models (e.g., TensorFlow, PyTorch).

[0421] 3. Result generation:

[0422] Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated, which are customized to take into account the user's current health status and security risks.

[0423] (User Notification)

[0424] The cloud server sends the acquired health assessment results and recommended actions to the user's smartphone, which then notifies the user of the results via pop-up messages, alerts, dashboard displays, and other methods.

[0425] (Example)

[0426] For example, if a user has slept less than usual and the emotion engine recognizes that they are under high levels of stress during the day, the system can analyze that data and provide specific recommended actions, such as "go to bed earlier tonight" or "take some relaxing breaks during the day."

[0427] Example prompt sentence:

[0428] If the user is found to have slept less than 6 hours last night and their daytime stress level is above 70:

[0429] "Security risk is high. We recommend you go home early today and get plenty of rest. Avoid any important work."

[0430] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0431] Step 1:

[0432] The user's device collects sleep data.

[0433] Input: Accelerometer and proximity sensor data from your smartphone

[0434] How it works: When a user goes to bed, the smartphone's built-in sensor records the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed. This data is then stored locally at regular intervals.

[0435] Output: Sleep data (sleep time, percentage of deep sleep, frequency of turning over)

[0436] Step 2:

[0437] User devices collect behavioral data during the day.

[0438] Input: Data from smartphone's GPS, pedometer sensor, and heart rate sensor

[0439] How it works: When you wake up and take your smartphone with you, the built-in GPS, pedometer, and heart rate sensor will record your steps, heart rate, activity level, and location. This data is collected in real time and stored locally.

[0440] Output: Daily activity data (step count, heart rate, activity level, GPS location information)

[0441] Step 3:

[0442] The user's device collects emotional data.

[0443] Input: Audio and image data obtained from the smartphone's microphone and camera

[0444] How it works: The emotion engine uses the smartphone's microphone and camera to analyze the user's voice and facial expressions, recognizing their emotional state in real time and recording it as emotional data.

[0445] Output: Emotion data (voice analysis results, facial expression analysis results)

[0446] Step 4:

[0447] The user device sends data to the cloud server.

[0448] Input: Sleep data, daytime behavior data, emotional data

[0449] Specific operation: Collected data is sent to a cloud server periodically or upon user instruction. The data is sent using a secure protocol (e.g., HTTPS).

[0450] Output: Data saved to a database on a cloud server

[0451] Step 5:

[0452] The cloud server pre-processes the data.

[0453] Input: Sleep data, daytime behavior data, emotional data

[0454] Specific operation: The cloud server cleans the received data, imputes missing values, and performs any necessary normalization. Preprocessing software (e.g., Python's pandas library) is used.

[0455] Output: Preprocessed data

[0456] Step 6:

[0457] Cloud servers run the AI ​​algorithms.

[0458] Input: Preprocessed data

[0459] Specific operation: Analyzes pre-processed data using AI algorithms (e.g., TensorFlow, PyTorch), and performs calculations to assess the user's health status and security risks.

[0460] Output: Health status assessment, security risk assessment

[0461] Step 7:

[0462] The cloud server generates the results and sends them to the user device.

[0463] Input: Health status assessment, security risk assessment

[0464] Specific operation: Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated for the user, and the generated results are sent to the user's smartphone.

[0465] Output: Health assessment and recommended actions

[0466] Step 8:

[0467] The user terminal notifies the received result.

[0468] Input: Health assessment and recommended actions

[0469] Specific behavior: The user device will display the results it receives. Notification methods can include pop-up messages, alerts, dashboard displays, etc. The user can then adjust their actions accordingly.

[0470] Output: Informational message to the user

[0471] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0472] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0473] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0474] [Second embodiment]

[0475] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0476] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0477] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0478] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0479] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0480] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0481] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0482] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0483] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0484] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0485] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0486] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0487] The present invention relates to a system that uses a user's sleep data and daytime behavior data to provide an individualized physical condition assessment and recommended actions. This system is mainly composed of a smartphone-type device and a cloud server.

[0488] System Configuration

[0489] 1. Smartphone devices

[0490] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0491] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[0492] (Terminal) Sends collected data to the cloud server at specified intervals.

[0493] 2. Cloud Server

[0494] (Server) Receives data sent from the smartphone device and stores it in a database.

[0495] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[0496] (Server) Runs AI algorithms using preprocessed data to assess the user's health status.

[0497] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[0498] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[0499] Program processing

[0500] Data collection

[0501] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[0502] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[0503] (Device) Collected data is sent to the cloud server at regular intervals.

[0504] Data analysis

[0505] (Server) The cloud server stores the received sleep data and daytime behavior data in a database.

[0506] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[0507] (Server) The pre-processed data is input into an AI algorithm to comprehensively assess the user's health status, including the user's sleep quality, fatigue level, and lifestyle risks.

[0508] (Server) Based on the evaluation results, a physical condition assessment and recommended actions for the next day (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[0509] Result notification

[0510] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[0511] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[0512] (User) The user can check the health assessment and recommended actions and adjust their actions based on the advice.

[0513] For example, if a user has slept less than usual, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a short break during the day." As a result, users can receive advice optimized for their individual health condition and improve their lifestyle habits.

[0514] The processing flow will be explained below.

[0515] Step 1: Start collecting data

[0516] (Device) When the user starts going to bed, the device begins collecting sleep data using its built-in sensors (accelerometer, heart rate sensor, etc.).

[0517] (Device) The user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting user inactivity.

[0518] Step 2: Collecting daytime behavioral data

[0519] (Device) When you wake up in the morning, sleep data collection will stop based on the user's manual input or a designed pattern.

[0520] (Device) When the user begins their activities, data on their daily activities such as number of steps, heart rate, amount of exercise, and GPS location information is collected.

[0521] Step 3: Sending data

[0522] (Device) The collected sleep data and daytime behavior data are sent to the cloud server in real time or at specified intervals.

[0523] (Terminal) After data transmission is complete, the terminal notifies the server of this fact.

[0524] Step 4: Receiving and storing data

[0525] (Server) Receives data sent from the device and stores it securely in a database, where it is identified and classified for each user.

[0526] Step 5: Preprocessing the data

[0527] (Server) Performs preprocessing such as noise removal, missing value imputation, and normalization on the received data, enabling accurate and consistent analysis.

[0528] Step 6: AI analysis

[0529] (Server) The preprocessed data is input into an AI model for analysis, which evaluates the user's sleep quality, daytime behavior patterns, risk of lifestyle-related diseases, etc.

[0530] (Server) As a result of the analysis, a detailed assessment of the user's health status is generated.

[0531] Step 7: Generate recommended actions

[0532] (Server) Based on the analysis results, specific recommended actions are generated, such as "go to bed early," "get moderate exercise," and "eat a specific diet."

[0533] (Server) Create a package that summarizes recommended actions and health assessments.

[0534] Step 8: Sending the results

[0535] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[0536] (Terminal) Properly receives the transmitted data and prepares it for presentation to the user.

[0537] Step 9: Notify users

[0538] The device application notifies the user of the received health assessment and recommended actions via pop-ups, alerts, and in-app dashboard displays.

[0539] (User) The user checks the notification and follows the recommended actions presented to improve their health.

[0540] Through this series of steps, users can receive an individually customized health assessment and recommended actions, which they can then incorporate into their daily lives to effectively manage their health.

[0541] Example 1

[0542] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0543] In recent years, with growing interest in health management, highly accurate data collection and analysis are required to provide specific recommended actions based on individual health conditions. However, conventional systems have had issues in providing appropriate advice to users due to limited types of collected data and insufficient analysis accuracy. The object of the present invention is to provide a system that collects a variety of biometric and activity data from users and analyzes them using advanced artificial intelligence to provide specific health assessments and recommended actions based on individual health conditions.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0545] In this invention, the server includes means for collecting a user's biometric data, means for collecting daytime activity data, means for transmitting the collected biometric data and activity data to a network server, means for preprocessing the data received by the network server, means for executing artificial intelligence for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the artificial intelligence, means for transmitting the generated health condition assessment and recommended actions to a user terminal, and means for notifying the user of the transmitted health condition assessment and recommended actions. This makes it possible to collect and analyze a variety of user data with high accuracy and provide specific advice based on an individual's health condition.

[0546] "Biometric data" refers to information about the user's physical condition, specifically including the amount of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0547] "Activity data" refers to information about a user's daily activities, and specifically includes the number of steps taken, heart rate, amount of exercise, location information, etc.

[0548] "Network server" refers to a server that receives, stores, and processes data sent from a user terminal.

[0549] "Preprocessing" refers to processing of received data, such as removing noise, filling in missing values, and normalizing data.

[0550] "Artificial intelligence" refers to algorithms that assess a user's health status based on pre-processed data.

[0551] "Health assessment" refers to the results of an assessment of the user's health condition based on the analysis results of artificial intelligence.

[0552] "Recommended actions" refer to actions that are specifically recommended to the user based on the health assessment.

[0553] "User terminal" refers to a device that collects biometric data and activity data and transmits it to a network server, and specifically includes a smartphone.

[0554] "Notification" refers to a means for informing a user of the generated health assessment and recommended actions, including pop-up messages, alerts, dashboard displays, etc.

[0555] The present invention relates to a system that uses a user's biometric data and daily activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone-type device and a cloud server.

[0556] The system configuration includes the following elements:

[0557] Data collection by terminal

[0558] (Device) When the user goes to bed, the smartphone device collects sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, and the frequency of turning over in bed.

[0559] (Device) During daytime activities, data on daily activities such as steps taken, heart rate, exercise volume, and GPS location information is collected using the device's activity tracker and location services.

[0560] (Device) Collected data is sent to the cloud server at regular intervals. The sending interval can be changed in the system settings, but is generally set to every hour.

[0561] Data analysis using a cloud server

[0562] (Server) The cloud server receives biometric and activity data sent from the user's smartphone and stores it in a database. Each data item is time-stamped for later analysis.

[0563] (Server) Performs preprocessing on the received data. Specifically, it performs filtering to remove noise, fills in missing values, and normalizes the data. For example, it filters out abnormal heart rate data and fills in missing data using the historical average.

[0564] (Server) The pre-processed data is used to run an artificial intelligence (AI) algorithm, which then compares the data with previously collected data and general health indicators to comprehensively assess the user's health, including the user's sleep quality, fatigue level, and lifestyle risks.

[0565] (Server) Based on the AI's evaluation results, a physical condition assessment and specific recommended actions for the next day are generated. Specific recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients."

[0566] Result notification

[0567] (Server) The generated health assessment and recommended action plan are sent to the user's smartphone, where the user can then receive push notifications and alerts.

[0568] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The user can check the notification content and follow the recommended actions.

[0569] Specific examples

[0570] If a user sleeps less than usual, the sensor records this information and sends the data to the cloud. The cloud server analyzes the data and generates specific advice, such as "go to bed earlier tonight" or "take a short break during the day." This advice is immediately sent to the user's device and notified to the user's application. The user can then adjust their daily schedule accordingly.

[0571] Prompt Sentence Examples

[0572] You can ask the generative AI model for a health assessment and recommended actions using prompts like the following:

[0573] Use the following sleep and daytime activity data from your users to assess their health and provide specific recommendations.

[0574] (Sleep data)

[0575] Sleep time: 6 hours

[0576] Deep sleep percentage: 20%

[0577] Turning frequency: 15 times

[0578] (Daytime behavior data)

[0579] Steps: 8,000

[0580] Heart rate: 80 BPM average

[0581] Activity level: High

[0582] GPS location: Commute from home to work

[0583] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0584] Step 1:

[0585] (Device) When the user goes to bed, the smartphone device begins collecting sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, the frequency of turning over in bed, etc. The input is biosignal data obtained from the sensors, and the output is structured sleep data.

[0586] Step 2:

[0587] When the user wakes up, the device automatically switches modes and starts collecting daily activity data. The collected data includes the number of steps taken by the pedometer, heart rate data from the heart rate sensor, activity level data from the activity tracker, and location data from GPS. The input is signal data obtained from various sensors during the day, and the output is structured activity data.

[0588] Step 3:

[0589] (Device) The collected biometric data and activity data are sent to the cloud server at regular intervals (e.g., every hour). The data is encoded and transmitted securely over the network. The input is data obtained from various data collection modules, and the output is the transmitted data arriving at the cloud server.

[0590] Step 4:

[0591] (Server) The cloud server receives the biometric data and activity data sent from the user device and stores them in a database. The input is structured data sent via the network, and the output is the biometric data and activity data stored in the database.

[0592] Step 5:

[0593] (Server) Performs preprocessing on the received data. Preprocessing includes filtering to remove noise, imputing missing values, and normalizing the data. For example, filtering out abnormal heart rate data and imputing missing values ​​using the historical average. The input is raw data, and after preprocessing, clean, normalized data is output.

[0594] Step 6:

[0595] (Server) Using the preprocessed data, an AI algorithm is run to evaluate the user's health status. The AI ​​evaluates the user's health status by comparing it with past data and health indicators. The evaluation includes sleep quality, fatigue level, lifestyle risks, etc. The input is the preprocessed data, and the output is a health status evaluation result.

[0596] Step 7:

[0597] (Server) Based on the AI ​​evaluation results, a health assessment and specific recommended actions for the next day are generated. Recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients." The input is the health assessment results, and the output is a health assessment and a recommended action plan.

[0598] Step 8:

[0599] (Server) The generated health assessment and recommended actions are sent to the user's smartphone. The input is the AI ​​analysis results, and the output is the recommended actions and assessment sent to the user's device.

[0600] Step 9:

[0601] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The input is the recommended action and evaluation data sent from the server, and the output is a visual or audio notification to the user. The user checks the notification and follows the recommended action.

[0602] (Application example 1)

[0603] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0604] In recent years, there has been growing interest in managing users' health status. However, there are only a limited number of systems that provide specific recommendations based on individual users' health status. Especially for in-store use, there is a need for systems that utilize users' location information to make appropriate recommendations. Furthermore, a mechanism for improving recommendations using user feedback is also needed, but this is similarly lacking. Given this background, there is a need for the development of a system that can provide appropriate in-store recommendations for products and facility use based on the user's health status and collect feedback for further improvement.

[0605] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0606] In this invention, the server includes means for collecting a user's sleep data, means for collecting daytime behavior data, means for transmitting the collected sleep data and daytime behavior data to a cloud server, means for preprocessing the received data in the cloud server, means for executing an AI algorithm for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the AI ​​algorithm, means for transmitting the generated health condition assessment and recommended actions to a user terminal, means for notifying the user of the transmitted health condition assessment and recommended actions, means for presenting recommended products and facilities suitable for the user using in-store location information, means for encouraging the user to take action based on the recommended products and facilities, and means for collecting feedback and improving the health assessment and recommendations based on the usage history of the recommended products and facilities. This makes it possible to make appropriate recommendations based on the user's health condition, improve the user experience in the physical store, and continuously improve the accuracy and effectiveness of the system using the feedback function.

[0607] "User" refers to an individual who uses the system to manage their own health condition.

[0608] "Sleep data" refers to information collected while a user is asleep, and specifically includes the amount of sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[0609] "Daytime activity data" refers to information about the activities a user engages in during the day, including, for example, the number of steps taken, heart rate, activity level, and GPS location information.

[0610] "Cloud server" refers to a remote server that stores and processes data over the Internet.

[0611] "Preprocessing" refers to the processing that the cloud server performs on the raw data it receives, and includes data cleaning, missing value imputation, normalization, etc.

[0612] "Health Status" refers to the results of an assessment of the user's physical and mental state.

[0613] "AI Algorithm" refers to the artificial intelligence algorithm used to assess a user's health status based on collected data.

[0614] "Recommended actions" refers to specific actions or options suggested to users based on the evaluation results of the AI ​​algorithm.

[0615] "Notification" refers to the act of sending a message or alert to a user's device to convey information to the user.

[0616] "In-store location information" refers to information used to identify a user's current location within a physical store.

[0617] "Recommended Products" refers to products and services suggested to you based on your health status.

[0618] "Facilities" refers to the locations of facilities and services that users are encouraged to use within the physical store.

[0619] "Means to encourage behavior" refers to mechanisms that encourage users to use recommended products or facilities.

[0620] "Feedback" refers to the opinions and ratings provided by users after using the system.

[0621] "Usage history" refers to the history of products and services a user has used in the past and recommended actions.

[0622] This invention relates to a system that uses a user's sleep data and daytime activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone device and a cloud server.

[0623] 1. Smartphone devices

[0624] The smartphone device has the following features:

[0625] Sleep data collection: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0626] Collection of daytime activity data: During daytime activities, we collect daytime activity data such as steps, heart rate, activity level, and GPS location information.

[0627] Data transmission: The collected data is sent to the cloud server at the specified interval.

[0628] 2. Cloud Server

[0629] The cloud server has the following functions:

[0630] Data reception and storage: Receives data sent from the smartphone device and stores it in a database.

[0631] Data preprocessing: Check the received data for noise and missing data, and preprocess it if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[0632] Data analysis: Using the pre-processed data, AI algorithms are run to assess the user's health status, including sleep quality, fatigue level, and lifestyle risks.

[0633] Generation of recommended actions: Based on the assessment results, a physical condition assessment and specific recommended actions based on that assessment (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[0634] Notification of results: The generated physical condition assessment and recommended action plan are sent to the user's device.

[0635] Gathering feedback: Gathering user feedback to improve our ratings and recommendations.

[0636] 3. Use in physical stores

[0637] This system can also be used in physical stores, where it can utilize the user's location information to provide the following services:

[0638] Location information identification: The user's current location is identified using in-store Wi-Fi and Bluetooth beacons.

[0639] Recommended products: Based on the user's health data, the app will suggest suitable products and facilities to use. For example, if you need to relax, it will display recommended menu items at a cafe.

[0640] Action promotion: Send notifications to your smartphone encouraging you to use recommended products or facilities.

[0641] Feedback collection: Collect feedback from users after using the recommended products to help improve the system.

[0642] Hardware and software used

[0643] Smartphone device: Collecting user data and running applications

[0644] Cloud Server: Data analysis and generation of recommended actions

[0645] Data storage and processing: Using AWS (Amazon Web Services) or GCP (Google Cloud Platform)

[0646] Running AI algorithms: using TensorFlow

[0647] Specific examples

[0648] If a user has slept less than usual, the system can analyze the data and provide specific recommendations such as "go to bed earlier tonight" or "take a short break during the day." If the user is in a physical store, the system can determine that they need to relax and recommend a relaxation menu for the cafe.

[0649] Prompt Sentence Examples

[0650] "Please propose a health recommendation application to be provided in physical stores based on the user's sleep data and daytime behavior data. For example, an application that makes recommendations on what items to purchase, which facilities to use, etc. Please also provide detailed information on specific features and how to implement them."

[0651] The above are details of a specific mode for carrying out the invention. The system allows users to receive personalized health advice and improve their lifestyle habits while enhancing their in-store experience.

[0652] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0653] Step 1: Data collection

[0654] The device collects the user's sleep data while they sleep. When the user goes to bed, the device's built-in sensors and external sensor devices are activated to collect data such as sleep time, percentage of deep sleep, and frequency of turning over in bed. The collected data is temporarily stored in a format that can be used immediately.

[0655] Step 2: Collecting daytime behavioral data

[0656] The device collects user behavior data during daytime activities. After the user wakes up, the device automatically switches to daytime activity data collection mode. Step count, heart rate, activity level, GPS location information, etc. are collected and temporarily stored.

[0657] Step 3: Send data

[0658] The device sends the collected sleep data and daytime behavior data to a cloud server. At specified intervals, the data is sent in batches to the cloud server for processing on the server side. The data is encrypted before transmission and decrypted upon arrival.

[0659] Step 4: Receiving and storing data

[0660] The server receives the data sent from the device and stores it in a database. To securely store the received data, it uses cloud storage services such as AWS and GCP. The data is then classified by user and prepared for analysis.

[0661] Step 5: Data Preprocessing

[0662] The server preprocesses the data it receives. Preprocessing includes data cleaning, missing value imputation, normalization, etc. This removes noise from the data and organizes it into a consistent format. Libraries such as Pandas are sometimes used for data cleaning.

[0663] Step 6: Data analysis

[0664] The server uses the preprocessed data to run AI algorithms to assess the user's health. Specifically, collected sleep data and daytime behavior data are input into a generative AI model such as TensorFlow to evaluate sleep quality, fatigue level, lifestyle risks, etc.

[0665] Step 7: Generate recommended actions

[0666] Based on the results of the AI ​​algorithm's evaluation, the server generates a physical condition assessment and specific recommended actions, such as performing relaxation exercises, going to bed earlier, and specific dietary advice.

[0667] Step 8: Notification of results

[0668] The server then sends the generated health assessment and recommended actions to the user's device, and notifications are sent in real time as pop-up messages or alerts on the user's smartphone.

[0669] Step 9: Identify your location

[0670] The device identifies the user's location within the physical store, detects the user's current location using Wi-Fi or Bluetooth beacons, and sends the location information to a cloud server.

[0671] Step 10: Recommendations

[0672] The server generates recommended products and facilities based on the user's health data and location information and sends them to the device. For example, a user who needs to relax will be shown recommended menu items at a cafe.

[0673] Step 11: Drive action

[0674] The device will send notifications to the user encouraging them to use the recommended products and facilities. These notifications will be displayed on the user's device and will serve as a guide for actions in the physical store.

[0675] Step 12: Gather feedback

[0676] The server collects user feedback to improve the system's ratings and recommendations. Feedback is entered by users through the application and is reflected in future recommendations.

[0677] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0678] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[0679] System Configuration

[0680] 1. Smartphone devices

[0681] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0682] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[0683] (Device) Equipped with an emotion engine that analyzes the user's voice and facial expressions, it collects emotional data from the user. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[0684] (Terminal) Sends collected data to the cloud server at specified intervals.

[0685] 2. Cloud Server

[0686] (Server) Receives data sent from the smartphone device and stores it in a database.

[0687] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[0688] (Server) Runs AI algorithms using the preprocessed data to assess the user's health, taking into account emotional data.

[0689] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[0690] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[0691] Program processing

[0692] Data collection

[0693] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[0694] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[0695] (Device) During the day, the emotion engine analyzes voice and facial expressions, recognizes the user's emotions in real time, and collects them as data.

[0696] (Device) Collected data is sent to the cloud server at regular intervals.

[0697] Data analysis

[0698] (Server) The cloud server stores the received sleep data, daytime behavior data, and emotion data in a database.

[0699] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[0700] (Server) The pre-processed data is fed into an AI algorithm to provide a comprehensive assessment of the user's health status, including the user's sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[0701] (Server) Adjusts health assessment based on emotional data, for example, if the user is feeling stressed, prioritizes recommended actions to reduce that stress.

[0702] Result notification

[0703] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[0704] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[0705] (User) The user checks the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[0706] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[0707] The processing flow will be explained below.

[0708] Step 1: Start collecting sleep data

[0709] (Device) When the user goes to bed, the device starts collecting sleep data using built-in sensors (accelerometer, heart rate sensor, etc.).

[0710] (Device) A user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting inactivity.

[0711] Step 2: Collecting daytime behavioral data

[0712] (Device) When you wake up in the morning, stop collecting sleep data and switch to daytime activity data collection mode.

[0713] (Device) Measures the user's daytime activity and records steps, heart rate, activity level, and GPS location.

[0714] Step 3: Collecting emotion data

[0715] (Device) The user's voice data and facial expression data are collected during the day using a microphone and camera, and the emotion engine analyzes this data to recognize the user's emotions.

[0716] (Device) Records emotional data at regular intervals during the day and transmits it to the cloud server as needed.

[0717] Step 4: Sending data

[0718] (Device) The collected sleep data, daytime behavior data, and emotional data are sent to a cloud server periodically or in real time.

[0719] Step 5: Receiving and storing data

[0720] (Server) Receives data sent from the device and stores it in a secure database. Data is identified for each user.

[0721] Step 6: Preprocessing the data

[0722] (Server) Preprocessing is performed on the received data, including noise removal, missing value completion, and normalization. This allows for highly accurate analysis.

[0723] Step 7: AI analysis

[0724] (Server) The preprocessed data is input into an AI model to comprehensively evaluate the user's health condition, including the risk of lifestyle-related diseases based on sleep quality, daytime behavior patterns, and emotional state.

[0725] (Server) Adjusts the physical condition assessment on a case-by-case basis based on emotional data. For example, if the user is in a high stress state, prioritizing stress relief.

[0726] Step 8: Generate recommended actions

[0727] (Server) Based on the analysis results, specific recommended actions are generated for the user (e.g., "go to bed early," "do some light exercise," "take time to relax," etc.).

[0728] (Server) The generated health assessment and recommended actions are compiled into a single package.

[0729] Step 9: Sending the results

[0730] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[0731] (Terminal) Receives transmitted data and prepares analysis results.

[0732] Step 10: Notify users

[0733] The device will notify the user of their health status assessment and recommended actions via the application, which may include pop-up messages, alerts, or in-app dashboard displays.

[0734] (User) The user reviews the notification and adjusts their behavior according to the recommended actions and advice provided.

[0735] This step allows users to receive a detailed health assessment including emotional data and a personalized improvement plan, helping them take greater control of their health in their daily lives.

[0736] Example 2

[0737] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0738] Current health management systems can collect a user's sleep data and daytime behavior data individually, but it is difficult to analyze them comprehensively and provide a comprehensive health assessment that includes emotional data. They also lack the ability to provide specific recommended actions that take into account the user's emotional state. This can lead to issues such as users being unable to take appropriate actions and making it difficult to maintain their health.

[0739] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0740] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for evaluating the health status based on the preprocessed data, thereby enabling a comprehensive health assessment including the user's emotional state and providing specific and individual recommended actions based on the assessment.

[0741] "Sleep data" refers to data related to the user's sleep, such as the user's sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[0742] "Daytime activity data" refers to data about a user's daytime activities, such as the user's steps, heart rate, activity level, and GPS location information.

[0743] "Emotional Data" is data about a user's emotional state collected through analysis of the user's voice and facial expressions.

[0744] A "cloud server" is a remote server that can receive collected data, store it, analyze it as needed, and send the results to a user terminal.

[0745] "Preprocessing" refers to performing processes such as data cleaning, missing value imputation, and normalization on collected data.

[0746] "AI algorithm" is an artificial intelligence technology that uses pre-processed data to assess a user's health status and generate a physical condition assessment and recommended actions.

[0747] "Health Assessment" is the result of an assessment of the user's health condition, generated based on an AI algorithm.

[0748] "Recommended actions" are specific actions that users should take based on their physical condition assessment.

[0749] "User terminal" means a device that can be operated by a user, which transmits collected data and receives and notifies health assessments and recommended actions.

[0750] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[0751] System Configuration

[0752] 1. Smartphone devices

[0753] Device: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0754] Device: During daytime activities, data on daily activities such as steps, heart rate, activity level, and GPS location is collected.

[0755] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions to collect emotional data. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[0756] Terminal: Sends collected data to the cloud server at specified intervals.

[0757] 2. Cloud Server

[0758] Server: Receives data sent from the smartphone device and stores it in a database.

[0759] Server: Checks the received data for noise and missing data, and performs preprocessing if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[0760] Server: Runs AI algorithms using pre-processed data to assess the user's health, taking emotional data into account.

[0761] Server: Based on the evaluation results, a physical condition evaluation and specific recommended actions are generated.

[0762] Server: Sends the generated physical condition assessment and recommended action plan to the user's terminal.

[0763] Specific examples

[0764] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[0765] Prompt Sentence Examples

[0766] An example of a prompt might be something like the following, given to a generative AI model:

[0767] If a user has had less sleep than usual and the emotion engine detects high stress during the day, what recommended actions should the system provide?

[0768] In this way, a system can be provided that enables individual health management based on the user's multifaceted data.

[0769] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0770] Step 1:

[0771] Data Collection - Sleep Data

[0772] Device: When a user goes to bed, sleep mode is initiated using the smartphone's built-in sensors (e.g., accelerometer, microphone) or an external sensor device (e.g., smartwatch).

[0773] Input: User's bedtime input.

[0774] Processing: Detects when sleep begins from bedtime and records sleep time, percentage of deep sleep, and frequency of turning over.

[0775] Output: Recorded sleep data. Specifically, data is collected every minute after falling asleep.

[0776] Step 2:

[0777] Data collection - daytime behavior data

[0778] Device: When the user wakes up in the morning, the smartphone switches to daytime activity data collection mode. The smartphone acquires data using built-in sensors (e.g., accelerometer, GPS, heart rate monitor).

[0779] Input: Enter the user's wake-up time.

[0780] Processing: Collection of daytime activity data begins from the time you wake up, and records steps, heart rate, activity level, and GPS location.

[0781] Output: Recorded daytime behavior data. Specifically, data is updated and recorded every 5 minutes.

[0782] Step 3:

[0783] Data Collection - Emotional Data

[0784] Device: During daytime activities, the emotion engine analyzes voice and facial expressions in real time, using the microphone and camera to detect changes in voice tone and facial expressions.

[0785] Input: Voice and facial expression data from the user during daytime activities.

[0786] Processing: Analyzes voice and facial expressions in real time to determine emotional states such as stress, joy, and sadness.

[0787] Output: Recorded emotional data. Specifically, the data is collected the moment the user starts talking, and the emotional state is analyzed every minute.

[0788] Step 4:

[0789] Data transmission

[0790] Device: The device transmits collected sleep data, daytime behavior data, and emotional data to a cloud server at regular intervals.

[0791] Input: All recorded data (sleep data, daytime behavior data, emotional data).

[0792] Processing: Data is compressed, encrypted and sent to the cloud server.

[0793] Output: Data sent to the cloud server. Specifically, data is sent once every hour.

[0794] Step 5:

[0795] Receiving and storing data

[0796] Server: The cloud server receives the data sent from the device and stores it in a database.

[0797] Input: Data sent from the terminal.

[0798] Processing: Receiving data and saving it to a database.

[0799] Output: Stored data. The specific operation is to execute the procedure to store the received data in real time.

[0800] Step 6:

[0801] Data Preprocessing

[0802] Server: Performs preprocessing on the received data.

[0803] Input: Saved data.

[0804] Processing: Data cleaning, missing value imputation, and normalization. Specifically, outliers are removed, missing data is imputed with the mean, and all data is normalized to the range 0 to 1.

[0805] Output: Preprocessed data. The specific operation is to run a process to preprocess the data every hour.

[0806] Step 7:

[0807] Health assessment

[0808] Server: Inputs preprocessed data into the AI ​​algorithm to assess the user's health status.

[0809] Input: Preprocessed data.

[0810] Processing: AI algorithms assess sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[0811] Output: Health assessment results. Specific actions include assessing health status once a day at night.

[0812] Step 8:

[0813] Tailoring recommended actions with emotional data

[0814] Server: Considers emotional data and adjusts health assessment based on the user's emotional state.

[0815] Input: Health assessment results, emotion data.

[0816] Processing: Adjusting health assessment based on emotional data and generating specific recommended actions. For example, if stress levels are high, recommending relaxation activities.

[0817] Output: Adjusted health assessment and recommended actions. The specific behavior is to adjust the recommended actions every 30 minutes based on the assessment results.

[0818] Step 9:

[0819] Generate and send recommended actions

[0820] Server: Sends the health assessment and recommended actions generated using an AI algorithm to the user's device.

[0821] Input: Tailored health assessment and recommended actions.

[0822] Processing: Generate recommended actions and send them to the device.

[0823] Output: Data sent to the user terminal. Specifically, it is sent to the user terminal once at night.

[0824] Step 10:

[0825] User Notification

[0826] Device: The smartphone uses notifications to inform the user of their health assessment and recommended actions.

[0827] Input: Received health assessment and recommended action.

[0828] Actions: Notifications via pop-up messages, alerts, and dashboard displays.

[0829] Output: A notification message to the user. The specific behavior is to notify the user immediately to provide timely follow-up.

[0830] Step 11:

[0831] Implementing the recommended actions

[0832] User: The user reviews the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[0833] Input: Health assessment and recommended actions provided by the device.

[0834] Treatment: Adjust your daily schedule and activities according to the recommended actions.

[0835] Output: The recommended actions that were taken. Specific actions include adjusting your bedtime or practicing relaxation techniques.

[0836] (Application example 2)

[0837] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0838] Conventional health assessment systems could only assess a user's health based on their sleep data and daytime behavior data, and did not take emotional data into account. As a result, the health assessment was sometimes inaccurate. Furthermore, because no assessment related to security risks was performed, appropriate security recommendations were not made, especially for users in a physically or mentally unstable state.

[0839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0840] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for assessing health status and security risks based on the preprocessed data, thereby enabling a comprehensive assessment of the user's physical condition and security risks, including their emotional state.

[0841] "Sleep Data" refers to information collected while a user is asleep, including, specifically, the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0842] "Daytime Activity Data" refers to information about a user's activities during the day, including steps taken, heart rate, activity level, and GPS location information.

[0843] "Emotional data" refers to information that indicates a user's emotional state, including data collected through voice analysis and facial expression analysis.

[0844] A "cloud server" refers to a remote server for storing, managing, and processing data via the Internet.

[0845] "Preprocessing" refers to processes such as cleaning, missing value imputation, and normalization performed on data, and is a basic process for improving the accuracy of analysis.

[0846] "AI algorithm" refers to a computational method that uses artificial intelligence technology to analyze data and make specific evaluations.

[0847] "Health assessment" refers to the process of comprehensively assessing a user's health status based on collected data.

[0848] "Security risk assessment" refers to the process of comprehensively assessing security risks, taking into account the user's emotional state and physical condition.

[0849] "Recommended actions" refers to advice on actions that a user should take based on the results of the health assessment and security risk assessment.

[0850] "Notification" refers to the process of sending alerts or messages to users to provide information.

[0851] This invention is a system that collects and analyzes a user's sleep data, daytime behavior data, and emotional data to provide an individualized physical condition assessment and recommended actions. The system aims to evaluate the user's health status and security risks and recommend appropriate actions.

[0852] The system mainly consists of a user terminal, a cloud server, and an AI algorithm.

[0853] (user device)

[0854] A smartphone is used as the user terminal. The smartphone collects the following data using built-in sensors and external sensor devices (e.g., wearable devices).

[0855] 1. Sleep data:

[0856] This data is collected when the user goes to bed, specifically including the duration of sleep, the percentage of deep sleep, the frequency of turning over in bed, etc. This data is recorded using the smartphone's accelerometer and proximity sensor.

[0857] 2. Daytime behavior data:

[0858] This includes steps, heart rate, activity level, GPS location, etc. This data is recorded using the smartphone's built-in GPS and pedometer sensors.

[0859] 3. Emotional Data:

[0860] This data is collected through voice and facial expression analysis, and is analyzed in real time by an emotion engine (e.g., voice recognition API or facial recognition software) using the smartphone's microphone and camera.

[0861] (Cloud server)

[0862] The cloud server is the main hardware that receives and stores data sent from smartphone devices and analyzes that data. The cloud server performs the following processes:

[0863] 1. Pretreatment:

[0864] The received data is cleaned, filled in with missing values, normalized, etc.

[0865] 2. AI algorithms:

[0866] The preprocessed data is used to run AI algorithms that assess the user's health and security risks, using generative AI models (e.g., TensorFlow, PyTorch).

[0867] 3. Result generation:

[0868] Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated, which are customized to take into account the user's current health status and security risks.

[0869] (User Notification)

[0870] The cloud server sends the acquired health assessment results and recommended actions to the user's smartphone, which then notifies the user of the results via pop-up messages, alerts, dashboard displays, and other methods.

[0871] (Example)

[0872] For example, if a user has slept less than usual and the emotion engine recognizes that they are under high levels of stress during the day, the system can analyze that data and provide specific recommended actions, such as "go to bed earlier tonight" or "take some relaxing breaks during the day."

[0873] Example prompt sentence:

[0874] If the user is found to have slept less than 6 hours last night and their daytime stress level is above 70:

[0875] "Security risk is high. We recommend you go home early today and get plenty of rest. Avoid any important work."

[0876] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0877] Step 1:

[0878] The user's device collects sleep data.

[0879] Input: Accelerometer and proximity sensor data from your smartphone

[0880] How it works: When a user goes to bed, the smartphone's built-in sensor records the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed. This data is then stored locally at regular intervals.

[0881] Output: Sleep data (sleep time, percentage of deep sleep, frequency of turning over)

[0882] Step 2:

[0883] User devices collect behavioral data during the day.

[0884] Input: Data from smartphone's GPS, pedometer sensor, and heart rate sensor

[0885] How it works: When you wake up and take your smartphone with you, the built-in GPS, pedometer, and heart rate sensor will record your steps, heart rate, activity level, and location. This data is collected in real time and stored locally.

[0886] Output: Daily activity data (step count, heart rate, activity level, GPS location information)

[0887] Step 3:

[0888] The user's device collects emotional data.

[0889] Input: Audio and image data obtained from the smartphone's microphone and camera

[0890] How it works: The emotion engine uses the smartphone's microphone and camera to analyze the user's voice and facial expressions, recognizing their emotional state in real time and recording it as emotional data.

[0891] Output: Emotion data (voice analysis results, facial expression analysis results)

[0892] Step 4:

[0893] The user device sends data to the cloud server.

[0894] Input: Sleep data, daytime behavior data, emotional data

[0895] Specific operation: Collected data is sent to a cloud server periodically or upon user instruction. The data is sent using a secure protocol (e.g., HTTPS).

[0896] Output: Data saved to a database on a cloud server

[0897] Step 5:

[0898] The cloud server pre-processes the data.

[0899] Input: Sleep data, daytime behavior data, emotional data

[0900] Specific operation: The cloud server cleans the received data, imputes missing values, and performs any necessary normalization. Preprocessing software (e.g., Python's pandas library) is used.

[0901] Output: Preprocessed data

[0902] Step 6:

[0903] Cloud servers run the AI ​​algorithms.

[0904] Input: Preprocessed data

[0905] Specific operation: Analyzes pre-processed data using AI algorithms (e.g., TensorFlow, PyTorch), and performs calculations to assess the user's health status and security risks.

[0906] Output: Health status assessment, security risk assessment

[0907] Step 7:

[0908] The cloud server generates the results and sends them to the user device.

[0909] Input: Health status assessment, security risk assessment

[0910] Specific operation: Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated for the user, and the generated results are sent to the user's smartphone.

[0911] Output: Health assessment and recommended actions

[0912] Step 8:

[0913] The user terminal notifies the received result.

[0914] Input: Health assessment and recommended actions

[0915] Specific behavior: The user device will display the results it receives. Notification methods can include pop-up messages, alerts, dashboard displays, etc. The user can then adjust their actions accordingly.

[0916] Output: Informational message to the user

[0917] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0918] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0919] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0920] [Third embodiment]

[0921] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0922] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0923] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0924] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0925] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0926] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0927] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0928] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0929] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0930] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0931] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0932] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0933] The present invention relates to a system that uses a user's sleep data and daytime behavior data to provide an individualized physical condition assessment and recommended actions. This system is mainly composed of a smartphone-type device and a cloud server.

[0934] System Configuration

[0935] 1. Smartphone devices

[0936] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[0937] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[0938] (Terminal) Sends collected data to the cloud server at specified intervals.

[0939] 2. Cloud Server

[0940] (Server) Receives data sent from the smartphone device and stores it in a database.

[0941] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[0942] (Server) Runs AI algorithms using preprocessed data to assess the user's health status.

[0943] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[0944] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[0945] Program processing

[0946] Data collection

[0947] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[0948] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[0949] (Device) Collected data is sent to the cloud server at regular intervals.

[0950] Data analysis

[0951] (Server) The cloud server stores the received sleep data and daytime behavior data in a database.

[0952] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[0953] (Server) The pre-processed data is input into an AI algorithm to comprehensively assess the user's health status, including the user's sleep quality, fatigue level, and lifestyle risks.

[0954] (Server) Based on the evaluation results, a physical condition assessment and recommended actions for the next day (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[0955] Result notification

[0956] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[0957] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[0958] (User) The user can check the health assessment and recommended actions and adjust their actions based on the advice.

[0959] For example, if a user has slept less than usual, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a short break during the day." As a result, users can receive advice optimized for their individual health condition and improve their lifestyle habits.

[0960] The processing flow will be explained below.

[0961] Step 1: Start collecting data

[0962] (Device) When the user starts going to bed, the device begins collecting sleep data using its built-in sensors (accelerometer, heart rate sensor, etc.).

[0963] (Device) The user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting user inactivity.

[0964] Step 2: Collecting daytime behavioral data

[0965] (Device) When you wake up in the morning, sleep data collection will stop based on the user's manual input or a designed pattern.

[0966] (Device) When the user begins their activities, data on their daily activities such as number of steps, heart rate, amount of exercise, and GPS location information is collected.

[0967] Step 3: Sending data

[0968] (Device) The collected sleep data and daytime behavior data are sent to the cloud server in real time or at specified intervals.

[0969] (Terminal) After data transmission is complete, the terminal notifies the server of this fact.

[0970] Step 4: Receiving and storing data

[0971] (Server) Receives data sent from the device and stores it securely in a database, where it is identified and classified for each user.

[0972] Step 5: Preprocessing the data

[0973] (Server) Performs preprocessing such as noise removal, missing value imputation, and normalization on the received data, enabling accurate and consistent analysis.

[0974] Step 6: AI analysis

[0975] (Server) The preprocessed data is input into an AI model for analysis, which evaluates the user's sleep quality, daytime behavior patterns, risk of lifestyle-related diseases, etc.

[0976] (Server) As a result of the analysis, a detailed assessment of the user's health status is generated.

[0977] Step 7: Generate recommended actions

[0978] (Server) Based on the analysis results, specific recommended actions are generated, such as "go to bed early," "get moderate exercise," and "eat a specific diet."

[0979] (Server) Create a package that summarizes recommended actions and health assessments.

[0980] Step 8: Sending the results

[0981] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[0982] (Terminal) Properly receives the transmitted data and prepares it for presentation to the user.

[0983] Step 9: Notify users

[0984] The device application notifies the user of the received health assessment and recommended actions via pop-ups, alerts, and in-app dashboard displays.

[0985] (User) The user checks the notification and follows the recommended actions presented to improve their health.

[0986] Through this series of steps, users can receive an individually customized health assessment and recommended actions, which they can then incorporate into their daily lives to effectively manage their health.

[0987] Example 1

[0988] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0989] In recent years, with growing interest in health management, highly accurate data collection and analysis are required to provide specific recommended actions based on individual health conditions. However, conventional systems have had issues in providing appropriate advice to users due to limited types of collected data and insufficient analysis accuracy. The object of the present invention is to provide a system that collects a variety of biometric and activity data from users and analyzes them using advanced artificial intelligence to provide specific health assessments and recommended actions based on individual health conditions.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0991] In this invention, the server includes means for collecting a user's biometric data, means for collecting daytime activity data, means for transmitting the collected biometric data and activity data to a network server, means for preprocessing the data received by the network server, means for executing artificial intelligence for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the artificial intelligence, means for transmitting the generated health condition assessment and recommended actions to a user terminal, and means for notifying the user of the transmitted health condition assessment and recommended actions. This makes it possible to collect and analyze a variety of user data with high accuracy and provide specific advice based on an individual's health condition.

[0992] "Biometric data" refers to information about the user's physical condition, specifically including the amount of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[0993] "Activity data" refers to information about a user's daily activities, and specifically includes the number of steps taken, heart rate, amount of exercise, location information, etc.

[0994] "Network server" refers to a server that receives, stores, and processes data sent from a user terminal.

[0995] "Preprocessing" refers to processing of received data, such as removing noise, filling in missing values, and normalizing data.

[0996] "Artificial intelligence" refers to algorithms that assess a user's health status based on pre-processed data.

[0997] "Health assessment" refers to the results of an assessment of the user's health condition based on the analysis results of artificial intelligence.

[0998] "Recommended actions" refer to actions that are specifically recommended to the user based on the health assessment.

[0999] "User terminal" refers to a device that collects biometric data and activity data and transmits it to a network server, and specifically includes a smartphone.

[1000] "Notification" refers to a means for informing a user of the generated health assessment and recommended actions, including pop-up messages, alerts, dashboard displays, etc.

[1001] The present invention relates to a system that uses a user's biometric data and daily activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone-type device and a cloud server.

[1002] The system configuration includes the following elements:

[1003] Data collection by terminal

[1004] (Device) When the user goes to bed, the smartphone device collects sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, and the frequency of turning over in bed.

[1005] (Device) During daytime activities, data on daily activities such as steps taken, heart rate, exercise volume, and GPS location information is collected using the device's activity tracker and location services.

[1006] (Device) Collected data is sent to the cloud server at regular intervals. The sending interval can be changed in the system settings, but is generally set to every hour.

[1007] Data analysis using a cloud server

[1008] (Server) The cloud server receives biometric and activity data sent from the user's smartphone and stores it in a database. Each data item is time-stamped for later analysis.

[1009] (Server) Performs preprocessing on the received data. Specifically, it performs filtering to remove noise, fills in missing values, and normalizes the data. For example, it filters out abnormal heart rate data and fills in missing data using the historical average.

[1010] (Server) The pre-processed data is used to run an artificial intelligence (AI) algorithm, which then compares the data with previously collected data and general health indicators to comprehensively assess the user's health, including the user's sleep quality, fatigue level, and lifestyle risks.

[1011] (Server) Based on the AI's evaluation results, a physical condition assessment and specific recommended actions for the next day are generated. Specific recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients."

[1012] Result notification

[1013] (Server) The generated health assessment and recommended action plan are sent to the user's smartphone, where the user can then receive push notifications and alerts.

[1014] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The user can check the notification content and follow the recommended actions.

[1015] Specific examples

[1016] If a user sleeps less than usual, the sensor records this information and sends the data to the cloud. The cloud server analyzes the data and generates specific advice, such as "go to bed earlier tonight" or "take a short break during the day." This advice is immediately sent to the user's device and notified to the user's application. The user can then adjust their daily schedule accordingly.

[1017] Prompt Sentence Examples

[1018] You can ask the generative AI model for a health assessment and recommended actions using prompts like the following:

[1019] Use the following sleep and daytime activity data from your users to assess their health and provide specific recommendations.

[1020] (Sleep data)

[1021] Sleep time: 6 hours

[1022] Deep sleep percentage: 20%

[1023] Turning frequency: 15 times

[1024] (Daytime behavior data)

[1025] Steps: 8,000

[1026] Heart rate: 80 BPM average

[1027] Activity level: High

[1028] GPS location: Commute from home to work

[1029] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1030] Step 1:

[1031] (Device) When the user goes to bed, the smartphone device begins collecting sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, the frequency of turning over in bed, etc. The input is biosignal data obtained from the sensors, and the output is structured sleep data.

[1032] Step 2:

[1033] When the user wakes up, the device automatically switches modes and starts collecting daily activity data. The collected data includes the number of steps taken by the pedometer, heart rate data from the heart rate sensor, activity level data from the activity tracker, and location data from GPS. The input is signal data obtained from various sensors during the day, and the output is structured activity data.

[1034] Step 3:

[1035] (Device) The collected biometric data and activity data are sent to the cloud server at regular intervals (e.g., every hour). The data is encoded and transmitted securely over the network. The input is data obtained from various data collection modules, and the output is the transmitted data arriving at the cloud server.

[1036] Step 4:

[1037] (Server) The cloud server receives the biometric data and activity data sent from the user device and stores them in a database. The input is structured data sent via the network, and the output is the biometric data and activity data stored in the database.

[1038] Step 5:

[1039] (Server) Performs preprocessing on the received data. Preprocessing includes filtering to remove noise, imputing missing values, and normalizing the data. For example, filtering out abnormal heart rate data and imputing missing values ​​using the historical average. The input is raw data, and after preprocessing, clean, normalized data is output.

[1040] Step 6:

[1041] (Server) Using the preprocessed data, an AI algorithm is run to evaluate the user's health status. The AI ​​evaluates the user's health status by comparing it with past data and health indicators. The evaluation includes sleep quality, fatigue level, lifestyle risks, etc. The input is the preprocessed data, and the output is a health status evaluation result.

[1042] Step 7:

[1043] (Server) Based on the AI ​​evaluation results, a health assessment and specific recommended actions for the next day are generated. Recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients." The input is the health assessment results, and the output is a health assessment and a recommended action plan.

[1044] Step 8:

[1045] (Server) The generated health assessment and recommended actions are sent to the user's smartphone. The input is the AI ​​analysis results, and the output is the recommended actions and assessment sent to the user's device.

[1046] Step 9:

[1047] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The input is the recommended action and evaluation data sent from the server, and the output is a visual or audio notification to the user. The user checks the notification and follows the recommended action.

[1048] (Application example 1)

[1049] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1050] In recent years, there has been growing interest in managing users' health status. However, there are only a limited number of systems that provide specific recommendations based on individual users' health status. Especially for in-store use, there is a need for systems that utilize users' location information to make appropriate recommendations. Furthermore, a mechanism for improving recommendations using user feedback is also needed, but this is similarly lacking. Given this background, there is a need for the development of a system that can provide appropriate in-store recommendations for products and facility use based on the user's health status and collect feedback for further improvement.

[1051] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1052] In this invention, the server includes means for collecting a user's sleep data, means for collecting daytime behavior data, means for transmitting the collected sleep data and daytime behavior data to a cloud server, means for preprocessing the received data in the cloud server, means for executing an AI algorithm for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the AI ​​algorithm, means for transmitting the generated health condition assessment and recommended actions to a user terminal, means for notifying the user of the transmitted health condition assessment and recommended actions, means for presenting recommended products and facilities suitable for the user using in-store location information, means for encouraging the user to take action based on the recommended products and facilities, and means for collecting feedback and improving the health assessment and recommendations based on the usage history of the recommended products and facilities. This makes it possible to make appropriate recommendations based on the user's health condition, improve the user experience in the physical store, and continuously improve the accuracy and effectiveness of the system using the feedback function.

[1053] "User" refers to an individual who uses the system to manage their own health condition.

[1054] "Sleep data" refers to information collected while a user is asleep, and specifically includes the amount of sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[1055] "Daytime activity data" refers to information about the activities a user engages in during the day, including, for example, the number of steps taken, heart rate, activity level, and GPS location information.

[1056] "Cloud server" refers to a remote server that stores and processes data over the Internet.

[1057] "Preprocessing" refers to the processing that the cloud server performs on the raw data it receives, and includes data cleaning, missing value imputation, normalization, etc.

[1058] "Health Status" refers to the results of an assessment of the user's physical and mental state.

[1059] "AI Algorithm" refers to the artificial intelligence algorithm used to assess a user's health status based on collected data.

[1060] "Recommended actions" refers to specific actions or options suggested to users based on the evaluation results of the AI ​​algorithm.

[1061] "Notification" refers to the act of sending a message or alert to a user's device to convey information to the user.

[1062] "In-store location information" refers to information used to identify a user's current location within a physical store.

[1063] "Recommended Products" refers to products and services suggested to you based on your health status.

[1064] "Facilities" refers to the locations of facilities and services that users are encouraged to use within the physical store.

[1065] "Means to encourage behavior" refers to mechanisms that encourage users to use recommended products or facilities.

[1066] "Feedback" refers to the opinions and ratings provided by users after using the system.

[1067] "Usage history" refers to the history of products and services a user has used in the past and recommended actions.

[1068] This invention relates to a system that uses a user's sleep data and daytime activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone device and a cloud server.

[1069] 1. Smartphone devices

[1070] The smartphone device has the following features:

[1071] Sleep data collection: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[1072] Collection of daytime activity data: During daytime activities, we collect daytime activity data such as steps, heart rate, activity level, and GPS location information.

[1073] Data transmission: The collected data is sent to the cloud server at the specified interval.

[1074] 2. Cloud Server

[1075] The cloud server has the following functions:

[1076] Data reception and storage: Receives data sent from the smartphone device and stores it in a database.

[1077] Data preprocessing: Check the received data for noise and missing data, and preprocess it if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[1078] Data analysis: Using the pre-processed data, AI algorithms are run to assess the user's health status, including sleep quality, fatigue level, and lifestyle risks.

[1079] Generation of recommended actions: Based on the assessment results, a physical condition assessment and specific recommended actions based on that assessment (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[1080] Notification of results: The generated physical condition assessment and recommended action plan are sent to the user's device.

[1081] Gathering feedback: Gathering user feedback to improve our ratings and recommendations.

[1082] 3. Use in physical stores

[1083] This system can also be used in physical stores, where it can utilize the user's location information to provide the following services:

[1084] Location information identification: The user's current location is identified using in-store Wi-Fi and Bluetooth beacons.

[1085] Recommended products: Based on the user's health data, the app will suggest suitable products and facilities to use. For example, if you need to relax, it will display recommended menu items at a cafe.

[1086] Action promotion: Send notifications to your smartphone encouraging you to use recommended products or facilities.

[1087] Feedback collection: Collect feedback from users after using the recommended products to help improve the system.

[1088] Hardware and software used

[1089] Smartphone device: Collecting user data and running applications

[1090] Cloud Server: Data analysis and generation of recommended actions

[1091] Data storage and processing: Using AWS (Amazon Web Services) or GCP (Google Cloud Platform)

[1092] Running AI algorithms: using TensorFlow

[1093] Specific examples

[1094] If a user has slept less than usual, the system can analyze the data and provide specific recommendations such as "go to bed earlier tonight" or "take a short break during the day." If the user is in a physical store, the system can determine that they need to relax and recommend a relaxation menu for the cafe.

[1095] Prompt Sentence Examples

[1096] "Please propose a health recommendation application to be provided in physical stores based on the user's sleep data and daytime behavior data. For example, an application that makes recommendations on what items to purchase, which facilities to use, etc. Please also provide detailed information on specific features and how to implement them."

[1097] The above are details of a specific mode for carrying out the invention. The system allows users to receive personalized health advice and improve their lifestyle habits while enhancing their in-store experience.

[1098] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1099] Step 1: Data collection

[1100] The device collects the user's sleep data while they sleep. When the user goes to bed, the device's built-in sensors and external sensor devices are activated to collect data such as sleep time, percentage of deep sleep, and frequency of turning over in bed. The collected data is temporarily stored in a format that can be used immediately.

[1101] Step 2: Collecting daytime behavioral data

[1102] The device collects user behavior data during daytime activities. After the user wakes up, the device automatically switches to daytime activity data collection mode. Step count, heart rate, activity level, GPS location information, etc. are collected and temporarily stored.

[1103] Step 3: Send data

[1104] The device sends the collected sleep data and daytime behavior data to a cloud server. At specified intervals, the data is sent in batches to the cloud server for processing on the server side. The data is encrypted before transmission and decrypted upon arrival.

[1105] Step 4: Receiving and storing data

[1106] The server receives the data sent from the device and stores it in a database. To securely store the received data, it uses cloud storage services such as AWS and GCP. The data is then classified by user and prepared for analysis.

[1107] Step 5: Data Preprocessing

[1108] The server preprocesses the data it receives. Preprocessing includes data cleaning, missing value imputation, normalization, etc. This removes noise from the data and organizes it into a consistent format. Libraries such as Pandas are sometimes used for data cleaning.

[1109] Step 6: Data analysis

[1110] The server uses the preprocessed data to run AI algorithms to assess the user's health. Specifically, collected sleep data and daytime behavior data are input into a generative AI model such as TensorFlow to evaluate sleep quality, fatigue level, lifestyle risks, etc.

[1111] Step 7: Generate recommended actions

[1112] Based on the results of the AI ​​algorithm's evaluation, the server generates a physical condition assessment and specific recommended actions, such as performing relaxation exercises, going to bed earlier, and specific dietary advice.

[1113] Step 8: Notification of results

[1114] The server then sends the generated health assessment and recommended actions to the user's device, and notifications are sent in real time as pop-up messages or alerts on the user's smartphone.

[1115] Step 9: Identify your location

[1116] The device identifies the user's location within the physical store, detects the user's current location using Wi-Fi or Bluetooth beacons, and sends the location information to a cloud server.

[1117] Step 10: Recommendations

[1118] The server generates recommended products and facilities based on the user's health data and location information and sends them to the device. For example, a user who needs to relax will be shown recommended menu items at a cafe.

[1119] Step 11: Drive action

[1120] The device will send notifications to the user encouraging them to use the recommended products and facilities. These notifications will be displayed on the user's device and will serve as a guide for actions in the physical store.

[1121] Step 12: Gather feedback

[1122] The server collects user feedback to improve the system's ratings and recommendations. Feedback is entered by users through the application and is reflected in future recommendations.

[1123] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1124] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[1125] System Configuration

[1126] 1. Smartphone devices

[1127] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[1128] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[1129] (Device) Equipped with an emotion engine that analyzes the user's voice and facial expressions, it collects emotional data from the user. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[1130] (Terminal) Sends collected data to the cloud server at specified intervals.

[1131] 2. Cloud Server

[1132] (Server) Receives data sent from the smartphone device and stores it in a database.

[1133] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[1134] (Server) Runs AI algorithms using the preprocessed data to assess the user's health, taking into account emotional data.

[1135] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[1136] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[1137] Program processing

[1138] Data collection

[1139] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[1140] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[1141] (Device) During the day, the emotion engine analyzes voice and facial expressions, recognizes the user's emotions in real time, and collects them as data.

[1142] (Device) Collected data is sent to the cloud server at regular intervals.

[1143] Data analysis

[1144] (Server) The cloud server stores the received sleep data, daytime behavior data, and emotion data in a database.

[1145] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[1146] (Server) The pre-processed data is fed into an AI algorithm to provide a comprehensive assessment of the user's health status, including the user's sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[1147] (Server) Adjusts health assessment based on emotional data, for example, if the user is feeling stressed, prioritizes recommended actions to reduce that stress.

[1148] Result notification

[1149] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[1150] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[1151] (User) The user checks the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[1152] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[1153] The processing flow will be explained below.

[1154] Step 1: Start collecting sleep data

[1155] (Device) When the user goes to bed, the device starts collecting sleep data using built-in sensors (accelerometer, heart rate sensor, etc.).

[1156] (Device) A user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting inactivity.

[1157] Step 2: Collecting daytime behavioral data

[1158] (Device) When you wake up in the morning, stop collecting sleep data and switch to daytime activity data collection mode.

[1159] (Device) Measures the user's daytime activity and records steps, heart rate, activity level, and GPS location.

[1160] Step 3: Collecting emotion data

[1161] (Device) The user's voice data and facial expression data are collected during the day using a microphone and camera, and the emotion engine analyzes this data to recognize the user's emotions.

[1162] (Device) Records emotional data at regular intervals during the day and transmits it to the cloud server as needed.

[1163] Step 4: Sending data

[1164] (Device) The collected sleep data, daytime behavior data, and emotional data are sent to a cloud server periodically or in real time.

[1165] Step 5: Receiving and storing data

[1166] (Server) Receives data sent from the device and stores it in a secure database. Data is identified for each user.

[1167] Step 6: Preprocessing the data

[1168] (Server) Preprocessing is performed on the received data, including noise removal, missing value completion, and normalization. This allows for highly accurate analysis.

[1169] Step 7: AI analysis

[1170] (Server) The preprocessed data is input into an AI model to comprehensively evaluate the user's health condition, including the risk of lifestyle-related diseases based on sleep quality, daytime behavior patterns, and emotional state.

[1171] (Server) Adjusts the physical condition assessment on a case-by-case basis based on emotional data. For example, if the user is in a high stress state, prioritizing stress relief.

[1172] Step 8: Generate recommended actions

[1173] (Server) Based on the analysis results, specific recommended actions are generated for the user (e.g., "go to bed early," "do some light exercise," "take time to relax," etc.).

[1174] (Server) The generated health assessment and recommended actions are compiled into a single package.

[1175] Step 9: Sending the results

[1176] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[1177] (Terminal) Receives transmitted data and prepares analysis results.

[1178] Step 10: Notify users

[1179] The device will notify the user of their health status assessment and recommended actions via the application, which may include pop-up messages, alerts, or in-app dashboard displays.

[1180] (User) The user reviews the notification and adjusts their behavior according to the recommended actions and advice provided.

[1181] This step allows users to receive a detailed health assessment including emotional data and a personalized improvement plan, helping them take greater control of their health in their daily lives.

[1182] Example 2

[1183] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] Current health management systems can collect a user's sleep data and daytime behavior data individually, but it is difficult to analyze them comprehensively and provide a comprehensive health assessment that includes emotional data. They also lack the ability to provide specific recommended actions that take into account the user's emotional state. This can lead to issues such as users being unable to take appropriate actions and making it difficult to maintain their health.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1186] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for evaluating the health status based on the preprocessed data, thereby enabling a comprehensive health assessment including the user's emotional state and providing specific and individual recommended actions based on the assessment.

[1187] "Sleep data" refers to data related to the user's sleep, such as the user's sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[1188] "Daytime activity data" refers to data about a user's daytime activities, such as the user's steps, heart rate, activity level, and GPS location information.

[1189] "Emotional Data" is data about a user's emotional state collected through analysis of the user's voice and facial expressions.

[1190] A "cloud server" is a remote server that can receive collected data, store it, analyze it as needed, and send the results to a user terminal.

[1191] "Preprocessing" refers to performing processes such as data cleaning, missing value imputation, and normalization on collected data.

[1192] "AI algorithm" is an artificial intelligence technology that uses pre-processed data to assess a user's health status and generate a physical condition assessment and recommended actions.

[1193] "Health Assessment" is the result of an assessment of the user's health condition, generated based on an AI algorithm.

[1194] "Recommended actions" are specific actions that users should take based on their physical condition assessment.

[1195] "User terminal" means a device that can be operated by a user, which transmits collected data and receives and notifies health assessments and recommended actions.

[1196] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[1197] System Configuration

[1198] 1. Smartphone devices

[1199] Device: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[1200] Device: During daytime activities, data on daily activities such as steps, heart rate, activity level, and GPS location is collected.

[1201] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions to collect emotional data. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[1202] Terminal: Sends collected data to the cloud server at specified intervals.

[1203] 2. Cloud Server

[1204] Server: Receives data sent from the smartphone device and stores it in a database.

[1205] Server: Checks the received data for noise and missing data, and performs preprocessing if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[1206] Server: Runs AI algorithms using pre-processed data to assess the user's health, taking emotional data into account.

[1207] Server: Based on the evaluation results, a physical condition evaluation and specific recommended actions are generated.

[1208] Server: Sends the generated physical condition assessment and recommended action plan to the user's terminal.

[1209] Specific examples

[1210] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[1211] Prompt Sentence Examples

[1212] An example of a prompt might be something like the following, given to a generative AI model:

[1213] If a user has had less sleep than usual and the emotion engine detects high stress during the day, what recommended actions should the system provide?

[1214] In this way, a system can be provided that enables individual health management based on the user's multifaceted data.

[1215] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1216] Step 1:

[1217] Data Collection - Sleep Data

[1218] Device: When a user goes to bed, sleep mode is initiated using the smartphone's built-in sensors (e.g., accelerometer, microphone) or an external sensor device (e.g., smartwatch).

[1219] Input: User's bedtime input.

[1220] Processing: Detects when sleep begins from bedtime and records sleep time, percentage of deep sleep, and frequency of turning over.

[1221] Output: Recorded sleep data. Specifically, data is collected every minute after falling asleep.

[1222] Step 2:

[1223] Data collection - daytime behavior data

[1224] Device: When the user wakes up in the morning, the smartphone switches to daytime activity data collection mode. The smartphone acquires data using built-in sensors (e.g., accelerometer, GPS, heart rate monitor).

[1225] Input: Enter the user's wake-up time.

[1226] Processing: Collection of daytime activity data begins from the time you wake up, and records steps, heart rate, activity level, and GPS location.

[1227] Output: Recorded daytime behavior data. Specifically, data is updated and recorded every 5 minutes.

[1228] Step 3:

[1229] Data Collection - Emotional Data

[1230] Device: During daytime activities, the emotion engine analyzes voice and facial expressions in real time, using the microphone and camera to detect changes in voice tone and facial expressions.

[1231] Input: Voice and facial expression data from the user during daytime activities.

[1232] Processing: Analyzes voice and facial expressions in real time to determine emotional states such as stress, joy, and sadness.

[1233] Output: Recorded emotional data. Specifically, the data is collected the moment the user starts talking, and the emotional state is analyzed every minute.

[1234] Step 4:

[1235] Data transmission

[1236] Device: The device transmits collected sleep data, daytime behavior data, and emotional data to a cloud server at regular intervals.

[1237] Input: All recorded data (sleep data, daytime behavior data, emotional data).

[1238] Processing: Data is compressed, encrypted and sent to the cloud server.

[1239] Output: Data sent to the cloud server. Specifically, data is sent once every hour.

[1240] Step 5:

[1241] Receiving and storing data

[1242] Server: The cloud server receives the data sent from the device and stores it in a database.

[1243] Input: Data sent from the terminal.

[1244] Processing: Receiving data and saving it to a database.

[1245] Output: Stored data. The specific operation is to execute the procedure to store the received data in real time.

[1246] Step 6:

[1247] Data Preprocessing

[1248] Server: Performs preprocessing on the received data.

[1249] Input: Saved data.

[1250] Processing: Data cleaning, missing value imputation, and normalization. Specifically, outliers are removed, missing data is imputed with the mean, and all data is normalized to the range 0 to 1.

[1251] Output: Preprocessed data. The specific operation is to run a process to preprocess the data every hour.

[1252] Step 7:

[1253] Health assessment

[1254] Server: Inputs preprocessed data into the AI ​​algorithm to assess the user's health status.

[1255] Input: Preprocessed data.

[1256] Processing: AI algorithms assess sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[1257] Output: Health assessment results. Specific actions include assessing health status once a day at night.

[1258] Step 8:

[1259] Tailoring recommended actions with emotional data

[1260] Server: Considers emotional data and adjusts health assessment based on the user's emotional state.

[1261] Input: Health assessment results, emotion data.

[1262] Processing: Adjusting health assessment based on emotional data and generating specific recommended actions. For example, if stress levels are high, recommending relaxation activities.

[1263] Output: Adjusted health assessment and recommended actions. The specific behavior is to adjust the recommended actions every 30 minutes based on the assessment results.

[1264] Step 9:

[1265] Generate and send recommended actions

[1266] Server: Sends the health assessment and recommended actions generated using an AI algorithm to the user's device.

[1267] Input: Tailored health assessment and recommended actions.

[1268] Processing: Generate recommended actions and send them to the device.

[1269] Output: Data sent to the user terminal. Specifically, it is sent to the user terminal once at night.

[1270] Step 10:

[1271] User Notification

[1272] Device: The smartphone uses notifications to inform the user of their health assessment and recommended actions.

[1273] Input: Received health assessment and recommended action.

[1274] Actions: Notifications via pop-up messages, alerts, and dashboard displays.

[1275] Output: A notification message to the user. The specific behavior is to notify the user immediately to provide timely follow-up.

[1276] Step 11:

[1277] Implementing the recommended actions

[1278] User: The user reviews the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[1279] Input: Health assessment and recommended actions provided by the device.

[1280] Treatment: Adjust your daily schedule and activities according to the recommended actions.

[1281] Output: The recommended actions that were taken. Specific actions include adjusting your bedtime or practicing relaxation techniques.

[1282] (Application example 2)

[1283] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1284] Conventional health assessment systems could only assess a user's health based on their sleep data and daytime behavior data, and did not take emotional data into account. As a result, the health assessment was sometimes inaccurate. Furthermore, because no assessment related to security risks was performed, appropriate security recommendations were not made, especially for users in a physically or mentally unstable state.

[1285] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1286] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for assessing health status and security risks based on the preprocessed data, thereby enabling a comprehensive assessment of the user's physical condition and security risks, including their emotional state.

[1287] "Sleep Data" refers to information collected while a user is asleep, including, specifically, the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[1288] "Daytime Activity Data" refers to information about a user's activities during the day, including steps taken, heart rate, activity level, and GPS location information.

[1289] "Emotional data" refers to information that indicates a user's emotional state, including data collected through voice analysis and facial expression analysis.

[1290] A "cloud server" refers to a remote server for storing, managing, and processing data via the Internet.

[1291] "Preprocessing" refers to processes such as cleaning, missing value imputation, and normalization performed on data, and is a basic process for improving the accuracy of analysis.

[1292] "AI algorithm" refers to a computational method that uses artificial intelligence technology to analyze data and make specific evaluations.

[1293] "Health assessment" refers to the process of comprehensively assessing a user's health status based on collected data.

[1294] "Security risk assessment" refers to the process of comprehensively assessing security risks, taking into account the user's emotional state and physical condition.

[1295] "Recommended actions" refers to advice on actions that a user should take based on the results of the health assessment and security risk assessment.

[1296] "Notification" refers to the process of sending alerts or messages to users to provide information.

[1297] This invention is a system that collects and analyzes a user's sleep data, daytime behavior data, and emotional data to provide an individualized physical condition assessment and recommended actions. The system aims to evaluate the user's health status and security risks and recommend appropriate actions.

[1298] The system mainly consists of a user terminal, a cloud server, and an AI algorithm.

[1299] (user device)

[1300] A smartphone is used as the user terminal. The smartphone collects the following data using built-in sensors and external sensor devices (e.g., wearable devices).

[1301] 1. Sleep data:

[1302] This data is collected when the user goes to bed, specifically including the duration of sleep, the percentage of deep sleep, the frequency of turning over in bed, etc. This data is recorded using the smartphone's accelerometer and proximity sensor.

[1303] 2. Daytime behavior data:

[1304] This includes steps, heart rate, activity level, GPS location, etc. This data is recorded using the smartphone's built-in GPS and pedometer sensors.

[1305] 3. Emotional Data:

[1306] This data is collected through voice and facial expression analysis, and is analyzed in real time by an emotion engine (e.g., voice recognition API or facial recognition software) using the smartphone's microphone and camera.

[1307] (Cloud server)

[1308] The cloud server is the main hardware that receives and stores data sent from smartphone devices and analyzes that data. The cloud server performs the following processes:

[1309] 1. Pretreatment:

[1310] The received data is cleaned, filled in with missing values, normalized, etc.

[1311] 2. AI algorithms:

[1312] The preprocessed data is used to run AI algorithms that assess the user's health and security risks, using generative AI models (e.g., TensorFlow, PyTorch).

[1313] 3. Result generation:

[1314] Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated, which are customized to take into account the user's current health status and security risks.

[1315] (User Notification)

[1316] The cloud server sends the acquired health assessment results and recommended actions to the user's smartphone, which then notifies the user of the results via pop-up messages, alerts, dashboard displays, and other methods.

[1317] (Example)

[1318] For example, if a user has slept less than usual and the emotion engine recognizes that they are under high levels of stress during the day, the system can analyze that data and provide specific recommended actions, such as "go to bed earlier tonight" or "take some relaxing breaks during the day."

[1319] Example prompt sentence:

[1320] If the user is found to have slept less than 6 hours last night and their daytime stress level is above 70:

[1321] "Security risk is high. We recommend you go home early today and get plenty of rest. Avoid any important work."

[1322] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1323] Step 1:

[1324] The user's device collects sleep data.

[1325] Input: Accelerometer and proximity sensor data from your smartphone

[1326] How it works: When a user goes to bed, the smartphone's built-in sensor records the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed. This data is then stored locally at regular intervals.

[1327] Output: Sleep data (sleep time, percentage of deep sleep, frequency of turning over)

[1328] Step 2:

[1329] User devices collect behavioral data during the day.

[1330] Input: Data from smartphone's GPS, pedometer sensor, and heart rate sensor

[1331] How it works: When you wake up and take your smartphone with you, the built-in GPS, pedometer, and heart rate sensor will record your steps, heart rate, activity level, and location. This data is collected in real time and stored locally.

[1332] Output: Daily activity data (step count, heart rate, activity level, GPS location information)

[1333] Step 3:

[1334] The user's device collects emotional data.

[1335] Input: Audio and image data obtained from the smartphone's microphone and camera

[1336] How it works: The emotion engine uses the smartphone's microphone and camera to analyze the user's voice and facial expressions, recognizing their emotional state in real time and recording it as emotional data.

[1337] Output: Emotion data (voice analysis results, facial expression analysis results)

[1338] Step 4:

[1339] The user device sends data to the cloud server.

[1340] Input: Sleep data, daytime behavior data, emotional data

[1341] Specific operation: Collected data is sent to a cloud server periodically or upon user instruction. The data is sent using a secure protocol (e.g., HTTPS).

[1342] Output: Data saved to a database on a cloud server

[1343] Step 5:

[1344] The cloud server pre-processes the data.

[1345] Input: Sleep data, daytime behavior data, emotional data

[1346] Specific operation: The cloud server cleans the received data, imputes missing values, and performs any necessary normalization. Preprocessing software (e.g., Python's pandas library) is used.

[1347] Output: Preprocessed data

[1348] Step 6:

[1349] Cloud servers run the AI ​​algorithms.

[1350] Input: Preprocessed data

[1351] Specific operation: Analyzes pre-processed data using AI algorithms (e.g., TensorFlow, PyTorch), and performs calculations to assess the user's health status and security risks.

[1352] Output: Health status assessment, security risk assessment

[1353] Step 7:

[1354] The cloud server generates the results and sends them to the user device.

[1355] Input: Health status assessment, security risk assessment

[1356] Specific operation: Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated for the user, and the generated results are sent to the user's smartphone.

[1357] Output: Health assessment and recommended actions

[1358] Step 8:

[1359] The user terminal notifies the received result.

[1360] Input: Health assessment and recommended actions

[1361] Specific behavior: The user device will display the results it receives. Notification methods can include pop-up messages, alerts, dashboard displays, etc. The user can then adjust their actions accordingly.

[1362] Output: Informational message to the user

[1363] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1364] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1365] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1366] [Fourth embodiment]

[1367] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1368] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1369] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1370] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1371] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1372] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1373] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1374] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1375] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1376] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1377] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1378] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1379] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1380] The present invention relates to a system that uses a user's sleep data and daytime behavior data to provide an individualized physical condition assessment and recommended actions. This system is mainly composed of a smartphone-type device and a cloud server.

[1381] System Configuration

[1382] 1. Smartphone devices

[1383] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[1384] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[1385] (Terminal) Sends collected data to the cloud server at specified intervals.

[1386] 2. Cloud Server

[1387] (Server) Receives data sent from the smartphone device and stores it in a database.

[1388] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[1389] (Server) Runs AI algorithms using preprocessed data to assess the user's health status.

[1390] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[1391] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[1392] Program processing

[1393] Data collection

[1394] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[1395] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[1396] (Device) Collected data is sent to the cloud server at regular intervals.

[1397] Data analysis

[1398] (Server) The cloud server stores the received sleep data and daytime behavior data in a database.

[1399] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[1400] (Server) The pre-processed data is input into an AI algorithm to comprehensively assess the user's health status, including the user's sleep quality, fatigue level, and lifestyle risks.

[1401] (Server) Based on the evaluation results, a physical condition assessment and recommended actions for the next day (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[1402] Result notification

[1403] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[1404] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[1405] (User) The user can check the health assessment and recommended actions and adjust their actions based on the advice.

[1406] For example, if a user has slept less than usual, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a short break during the day." As a result, users can receive advice optimized for their individual health condition and improve their lifestyle habits.

[1407] The processing flow will be explained below.

[1408] Step 1: Start collecting data

[1409] (Device) When the user starts going to bed, the device begins collecting sleep data using its built-in sensors (accelerometer, heart rate sensor, etc.).

[1410] (Device) The user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting user inactivity.

[1411] Step 2: Collecting daytime behavioral data

[1412] (Device) When you wake up in the morning, sleep data collection will stop based on the user's manual input or a designed pattern.

[1413] (Device) When the user begins their activities, data on their daily activities such as number of steps, heart rate, amount of exercise, and GPS location information is collected.

[1414] Step 3: Sending data

[1415] (Device) The collected sleep data and daytime behavior data are sent to the cloud server in real time or at specified intervals.

[1416] (Terminal) After data transmission is complete, the terminal notifies the server of this fact.

[1417] Step 4: Receiving and storing data

[1418] (Server) Receives data sent from the device and stores it securely in a database, where it is identified and classified for each user.

[1419] Step 5: Preprocessing the data

[1420] (Server) Performs preprocessing such as noise removal, missing value imputation, and normalization on the received data, enabling accurate and consistent analysis.

[1421] Step 6: AI analysis

[1422] (Server) The preprocessed data is input into an AI model for analysis, which evaluates the user's sleep quality, daytime behavior patterns, risk of lifestyle-related diseases, etc.

[1423] (Server) As a result of the analysis, a detailed assessment of the user's health status is generated.

[1424] Step 7: Generate recommended actions

[1425] (Server) Based on the analysis results, specific recommended actions are generated, such as "go to bed early," "get moderate exercise," and "eat a specific diet."

[1426] (Server) Create a package that summarizes recommended actions and health assessments.

[1427] Step 8: Sending the results

[1428] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[1429] (Terminal) Properly receives the transmitted data and prepares it for presentation to the user.

[1430] Step 9: Notify users

[1431] The device application notifies the user of the received health assessment and recommended actions via pop-ups, alerts, and in-app dashboard displays.

[1432] (User) The user checks the notification and follows the recommended actions presented to improve their health.

[1433] Through this series of steps, users can receive an individually customized health assessment and recommended actions, which they can then incorporate into their daily lives to effectively manage their health.

[1434] Example 1

[1435] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1436] In recent years, with growing interest in health management, highly accurate data collection and analysis are required to provide specific recommended actions based on individual health conditions. However, conventional systems have had issues in providing appropriate advice to users due to limited types of collected data and insufficient analysis accuracy. The object of the present invention is to provide a system that collects a variety of biometric and activity data from users and analyzes them using advanced artificial intelligence to provide specific health assessments and recommended actions based on individual health conditions.

[1437] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1438] In this invention, the server includes means for collecting a user's biometric data, means for collecting daytime activity data, means for transmitting the collected biometric data and activity data to a network server, means for preprocessing the data received by the network server, means for executing artificial intelligence for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the artificial intelligence, means for transmitting the generated health condition assessment and recommended actions to a user terminal, and means for notifying the user of the transmitted health condition assessment and recommended actions. This makes it possible to collect and analyze a variety of user data with high accuracy and provide specific advice based on an individual's health condition.

[1439] "Biometric data" refers to information about the user's physical condition, specifically including the amount of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[1440] "Activity data" refers to information about a user's daily activities, and specifically includes the number of steps taken, heart rate, amount of exercise, location information, etc.

[1441] "Network server" refers to a server that receives, stores, and processes data sent from a user terminal.

[1442] "Preprocessing" refers to processing of received data, such as removing noise, filling in missing values, and normalizing data.

[1443] "Artificial intelligence" refers to algorithms that assess a user's health status based on pre-processed data.

[1444] "Health assessment" refers to the results of an assessment of the user's health condition based on the analysis results of artificial intelligence.

[1445] "Recommended actions" refer to actions that are specifically recommended to the user based on the health assessment.

[1446] "User terminal" refers to a device that collects biometric data and activity data and transmits it to a network server, and specifically includes a smartphone.

[1447] "Notification" refers to a means for informing a user of the generated health assessment and recommended actions, including pop-up messages, alerts, dashboard displays, etc.

[1448] The present invention relates to a system that uses a user's biometric data and daily activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone-type device and a cloud server.

[1449] The system configuration includes the following elements:

[1450] Data collection by terminal

[1451] (Device) When the user goes to bed, the smartphone device collects sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, and the frequency of turning over in bed.

[1452] (Device) During daytime activities, data on daily activities such as steps taken, heart rate, exercise volume, and GPS location information is collected using the device's activity tracker and location services.

[1453] (Device) Collected data is sent to the cloud server at regular intervals. The sending interval can be changed in the system settings, but is generally set to every hour.

[1454] Data analysis using a cloud server

[1455] (Server) The cloud server receives biometric and activity data sent from the user's smartphone and stores it in a database. Each data item is time-stamped for later analysis.

[1456] (Server) Performs preprocessing on the received data. Specifically, it performs filtering to remove noise, fills in missing values, and normalizes the data. For example, it filters out abnormal heart rate data and fills in missing data using the historical average.

[1457] (Server) The pre-processed data is used to run an artificial intelligence (AI) algorithm, which then compares the data with previously collected data and general health indicators to comprehensively assess the user's health, including the user's sleep quality, fatigue level, and lifestyle risks.

[1458] (Server) Based on the AI's evaluation results, a physical condition assessment and specific recommended actions for the next day are generated. Specific recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients."

[1459] Result notification

[1460] (Server) The generated health assessment and recommended action plan are sent to the user's smartphone, where the user can then receive push notifications and alerts.

[1461] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The user can check the notification content and follow the recommended actions.

[1462] Specific examples

[1463] If a user sleeps less than usual, the sensor records this information and sends the data to the cloud. The cloud server analyzes the data and generates specific advice, such as "go to bed earlier tonight" or "take a short break during the day." This advice is immediately sent to the user's device and notified to the user's application. The user can then adjust their daily schedule accordingly.

[1464] Prompt Sentence Examples

[1465] You can ask the generative AI model for a health assessment and recommended actions using prompts like the following:

[1466] Use the following sleep and daytime activity data from your users to assess their health and provide specific recommendations.

[1467] (Sleep data)

[1468] Sleep time: 6 hours

[1469] Deep sleep percentage: 20%

[1470] Turning frequency: 15 times

[1471] (Daytime behavior data)

[1472] Steps: 8,000

[1473] Heart rate: 80 BPM average

[1474] Activity level: High

[1475] GPS location: Commute from home to work

[1476] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1477] Step 1:

[1478] (Device) When the user goes to bed, the smartphone device begins collecting sleep data using built-in sensors and external sensor devices. Specifically, it uses an accelerometer and heart rate sensor to record sleep time, the percentage of deep sleep, the frequency of turning over in bed, etc. The input is biosignal data obtained from the sensors, and the output is structured sleep data.

[1479] Step 2:

[1480] When the user wakes up, the device automatically switches modes and starts collecting daily activity data. The collected data includes the number of steps taken by the pedometer, heart rate data from the heart rate sensor, activity level data from the activity tracker, and location data from GPS. The input is signal data obtained from various sensors during the day, and the output is structured activity data.

[1481] Step 3:

[1482] (Device) The collected biometric data and activity data are sent to the cloud server at regular intervals (e.g., every hour). The data is encoded and transmitted securely over the network. The input is data obtained from various data collection modules, and the output is the transmitted data arriving at the cloud server.

[1483] Step 4:

[1484] (Server) The cloud server receives the biometric data and activity data sent from the user device and stores them in a database. The input is structured data sent via the network, and the output is the biometric data and activity data stored in the database.

[1485] Step 5:

[1486] (Server) Performs preprocessing on the received data. Preprocessing includes filtering to remove noise, imputing missing values, and normalizing the data. For example, filtering out abnormal heart rate data and imputing missing values ​​using the historical average. The input is raw data, and after preprocessing, clean, normalized data is output.

[1487] Step 6:

[1488] (Server) Using the preprocessed data, an AI algorithm is run to evaluate the user's health status. The AI ​​evaluates the user's health status by comparing it with past data and health indicators. The evaluation includes sleep quality, fatigue level, lifestyle risks, etc. The input is the preprocessed data, and the output is a health status evaluation result.

[1489] Step 7:

[1490] (Server) Based on the AI ​​evaluation results, a health assessment and specific recommended actions for the next day are generated. Recommended actions include "go to bed early tonight," "take a short break during the day," and "take in specific nutrients." The input is the health assessment results, and the output is a health assessment and a recommended action plan.

[1491] Step 8:

[1492] (Server) The generated health assessment and recommended actions are sent to the user's smartphone. The input is the AI ​​analysis results, and the output is the recommended actions and assessment sent to the user's device.

[1493] Step 9:

[1494] (Device) Based on the received data, the user is notified through the application. Notification methods include pop-up messages, alerts, and dashboard displays. The input is the recommended action and evaluation data sent from the server, and the output is a visual or audio notification to the user. The user checks the notification and follows the recommended action.

[1495] (Application example 1)

[1496] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1497] In recent years, there has been growing interest in managing users' health status. However, there are only a limited number of systems that provide specific recommendations based on individual users' health status. Especially for in-store use, there is a need for systems that utilize users' location information to make appropriate recommendations. Furthermore, a mechanism for improving recommendations using user feedback is also needed, but this is similarly lacking. Given this background, there is a need for the development of a system that can provide appropriate in-store recommendations for products and facility use based on the user's health status and collect feedback for further improvement.

[1498] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1499] In this invention, the server includes means for collecting a user's sleep data, means for collecting daytime behavior data, means for transmitting the collected sleep data and daytime behavior data to a cloud server, means for preprocessing the received data in the cloud server, means for executing an AI algorithm for evaluating a health condition based on the preprocessed data, means for generating a health condition assessment and recommended actions based on the results of the AI ​​algorithm, means for transmitting the generated health condition assessment and recommended actions to a user terminal, means for notifying the user of the transmitted health condition assessment and recommended actions, means for presenting recommended products and facilities suitable for the user using in-store location information, means for encouraging the user to take action based on the recommended products and facilities, and means for collecting feedback and improving the health assessment and recommendations based on the usage history of the recommended products and facilities. This makes it possible to make appropriate recommendations based on the user's health condition, improve the user experience in the physical store, and continuously improve the accuracy and effectiveness of the system using the feedback function.

[1500] "User" refers to an individual who uses the system to manage their own health condition.

[1501] "Sleep data" refers to information collected while a user is asleep, and specifically includes the amount of sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[1502] "Daytime activity data" refers to information about the activities a user engages in during the day, including, for example, the number of steps taken, heart rate, activity level, and GPS location information.

[1503] "Cloud server" refers to a remote server that stores and processes data over the Internet.

[1504] "Preprocessing" refers to the processing that the cloud server performs on the raw data it receives, and includes data cleaning, missing value imputation, normalization, etc.

[1505] "Health Status" refers to the results of an assessment of the user's physical and mental state.

[1506] "AI Algorithm" refers to the artificial intelligence algorithm used to assess a user's health status based on collected data.

[1507] "Recommended actions" refers to specific actions or options suggested to users based on the evaluation results of the AI ​​algorithm.

[1508] "Notification" refers to the act of sending a message or alert to a user's device to convey information to the user.

[1509] "In-store location information" refers to information used to identify a user's current location within a physical store.

[1510] "Recommended Products" refers to products and services suggested to you based on your health status.

[1511] "Facilities" refers to the locations of facilities and services that users are encouraged to use within the physical store.

[1512] "Means to encourage behavior" refers to mechanisms that encourage users to use recommended products or facilities.

[1513] "Feedback" refers to the opinions and ratings provided by users after using the system.

[1514] "Usage history" refers to the history of products and services a user has used in the past and recommended actions.

[1515] This invention relates to a system that uses a user's sleep data and daytime activity data to provide personalized health assessments and recommended actions. This system is primarily composed of a smartphone device and a cloud server.

[1516] 1. Smartphone devices

[1517] The smartphone device has the following features:

[1518] Sleep data collection: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[1519] Collection of daytime activity data: During daytime activities, we collect daytime activity data such as steps, heart rate, activity level, and GPS location information.

[1520] Data transmission: The collected data is sent to the cloud server at the specified interval.

[1521] 2. Cloud Server

[1522] The cloud server has the following functions:

[1523] Data reception and storage: Receives data sent from the smartphone device and stores it in a database.

[1524] Data preprocessing: Check the received data for noise and missing data, and preprocess it if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[1525] Data analysis: Using the pre-processed data, AI algorithms are run to assess the user's health status, including sleep quality, fatigue level, and lifestyle risks.

[1526] Generation of recommended actions: Based on the assessment results, a physical condition assessment and specific recommended actions based on that assessment (e.g., relaxation exercise, early bedtime, specific dietary advice, etc.) are generated.

[1527] Notification of results: The generated physical condition assessment and recommended action plan are sent to the user's device.

[1528] Gathering feedback: Gathering user feedback to improve our ratings and recommendations.

[1529] 3. Use in physical stores

[1530] This system can also be used in physical stores, where it can utilize the user's location information to provide the following services:

[1531] Location information identification: The user's current location is identified using in-store Wi-Fi and Bluetooth beacons.

[1532] Recommended products: Based on the user's health data, the app will suggest suitable products and facilities to use. For example, if you need to relax, it will display recommended menu items at a cafe.

[1533] Action promotion: Send notifications to your smartphone encouraging you to use recommended products or facilities.

[1534] Feedback collection: Collect feedback from users after using the recommended products to help improve the system.

[1535] Hardware and software used

[1536] Smartphone device: Collecting user data and running applications

[1537] Cloud Server: Data analysis and generation of recommended actions

[1538] Data storage and processing: Using AWS (Amazon Web Services) or GCP (Google Cloud Platform)

[1539] Running AI algorithms: using TensorFlow

[1540] Specific examples

[1541] If a user has slept less than usual, the system can analyze the data and provide specific recommendations such as "go to bed earlier tonight" or "take a short break during the day." If the user is in a physical store, the system can determine that they need to relax and recommend a relaxation menu for the cafe.

[1542] Prompt Sentence Examples

[1543] "Please propose a health recommendation application to be provided in physical stores based on the user's sleep data and daytime behavior data. For example, an application that makes recommendations on what items to purchase, which facilities to use, etc. Please also provide detailed information on specific features and how to implement them."

[1544] The above are details of a specific mode for carrying out the invention. The system allows users to receive personalized health advice and improve their lifestyle habits while enhancing their in-store experience.

[1545] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1546] Step 1: Data collection

[1547] The device collects the user's sleep data while they sleep. When the user goes to bed, the device's built-in sensors and external sensor devices are activated to collect data such as sleep time, percentage of deep sleep, and frequency of turning over in bed. The collected data is temporarily stored in a format that can be used immediately.

[1548] Step 2: Collecting daytime behavioral data

[1549] The device collects user behavior data during daytime activities. After the user wakes up, the device automatically switches to daytime activity data collection mode. Step count, heart rate, activity level, GPS location information, etc. are collected and temporarily stored.

[1550] Step 3: Send data

[1551] The device sends the collected sleep data and daytime behavior data to a cloud server. At specified intervals, the data is sent in batches to the cloud server for processing on the server side. The data is encrypted before transmission and decrypted upon arrival.

[1552] Step 4: Receiving and storing data

[1553] The server receives the data sent from the device and stores it in a database. To securely store the received data, it uses cloud storage services such as AWS and GCP. The data is then classified by user and prepared for analysis.

[1554] Step 5: Data Preprocessing

[1555] The server preprocesses the data it receives. Preprocessing includes data cleaning, missing value imputation, normalization, etc. This removes noise from the data and organizes it into a consistent format. Libraries such as Pandas are sometimes used for data cleaning.

[1556] Step 6: Data analysis

[1557] The server uses the preprocessed data to run AI algorithms to assess the user's health. Specifically, collected sleep data and daytime behavior data are input into a generative AI model such as TensorFlow to evaluate sleep quality, fatigue level, lifestyle risks, etc.

[1558] Step 7: Generate recommended actions

[1559] Based on the results of the AI ​​algorithm's evaluation, the server generates a physical condition assessment and specific recommended actions, such as performing relaxation exercises, going to bed earlier, and specific dietary advice.

[1560] Step 8: Notification of results

[1561] The server then sends the generated health assessment and recommended actions to the user's device, and notifications are sent in real time as pop-up messages or alerts on the user's smartphone.

[1562] Step 9: Identify your location

[1563] The device identifies the user's location within the physical store, detects the user's current location using Wi-Fi or Bluetooth beacons, and sends the location information to a cloud server.

[1564] Step 10: Recommendations

[1565] The server generates recommended products and facilities based on the user's health data and location information and sends them to the device. For example, a user who needs to relax will be shown recommended menu items at a cafe.

[1566] Step 11: Drive action

[1567] The device will send notifications to the user encouraging them to use the recommended products and facilities. These notifications will be displayed on the user's device and will serve as a guide for actions in the physical store.

[1568] Step 12: Gather feedback

[1569] The server collects user feedback to improve the system's ratings and recommendations. Feedback is entered by users through the application and is reflected in future recommendations.

[1570] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1571] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[1572] System Configuration

[1573] 1. Smartphone devices

[1574] (Device) When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[1575] (Device) During daytime activities, data on daytime activities such as steps, heart rate, activity level, and GPS location information is collected.

[1576] (Device) Equipped with an emotion engine that analyzes the user's voice and facial expressions, it collects emotional data from the user. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[1577] (Terminal) Sends collected data to the cloud server at specified intervals.

[1578] 2. Cloud Server

[1579] (Server) Receives data sent from the smartphone device and stores it in a database.

[1580] (Server) Checks the received data for noise or missing parts, and performs preprocessing if necessary.

[1581] (Server) Runs AI algorithms using the preprocessed data to assess the user's health, taking into account emotional data.

[1582] (Server) Based on the evaluation results, a physical condition evaluation and specific recommended actions based on the evaluation are generated.

[1583] (Server) The generated physical condition assessment and recommended action plan are sent to the user's terminal.

[1584] Program processing

[1585] Data collection

[1586] (Device) The user goes to bed and the device's sensors initiate sleep mode, collecting data such as sleep duration, deep sleep, and frequency of tossing and turning.

[1587] When the device wakes up in the morning, it switches to a mode for collecting daytime activity data, which includes the number of steps taken, heart rate, amount of exercise, and GPS tracking.

[1588] (Device) During the day, the emotion engine analyzes voice and facial expressions, recognizes the user's emotions in real time, and collects them as data.

[1589] (Device) Collected data is sent to the cloud server at regular intervals.

[1590] Data analysis

[1591] (Server) The cloud server stores the received sleep data, daytime behavior data, and emotion data in a database.

[1592] (Server) Perform preprocessing steps on the stored data, such as data cleaning, missing value imputation, and normalization.

[1593] (Server) The pre-processed data is fed into an AI algorithm to provide a comprehensive assessment of the user's health status, including the user's sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[1594] (Server) Adjusts health assessment based on emotional data, for example, if the user is feeling stressed, prioritizes recommended actions to reduce that stress.

[1595] Result notification

[1596] (Server) The generated health assessment and recommended actions are sent to the user's smartphone device.

[1597] (Device) The data received by the user device is notified to the user through the application. Notifications are made in the form of pop-up messages, alerts, dashboard displays, etc.

[1598] (User) The user checks the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[1599] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[1600] The processing flow will be explained below.

[1601] Step 1: Start collecting sleep data

[1602] (Device) When the user goes to bed, the device starts collecting sleep data using built-in sensors (accelerometer, heart rate sensor, etc.).

[1603] (Device) A user may manually initiate sleep mode, or the device may automatically activate sleep mode upon detecting inactivity.

[1604] Step 2: Collecting daytime behavioral data

[1605] (Device) When you wake up in the morning, stop collecting sleep data and switch to daytime activity data collection mode.

[1606] (Device) Measures the user's daytime activity and records steps, heart rate, activity level, and GPS location.

[1607] Step 3: Collecting emotion data

[1608] (Device) The user's voice data and facial expression data are collected during the day using a microphone and camera, and the emotion engine analyzes this data to recognize the user's emotions.

[1609] (Device) Records emotional data at regular intervals during the day and transmits it to the cloud server as needed.

[1610] Step 4: Sending data

[1611] (Device) The collected sleep data, daytime behavior data, and emotional data are sent to a cloud server periodically or in real time.

[1612] Step 5: Receiving and storing data

[1613] (Server) Receives data sent from the device and stores it in a secure database. Data is identified for each user.

[1614] Step 6: Preprocessing the data

[1615] (Server) Preprocessing is performed on the received data, including noise removal, missing value completion, and normalization. This allows for highly accurate analysis.

[1616] Step 7: AI analysis

[1617] (Server) The preprocessed data is input into an AI model to comprehensively evaluate the user's health condition, including the risk of lifestyle-related diseases based on sleep quality, daytime behavior patterns, and emotional state.

[1618] (Server) Adjusts the physical condition assessment on a case-by-case basis based on emotional data. For example, if the user is in a high stress state, prioritizing stress relief.

[1619] Step 8: Generate recommended actions

[1620] (Server) Based on the analysis results, specific recommended actions are generated for the user (e.g., "go to bed early," "do some light exercise," "take time to relax," etc.).

[1621] (Server) The generated health assessment and recommended actions are compiled into a single package.

[1622] Step 9: Sending the results

[1623] (Server) The generated health assessment and recommended action package is sent to the user's smartphone device.

[1624] (Terminal) Receives transmitted data and prepares analysis results.

[1625] Step 10: Notify users

[1626] The device will notify the user of their health status assessment and recommended actions via the application, which may include pop-up messages, alerts, or in-app dashboard displays.

[1627] (User) The user reviews the notification and adjusts their behavior according to the recommended actions and advice provided.

[1628] This step allows users to receive a detailed health assessment including emotional data and a personalized improvement plan, helping them take greater control of their health in their daily lives.

[1629] Example 2

[1630] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1631] Current health management systems can collect a user's sleep data and daytime behavior data individually, but it is difficult to analyze them comprehensively and provide a comprehensive health assessment that includes emotional data. They also lack the ability to provide specific recommended actions that take into account the user's emotional state. This can lead to issues such as users being unable to take appropriate actions and making it difficult to maintain their health.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1633] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for evaluating the health status based on the preprocessed data, thereby enabling a comprehensive health assessment including the user's emotional state and providing specific and individual recommended actions based on the assessment.

[1634] "Sleep data" refers to data related to the user's sleep, such as the user's sleep time, the percentage of deep sleep, and the frequency of turning over in sleep.

[1635] "Daytime activity data" refers to data about a user's daytime activities, such as the user's steps, heart rate, activity level, and GPS location information.

[1636] "Emotional Data" is data about a user's emotional state collected through analysis of the user's voice and facial expressions.

[1637] A "cloud server" is a remote server that can receive collected data, store it, analyze it as needed, and send the results to a user terminal.

[1638] "Preprocessing" refers to performing processes such as data cleaning, missing value imputation, and normalization on collected data.

[1639] "AI algorithm" is an artificial intelligence technology that uses pre-processed data to assess a user's health status and generate a physical condition assessment and recommended actions.

[1640] "Health Assessment" is the result of an assessment of the user's health condition, generated based on an AI algorithm.

[1641] "Recommended actions" are specific actions that users should take based on their physical condition assessment.

[1642] "User terminal" means a device that can be operated by a user, which transmits collected data and receives and notifies health assessments and recommended actions.

[1643] The present invention relates to a system that uses a smartphone-type device and a cloud server to collect and analyze a user's sleep data, daytime behavior data, and emotional data, and provides an individual physical condition assessment and recommended actions.

[1644] System Configuration

[1645] 1. Smartphone devices

[1646] Device: When the user goes to bed, the device uses built-in sensors and external sensor devices to collect sleep data, such as the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed.

[1647] Device: During daytime activities, data on daily activities such as steps, heart rate, activity level, and GPS location is collected.

[1648] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions to collect emotional data. This emotion engine uses a microphone and camera to analyze voice and facial expressions in real time.

[1649] Terminal: Sends collected data to the cloud server at specified intervals.

[1650] 2. Cloud Server

[1651] Server: Receives data sent from the smartphone device and stores it in a database.

[1652] Server: Checks the received data for noise and missing data, and performs preprocessing if necessary. Preprocessing includes data cleaning, missing value imputation, normalization, etc.

[1653] Server: Runs AI algorithms using pre-processed data to assess the user's health, taking emotional data into account.

[1654] Server: Based on the evaluation results, a physical condition evaluation and specific recommended actions are generated.

[1655] Server: Sends the generated physical condition assessment and recommended action plan to the user's terminal.

[1656] Specific examples

[1657] For example, if the emotion engine detects that a user has slept less than usual and is under high stress during the day, the system can analyze the data and provide specific recommended actions such as "go to bed earlier tonight" or "take a relaxing break during the day." As a result, users can receive optimal advice based on their individual health condition and emotions, and improve their lifestyle habits.

[1658] Prompt Sentence Examples

[1659] An example of a prompt might be something like the following, given to a generative AI model:

[1660] If a user has had less sleep than usual and the emotion engine detects high stress during the day, what recommended actions should the system provide?

[1661] In this way, a system can be provided that enables individual health management based on the user's multifaceted data.

[1662] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1663] Step 1:

[1664] Data Collection - Sleep Data

[1665] Device: When a user goes to bed, sleep mode is initiated using the smartphone's built-in sensors (e.g., accelerometer, microphone) or an external sensor device (e.g., smartwatch).

[1666] Input: User's bedtime input.

[1667] Processing: Detects when sleep begins from bedtime and records sleep time, percentage of deep sleep, and frequency of turning over.

[1668] Output: Recorded sleep data. Specifically, data is collected every minute after falling asleep.

[1669] Step 2:

[1670] Data collection - daytime behavior data

[1671] Device: When the user wakes up in the morning, the smartphone switches to daytime activity data collection mode. The smartphone acquires data using built-in sensors (e.g., accelerometer, GPS, heart rate monitor).

[1672] Input: Enter the user's wake-up time.

[1673] Processing: Collection of daytime activity data begins from the time you wake up, and records steps, heart rate, activity level, and GPS location.

[1674] Output: Recorded daytime behavior data. Specifically, data is updated and recorded every 5 minutes.

[1675] Step 3:

[1676] Data Collection - Emotional Data

[1677] Device: During daytime activities, the emotion engine analyzes voice and facial expressions in real time, using the microphone and camera to detect changes in voice tone and facial expressions.

[1678] Input: Voice and facial expression data from the user during daytime activities.

[1679] Processing: Analyzes voice and facial expressions in real time to determine emotional states such as stress, joy, and sadness.

[1680] Output: Recorded emotional data. Specifically, the data is collected the moment the user starts talking, and the emotional state is analyzed every minute.

[1681] Step 4:

[1682] Data transmission

[1683] Device: The device transmits collected sleep data, daytime behavior data, and emotional data to a cloud server at regular intervals.

[1684] Input: All recorded data (sleep data, daytime behavior data, emotional data).

[1685] Processing: Data is compressed, encrypted and sent to the cloud server.

[1686] Output: Data sent to the cloud server. Specifically, data is sent once every hour.

[1687] Step 5:

[1688] Receiving and storing data

[1689] Server: The cloud server receives the data sent from the device and stores it in a database.

[1690] Input: Data sent from the terminal.

[1691] Processing: Receiving data and saving it to a database.

[1692] Output: Stored data. The specific operation is to execute the procedure to store the received data in real time.

[1693] Step 6:

[1694] Data Preprocessing

[1695] Server: Performs preprocessing on the received data.

[1696] Input: Saved data.

[1697] Processing: Data cleaning, missing value imputation, and normalization. Specifically, outliers are removed, missing data is imputed with the mean, and all data is normalized to the range 0 to 1.

[1698] Output: Preprocessed data. The specific operation is to run a process to preprocess the data every hour.

[1699] Step 7:

[1700] Health assessment

[1701] Server: Inputs preprocessed data into the AI ​​algorithm to assess the user's health status.

[1702] Input: Preprocessed data.

[1703] Processing: AI algorithms assess sleep quality, daytime behavior patterns, emotional state, and lifestyle risks.

[1704] Output: Health assessment results. Specific actions include assessing health status once a day at night.

[1705] Step 8:

[1706] Tailoring recommended actions with emotional data

[1707] Server: Considers emotional data and adjusts health assessment based on the user's emotional state.

[1708] Input: Health assessment results, emotion data.

[1709] Processing: Adjusting health assessment based on emotional data and generating specific recommended actions. For example, if stress levels are high, recommending relaxation activities.

[1710] Output: Adjusted health assessment and recommended actions. The specific behavior is to adjust the recommended actions every 30 minutes based on the assessment results.

[1711] Step 9:

[1712] Generate and send recommended actions

[1713] Server: Sends the health assessment and recommended actions generated using an AI algorithm to the user's device.

[1714] Input: Tailored health assessment and recommended actions.

[1715] Processing: Generate recommended actions and send them to the device.

[1716] Output: Data sent to the user terminal. Specifically, it is sent to the user terminal once at night.

[1717] Step 10:

[1718] User Notification

[1719] Device: The smartphone uses notifications to inform the user of their health assessment and recommended actions.

[1720] Input: Received health assessment and recommended action.

[1721] Actions: Notifications via pop-up messages, alerts, and dashboard displays.

[1722] Output: A notification message to the user. The specific behavior is to notify the user immediately to provide timely follow-up.

[1723] Step 11:

[1724] Implementing the recommended actions

[1725] User: The user reviews the health assessment and recommended actions and adjusts their behavior according to the advice provided.

[1726] Input: Health assessment and recommended actions provided by the device.

[1727] Treatment: Adjust your daily schedule and activities according to the recommended actions.

[1728] Output: The recommended actions that were taken. Specific actions include adjusting your bedtime or practicing relaxation techniques.

[1729] (Application example 2)

[1730] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1731] Conventional health assessment systems could only assess a user's health based on their sleep data and daytime behavior data, and did not take emotional data into account. As a result, the health assessment was sometimes inaccurate. Furthermore, because no assessment related to security risks was performed, appropriate security recommendations were not made, especially for users in a physically or mentally unstable state.

[1732] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1733] In this invention, the server includes a means for collecting user emotional data, a means for preprocessing the received data in the cloud server, and a means for executing an AI algorithm for assessing health status and security risks based on the preprocessed data, thereby enabling a comprehensive assessment of the user's physical condition and security risks, including their emotional state.

[1734] "Sleep Data" refers to information collected while a user is asleep, including, specifically, the duration of sleep, the percentage of deep sleep, and the frequency of turning over in sleep.

[1735] "Daytime Activity Data" refers to information about a user's activities during the day, including steps taken, heart rate, activity level, and GPS location information.

[1736] "Emotional data" refers to information that indicates a user's emotional state, including data collected through voice analysis and facial expression analysis.

[1737] A "cloud server" refers to a remote server for storing, managing, and processing data via the Internet.

[1738] "Preprocessing" refers to processes such as cleaning, missing value imputation, and normalization performed on data, and is a basic process for improving the accuracy of analysis.

[1739] "AI algorithm" refers to a computational method that uses artificial intelligence technology to analyze data and make specific evaluations.

[1740] "Health assessment" refers to the process of comprehensively assessing a user's health status based on collected data.

[1741] "Security risk assessment" refers to the process of comprehensively assessing security risks, taking into account the user's emotional state and physical condition.

[1742] "Recommended actions" refers to advice on actions that a user should take based on the results of the health assessment and security risk assessment.

[1743] "Notification" refers to the process of sending alerts or messages to users to provide information.

[1744] This invention is a system that collects and analyzes a user's sleep data, daytime behavior data, and emotional data to provide an individualized physical condition assessment and recommended actions. The system aims to evaluate the user's health status and security risks and recommend appropriate actions.

[1745] The system mainly consists of a user terminal, a cloud server, and an AI algorithm.

[1746] (user device)

[1747] A smartphone is used as the user terminal. The smartphone collects the following data using built-in sensors and external sensor devices (e.g., wearable devices).

[1748] 1. Sleep data:

[1749] This data is collected when the user goes to bed, specifically including the duration of sleep, the percentage of deep sleep, the frequency of turning over in bed, etc. This data is recorded using the smartphone's accelerometer and proximity sensor.

[1750] 2. Daytime behavior data:

[1751] This includes steps, heart rate, activity level, GPS location, etc. This data is recorded using the smartphone's built-in GPS and pedometer sensors.

[1752] 3. Emotional Data:

[1753] This data is collected through voice and facial expression analysis, and is analyzed in real time by an emotion engine (e.g., voice recognition API or facial recognition software) using the smartphone's microphone and camera.

[1754] (Cloud server)

[1755] The cloud server is the main hardware that receives and stores data sent from smartphone devices and analyzes that data. The cloud server performs the following processes:

[1756] 1. Pretreatment:

[1757] The received data is cleaned, filled in with missing values, normalized, etc.

[1758] 2. AI algorithms:

[1759] The preprocessed data is used to run AI algorithms that assess the user's health and security risks, using generative AI models (e.g., TensorFlow, PyTorch).

[1760] 3. Result generation:

[1761] Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated, which are customized to take into account the user's current health status and security risks.

[1762] (User Notification)

[1763] The cloud server sends the acquired health assessment results and recommended actions to the user's smartphone, which then notifies the user of the results via pop-up messages, alerts, dashboard displays, and other methods.

[1764] (Example)

[1765] For example, if a user has slept less than usual and the emotion engine recognizes that they are under high levels of stress during the day, the system can analyze that data and provide specific recommended actions, such as "go to bed earlier tonight" or "take some relaxing breaks during the day."

[1766] Example prompt sentence:

[1767] If the user is found to have slept less than 6 hours last night and their daytime stress level is above 70:

[1768] "Security risk is high. We recommend you go home early today and get plenty of rest. Avoid any important work."

[1769] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1770] Step 1:

[1771] The user's device collects sleep data.

[1772] Input: Accelerometer and proximity sensor data from your smartphone

[1773] How it works: When a user goes to bed, the smartphone's built-in sensor records the duration of sleep, the percentage of deep sleep, and the frequency of turning over in bed. This data is then stored locally at regular intervals.

[1774] Output: Sleep data (sleep time, percentage of deep sleep, frequency of turning over)

[1775] Step 2:

[1776] User devices collect behavioral data during the day.

[1777] Input: Data from smartphone's GPS, pedometer sensor, and heart rate sensor

[1778] How it works: When you wake up and take your smartphone with you, the built-in GPS, pedometer, and heart rate sensor will record your steps, heart rate, activity level, and location. This data is collected in real time and stored locally.

[1779] Output: Daily activity data (step count, heart rate, activity level, GPS location information)

[1780] Step 3:

[1781] The user's device collects emotional data.

[1782] Input: Audio and image data obtained from the smartphone's microphone and camera

[1783] How it works: The emotion engine uses the smartphone's microphone and camera to analyze the user's voice and facial expressions, recognizing their emotional state in real time and recording it as emotional data.

[1784] Output: Emotion data (voice analysis results, facial expression analysis results)

[1785] Step 4:

[1786] The user device sends data to the cloud server.

[1787] Input: Sleep data, daytime behavior data, emotional data

[1788] Specific operation: Collected data is sent to a cloud server periodically or upon user instruction. The data is sent using a secure protocol (e.g., HTTPS).

[1789] Output: Data saved to a database on a cloud server

[1790] Step 5:

[1791] The cloud server pre-processes the data.

[1792] Input: Sleep data, daytime behavior data, emotional data

[1793] Specific operation: The cloud server cleans the received data, imputes missing values, and performs any necessary normalization. Preprocessing software (e.g., Python's pandas library) is used.

[1794] Output: Preprocessed data

[1795] Step 6:

[1796] Cloud servers run the AI ​​algorithms.

[1797] Input: Preprocessed data

[1798] Specific operation: Analyzes pre-processed data using AI algorithms (e.g., TensorFlow, PyTorch), and performs calculations to assess the user's health status and security risks.

[1799] Output: Health status assessment, security risk assessment

[1800] Step 7:

[1801] The cloud server generates the results and sends them to the user device.

[1802] Input: Health status assessment, security risk assessment

[1803] Specific operation: Based on the results of the AI ​​algorithm, a health assessment and recommended actions are generated for the user, and the generated results are sent to the user's smartphone.

[1804] Output: Health assessment and recommended actions

[1805] Step 8:

[1806] The user terminal notifies the received result.

[1807] Input: Health assessment and recommended actions

[1808] Specific behavior: The user device will display the results it receives. Notification methods can include pop-up messages, alerts, dashboard displays, etc. The user can then adjust their actions accordingly.

[1809] Output: Informational message to the user

[1810] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1812] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1813] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1814] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1815] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1816] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1817] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1818] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1819] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1820] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1821] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1822] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1823] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1824] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1825] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1826] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1827] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1828] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1829] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1830] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1831] The following is further disclosed regarding the above embodiment.

[1832] (Claim 1)

[1833] a means for collecting user sleep data;

[1834] a means for collecting intraday behavioral data;

[1835] means for transmitting the collected sleep data and daytime behavior data to a cloud server;

[1836] means for preprocessing the received data in the cloud server;

[1837] A means for running AI algorithms that assess health status based on pre-processed data; and

[1838] A means for generating a health assessment and recommended actions based on the results of the AI ​​algorithm;

[1839] means for transmitting the generated health assessment and recommended actions to a user terminal;

[1840] A means of informing the user of the submitted health assessment and recommended actions

[1841] A system including:

[1842] (Claim 2)

[1843] 10. The system of claim 1, wherein the sleep data includes the user's sleep time, percentage of deep sleep, and frequency of tossing and turning.

[1844] (Claim 3)

[1845] 10. The system of claim 1, wherein the daytime activity data includes the user's steps, heart rate, activity level, and GPS location information.

[1846] "Example 1"

[1847] (Claim 1)

[1848] a means for collecting biometric data of a user;

[1849] a means for collecting daytime activity data;

[1850] means for transmitting the collected biometric and activity data to a network server;

[1851] means for preprocessing the received data at the network server;

[1852] means for implementing artificial intelligence to assess health status based on the preprocessed data;

[1853] a means for generating a health assessment and recommended actions based on the results of the artificial intelligence;

[1854] means for transmitting the generated health assessment and recommended actions to a user terminal;

[1855] A means of informing the user of the submitted health assessment and recommended actions

[1856] A system including:

[1857] (Claim 2)

[1858] 10. The system of claim 1, wherein the biometric data includes the user's sleep duration, percentage of deep sleep, and frequency of tossing and turning.

[1859] (Claim 3)

[1860] 2. The system of claim 1, wherein the activity data includes the user's steps, heart rate, exercise volume, and location information.

[1861] "Application Example 1"

[1862] (Claim 1)

[1863] a means for collecting user sleep data;

[1864] a means for collecting intraday behavioral data;

[1865] means for transmitting the collected sleep data and daytime behavior data to a cloud server;

[1866] means for preprocessing the received data in the cloud server;

[1867] A means for running AI algorithms that assess health status based on pre-processed data; and

[1868] A means for generating a health assessment and recommended actions based on the results of the AI ​​algorithm;

[1869] means for transmitting the generated health assessment and recommended actions to a user terminal;

[1870] a means for notifying the user of the submitted health assessment and recommended actions;

[1871] A method of presenting recommended products and facilities suitable for users using in-store location information

[1872] A system including:

[1873] (Claim 2)

[1874] 10. The system of claim 1, further comprising means for encouraging user behavior based on recommended products and facilities.

[1875] (Claim 3)

[1876] 10. The system of claim 1, further comprising means for collecting feedback and improving the health assessment and recommendations based on a history of use of recommended products and facilities.

[1877] "Example 2: Combining Emotion Engines"

[1878] (Claim 1)

[1879] a means for collecting user sleep data;

[1880] a means for collecting intraday behavioral data;

[1881] a means for collecting user emotional data;

[1882] means for transmitting the collected sleep data, daytime behavior data, and emotion data to a cloud server;

[1883] means for preprocessing the received data in the cloud server;

[1884] A means for running AI algorithms that assess health status based on pre-processed data; and

[1885] A means for generating a health assessment and recommended actions based on the results of the AI ​​algorithm;

[1886] means for transmitting the generated health assessment and recommended actions to a user terminal;

[1887] A means of informing the user of the submitted health assessment and recommended actions

[1888] A system including:

[1889] (Claim 2)

[1890] 10. The system of claim 1, wherein the sleep data includes the user's sleep time, percentage of deep sleep, and frequency of tossing and turning.

[1891] (Claim 3)

[1892] 10. The system of claim 1, wherein the daytime activity data includes the user's steps, heart rate, activity level, and GPS location information.

[1893] (Claim 4)

[1894] 10. The system of claim 1, wherein the emotional data includes a voice analysis and a facial expression analysis of the user.

[1895] (Claim 5)

[1896] 2. The system of claim 1, wherein the preprocessing includes data cleaning, missing value imputation, and normalization.

[1897] "Application example 2 when combining emotion engines"

[1898] (Claim 1)

[1899] a means for collecting user sleep data;

[1900] a means for collecting intraday behavioral data;

[1901] a means for collecting user emotional data;

[1902] means for transmitting the collected sleep data, daytime behavior data, and emotion data to a cloud server;

[1903] means for preprocessing the received data in the cloud server;

[1904] A means for running AI algorithms that assess health status and security risks based on the pre-processed data; and

[1905] A means for generating a health assessment, a security risk assessment and recommended actions based on the results of the AI ​​algorithm;

[1906] means for transmitting the generated health assessment and recommended actions to a user terminal;

[1907] A means of informing the user of the submitted health assessment and recommended actions

[1908] A system including:

[1909] (Claim 2)

[1910] 10. The system of claim 1, wherein the sleep data includes the user's sleep time, percentage of deep sleep, and frequency of tossing and turning.

[1911] (Claim 3)

[1912] 10. The system of claim 1, wherein the daytime activity data includes the user's steps, heart rate, activity level, and GPS location information. [Explanation of symbols]

[1913] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting user sleep data; a means for collecting intraday behavioral data; means for transmitting the collected sleep data and daytime behavior data to a cloud server; means for preprocessing the received data in the cloud server; A means for running AI algorithms that assess health status based on pre-processed data; and A means for generating a health assessment and recommended actions based on the results of the AI ​​algorithm; means for transmitting the generated health assessment and recommended actions to a user terminal; A means of informing the user of the submitted health assessment and recommended actions A system including:

2. The system of claim 1 , wherein the sleep data includes the user's sleep time, percentage of deep sleep, and frequency of tossing and turning.

3. The system of claim 1 , wherein the daytime activity data includes the user's steps, heart rate, activity level, and GPS location information.

Images