System

Abstract

A system is provided.SOLUTION: A system, comprising: means for measuring a user's physical health indicators in real time; means for storing the measured health indicators in a database; means for analyzing the stored data to evaluate the user's health condition; means for sending a reminder notification when the user's emotional state deteriorates; means for playing relaxing music as an environment adjustment means; means for connecting the user to a social support group; and means for recording and analyzing the user's emotional diary.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many users today face mental health problems. However, when it comes to receiving psychological counseling, problems include difficulty in making appointments, a shortage of medical professionals, limited consultation time, and difficulty in explaining symptoms. This makes it difficult for users to receive early mental care, resulting in a decrease in the effectiveness of treatment. Furthermore, there are only a limited number of systems that support daily stress management and emotional relaxation, leaving users with insufficient support to improve their mental state on their own. Therefore, a system that allows users to easily and effectively receive psychological support is needed. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes a means for measuring a user's physical health indicators in real time, a means for storing the measured health indicators in a database, a means for analyzing the stored data to evaluate the user's health status, a means for sending a reminder notification when the user's emotional state declines, a means for playing relaxing music as an environmental adjustment method, a means for connecting the user to a social support group, and a means for recording and analyzing the user's emotional diary. This allows users to receive care and relaxation at appropriate times, making it easier to maintain their mental health. Furthermore, the use of social support groups promotes communication between users and strengthens their mental support network.

[0006] "User" refers to an individual who uses the System to manage their own mental and physical well-being.

[0007] "Physical health indicators" refers to physiological data that indicates a user's health status, such as heart rate, blood pressure, and body temperature.

[0008] "Real-time measurement means" refers to devices or software that continuously capture a user's physical health indicators and generate data instantly.

[0009] "Database" refers to a system that stores measured physical and emotional data and stores it in a form that can be analyzed at a later date.

[0010] "Means for analysis" refers to algorithms or programs that assess the user's health and emotional state based on the stored data.

[0011] "Reminder notifications" refer to messages or alerts that prompt users to take appropriate action when their emotional state declines.

[0012] "Environmental adjustment means" refers to a method or device for providing a relaxing environment according to the user's emotional state.

[0013] "Relaxing music" refers to music selected to help a user relax.

[0014] A "social support group" refers to an online or offline community that users join to receive emotional support.

[0015] An "emotion diary" refers to a tool or application that allows users to record their emotions and mood changes.

[0016] "Analyzing" refers to the process of generating appropriate assessments and recommendations based on the recorded emotional diary and health indicator data. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system for supporting a user's mental and physical health. The system measures a user's physical health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group.

[0039] Overall system configuration

[0040] health measurement

[0041] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[0042] Data analysis

[0043] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[0044] Reminders

[0045] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[0046] Play relaxing music

[0047] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[0048] Connecting to social support groups

[0049] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[0050] Emotional diary recording

[0051] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[0052] Specific examples

[0053] Specific examples of health measurements

[0054] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[0055] Examples of reminder notifications

[0056] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[0057] Example of relaxing music playback

[0058] If the server determines that the user is feeling stressed, it sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music including the sounds of a quiet forest or waves can be played to help the user relax.

[0059] Examples of connecting with social support groups

[0060] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[0061] Examples of emotional diary entries

[0062] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[0063] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate psychological support as needed.

[0064] The processing flow will be explained below.

[0065] health measurement

[0066] Program processing steps

[0067] Step 1:

[0068] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[0069] Step 2:

[0070] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[0071] Step 3:

[0072] The device sends the measurement data to the server, which then sends the data to the server via API.

[0073] Step 4:

[0074] The server analyzes the received data and stores it in a real-time database.

[0075] Data analysis

[0076] Program processing steps

[0077] Step 1:

[0078] The server periodically queries the database for the latest heart rate data.

[0079] Step 2:

[0080] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[0081] Step 3:

[0082] The server stores the analysis results in a database and prepares notifications as needed.

[0083] Reminders

[0084] Program processing steps

[0085] Step 1:

[0086] The server monitors the user's health data and detects a decline in emotional state.

[0087] Step 2:

[0088] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[0089] Step 3:

[0090] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[0091] Play relaxing music

[0092] Program processing steps

[0093] Step 1:

[0094] The server analyzes the user's emotional state and determines that relaxing music is required.

[0095] Step 2:

[0096] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[0097] Step 3:

[0098] The server sends a music playback command to the device, including the music file information and playback command.

[0099] Step 4:

[0100] The device receives the instruction and plays the selected relaxing music.

[0101] Connecting to social support groups

[0102] Program processing steps

[0103] Step 1:

[0104] A user sends a request from the device to connect to a social support group.

[0105] Step 2:

[0106] The device sends the user's request to the server, which also includes the user's profile information.

[0107] Step 3:

[0108] The server selects appropriate social support groups based on the user's profile information.

[0109] Step 4:

[0110] The server transmits information about the selected social support group to the terminal.

[0111] Step 5:

[0112] The terminal displays the received group information and connects the user to the group.

[0113] Emotional diary recording

[0114] Program processing steps

[0115] Step 1:

[0116] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[0117] Step 2:

[0118] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[0119] Step 3:

[0120] The server stores the received data in a database for later analysis.

[0121] Step 4:

[0122] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[0123] Step 5:

[0124] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] In modern society, many people live with stress and fatigue. Conventional health management systems are primarily limited to measuring physical health indicators and lack the means to properly monitor changes in users' mental state and emotions. As a result, it is difficult for users to accurately understand their own mental health status and take prompt action. In addition, there are limited ways to provide relaxation and mental support when needed, making comprehensive health management difficult.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for analyzing the user's emotional patterns based on the data recorded by the user in the emotional diary and providing feedback and suggestions, and means for analyzing the user's health data using a generative AI model and providing the user with optimal reminders and relaxing music, thereby enabling the user to comprehensively manage their health from both physical and mental perspectives.

[0130] "User physical health indicators" are data that indicate the user's physical condition or health in real time, such as heart rate, blood pressure, and body temperature.

[0131] "Real-time measurement means" refers to a mechanism that obtains a user's health indicators continuously or periodically and collects data instantly.

[0132] The "database" is a system that systematically stores and manages received data such as health indicators and emotional diaries.

[0133] "Means for storing data in a database" refers to the methods and technologies for efficiently storing collected data in a database.

[0134] "Means for analyzing and assessing the user's health status" means algorithms or methods for using the collected data to assess the user's physical and mental health status.

[0135] The "means for sending reminder notifications" is a mechanism that generates messages encouraging breaks or attention based on the user's health and emotional state and sends them to the user's device.

[0136] "Environmental adjustment means" is a mechanism that provides an appropriate music and audio environment for the purpose of user relaxation and stress relief.

[0137] "Means for playing relaxing music" refers to a system that allows users to select music that helps them relax and play it on their devices.

[0138] "Means of connecting users to social support groups" means mechanisms that allow users to join appropriate groups or communities to receive emotional support.

[0139] "User's Emotion Diary" is data that allows users to record their daily emotions and moods.

[0140] The "means for recording and analyzing an emotional diary" is a technology for saving an emotional diary entered by a user and analyzing the data to understand the user's emotional patterns.

[0141] A "generative AI model" is an artificial intelligence algorithm used to analyze collected data and provide optimal feedback and suggestions to users.

[0142] The present invention is a system for supporting a user's physical and mental health. The system measures a user's health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system can then provide the user with reminders, relaxing music, and even connect them to social support groups.

[0143] Overall system configuration

[0144] health measurement

[0145] While the user is wearing the smart glasses, health indicators such as heart rate and blood pressure are measured in real time using the smart glasses' built-in sensors, and the data is sent to a server via Bluetooth or Wi-Fi.

[0146] Data analysis

[0147] The server stores the received health index data in a database and periodically evaluates the user's health status using a data analysis algorithm. For example, if the user's heart rate is higher than normal, the server determines that the user is feeling stressed.

[0148] Reminders

[0149] If the server determines that the user's emotional state is declining, it generates a reminder notification to encourage them to take a break and sends it to the user's device. Specifically, it sends a message saying, "Your emotional state is declining. Please take a break."

[0150] Play relaxing music

[0151] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends a command to play it to the device. The device then follows the command and plays music to help the user relax. For example, relaxing music such as the sounds of a quiet forest or the sound of waves can be played.

[0152] Connecting to social support groups

[0153] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information and sends the information to the device, where the user can join and communicate with other members in real time.

[0154] Emotional diary recording

[0155] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[0156] Using generative AI models

[0157] The server uses a generative AI model to analyze the received health index data and emotional diary data. This analysis allows it to provide optimal reminder notifications and relaxing music to the user. Specifically, it analyzes the user's stress level and suggests optimal measures accordingly.

[0158] Specific examples

[0159] Specific examples of health measurements

[0160] While the user is wearing the smart glasses, their heart rate is measured every second and sent to a server. The data is then recorded in a database and compared to their normal heart rate. For example, if their heart rate is higher than normal, the server will determine that the user's stress level is high.

[0161] Examples of reminder notifications

[0162] If the server determines that the user's emotional state is declining based on the user's emotional diary and health data, it generates a reminder notification saying, "Your emotional state is declining. Please take a break." The device receives this notification and displays it to the user.

[0163] Example of relaxing music playback

[0164] If the server determines that the user is feeling stressed, it sends a command to play relaxing music to the device. The device then follows this command and plays relaxing music such as the sound of a quiet forest or the sound of waves, allowing the user to relax.

[0165] Examples of connecting with social support groups

[0166] When a user requests to connect to a support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members in real time.

[0167] Examples of emotional diary entries

[0168] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[0169] Example prompts for generative AI models

[0170] Analyze the user's heart rate data and emotional diary to determine whether they are feeling stressed and suggest relaxing music to reduce stress.

[0171] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0172] Step 1:

[0173] User wear of the device and measurement of health indicators

[0174] The user wears smart glasses, which use built-in sensors to measure health indicators such as heart rate and blood pressure in real time. The input data is the user's health indicators, such as heart rate and blood pressure, which are collected by the smart glasses every second. The collected data is then sent to a server via Bluetooth or WiFi.

[0175] Step 2:

[0176] Sending and saving data to the server

[0177] The server receives the data sent from the smart glasses. The input data is the health index data sent from the smart glasses. The server receives this data and first checks the integrity of the data. Then, it stores it in the database. The output is the health index data whose integrity has been confirmed, which is then stored in the database.

[0178] Step 3:

[0179] Regular analysis of health data

[0180] The server periodically analyzes the health index data stored in the database. The input data is the past and current health index data stored in the database. The server uses a data analysis algorithm to evaluate the user's health status. Specifically, it analyzes fluctuations in heart rate and blood pressure to detect signs of stress and fatigue. The output is an evaluation result of the user's health status.

[0181] Step 4:

[0182] Evaluating emotional state and generating reminder notifications

[0183] The server evaluates the user's emotional state based on the analysis results. The input data is the analysis results of health data. For example, if the heart rate is continuously high, it is determined that the emotional state is declining. The server generates a reminder notification such as "Your emotional state is declining. Please take a break." The output is the generated reminder notification.

[0184] Step 5:

[0185] Sending reminders

[0186] The server sends the generated reminder notification to the user's device. The input data is the reminder notification generated by the server. The server sends the notification and the device receives it. The output is the reminder notification received by the user's device and displayed on the device screen.

[0187] Step 6:

[0188] Selecting relaxing music and sending playback instructions

[0189] The server selects relaxing music based on the reminder notification. The input data is the user's current health and emotional state, and the user's music preferences. The server uses a generative AI model to select the most suitable relaxing music for the user. It sends a play instruction for the selected music to the device. The output is the music play instruction sent to the device.

[0190] Step 7:

[0191] Playing relaxing music on your device

[0192] The terminal plays relaxing music based on the music playback instruction received from the server. The input data is the music playback instruction sent from the server. The terminal receives the instruction and plays the specified relaxing music to the user. The output is music that the user can listen to and feel relaxed.

[0193] Step 8:

[0194] Connecting to social support groups

[0195] When a user wants to connect to a social support group, the server selects an appropriate group based on the user's profile information. The input data is the user's profile information and a database of groups. The server selects an appropriate group and sends the information to the user's device. The output is the group information to connect to.

[0196] Step 9:

[0197] Record and send your emotional diary

[0198] Users record their emotion diary using smart glasses or a smartphone app. The input data is the diary of emotions recorded by the user. The recorded diary data is sent to a server and stored in a database. The output is the emotion diary data sent to and stored on the server.

[0199] Step 10:

[0200] Emotion diary analysis and feedback

[0201] The server analyzes the emotion diary data and provides feedback and suggestions. The input data is the emotion diary data stored in the database. The server analyzes it and generates feedback such as, "You seem to have been feeling tired lately. I recommend you take a rest." The output is the feedback and suggestions sent to the user.

[0202] Above are the specific processing steps and detailed explanation of this system, which allows users to comprehensively manage their physical and mental health.

[0203] (Application example 1)

[0204] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0205] In factory work environments, managing employee stress and health is important, but conventional methods make it difficult to grasp the situation of individual employees in real time and implement appropriate measures. Also, when employees need psychological support, there are limited ways to connect them to appropriate social support groups. To solve these issues, real-time analysis of data and automatic implementation of appropriate measures are required.

[0206] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0207] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for measuring the stress level and heart rate of factory workers and sending a health notification and playing relaxing music based on the analysis results, means for connecting the factory workers to an appropriate social support group, and means for sending notifications and playing music via a robot. This enables real-time management of the health and emotional states of employees in a factory work environment, and makes it possible to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[0208] "User" refers to an individual who uses this system.

[0209] "Physical health indicators" are data that indicate the user's physical condition, such as heart rate and blood pressure.

[0210] "Real-time measurement means" refers to technology that uses smart glasses or wearable devices to instantly collect health indicators.

[0211] The "means for storing in a database" refers to a technique for recording and storing the measured health indicators in a database.

[0212] "Means for assessing health status" refers to technology that analyzes stored data and determines the user's physical and mental status.

[0213] "Means for sending reminder notifications" refers to technology that sends notifications to users to warn them or suggest they take a break.

[0214] The "means for playing relaxing music as an environmental adjustment means" is a technology for selecting and playing appropriate relaxing music to improve the user's emotional state.

[0215] "Means for connecting users to social support groups" refers to technology that connects users to appropriate support groups when desired.

[0216] "Means for recording and analyzing emotional diaries" refers to a technology that allows users to record their own emotional state and then analyze that data.

[0217] "Means for measuring and analyzing stress levels and heart rates" refers to technology that measures and analyzes health indicators such as the heart rate of factory workers in real time.

[0218] The "means for sending health notifications and playing relaxing music" is a technology that sends notifications and plays relaxing music when stress or overwork is detected.

[0219] "Means for sending notifications and playing music via a robot" refers to a technology in which a robot sends reminder notifications to the user and plays relaxing music.

[0220] System Overview

[0221] The system of the present invention is designed to manage the physical and mental health of users and support stress management in factory work environments. Users wear smart glasses or wearable devices to measure health indicators such as heart rate in real time. The measured data is sent to a server, where it is recorded and analyzed in a database to evaluate the user's health. Monitoring the health of employees is particularly important in factory environments where workers are under heavy stress.

[0222] Hardware and software used

[0223] The system is implemented using the following hardware and software.

[0224] Wearable devices: Smart glasses and smart watches are used to measure data such as heart rate and stress levels.

[0225] Server: Stores the received health data in a database and analyzes it. Implemented in a programming language such as Python.

[0226] Database: A database system such as MySQL or PostgreSQL will be used to record and store health index data.

[0227] Robot: Use a robot to send reminder notifications to users and play relaxing music.

[0228] Data collection and analysis process

[0229] 1. Data collection: Users wear smart glasses or smartwatches, which measure their heart rate and stress levels in real time. The data is then sent to a server via Bluetooth or Wi-Fi.

[0230] 2. Data storage: The received data is stored on the server and recorded in a database.

[0231] 3. Data analysis: The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate is higher than normal, it determines that the user is feeling stressed.

[0232] 4. Reminder notification: If the user's emotional state is determined to be declining, the server generates a reminder notification and sends a message to the user encouraging them to take a break.

[0233] 5. Relaxing music playback: The server sends a reminder notification and a command to play relaxing music to the robot, allowing the user to relax while listening to the music.

[0234] Social support function

[0235] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[0236] Specific examples

[0237] As a specific example of operation, consider the case where a factory worker feels fatigued or stressed. If the factory worker wears a smartwatch and their heart rate exceeds the normal level, the server will send a notification via a robot saying, "High stress level detected. Please take a break." In addition, relaxing music will be played automatically.

[0238] Prompt Sentence Examples

[0239] "Write a program that notifies you and plays relaxing music if it determines that a factory worker is stressed."

[0240] In this way, this invention enables real-time management of employee health and emotional states in a factory environment, and is expected to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[0241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0242] Program processing steps

[0243] Step 1: Data collection

[0244] Health index data such as heart rate and stress level is collected in real time while the user is wearing the wearable device. Data is sent from the device to a server via Bluetooth or WiFi. Input: Measurement data of heart rate and stress level. Output: Health index data sent to the server.

[0245] Step 2: Save data

[0246] The server stores the received data in a database. The stored data is used for later analysis. Input: Health index data received from the device. Output: Stored data recorded in the database.

[0247] Step 3: Data analysis

[0248] The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate exceeds normal values, it determines that the user is feeling stressed. Data analysis is performed using Python and other tools. Input: Health index data in the database. Output: Analysis results (user's stress and health state).

[0249] Step 4: Generate reminder notifications

[0250] If the data analysis determines that the user's emotional state is declining, the server generates a reminder notification. The notification may contain a text message such as "High stress detected. Please take a break." Input: Analysis results. Output: Reminder notification text message.

[0251] Step 5: Select and play relaxation music

[0252] The server selects appropriate relaxation music based on the user's emotional state and sends playback instructions to the robot. For example, it may select forest sounds or the sound of waves. The robot plays the music based on the instructions. Input: Reminder notification and emotional state data. Output: Relaxation music to be played.

[0253] Step 6: Social support connections

[0254] When a user requests a social support connection, the server selects an appropriate social support group based on the user's profile information and establishes a connection. Input: User's profile information and connection request. Output: Selected social support group information.

[0255] Specific actions

[0256] For example, when a factory worker wears a smartwatch, their heart rate is measured in real time. If their heart rate exceeds 110 and their stress level is high at 8, the server generates a notification saying "High stress detected. Please take a break" and sends it to the worker via a robot. The robot also automatically plays relaxation music.

[0257] Example prompt sentence:

[0258] "Write a program that notifies you and plays relaxation music if it determines that a factory worker is stressed."

[0259] In this way, the processing at each step is carried out specifically, and real-time management of the user's health condition is realized.

[0260] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0261] The present invention is a system for supporting a user's mental and physical health, and in particular, aims to incorporate an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine also recognizes the user's emotions and provides accurate feedback based on those emotions.

[0262] Overall system configuration

[0263] health measurement

[0264] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[0265] Data analysis

[0266] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[0267] Use of emotion engine

[0268] The emotion engine analyzes the user's facial expressions, voice, input text data, etc. to recognize the user's emotional state. The emotion data obtained by the emotion engine is sent to the server and used as part of the analysis.

[0269] Reminders

[0270] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[0271] Play relaxing music

[0272] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[0273] Connecting to social support groups

[0274] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[0275] Emotional diary recording

[0276] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[0277] Specific examples

[0278] Specific examples of health measurements

[0279] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[0280] Examples of emotion engines

[0281] When a user speaks using the smartphone app, the voice data is analyzed by the emotion engine. The emotion engine recognizes the user's emotional state from the tone of the voice and the way they speak, and determines that they are in a "stressed state." As a result, the server sends appropriate instructions, such as sending a reminder notification or playing relaxing music.

[0282] Examples of reminder notifications

[0283] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[0284] Example of relaxing music playback

[0285] When the emotion engine recognizes that the user is feeling stressed, the server sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music that includes the sounds of a quiet forest or waves can help the user relax.

[0286] Examples of connecting with social support groups

[0287] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[0288] Examples of emotional diary entries

[0289] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[0290] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate mental support as needed. In addition, the emotion engine helps recognize emotional states and provides more accurate feedback, thereby more effectively supporting the user's mental health.

[0291] The processing flow will be explained below.

[0292] health measurement

[0293] Program processing steps

[0294] Step 1:

[0295] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[0296] Step 2:

[0297] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[0298] Step 3:

[0299] The device sends the measurement data to the server, which then sends the data to the server via API.

[0300] Step 4:

[0301] The server analyzes the received data and stores it in a real-time database.

[0302] Data analysis

[0303] Program processing steps

[0304] Step 1:

[0305] The server periodically queries the database for the latest heart rate data.

[0306] Step 2:

[0307] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[0308] Step 3:

[0309] The server stores the analysis results in a database and prepares notifications as needed.

[0310] Use of emotion engine

[0311] Program processing steps

[0312] Step 1:

[0313] The user starts inputting emotions through a smartphone app or smart glasses, and the user's facial, voice, and text data are sent to the emotion engine.

[0314] Step 2:

[0315] The emotion engine analyzes the received data and recognizes the user's emotional state. The recognized emotion data is sent to the server.

[0316] Step 3:

[0317] The server uses the data from the emotion engine as part of its analysis to assess the user's current emotional state.

[0318] Reminders

[0319] Program processing steps

[0320] Step 1:

[0321] The server monitors the user's health status data and the analysis results of the emotion engine to detect a decline in the emotional state.

[0322] Step 2:

[0323] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[0324] Step 3:

[0325] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[0326] Play relaxing music

[0327] Program processing steps

[0328] Step 1:

[0329] The server analyzes the user's emotional state and determines that relaxing music is required.

[0330] Step 2:

[0331] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[0332] Step 3:

[0333] The server sends a music playback command to the device, including the music file information and playback command.

[0334] Step 4:

[0335] The device receives the instruction and plays the selected relaxing music.

[0336] Connecting to social support groups

[0337] Program processing steps

[0338] Step 1:

[0339] A user sends a request from the device to connect to a social support group.

[0340] Step 2:

[0341] The device sends the user's request to the server, which also includes the user's profile information.

[0342] Step 3:

[0343] The server selects appropriate social support groups based on the user's profile information.

[0344] Step 4:

[0345] The server transmits information about the selected social support group to the terminal.

[0346] Step 5:

[0347] The terminal displays the received group information and connects the user to the group.

[0348] Emotional diary recording

[0349] Program processing steps

[0350] Step 1:

[0351] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[0352] Step 2:

[0353] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[0354] Step 3:

[0355] The server stores the received data in a database for later analysis.

[0356] Step 4:

[0357] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[0358] Step 5:

[0359] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[0360] Example 2

[0361] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0362] In today's world, lifestyle habits and work stress can lead to a decline in users' mental and physical health. These health problems are particularly difficult to recognize, making it challenging to address them at the appropriate time. It is also difficult to provide users with appropriate feedback regarding a decline in mental health. Conventional systems have struggled to monitor a user's health and emotional state in real time and provide appropriate support. To address these issues, it is necessary to monitor a user's health and emotional state in real time and take appropriate measures.

[0363] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0364] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health condition, means for analyzing the user's facial expressions, voice, and text data to recognize the user's emotional state, means for storing the recognized emotional state in a database, means for sending a reminder notification when the user's emotional state deteriorates, means for selecting and playing relaxing music together with the reminder notification, means for connecting the user to a social support group, and means for recording and analyzing the user's emotional diary, thereby making it possible to monitor the user's health condition and emotional state in real time and provide appropriate feedback and support.

[0365] A "user" is an individual or group that uses the system and provides data on health indicators and emotional states.

[0366] "Physical health indicators" are measurement data that indicate an individual's physical health status, such as heart rate, body temperature, blood pressure, and respiratory rate.

[0367] "Real-time" refers to the state in which data is processed immediately from the moment it is generated, without any delay.

[0368] The "database" is an electronic record system for systematically organizing and storing data on users' health indicators and emotional states.

[0369] "Analysis" is the process of extracting information from collected data using statistical methods and algorithms, and then evaluating and judging it.

[0370] "Emotional state" is information that indicates the user's psychological state and mood, and is obtained from facial expressions, voice, and text data.

[0371] A "reminder notification" is a notification that includes a message or advice that urges the user to take a break or take measures.

[0372] "Relaxing music" is music selected for the purpose of relieving the user's mental tension and has the role of promoting relaxation.

[0373] A "social support group" is an online or offline group that users can join to receive emotional support and communication from other members.

[0374] An "emotion diary" is an electronic diary that allows users to record their emotions, moods, physical conditions, etc.

[0375] The present invention provides a system for supporting a user's mental and physical health, particularly by incorporating an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine is also used to recognize the user's emotions and provide accurate feedback based on those emotions.

[0376] The components of this system are:

[0377] 1. A means of measuring the user's physical health indicators:

[0378] When a user wears smart glasses, health indicators such as heart rate are measured in real time. A specific example is a heart rate sensor built into smart glasses such as Google Glass.

[0379] 2. Means of storing health indicator data in the database:

[0380] The smart glasses transmit the measured heart rate data to a server, which stores the data in a database (e.g., MySQL). The data is transmitted via Wi-Fi or Bluetooth and recorded in real time.

[0381] 3. Means of analyzing health indicator data:

[0382] The server periodically analyzes the received health index data and evaluates the user's health condition. If an abnormal value (e.g., a higher heart rate than normal) is detected at this stage, an analysis result is generated indicating that the user may be experiencing stress.

[0383] 4. Means of recognizing emotional states:

[0384] The user inputs voice and facial expression data using a smartphone app. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes this data and recognizes the user's emotional state. The analysis results are sent to a server and stored in a database.

[0385] 5. Send reminder notifications by:

[0386] If the server determines that the user's emotional state is declining, it generates and sends a reminder notification to the device, which may include a message such as "Your emotional state is declining. Please take a break."

[0387] 6. How to play relaxing music:

[0388] The server selects relaxing music along with the reminder notification. The selection process refers to the analysis results of the emotion engine and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[0389] 7. Ways to connect with social support groups:

[0390] When a user requests to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[0391] 8. How to record and analyze your emotional diary:

[0392] Users use a smartphone app to record their emotions in a diary. Records such as "I'm tired today" are sent to a server and stored in a database. The server analyzes this data and provides feedback to the user suggesting they take a rest if fatigue persists.

[0393] Specific actions

[0394] Specific examples of health measurements

[0395] While the user is wearing Google Glass, heart rate data is measured every second and sent to a server, which receives the data and records it in a database. If the data shows abnormal values, the user's stress level is evaluated.

[0396] Examples of emotion engines

[0397] When a user inputs a voice message using a smartphone app, the emotion engine analyzes the voice data and determines that the user is in a "stressed state." This analysis result is sent to the server, which then sends a reminder notification to the device along with an instruction to play relaxing music.

[0398] Prompt Sentence Examples

[0399] For example, by inputting to the generative AI model, "Please tell me the code to generate a specific music list for the user to relax and send the playback instructions to the server," an appropriate program can be generated.

[0400] This system allows users to manage their health in real time and receive psychological support when necessary. The emotional engine enables the system to accurately recognize the user's emotional state and provide effective feedback.

[0401] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0402] Step 1:

[0403] Users wear smart glasses to measure their health data.

[0404] The user wears the smart glasses. The smart glasses have a built-in heart rate sensor that measures heart rate data every second. The measured data is not stored directly, but is sent to the server in the next step.

[0405] Input: Smart glasses worn by the user, user's heart rate

[0406] Output: Heart rate data measured every second

[0407] Step 2:

[0408] The smart glasses send the data to the server.

[0409] The smart glasses transmit the measured heart rate data to a server in real time via Wi-Fi or Bluetooth, allowing the data to be analyzed immediately.

[0410] Input: Heart rate data measured every second

[0411] Output: Heart rate data sent to the server

[0412] Step 3:

[0413] The server stores health data in a database and periodically analyzes it.

[0414] The server stores the received heart rate data in a database. The data stored in the database (e.g., MySQL) is periodically analyzed using an analysis algorithm (e.g., an anomaly detection algorithm). If an abnormal value is detected, the user's health condition is evaluated.

[0415] Input: Heart rate data sent to the server

[0416] Output: Data stored in a database, analysis results (health status is evaluated)

[0417] Step 4:

[0418] The user generates emotion data using the emotion engine.

[0419] Users use a smartphone app to input voice messages or text data. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes the input voice or text data and recognizes the user's emotional state. The results are sent to the server.

[0420] Input: Voice messages and text data

[0421] Output: Parsed emotional state data

[0422] Step 5:

[0423] The server analyzes the emotion data and generates reminder notifications as needed.

[0424] The server analyzes the emotion data sent from the emotion engine. If the user's emotional state is recognized as "stressed," it generates a reminder notification and sends it to the device. The reminder notification may include a message such as "Your emotional state is declining. Please take a break."

[0425] Input: Parsed emotional state data

[0426] Output: Reminder notification

[0427] Step 6:

[0428] The server selects relaxing music along with the reminder notification and sends it to the device.

[0429] The server selects relaxing music based on the user's current emotional state and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[0430] Input: Reminders, emotional state data, historical data

[0431] Output: Relaxing music selection, music playback instructions

[0432] Step 7:

[0433] When a user wants to connect to a social support group, the server provides the information and helps them connect.

[0434] When a user requests to join a social support group through a smartphone app, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[0435] Input: User profile information, desired connection information

[0436] Output: Information on suitable social support groups

[0437] Step 8:

[0438] Users record their emotions in a diary, and the server analyzes the data and provides feedback.

[0439] Users use a smartphone app to record their emotions in a diary. The recorded data (e.g., "I feel tired today") is sent to a server and stored in a database. The server analyzes this data and analyzes the user's emotional patterns. If fatigue persists, the server provides feedback to the user suggesting that they take a rest.

[0440] Input: Emotion diary data

[0441] Output: Feedback notification based on analysis results

[0442] (Application example 2)

[0443] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0444] Conventional technologies for supporting users' mental and physical health have struggled to accurately recognize the user's emotional state, preventing them from providing appropriate support. Furthermore, providing services in physical stores has also been problematic, as it has been difficult to provide individualized support based on the user's current health and emotional state, making it difficult to improve user satisfaction. Therefore, there is a need for the development of a system that can more accurately recognize the user's emotional state and provide appropriate feedback and support in real time.

[0445] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0446] In this invention, the server includes means for measuring a user's physical health index in real time, means for storing the measured health index in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for providing appropriate services in a physical store based on the user's physical health index and emotional state, means for suggesting products according to the user's health state and emotional state, and means for playing relaxing music in a specific area when the user's stress state is determined to be high. This allows the user to manage their health in real time and receive appropriate psychological support and personalized services in a physical store as needed.

[0447] "User's physical health indicators" refers to physiological data such as the user's heart rate, steps, blood pressure, and body temperature.

[0448] "Means of measuring in real time" refers to technology that continuously acquires physical health indicators from the user's body and instantly converts them into data.

[0449] "Means for storing data in a database" refers to a system for systematically recording measured data and storing it in a format that can be managed and searched.

[0450] "Means for analyzing and assessing the user's health status" refers to algorithms and software that objectively assess the user's health status based on collected health indicator data.

[0451] "Means for sending reminder notifications" refers to a system that sends messages to users to encourage them to take a break or suggest ways to reduce stress.

[0452] "Means for playing relaxing music" refers to a function that plays music that relieves stress based on the user's emotional state.

[0453] "Means for connecting users to social support groups" refers to technology that allows users to access appropriate communities and groups to receive emotional support.

[0454] "A means for recording and analyzing a user's emotional diary" refers to a system that allows users to input the emotions they feel on a daily basis, and then records and analyzes that data.

[0455] "Means for providing appropriate services in physical stores" refers to a system for providing services in a physical store environment that take into account the user's current health and emotional state.

[0456] "Means for suggesting products based on health and emotional state" refers to technology that suggests optimal products and services based on the user's health and emotional data.

[0457] "Means for playing relaxing music in specific areas" refers to the function of playing music in specific areas within the store to help users relax.

[0458] The present invention is a system for supporting the mental and physical health of a user, and in particular incorporates an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. This system is realized mainly using the following hardware and software.

[0459] Hardware and software used:

[0460] Smart glasses (e.g., Google Glass, Vuzix Blade): measure the user's physical health indicators (heart rate, steps, etc.) in real time.

[0461] Smartphone / tablet (e.g. iPad, Android tablet): Displays and notifies the user's emotional state and health data.

[0462] Server (e.g. AWS EC2, Google Cloud VM): Analyzes and stores data, and generates notifications.

[0463] Emotion engine (e.g., Microsoft Azure Emotion API, IBM Watson Tone Analyzer): Evaluates the user's emotional state.

[0464] System features:

[0465] The system is equipped with the following functions, and analyzes the user's health and emotional state in real time and provides appropriate support based on that.

[0466] 1. Real-time health monitoring:

[0467] The smart glasses measure the user's heart rate, number of steps, and other data in real time and send it to a server, where it stores the data in a database and analyzes it, allowing the user's health status to be tracked.

[0468] 2. Recognition of emotional states:

[0469] The emotion engine analyzes the user's facial expressions and voice data to assess their emotional state. For example, it analyzes what the user says and their facial expressions using a smartphone app to determine their stress level. The data obtained by the emotion engine is also sent to the server and used as part of the analysis.

[0470] 3. Reminder Notification:

[0471] If the user's emotional state is determined to be declining, the server generates a reminder notification and sends it to the user's smartphone or tablet, which may include a message to take a break or specific stress reduction advice.

[0472] 4. Play relaxing music:

[0473] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends playback instructions to the smart glasses or smartphone, allowing the user to relax by listening to music such as the sound of a tranquil forest or the sound of waves.

[0474] 5. Connect with social support groups:

[0475] If a user wishes to join a social support group, the server will recommend and connect them to an appropriate group based on their profile information, allowing them to communicate with other members in real time and receive emotional support.

[0476] 6. Personalized services in-store:

[0477] When a user is in a physical store, real-time information is provided to store staff based on the user's health indicators and emotional state. For example, a tense user can be provided with a relaxing environment, and a stressed user can be suggested products that will have a relaxing effect.

[0478] Specific use cases:

[0479] 1. Health measurement and emotional state recognition:

[0480] When a user wears smart glasses and goes shopping, their heart rate is measured to be high, and the emotion engine detects a state of stress. In this case, the server issues a command to "play relaxing music," and nature music is played on the user's smartphone.

[0481] 2. Reminder Notification:

[0482] If the server analyzes health data and emotional state to be declining, a reminder notification such as "Take a break" will be sent to the user's device.

[0483] 3. Social support group connections:

[0484] When a user types "I need support" into the smartphone app, the server recommends appropriate social support groups and connects the user to them.

[0485] Example prompt sentence:

[0486] "Implement a system that recognizes customers' stress levels and suggests products that will help them relax. Use smart glasses and an emotion recognition engine to notify staff when a certain threshold is exceeded."

[0487] This allows users to receive appropriate mental and physical support in real time, making the shopping experience in physical stores more comfortable.

[0488] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0489] Step 1:

[0490] When a user wears smart glasses, physical health indicators (heart rate, number of steps, etc.) are measured in real time. The smart glasses instantly convert this data into digital data and send it to a server. The input data is health information such as heart rate and number of steps, and the output is the measurement data sent to the server. The smart glasses have built-in heart rate monitors and acceleration sensors, which are used to acquire data in real time.

[0491] Step 2:

[0492] The server receives the health index data sent from the smart glasses and stores it in a database. At this time, the server organizes the data so that it can be easily searched and analyzed later. The input data is the measurement data sent from the smart glasses, and the output is the health data stored in the database. The server receives the data using a data transfer protocol (e.g., HTTP, HTTPS) and stores it in a database (e.g., MySQL, PostgreSQL).

[0493] Step 3:

[0494] The server analyzes the stored data and evaluates the user's health condition. This analysis uses an algorithm that, for example, determines that an abnormally high heart rate indicates stress. The input data is the health data stored in the database, and the output is the evaluation result of the user's health condition. The server analyzes the data using a programming language such as Python and a data analysis library (e.g., Pandas, NumPy).

[0495] Step 4:

[0496] The emotion engine analyzes the user's facial expression and voice data to evaluate their emotional state. The emotion engine analyzes facial expression and voice data input via a smartphone or tablet. The input data is the user's facial expression and voice information, and the output is an evaluation of their emotional state. The emotion engine uses the Microsoft Azure Emotion API, IBM Watson Tone Analyzer, etc. to analyze facial expressions and voice emotions.

[0497] Step 5:

[0498] If the server determines that the user's health and emotional state is declining, it generates a reminder notification. The reminder notification includes a message encouraging the user to take a break and specific advice for reducing stress. The input data is the assessment result of the user's health and emotional state, and the output is the reminder notification message. The server creates the message using a notification generation algorithm and sends the notification using a notification service (e.g., Firebase Cloud Messaging).

[0499] Step 6:

[0500] The server sends a reminder notification and an instruction to play relaxing music to the smart glasses or smartphone. The relaxing music may include, for example, the sound of a quiet forest or the sound of waves. The input data is the instruction to play relaxing music, and the output is the relaxing music played from the smart glasses or smartphone. The server uses a music selection algorithm and sends the play instruction to the device.

[0501] Step 7:

[0502] When a user wishes to join a social support group, the server recommends and connects them to appropriate groups based on the user's profile information. The input data is the user's profile information, and the output is information about recommended social support groups. The server uses a recommendation algorithm and connects to communication platforms (e.g., Slack, Discord).

[0503] Step 8:

[0504] When a user is in a physical store, the server provides the user's health indicators and emotional state to store staff in real time. The staff obtains this information via tablets or smart devices and provides individualized support. The input data is the user's health indicators and emotional state, and the output is the information displayed to the staff. The server uses a data transfer protocol to send the data to the staff application.

[0505] Step 9:

[0506] In a physical store, the server makes product suggestions based on the user's health and emotional state. For example, a user experiencing high stress may be recommended a product with a relaxing effect. The input data is the user's health and emotional state, and the output is personalized product suggestions. The server uses a recommendation engine to generate the suggestions.

[0507] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0508] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0509] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0510] [Second embodiment]

[0511] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0512] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0513] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0514] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0515] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0516] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0517] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0518] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0519] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0520] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0521] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0522] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0523] The present invention is a system for supporting a user's mental and physical health. The system measures a user's physical health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group.

[0524] Overall system configuration

[0525] health measurement

[0526] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[0527] Data analysis

[0528] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[0529] Reminders

[0530] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[0531] Play relaxing music

[0532] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[0533] Connecting to social support groups

[0534] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[0535] Emotional diary recording

[0536] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[0537] Specific examples

[0538] Specific examples of health measurements

[0539] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[0540] Examples of reminder notifications

[0541] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[0542] Example of relaxing music playback

[0543] If the server determines that the user is feeling stressed, it sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music including the sounds of a quiet forest or waves can be played to help the user relax.

[0544] Examples of connecting with social support groups

[0545] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[0546] Examples of emotional diary entries

[0547] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[0548] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate psychological support as needed.

[0549] The processing flow will be explained below.

[0550] health measurement

[0551] Program processing steps

[0552] Step 1:

[0553] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[0554] Step 2:

[0555] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[0556] Step 3:

[0557] The device sends the measurement data to the server, which then sends the data to the server via API.

[0558] Step 4:

[0559] The server analyzes the received data and stores it in a real-time database.

[0560] Data analysis

[0561] Program processing steps

[0562] Step 1:

[0563] The server periodically queries the database for the latest heart rate data.

[0564] Step 2:

[0565] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[0566] Step 3:

[0567] The server stores the analysis results in a database and prepares notifications as needed.

[0568] Reminders

[0569] Program processing steps

[0570] Step 1:

[0571] The server monitors the user's health data and detects a decline in emotional state.

[0572] Step 2:

[0573] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[0574] Step 3:

[0575] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[0576] Play relaxing music

[0577] Program processing steps

[0578] Step 1:

[0579] The server analyzes the user's emotional state and determines that relaxing music is required.

[0580] Step 2:

[0581] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[0582] Step 3:

[0583] The server sends a music playback command to the device, including the music file information and playback command.

[0584] Step 4:

[0585] The device receives the instruction and plays the selected relaxing music.

[0586] Connecting to social support groups

[0587] Program processing steps

[0588] Step 1:

[0589] A user sends a request from the device to connect to a social support group.

[0590] Step 2:

[0591] The device sends the user's request to the server, which also includes the user's profile information.

[0592] Step 3:

[0593] The server selects appropriate social support groups based on the user's profile information.

[0594] Step 4:

[0595] The server transmits information about the selected social support group to the terminal.

[0596] Step 5:

[0597] The terminal displays the received group information and connects the user to the group.

[0598] Emotional diary recording

[0599] Program processing steps

[0600] Step 1:

[0601] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[0602] Step 2:

[0603] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[0604] Step 3:

[0605] The server stores the received data in a database for later analysis.

[0606] Step 4:

[0607] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[0608] Step 5:

[0609] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[0610] Example 1

[0611] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] In modern society, many people live with stress and fatigue. Conventional health management systems are primarily limited to measuring physical health indicators and lack the means to properly monitor changes in users' mental state and emotions. As a result, it is difficult for users to accurately understand their own mental health status and take prompt action. In addition, there are limited ways to provide relaxation and mental support when needed, making comprehensive health management difficult.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0614] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for analyzing the user's emotional patterns based on the data recorded by the user in the emotional diary and providing feedback and suggestions, and means for analyzing the user's health data using a generative AI model and providing the user with optimal reminders and relaxing music, thereby enabling the user to comprehensively manage their health from both physical and mental perspectives.

[0615] "User physical health indicators" are data that indicate the user's physical condition or health in real time, such as heart rate, blood pressure, and body temperature.

[0616] "Real-time measurement means" refers to a mechanism that obtains a user's health indicators continuously or periodically and collects data instantly.

[0617] The "database" is a system that systematically stores and manages received data such as health indicators and emotional diaries.

[0618] "Means for storing data in a database" refers to the methods and technologies for efficiently storing collected data in a database.

[0619] "Means for analyzing and assessing the user's health status" means algorithms or methods for using the collected data to assess the user's physical and mental health status.

[0620] The "means for sending reminder notifications" is a mechanism that generates messages encouraging breaks or attention based on the user's health and emotional state and sends them to the user's device.

[0621] "Environmental adjustment means" is a mechanism that provides an appropriate music and audio environment for the purpose of user relaxation and stress relief.

[0622] "Means for playing relaxing music" refers to a system that allows users to select music that helps them relax and play it on their devices.

[0623] "Means of connecting users to social support groups" means mechanisms that allow users to join appropriate groups or communities to receive emotional support.

[0624] "User's Emotion Diary" is data that allows users to record their daily emotions and moods.

[0625] The "means for recording and analyzing an emotional diary" is a technology for saving an emotional diary entered by a user and analyzing the data to understand the user's emotional patterns.

[0626] A "generative AI model" is an artificial intelligence algorithm used to analyze collected data and provide optimal feedback and suggestions to users.

[0627] The present invention is a system for supporting a user's physical and mental health. The system measures a user's health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system can then provide the user with reminders, relaxing music, and even connect them to social support groups.

[0628] Overall system configuration

[0629] health measurement

[0630] While the user is wearing the smart glasses, health indicators such as heart rate and blood pressure are measured in real time using the smart glasses' built-in sensors, and the data is sent to a server via Bluetooth or Wi-Fi.

[0631] Data analysis

[0632] The server stores the received health index data in a database and periodically evaluates the user's health status using a data analysis algorithm. For example, if the user's heart rate is higher than normal, the server determines that the user is feeling stressed.

[0633] Reminders

[0634] If the server determines that the user's emotional state is declining, it generates a reminder notification to encourage them to take a break and sends it to the user's device. Specifically, it sends a message saying, "Your emotional state is declining. Please take a break."

[0635] Play relaxing music

[0636] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends a command to play it to the device. The device then follows the command and plays music to help the user relax. For example, relaxing music such as the sounds of a quiet forest or the sound of waves can be played.

[0637] Connecting to social support groups

[0638] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information and sends the information to the device, where the user can join and communicate with other members in real time.

[0639] Emotional diary recording

[0640] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[0641] Using generative AI models

[0642] The server uses a generative AI model to analyze the received health index data and emotional diary data. This analysis allows it to provide optimal reminder notifications and relaxing music to the user. Specifically, it analyzes the user's stress level and suggests optimal measures accordingly.

[0643] Specific examples

[0644] Specific examples of health measurements

[0645] While the user is wearing the smart glasses, their heart rate is measured every second and sent to a server. The data is then recorded in a database and compared to their normal heart rate. For example, if their heart rate is higher than normal, the server will determine that the user's stress level is high.

[0646] Examples of reminder notifications

[0647] If the server determines that the user's emotional state is declining based on the user's emotional diary and health data, it generates a reminder notification saying, "Your emotional state is declining. Please take a break." The device receives this notification and displays it to the user.

[0648] Example of relaxing music playback

[0649] If the server determines that the user is feeling stressed, it sends a command to play relaxing music to the device. The device then follows this command and plays relaxing music such as the sound of a quiet forest or the sound of waves, allowing the user to relax.

[0650] Examples of connecting with social support groups

[0651] When a user requests to connect to a support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members in real time.

[0652] Examples of emotional diary entries

[0653] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[0654] Example prompts for generative AI models

[0655] Analyze the user's heart rate data and emotional diary to determine whether they are feeling stressed and suggest relaxing music to reduce stress.

[0656] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0657] Step 1:

[0658] User wear of the device and measurement of health indicators

[0659] The user wears smart glasses, which use built-in sensors to measure health indicators such as heart rate and blood pressure in real time. The input data is the user's health indicators, such as heart rate and blood pressure, which are collected by the smart glasses every second. The collected data is then sent to a server via Bluetooth or WiFi.

[0660] Step 2:

[0661] Sending and saving data to the server

[0662] The server receives the data sent from the smart glasses. The input data is the health index data sent from the smart glasses. The server receives this data and first checks the integrity of the data. Then, it stores it in the database. The output is the health index data whose integrity has been confirmed, which is then stored in the database.

[0663] Step 3:

[0664] Regular analysis of health data

[0665] The server periodically analyzes the health index data stored in the database. The input data is the past and current health index data stored in the database. The server uses a data analysis algorithm to evaluate the user's health status. Specifically, it analyzes fluctuations in heart rate and blood pressure to detect signs of stress and fatigue. The output is an evaluation result of the user's health status.

[0666] Step 4:

[0667] Evaluating emotional state and generating reminder notifications

[0668] The server evaluates the user's emotional state based on the analysis results. The input data is the analysis results of health data. For example, if the heart rate is continuously high, it is determined that the emotional state is declining. The server generates a reminder notification such as "Your emotional state is declining. Please take a break." The output is the generated reminder notification.

[0669] Step 5:

[0670] Sending reminders

[0671] The server sends the generated reminder notification to the user's device. The input data is the reminder notification generated by the server. The server sends the notification and the device receives it. The output is the reminder notification received by the user's device and displayed on the device screen.

[0672] Step 6:

[0673] Selecting relaxing music and sending playback instructions

[0674] The server selects relaxing music based on the reminder notification. The input data is the user's current health and emotional state, and the user's music preferences. The server uses a generative AI model to select the most suitable relaxing music for the user. It sends a play instruction for the selected music to the device. The output is the music play instruction sent to the device.

[0675] Step 7:

[0676] Playing relaxing music on your device

[0677] The terminal plays relaxing music based on the music playback instruction received from the server. The input data is the music playback instruction sent from the server. The terminal receives the instruction and plays the specified relaxing music to the user. The output is music that the user can listen to and feel relaxed.

[0678] Step 8:

[0679] Connecting to social support groups

[0680] When a user wants to connect to a social support group, the server selects an appropriate group based on the user's profile information. The input data is the user's profile information and a database of groups. The server selects an appropriate group and sends the information to the user's device. The output is the group information to connect to.

[0681] Step 9:

[0682] Record and send your emotional diary

[0683] Users record their emotion diary using smart glasses or a smartphone app. The input data is the diary of emotions recorded by the user. The recorded diary data is sent to a server and stored in a database. The output is the emotion diary data sent to and stored on the server.

[0684] Step 10:

[0685] Emotion diary analysis and feedback

[0686] The server analyzes the emotion diary data and provides feedback and suggestions. The input data is the emotion diary data stored in the database. The server analyzes it and generates feedback such as, "You seem to have been feeling tired lately. I recommend you take a rest." The output is the feedback and suggestions sent to the user.

[0687] Above are the specific processing steps and detailed explanation of this system, which allows users to comprehensively manage their physical and mental health.

[0688] (Application example 1)

[0689] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0690] In factory work environments, managing employee stress and health is important, but conventional methods make it difficult to grasp the situation of individual employees in real time and implement appropriate measures. Also, when employees need psychological support, there are limited ways to connect them to appropriate social support groups. To solve these issues, real-time analysis of data and automatic implementation of appropriate measures are required.

[0691] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0692] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for measuring the stress level and heart rate of factory workers and sending a health notification and playing relaxing music based on the analysis results, means for connecting the factory workers to an appropriate social support group, and means for sending notifications and playing music via a robot. This enables real-time management of the health and emotional states of employees in a factory work environment, and makes it possible to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[0693] "User" refers to an individual who uses this system.

[0694] "Physical health indicators" are data that indicate the user's physical condition, such as heart rate and blood pressure.

[0695] "Real-time measurement means" refers to technology that uses smart glasses or wearable devices to instantly collect health indicators.

[0696] The "means for storing in a database" refers to a technique for recording and storing the measured health indicators in a database.

[0697] "Means for assessing health status" refers to technology that analyzes stored data and determines the user's physical and mental status.

[0698] "Means for sending reminder notifications" refers to technology that sends notifications to users to warn them or suggest they take a break.

[0699] The "means for playing relaxing music as an environmental adjustment means" is a technology for selecting and playing appropriate relaxing music to improve the user's emotional state.

[0700] "Means for connecting users to social support groups" refers to technology that connects users to appropriate support groups when desired.

[0701] "Means for recording and analyzing emotional diaries" refers to a technology that allows users to record their own emotional state and then analyze that data.

[0702] "Means for measuring and analyzing stress levels and heart rates" refers to technology that measures and analyzes health indicators such as the heart rate of factory workers in real time.

[0703] The "means for sending health notifications and playing relaxing music" is a technology that sends notifications and plays relaxing music when stress or overwork is detected.

[0704] "Means for sending notifications and playing music via a robot" refers to a technology in which a robot sends reminder notifications to the user and plays relaxing music.

[0705] System Overview

[0706] The system of the present invention is designed to manage the physical and mental health of users and support stress management in factory work environments. Users wear smart glasses or wearable devices to measure health indicators such as heart rate in real time. The measured data is sent to a server, where it is recorded and analyzed in a database to evaluate the user's health. Monitoring the health of employees is particularly important in factory environments where workers are under heavy stress.

[0707] Hardware and software used

[0708] The system is implemented using the following hardware and software.

[0709] Wearable devices: Smart glasses and smart watches are used to measure data such as heart rate and stress levels.

[0710] Server: Stores the received health data in a database and analyzes it. Implemented in a programming language such as Python.

[0711] Database: A database system such as MySQL or PostgreSQL will be used to record and store health index data.

[0712] Robot: Use a robot to send reminder notifications to users and play relaxing music.

[0713] Data collection and analysis process

[0714] 1. Data collection: Users wear smart glasses or smartwatches, which measure their heart rate and stress levels in real time. The data is then sent to a server via Bluetooth or Wi-Fi.

[0715] 2. Data storage: The received data is stored on the server and recorded in a database.

[0716] 3. Data analysis: The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate is higher than normal, it determines that the user is feeling stressed.

[0717] 4. Reminder notification: If the user's emotional state is determined to be declining, the server generates a reminder notification and sends a message to the user encouraging them to take a break.

[0718] 5. Relaxing music playback: The server sends a reminder notification and a command to play relaxing music to the robot, allowing the user to relax while listening to the music.

[0719] Social support function

[0720] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[0721] Specific examples

[0722] As a specific example of operation, consider the case where a factory worker feels fatigued or stressed. If the factory worker wears a smartwatch and their heart rate exceeds the normal level, the server will send a notification via a robot saying, "High stress level detected. Please take a break." In addition, relaxing music will be played automatically.

[0723] Prompt Sentence Examples

[0724] "Write a program that notifies you and plays relaxing music if it determines that a factory worker is stressed."

[0725] In this way, this invention enables real-time management of employee health and emotional states in a factory environment, and is expected to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[0726] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0727] Program processing steps

[0728] Step 1: Data collection

[0729] Health index data such as heart rate and stress level is collected in real time while the user is wearing the wearable device. Data is sent from the device to a server via Bluetooth or WiFi. Input: Measurement data of heart rate and stress level. Output: Health index data sent to the server.

[0730] Step 2: Save data

[0731] The server stores the received data in a database. The stored data is used for later analysis. Input: Health index data received from the device. Output: Stored data recorded in the database.

[0732] Step 3: Data analysis

[0733] The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate exceeds normal values, it determines that the user is feeling stressed. Data analysis is performed using Python and other tools. Input: Health index data in the database. Output: Analysis results (user's stress and health state).

[0734] Step 4: Generate reminder notifications

[0735] If the data analysis determines that the user's emotional state is declining, the server generates a reminder notification. The notification may contain a text message such as "High stress detected. Please take a break." Input: Analysis results. Output: Reminder notification text message.

[0736] Step 5: Select and play relaxation music

[0737] The server selects appropriate relaxation music based on the user's emotional state and sends playback instructions to the robot. For example, it may select forest sounds or the sound of waves. The robot plays the music based on the instructions. Input: Reminder notification and emotional state data. Output: Relaxation music to be played.

[0738] Step 6: Social support connections

[0739] When a user requests a social support connection, the server selects an appropriate social support group based on the user's profile information and establishes a connection. Input: User's profile information and connection request. Output: Selected social support group information.

[0740] Specific actions

[0741] For example, when a factory worker wears a smartwatch, their heart rate is measured in real time. If their heart rate exceeds 110 and their stress level is high at 8, the server generates a notification saying "High stress detected. Please take a break" and sends it to the worker via a robot. The robot also automatically plays relaxation music.

[0742] Example prompt sentence:

[0743] "Write a program that notifies you and plays relaxation music if it determines that a factory worker is stressed."

[0744] In this way, the processing at each step is carried out specifically, and real-time management of the user's health condition is realized.

[0745] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0746] The present invention is a system for supporting a user's mental and physical health, and in particular, aims to incorporate an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine also recognizes the user's emotions and provides accurate feedback based on those emotions.

[0747] Overall system configuration

[0748] health measurement

[0749] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[0750] Data analysis

[0751] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[0752] Use of emotion engine

[0753] The emotion engine analyzes the user's facial expressions, voice, input text data, etc. to recognize the user's emotional state. The emotion data obtained by the emotion engine is sent to the server and used as part of the analysis.

[0754] Reminders

[0755] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[0756] Play relaxing music

[0757] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[0758] Connecting to social support groups

[0759] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[0760] Emotional diary recording

[0761] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[0762] Specific examples

[0763] Specific examples of health measurements

[0764] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[0765] Examples of emotion engines

[0766] When a user speaks using the smartphone app, the voice data is analyzed by the emotion engine. The emotion engine recognizes the user's emotional state from the tone of the voice and the way they speak, and determines that they are in a "stressed state." As a result, the server sends appropriate instructions, such as sending a reminder notification or playing relaxing music.

[0767] Examples of reminder notifications

[0768] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[0769] Example of relaxing music playback

[0770] When the emotion engine recognizes that the user is feeling stressed, the server sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music that includes the sounds of a quiet forest or waves can help the user relax.

[0771] Examples of connecting with social support groups

[0772] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[0773] Examples of emotional diary entries

[0774] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[0775] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate mental support as needed. In addition, the emotion engine helps recognize emotional states and provides more accurate feedback, thereby more effectively supporting the user's mental health.

[0776] The processing flow will be explained below.

[0777] health measurement

[0778] Program processing steps

[0779] Step 1:

[0780] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[0781] Step 2:

[0782] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[0783] Step 3:

[0784] The device sends the measurement data to the server, which then sends the data to the server via API.

[0785] Step 4:

[0786] The server analyzes the received data and stores it in a real-time database.

[0787] Data analysis

[0788] Program processing steps

[0789] Step 1:

[0790] The server periodically queries the database for the latest heart rate data.

[0791] Step 2:

[0792] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[0793] Step 3:

[0794] The server stores the analysis results in a database and prepares notifications as needed.

[0795] Use of emotion engine

[0796] Program processing steps

[0797] Step 1:

[0798] The user starts inputting emotions through a smartphone app or smart glasses, and the user's facial, voice, and text data are sent to the emotion engine.

[0799] Step 2:

[0800] The emotion engine analyzes the received data and recognizes the user's emotional state. The recognized emotion data is sent to the server.

[0801] Step 3:

[0802] The server uses the data from the emotion engine as part of its analysis to assess the user's current emotional state.

[0803] Reminders

[0804] Program processing steps

[0805] Step 1:

[0806] The server monitors the user's health status data and the analysis results of the emotion engine to detect a decline in the emotional state.

[0807] Step 2:

[0808] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[0809] Step 3:

[0810] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[0811] Play relaxing music

[0812] Program processing steps

[0813] Step 1:

[0814] The server analyzes the user's emotional state and determines that relaxing music is required.

[0815] Step 2:

[0816] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[0817] Step 3:

[0818] The server sends a music playback command to the device, including the music file information and playback command.

[0819] Step 4:

[0820] The device receives the instruction and plays the selected relaxing music.

[0821] Connecting to social support groups

[0822] Program processing steps

[0823] Step 1:

[0824] A user sends a request from the device to connect to a social support group.

[0825] Step 2:

[0826] The device sends the user's request to the server, which also includes the user's profile information.

[0827] Step 3:

[0828] The server selects appropriate social support groups based on the user's profile information.

[0829] Step 4:

[0830] The server transmits information about the selected social support group to the terminal.

[0831] Step 5:

[0832] The terminal displays the received group information and connects the user to the group.

[0833] Emotional diary recording

[0834] Program processing steps

[0835] Step 1:

[0836] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[0837] Step 2:

[0838] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[0839] Step 3:

[0840] The server stores the received data in a database for later analysis.

[0841] Step 4:

[0842] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[0843] Step 5:

[0844] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[0845] Example 2

[0846] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0847] In today's world, lifestyle habits and work stress can lead to a decline in users' mental and physical health. These health problems are particularly difficult to recognize, making it challenging to address them at the appropriate time. It is also difficult to provide users with appropriate feedback regarding a decline in mental health. Conventional systems have struggled to monitor a user's health and emotional state in real time and provide appropriate support. To address these issues, it is necessary to monitor a user's health and emotional state in real time and take appropriate measures.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0849] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health condition, means for analyzing the user's facial expressions, voice, and text data to recognize the user's emotional state, means for storing the recognized emotional state in a database, means for sending a reminder notification when the user's emotional state deteriorates, means for selecting and playing relaxing music together with the reminder notification, means for connecting the user to a social support group, and means for recording and analyzing the user's emotional diary, thereby making it possible to monitor the user's health condition and emotional state in real time and provide appropriate feedback and support.

[0850] A "user" is an individual or group that uses the system and provides data on health indicators and emotional states.

[0851] "Physical health indicators" are measurement data that indicate an individual's physical health status, such as heart rate, body temperature, blood pressure, and respiratory rate.

[0852] "Real-time" refers to the state in which data is processed immediately from the moment it is generated, without any delay.

[0853] The "database" is an electronic record system for systematically organizing and storing data on users' health indicators and emotional states.

[0854] "Analysis" is the process of extracting information from collected data using statistical methods and algorithms, and then evaluating and judging it.

[0855] "Emotional state" is information that indicates the user's psychological state and mood, and is obtained from facial expressions, voice, and text data.

[0856] A "reminder notification" is a notification that includes a message or advice that urges the user to take a break or take measures.

[0857] "Relaxing music" is music selected for the purpose of relieving the user's mental tension and has the role of promoting relaxation.

[0858] A "social support group" is an online or offline group that users can join to receive emotional support and communication from other members.

[0859] An "emotion diary" is an electronic diary that allows users to record their emotions, moods, physical conditions, etc.

[0860] The present invention provides a system for supporting a user's mental and physical health, particularly by incorporating an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine is also used to recognize the user's emotions and provide accurate feedback based on those emotions.

[0861] The components of this system are:

[0862] 1. A means of measuring the user's physical health indicators:

[0863] When a user wears smart glasses, health indicators such as heart rate are measured in real time. A specific example is a heart rate sensor built into smart glasses such as Google Glass.

[0864] 2. Means of storing health indicator data in the database:

[0865] The smart glasses transmit the measured heart rate data to a server, which stores the data in a database (e.g., MySQL). The data is transmitted via Wi-Fi or Bluetooth and recorded in real time.

[0866] 3. Means of analyzing health indicator data:

[0867] The server periodically analyzes the received health index data and evaluates the user's health condition. If an abnormal value (e.g., a higher heart rate than normal) is detected at this stage, an analysis result is generated indicating that the user may be experiencing stress.

[0868] 4. Means of recognizing emotional states:

[0869] The user inputs voice and facial expression data using a smartphone app. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes this data and recognizes the user's emotional state. The analysis results are sent to a server and stored in a database.

[0870] 5. Send reminder notifications by:

[0871] If the server determines that the user's emotional state is declining, it generates and sends a reminder notification to the device, which may include a message such as "Your emotional state is declining. Please take a break."

[0872] 6. How to play relaxing music:

[0873] The server selects relaxing music along with the reminder notification. The selection process refers to the analysis results of the emotion engine and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[0874] 7. Ways to connect with social support groups:

[0875] When a user requests to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[0876] 8. How to record and analyze your emotional diary:

[0877] Users use a smartphone app to record their emotions in a diary. Records such as "I'm tired today" are sent to a server and stored in a database. The server analyzes this data and provides feedback to the user suggesting they take a rest if fatigue persists.

[0878] Specific actions

[0879] Specific examples of health measurements

[0880] While the user is wearing Google Glass, heart rate data is measured every second and sent to a server, which receives the data and records it in a database. If the data shows abnormal values, the user's stress level is evaluated.

[0881] Examples of emotion engines

[0882] When a user inputs a voice message using a smartphone app, the emotion engine analyzes the voice data and determines that the user is in a "stressed state." This analysis result is sent to the server, which then sends a reminder notification to the device along with an instruction to play relaxing music.

[0883] Prompt Sentence Examples

[0884] For example, by inputting to the generative AI model, "Please tell me the code to generate a specific music list for the user to relax and send the playback instructions to the server," an appropriate program can be generated.

[0885] This system allows users to manage their health in real time and receive psychological support when necessary. The emotional engine enables the system to accurately recognize the user's emotional state and provide effective feedback.

[0886] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0887] Step 1:

[0888] Users wear smart glasses to measure their health data.

[0889] The user wears the smart glasses. The smart glasses have a built-in heart rate sensor that measures heart rate data every second. The measured data is not stored directly, but is sent to the server in the next step.

[0890] Input: Smart glasses worn by the user, user's heart rate

[0891] Output: Heart rate data measured every second

[0892] Step 2:

[0893] The smart glasses send the data to the server.

[0894] The smart glasses transmit the measured heart rate data to a server in real time via Wi-Fi or Bluetooth, allowing the data to be analyzed immediately.

[0895] Input: Heart rate data measured every second

[0896] Output: Heart rate data sent to the server

[0897] Step 3:

[0898] The server stores health data in a database and periodically analyzes it.

[0899] The server stores the received heart rate data in a database. The data stored in the database (e.g., MySQL) is periodically analyzed using an analysis algorithm (e.g., an anomaly detection algorithm). If an abnormal value is detected, the user's health condition is evaluated.

[0900] Input: Heart rate data sent to the server

[0901] Output: Data stored in a database, analysis results (health status is evaluated)

[0902] Step 4:

[0903] The user generates emotion data using the emotion engine.

[0904] Users use a smartphone app to input voice messages or text data. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes the input voice or text data and recognizes the user's emotional state. The results are sent to the server.

[0905] Input: Voice messages and text data

[0906] Output: Parsed emotional state data

[0907] Step 5:

[0908] The server analyzes the emotion data and generates reminder notifications as needed.

[0909] The server analyzes the emotion data sent from the emotion engine. If the user's emotional state is recognized as "stressed," it generates a reminder notification and sends it to the device. The reminder notification may include a message such as "Your emotional state is declining. Please take a break."

[0910] Input: Parsed emotional state data

[0911] Output: Reminder notification

[0912] Step 6:

[0913] The server selects relaxing music along with the reminder notification and sends it to the device.

[0914] The server selects relaxing music based on the user's current emotional state and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[0915] Input: Reminders, emotional state data, historical data

[0916] Output: Relaxing music selection, music playback instructions

[0917] Step 7:

[0918] When a user wants to connect to a social support group, the server provides the information and helps them connect.

[0919] When a user requests to join a social support group through a smartphone app, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[0920] Input: User profile information, desired connection information

[0921] Output: Information on suitable social support groups

[0922] Step 8:

[0923] Users record their emotions in a diary, and the server analyzes the data and provides feedback.

[0924] Users use a smartphone app to record their emotions in a diary. The recorded data (e.g., "I feel tired today") is sent to a server and stored in a database. The server analyzes this data and analyzes the user's emotional patterns. If fatigue persists, the server provides feedback to the user suggesting that they take a rest.

[0925] Input: Emotion diary data

[0926] Output: Feedback notification based on analysis results

[0927] (Application example 2)

[0928] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0929] Conventional technologies for supporting users' mental and physical health have struggled to accurately recognize the user's emotional state, preventing them from providing appropriate support. Furthermore, providing services in physical stores has also been problematic, as it has been difficult to provide individualized support based on the user's current health and emotional state, making it difficult to improve user satisfaction. Therefore, there is a need for the development of a system that can more accurately recognize the user's emotional state and provide appropriate feedback and support in real time.

[0930] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0931] In this invention, the server includes means for measuring a user's physical health index in real time, means for storing the measured health index in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for providing appropriate services in a physical store based on the user's physical health index and emotional state, means for suggesting products according to the user's health state and emotional state, and means for playing relaxing music in a specific area when the user's stress state is determined to be high. This allows the user to manage their health in real time and receive appropriate psychological support and personalized services in a physical store as needed.

[0932] "User's physical health indicators" refers to physiological data such as the user's heart rate, steps, blood pressure, and body temperature.

[0933] "Means of measuring in real time" refers to technology that continuously acquires physical health indicators from the user's body and instantly converts them into data.

[0934] "Means for storing data in a database" refers to a system for systematically recording measured data and storing it in a format that can be managed and searched.

[0935] "Means for analyzing and assessing the user's health status" refers to algorithms and software that objectively assess the user's health status based on collected health indicator data.

[0936] "Means for sending reminder notifications" refers to a system that sends messages to users to encourage them to take a break or suggest ways to reduce stress.

[0937] "Means for playing relaxing music" refers to a function that plays music that relieves stress based on the user's emotional state.

[0938] "Means for connecting users to social support groups" refers to technology that allows users to access appropriate communities and groups to receive emotional support.

[0939] "A means for recording and analyzing a user's emotional diary" refers to a system that allows users to input the emotions they feel on a daily basis, and then records and analyzes that data.

[0940] "Means for providing appropriate services in physical stores" refers to a system for providing services in a physical store environment that take into account the user's current health and emotional state.

[0941] "Means for suggesting products based on health and emotional state" refers to technology that suggests optimal products and services based on the user's health and emotional data.

[0942] "Means for playing relaxing music in specific areas" refers to the function of playing music in specific areas within the store to help users relax.

[0943] The present invention is a system for supporting the mental and physical health of a user, and in particular incorporates an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. This system is realized mainly using the following hardware and software.

[0944] Hardware and software used:

[0945] Smart glasses (e.g., Google Glass, Vuzix Blade): measure the user's physical health indicators (heart rate, steps, etc.) in real time.

[0946] Smartphone / tablet (e.g. iPad, Android tablet): Displays and notifies the user's emotional state and health data.

[0947] Server (e.g. AWS EC2, Google Cloud VM): Analyzes and stores data, and generates notifications.

[0948] Emotion engine (e.g., Microsoft Azure Emotion API, IBM Watson Tone Analyzer): Evaluates the user's emotional state.

[0949] System features:

[0950] The system is equipped with the following functions, and analyzes the user's health and emotional state in real time and provides appropriate support based on that.

[0951] 1. Real-time health monitoring:

[0952] The smart glasses measure the user's heart rate, number of steps, and other data in real time and send it to a server, where it stores the data in a database and analyzes it, allowing the user's health status to be tracked.

[0953] 2. Recognition of emotional states:

[0954] The emotion engine analyzes the user's facial expressions and voice data to assess their emotional state. For example, it analyzes what the user says and their facial expressions using a smartphone app to determine their stress level. The data obtained by the emotion engine is also sent to the server and used as part of the analysis.

[0955] 3. Reminder Notification:

[0956] If the user's emotional state is determined to be declining, the server generates a reminder notification and sends it to the user's smartphone or tablet, which may include a message to take a break or specific stress reduction advice.

[0957] 4. Play relaxing music:

[0958] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends playback instructions to the smart glasses or smartphone, allowing the user to relax by listening to music such as the sound of a tranquil forest or the sound of waves.

[0959] 5. Connect with social support groups:

[0960] If a user wishes to join a social support group, the server will recommend and connect them to an appropriate group based on their profile information, allowing them to communicate with other members in real time and receive emotional support.

[0961] 6. Personalized services in-store:

[0962] When a user is in a physical store, real-time information is provided to store staff based on the user's health indicators and emotional state. For example, a tense user can be provided with a relaxing environment, and a stressed user can be suggested products that will have a relaxing effect.

[0963] Specific use cases:

[0964] 1. Health measurement and emotional state recognition:

[0965] When a user wears smart glasses and goes shopping, their heart rate is measured to be high, and the emotion engine detects a state of stress. In this case, the server issues a command to "play relaxing music," and nature music is played on the user's smartphone.

[0966] 2. Reminder Notification:

[0967] If the server analyzes health data and emotional state to be declining, a reminder notification such as "Take a break" will be sent to the user's device.

[0968] 3. Social support group connections:

[0969] When a user types "I need support" into the smartphone app, the server recommends appropriate social support groups and connects the user to them.

[0970] Example prompt sentence:

[0971] "Implement a system that recognizes customers' stress levels and suggests products that will help them relax. Use smart glasses and an emotion recognition engine to notify staff when a certain threshold is exceeded."

[0972] This allows users to receive appropriate mental and physical support in real time, making the shopping experience in physical stores more comfortable.

[0973] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0974] Step 1:

[0975] When a user wears smart glasses, physical health indicators (heart rate, number of steps, etc.) are measured in real time. The smart glasses instantly convert this data into digital data and send it to a server. The input data is health information such as heart rate and number of steps, and the output is the measurement data sent to the server. The smart glasses have built-in heart rate monitors and acceleration sensors, which are used to acquire data in real time.

[0976] Step 2:

[0977] The server receives the health index data sent from the smart glasses and stores it in a database. At this time, the server organizes the data so that it can be easily searched and analyzed later. The input data is the measurement data sent from the smart glasses, and the output is the health data stored in the database. The server receives the data using a data transfer protocol (e.g., HTTP, HTTPS) and stores it in a database (e.g., MySQL, PostgreSQL).

[0978] Step 3:

[0979] The server analyzes the stored data and evaluates the user's health condition. This analysis uses an algorithm that, for example, determines that an abnormally high heart rate indicates stress. The input data is the health data stored in the database, and the output is the evaluation result of the user's health condition. The server analyzes the data using a programming language such as Python and a data analysis library (e.g., Pandas, NumPy).

[0980] Step 4:

[0981] The emotion engine analyzes the user's facial expression and voice data to evaluate their emotional state. The emotion engine analyzes facial expression and voice data input via a smartphone or tablet. The input data is the user's facial expression and voice information, and the output is an evaluation of their emotional state. The emotion engine uses the Microsoft Azure Emotion API, IBM Watson Tone Analyzer, etc. to analyze facial expressions and voice emotions.

[0982] Step 5:

[0983] If the server determines that the user's health and emotional state is declining, it generates a reminder notification. The reminder notification includes a message encouraging the user to take a break and specific advice for reducing stress. The input data is the assessment result of the user's health and emotional state, and the output is the reminder notification message. The server creates the message using a notification generation algorithm and sends the notification using a notification service (e.g., Firebase Cloud Messaging).

[0984] Step 6:

[0985] The server sends a reminder notification and an instruction to play relaxing music to the smart glasses or smartphone. The relaxing music may include, for example, the sound of a quiet forest or the sound of waves. The input data is the instruction to play relaxing music, and the output is the relaxing music played from the smart glasses or smartphone. The server uses a music selection algorithm and sends the play instruction to the device.

[0986] Step 7:

[0987] When a user wishes to join a social support group, the server recommends and connects them to appropriate groups based on the user's profile information. The input data is the user's profile information, and the output is information about recommended social support groups. The server uses a recommendation algorithm and connects to communication platforms (e.g., Slack, Discord).

[0988] Step 8:

[0989] When a user is in a physical store, the server provides the user's health indicators and emotional state to store staff in real time. The staff obtains this information via tablets or smart devices and provides individualized support. The input data is the user's health indicators and emotional state, and the output is the information displayed to the staff. The server uses a data transfer protocol to send the data to the staff application.

[0990] Step 9:

[0991] In a physical store, the server makes product suggestions based on the user's health and emotional state. For example, a user experiencing high stress may be recommended a product with a relaxing effect. The input data is the user's health and emotional state, and the output is personalized product suggestions. The server uses a recommendation engine to generate the suggestions.

[0992] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0993] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0994] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0995] [Third embodiment]

[0996] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0997] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0998] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0999] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1000] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1001] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1002] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1003] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1004] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1005] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1006] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1007] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1008] The present invention is a system for supporting a user's mental and physical health. The system measures a user's physical health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group.

[1009] Overall system configuration

[1010] health measurement

[1011] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[1012] Data analysis

[1013] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[1014] Reminders

[1015] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[1016] Play relaxing music

[1017] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[1018] Connecting to social support groups

[1019] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[1020] Emotional diary recording

[1021] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[1022] Specific examples

[1023] Specific examples of health measurements

[1024] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[1025] Examples of reminder notifications

[1026] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[1027] Example of relaxing music playback

[1028] If the server determines that the user is feeling stressed, it sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music including the sounds of a quiet forest or waves can be played to help the user relax.

[1029] Examples of connecting with social support groups

[1030] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[1031] Examples of emotional diary entries

[1032] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[1033] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate psychological support as needed.

[1034] The processing flow will be explained below.

[1035] health measurement

[1036] Program processing steps

[1037] Step 1:

[1038] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[1039] Step 2:

[1040] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[1041] Step 3:

[1042] The device sends the measurement data to the server, which then sends the data to the server via API.

[1043] Step 4:

[1044] The server analyzes the received data and stores it in a real-time database.

[1045] Data analysis

[1046] Program processing steps

[1047] Step 1:

[1048] The server periodically queries the database for the latest heart rate data.

[1049] Step 2:

[1050] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[1051] Step 3:

[1052] The server stores the analysis results in a database and prepares notifications as needed.

[1053] Reminders

[1054] Program processing steps

[1055] Step 1:

[1056] The server monitors the user's health data and detects a decline in emotional state.

[1057] Step 2:

[1058] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[1059] Step 3:

[1060] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[1061] Play relaxing music

[1062] Program processing steps

[1063] Step 1:

[1064] The server analyzes the user's emotional state and determines that relaxing music is required.

[1065] Step 2:

[1066] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[1067] Step 3:

[1068] The server sends a music playback command to the device, including the music file information and playback command.

[1069] Step 4:

[1070] The device receives the instruction and plays the selected relaxing music.

[1071] Connecting to social support groups

[1072] Program processing steps

[1073] Step 1:

[1074] A user sends a request from the device to connect to a social support group.

[1075] Step 2:

[1076] The device sends the user's request to the server, which also includes the user's profile information.

[1077] Step 3:

[1078] The server selects appropriate social support groups based on the user's profile information.

[1079] Step 4:

[1080] The server transmits information about the selected social support group to the terminal.

[1081] Step 5:

[1082] The terminal displays the received group information and connects the user to the group.

[1083] Emotional diary recording

[1084] Program processing steps

[1085] Step 1:

[1086] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[1087] Step 2:

[1088] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[1089] Step 3:

[1090] The server stores the received data in a database for later analysis.

[1091] Step 4:

[1092] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[1093] Step 5:

[1094] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[1095] Example 1

[1096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1097] In modern society, many people live with stress and fatigue. Conventional health management systems are primarily limited to measuring physical health indicators and lack the means to properly monitor changes in users' mental state and emotions. As a result, it is difficult for users to accurately understand their own mental health status and take prompt action. In addition, there are limited ways to provide relaxation and mental support when needed, making comprehensive health management difficult.

[1098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1099] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for analyzing the user's emotional patterns based on the data recorded by the user in the emotional diary and providing feedback and suggestions, and means for analyzing the user's health data using a generative AI model and providing the user with optimal reminders and relaxing music, thereby enabling the user to comprehensively manage their health from both physical and mental perspectives.

[1100] "User physical health indicators" are data that indicate the user's physical condition or health in real time, such as heart rate, blood pressure, and body temperature.

[1101] "Real-time measurement means" refers to a mechanism that obtains a user's health indicators continuously or periodically and collects data instantly.

[1102] The "database" is a system that systematically stores and manages received data such as health indicators and emotional diaries.

[1103] "Means for storing data in a database" refers to the methods and technologies for efficiently storing collected data in a database.

[1104] "Means for analyzing and assessing the user's health status" means algorithms or methods for using the collected data to assess the user's physical and mental health status.

[1105] The "means for sending reminder notifications" is a mechanism that generates messages encouraging breaks or attention based on the user's health and emotional state and sends them to the user's device.

[1106] "Environmental adjustment means" is a mechanism that provides an appropriate music and audio environment for the purpose of user relaxation and stress relief.

[1107] "Means for playing relaxing music" refers to a system that allows users to select music that helps them relax and play it on their devices.

[1108] "Means of connecting users to social support groups" means mechanisms that allow users to join appropriate groups or communities to receive emotional support.

[1109] "User's Emotion Diary" is data that allows users to record their daily emotions and moods.

[1110] The "means for recording and analyzing an emotional diary" is a technology for saving an emotional diary entered by a user and analyzing the data to understand the user's emotional patterns.

[1111] A "generative AI model" is an artificial intelligence algorithm used to analyze collected data and provide optimal feedback and suggestions to users.

[1112] The present invention is a system for supporting a user's physical and mental health. The system measures a user's health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system can then provide the user with reminders, relaxing music, and even connect them to social support groups.

[1113] Overall system configuration

[1114] health measurement

[1115] While the user is wearing the smart glasses, health indicators such as heart rate and blood pressure are measured in real time using the smart glasses' built-in sensors, and the data is sent to a server via Bluetooth or Wi-Fi.

[1116] Data analysis

[1117] The server stores the received health index data in a database and periodically evaluates the user's health status using a data analysis algorithm. For example, if the user's heart rate is higher than normal, the server determines that the user is feeling stressed.

[1118] Reminders

[1119] If the server determines that the user's emotional state is declining, it generates a reminder notification to encourage them to take a break and sends it to the user's device. Specifically, it sends a message saying, "Your emotional state is declining. Please take a break."

[1120] Play relaxing music

[1121] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends a command to play it to the device. The device then follows the command and plays music to help the user relax. For example, relaxing music such as the sounds of a quiet forest or the sound of waves can be played.

[1122] Connecting to social support groups

[1123] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information and sends the information to the device, where the user can join and communicate with other members in real time.

[1124] Emotional diary recording

[1125] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[1126] Using generative AI models

[1127] The server uses a generative AI model to analyze the received health index data and emotional diary data. This analysis allows it to provide optimal reminder notifications and relaxing music to the user. Specifically, it analyzes the user's stress level and suggests optimal measures accordingly.

[1128] Specific examples

[1129] Specific examples of health measurements

[1130] While the user is wearing the smart glasses, their heart rate is measured every second and sent to a server. The data is then recorded in a database and compared to their normal heart rate. For example, if their heart rate is higher than normal, the server will determine that the user's stress level is high.

[1131] Examples of reminder notifications

[1132] If the server determines that the user's emotional state is declining based on the user's emotional diary and health data, it generates a reminder notification saying, "Your emotional state is declining. Please take a break." The device receives this notification and displays it to the user.

[1133] Example of relaxing music playback

[1134] If the server determines that the user is feeling stressed, it sends a command to play relaxing music to the device. The device then follows this command and plays relaxing music such as the sound of a quiet forest or the sound of waves, allowing the user to relax.

[1135] Examples of connecting with social support groups

[1136] When a user requests to connect to a support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members in real time.

[1137] Examples of emotional diary entries

[1138] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[1139] Example prompts for generative AI models

[1140] Analyze the user's heart rate data and emotional diary to determine whether they are feeling stressed and suggest relaxing music to reduce stress.

[1141] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1142] Step 1:

[1143] User wear of the device and measurement of health indicators

[1144] The user wears smart glasses, which use built-in sensors to measure health indicators such as heart rate and blood pressure in real time. The input data is the user's health indicators, such as heart rate and blood pressure, which are collected by the smart glasses every second. The collected data is then sent to a server via Bluetooth or WiFi.

[1145] Step 2:

[1146] Sending and saving data to the server

[1147] The server receives the data sent from the smart glasses. The input data is the health index data sent from the smart glasses. The server receives this data and first checks the integrity of the data. Then, it stores it in the database. The output is the health index data whose integrity has been confirmed, which is then stored in the database.

[1148] Step 3:

[1149] Regular analysis of health data

[1150] The server periodically analyzes the health index data stored in the database. The input data is the past and current health index data stored in the database. The server uses a data analysis algorithm to evaluate the user's health status. Specifically, it analyzes fluctuations in heart rate and blood pressure to detect signs of stress and fatigue. The output is an evaluation result of the user's health status.

[1151] Step 4:

[1152] Evaluating emotional state and generating reminder notifications

[1153] The server evaluates the user's emotional state based on the analysis results. The input data is the analysis results of health data. For example, if the heart rate is continuously high, it is determined that the emotional state is declining. The server generates a reminder notification such as "Your emotional state is declining. Please take a break." The output is the generated reminder notification.

[1154] Step 5:

[1155] Sending reminders

[1156] The server sends the generated reminder notification to the user's device. The input data is the reminder notification generated by the server. The server sends the notification and the device receives it. The output is the reminder notification received by the user's device and displayed on the device screen.

[1157] Step 6:

[1158] Selecting relaxing music and sending playback instructions

[1159] The server selects relaxing music based on the reminder notification. The input data is the user's current health and emotional state, and the user's music preferences. The server uses a generative AI model to select the most suitable relaxing music for the user. It sends a play instruction for the selected music to the device. The output is the music play instruction sent to the device.

[1160] Step 7:

[1161] Playing relaxing music on your device

[1162] The terminal plays relaxing music based on the music playback instruction received from the server. The input data is the music playback instruction sent from the server. The terminal receives the instruction and plays the specified relaxing music to the user. The output is music that the user can listen to and feel relaxed.

[1163] Step 8:

[1164] Connecting to social support groups

[1165] When a user wants to connect to a social support group, the server selects an appropriate group based on the user's profile information. The input data is the user's profile information and a database of groups. The server selects an appropriate group and sends the information to the user's device. The output is the group information to connect to.

[1166] Step 9:

[1167] Record and send your emotional diary

[1168] Users record their emotion diary using smart glasses or a smartphone app. The input data is the diary of emotions recorded by the user. The recorded diary data is sent to a server and stored in a database. The output is the emotion diary data sent to and stored on the server.

[1169] Step 10:

[1170] Emotion diary analysis and feedback

[1171] The server analyzes the emotion diary data and provides feedback and suggestions. The input data is the emotion diary data stored in the database. The server analyzes it and generates feedback such as, "You seem to have been feeling tired lately. I recommend you take a rest." The output is the feedback and suggestions sent to the user.

[1172] Above are the specific processing steps and detailed explanation of this system, which allows users to comprehensively manage their physical and mental health.

[1173] (Application example 1)

[1174] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1175] In factory work environments, managing employee stress and health is important, but conventional methods make it difficult to grasp the situation of individual employees in real time and implement appropriate measures. Also, when employees need psychological support, there are limited ways to connect them to appropriate social support groups. To solve these issues, real-time analysis of data and automatic implementation of appropriate measures are required.

[1176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1177] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for measuring the stress level and heart rate of factory workers and sending a health notification and playing relaxing music based on the analysis results, means for connecting the factory workers to an appropriate social support group, and means for sending notifications and playing music via a robot. This enables real-time management of the health and emotional states of employees in a factory work environment, and makes it possible to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[1178] "User" refers to an individual who uses this system.

[1179] "Physical health indicators" are data that indicate the user's physical condition, such as heart rate and blood pressure.

[1180] "Real-time measurement means" refers to technology that uses smart glasses or wearable devices to instantly collect health indicators.

[1181] The "means for storing in a database" refers to a technique for recording and storing the measured health indicators in a database.

[1182] "Means for assessing health status" refers to technology that analyzes stored data and determines the user's physical and mental status.

[1183] "Means for sending reminder notifications" refers to technology that sends notifications to users to warn them or suggest they take a break.

[1184] The "means for playing relaxing music as an environmental adjustment means" is a technology for selecting and playing appropriate relaxing music to improve the user's emotional state.

[1185] "Means for connecting users to social support groups" refers to technology that connects users to appropriate support groups when desired.

[1186] "Means for recording and analyzing emotional diaries" refers to a technology that allows users to record their own emotional state and then analyze that data.

[1187] "Means for measuring and analyzing stress levels and heart rates" refers to technology that measures and analyzes health indicators such as the heart rate of factory workers in real time.

[1188] The "means for sending health notifications and playing relaxing music" is a technology that sends notifications and plays relaxing music when stress or overwork is detected.

[1189] "Means for sending notifications and playing music via a robot" refers to a technology in which a robot sends reminder notifications to the user and plays relaxing music.

[1190] System Overview

[1191] The system of the present invention is designed to manage the physical and mental health of users and support stress management in factory work environments. Users wear smart glasses or wearable devices to measure health indicators such as heart rate in real time. The measured data is sent to a server, where it is recorded and analyzed in a database to evaluate the user's health. Monitoring the health of employees is particularly important in factory environments where workers are under heavy stress.

[1192] Hardware and software used

[1193] The system is implemented using the following hardware and software.

[1194] Wearable devices: Smart glasses and smart watches are used to measure data such as heart rate and stress levels.

[1195] Server: Stores the received health data in a database and analyzes it. Implemented in a programming language such as Python.

[1196] Database: A database system such as MySQL or PostgreSQL will be used to record and store health index data.

[1197] Robot: Use a robot to send reminder notifications to users and play relaxing music.

[1198] Data collection and analysis process

[1199] 1. Data collection: Users wear smart glasses or smartwatches, which measure their heart rate and stress levels in real time. The data is then sent to a server via Bluetooth or Wi-Fi.

[1200] 2. Data storage: The received data is stored on the server and recorded in a database.

[1201] 3. Data analysis: The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate is higher than normal, it determines that the user is feeling stressed.

[1202] 4. Reminder notification: If the user's emotional state is determined to be declining, the server generates a reminder notification and sends a message to the user encouraging them to take a break.

[1203] 5. Relaxing music playback: The server sends a reminder notification and a command to play relaxing music to the robot, allowing the user to relax while listening to the music.

[1204] Social support function

[1205] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[1206] Specific examples

[1207] As a specific example of operation, consider the case where a factory worker feels fatigued or stressed. If the factory worker wears a smartwatch and their heart rate exceeds the normal level, the server will send a notification via a robot saying, "High stress level detected. Please take a break." In addition, relaxing music will be played automatically.

[1208] Prompt Sentence Examples

[1209] "Write a program that notifies you and plays relaxing music if it determines that a factory worker is stressed."

[1210] In this way, this invention enables real-time management of employee health and emotional states in a factory environment, and is expected to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[1211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1212] Program processing steps

[1213] Step 1: Data collection

[1214] Health index data such as heart rate and stress level is collected in real time while the user is wearing the wearable device. Data is sent from the device to a server via Bluetooth or WiFi. Input: Measurement data of heart rate and stress level. Output: Health index data sent to the server.

[1215] Step 2: Save data

[1216] The server stores the received data in a database. The stored data is used for later analysis. Input: Health index data received from the device. Output: Stored data recorded in the database.

[1217] Step 3: Data analysis

[1218] The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate exceeds normal values, it determines that the user is feeling stressed. Data analysis is performed using Python and other tools. Input: Health index data in the database. Output: Analysis results (user's stress and health state).

[1219] Step 4: Generate reminder notifications

[1220] If the data analysis determines that the user's emotional state is declining, the server generates a reminder notification. The notification may contain a text message such as "High stress detected. Please take a break." Input: Analysis results. Output: Reminder notification text message.

[1221] Step 5: Select and play relaxation music

[1222] The server selects appropriate relaxation music based on the user's emotional state and sends playback instructions to the robot. For example, it may select forest sounds or the sound of waves. The robot plays the music based on the instructions. Input: Reminder notification and emotional state data. Output: Relaxation music to be played.

[1223] Step 6: Social support connections

[1224] When a user requests a social support connection, the server selects an appropriate social support group based on the user's profile information and establishes a connection. Input: User's profile information and connection request. Output: Selected social support group information.

[1225] Specific actions

[1226] For example, when a factory worker wears a smartwatch, their heart rate is measured in real time. If their heart rate exceeds 110 and their stress level is high at 8, the server generates a notification saying "High stress detected. Please take a break" and sends it to the worker via a robot. The robot also automatically plays relaxation music.

[1227] Example prompt sentence:

[1228] "Write a program that notifies you and plays relaxation music if it determines that a factory worker is stressed."

[1229] In this way, the processing at each step is carried out specifically, and real-time management of the user's health condition is realized.

[1230] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1231] The present invention is a system for supporting a user's mental and physical health, and in particular, aims to incorporate an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine also recognizes the user's emotions and provides accurate feedback based on those emotions.

[1232] Overall system configuration

[1233] health measurement

[1234] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[1235] Data analysis

[1236] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[1237] Use of emotion engine

[1238] The emotion engine analyzes the user's facial expressions, voice, input text data, etc. to recognize the user's emotional state. The emotion data obtained by the emotion engine is sent to the server and used as part of the analysis.

[1239] Reminders

[1240] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[1241] Play relaxing music

[1242] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[1243] Connecting to social support groups

[1244] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[1245] Emotional diary recording

[1246] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[1247] Specific examples

[1248] Specific examples of health measurements

[1249] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[1250] Examples of emotion engines

[1251] When a user speaks using the smartphone app, the voice data is analyzed by the emotion engine. The emotion engine recognizes the user's emotional state from the tone of the voice and the way they speak, and determines that they are in a "stressed state." As a result, the server sends appropriate instructions, such as sending a reminder notification or playing relaxing music.

[1252] Examples of reminder notifications

[1253] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[1254] Example of relaxing music playback

[1255] When the emotion engine recognizes that the user is feeling stressed, the server sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music that includes the sounds of a quiet forest or waves can help the user relax.

[1256] Examples of connecting with social support groups

[1257] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[1258] Examples of emotional diary entries

[1259] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[1260] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate mental support as needed. In addition, the emotion engine helps recognize emotional states and provides more accurate feedback, thereby more effectively supporting the user's mental health.

[1261] The processing flow will be explained below.

[1262] health measurement

[1263] Program processing steps

[1264] Step 1:

[1265] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[1266] Step 2:

[1267] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[1268] Step 3:

[1269] The device sends the measurement data to the server, which then sends the data to the server via API.

[1270] Step 4:

[1271] The server analyzes the received data and stores it in a real-time database.

[1272] Data analysis

[1273] Program processing steps

[1274] Step 1:

[1275] The server periodically queries the database for the latest heart rate data.

[1276] Step 2:

[1277] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[1278] Step 3:

[1279] The server stores the analysis results in a database and prepares notifications as needed.

[1280] Use of emotion engine

[1281] Program processing steps

[1282] Step 1:

[1283] The user starts inputting emotions through a smartphone app or smart glasses, and the user's facial, voice, and text data are sent to the emotion engine.

[1284] Step 2:

[1285] The emotion engine analyzes the received data and recognizes the user's emotional state. The recognized emotion data is sent to the server.

[1286] Step 3:

[1287] The server uses the data from the emotion engine as part of its analysis to assess the user's current emotional state.

[1288] Reminders

[1289] Program processing steps

[1290] Step 1:

[1291] The server monitors the user's health status data and the analysis results of the emotion engine to detect a decline in the emotional state.

[1292] Step 2:

[1293] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[1294] Step 3:

[1295] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[1296] Play relaxing music

[1297] Program processing steps

[1298] Step 1:

[1299] The server analyzes the user's emotional state and determines that relaxing music is required.

[1300] Step 2:

[1301] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[1302] Step 3:

[1303] The server sends a music playback command to the device, including the music file information and playback command.

[1304] Step 4:

[1305] The device receives the instruction and plays the selected relaxing music.

[1306] Connecting to social support groups

[1307] Program processing steps

[1308] Step 1:

[1309] A user sends a request from the device to connect to a social support group.

[1310] Step 2:

[1311] The device sends the user's request to the server, which also includes the user's profile information.

[1312] Step 3:

[1313] The server selects appropriate social support groups based on the user's profile information.

[1314] Step 4:

[1315] The server transmits information about the selected social support group to the terminal.

[1316] Step 5:

[1317] The terminal displays the received group information and connects the user to the group.

[1318] Emotional diary recording

[1319] Program processing steps

[1320] Step 1:

[1321] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[1322] Step 2:

[1323] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[1324] Step 3:

[1325] The server stores the received data in a database for later analysis.

[1326] Step 4:

[1327] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[1328] Step 5:

[1329] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[1330] Example 2

[1331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1332] In today's world, lifestyle habits and work stress can lead to a decline in users' mental and physical health. These health problems are particularly difficult to recognize, making it challenging to address them at the appropriate time. It is also difficult to provide users with appropriate feedback regarding a decline in mental health. Conventional systems have struggled to monitor a user's health and emotional state in real time and provide appropriate support. To address these issues, it is necessary to monitor a user's health and emotional state in real time and take appropriate measures.

[1333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1334] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health condition, means for analyzing the user's facial expressions, voice, and text data to recognize the user's emotional state, means for storing the recognized emotional state in a database, means for sending a reminder notification when the user's emotional state deteriorates, means for selecting and playing relaxing music together with the reminder notification, means for connecting the user to a social support group, and means for recording and analyzing the user's emotional diary, thereby making it possible to monitor the user's health condition and emotional state in real time and provide appropriate feedback and support.

[1335] A "user" is an individual or group that uses the system and provides data on health indicators and emotional states.

[1336] "Physical health indicators" are measurement data that indicate an individual's physical health status, such as heart rate, body temperature, blood pressure, and respiratory rate.

[1337] "Real-time" refers to the state in which data is processed immediately from the moment it is generated, without any delay.

[1338] The "database" is an electronic record system for systematically organizing and storing data on users' health indicators and emotional states.

[1339] "Analysis" is the process of extracting information from collected data using statistical methods and algorithms, and then evaluating and judging it.

[1340] "Emotional state" is information that indicates the user's psychological state and mood, and is obtained from facial expressions, voice, and text data.

[1341] A "reminder notification" is a notification that includes a message or advice that urges the user to take a break or take measures.

[1342] "Relaxing music" is music selected for the purpose of relieving the user's mental tension and has the role of promoting relaxation.

[1343] A "social support group" is an online or offline group that users can join to receive emotional support and communication from other members.

[1344] An "emotion diary" is an electronic diary that allows users to record their emotions, moods, physical conditions, etc.

[1345] The present invention provides a system for supporting a user's mental and physical health, particularly by incorporating an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine is also used to recognize the user's emotions and provide accurate feedback based on those emotions.

[1346] The components of this system are:

[1347] 1. A means of measuring the user's physical health indicators:

[1348] When a user wears smart glasses, health indicators such as heart rate are measured in real time. A specific example is a heart rate sensor built into smart glasses such as Google Glass.

[1349] 2. Means of storing health indicator data in the database:

[1350] The smart glasses transmit the measured heart rate data to a server, which stores the data in a database (e.g., MySQL). The data is transmitted via Wi-Fi or Bluetooth and recorded in real time.

[1351] 3. Means of analyzing health indicator data:

[1352] The server periodically analyzes the received health index data and evaluates the user's health condition. If an abnormal value (e.g., a higher heart rate than normal) is detected at this stage, an analysis result is generated indicating that the user may be experiencing stress.

[1353] 4. Means of recognizing emotional states:

[1354] The user inputs voice and facial expression data using a smartphone app. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes this data and recognizes the user's emotional state. The analysis results are sent to a server and stored in a database.

[1355] 5. Send reminder notifications by:

[1356] If the server determines that the user's emotional state is declining, it generates and sends a reminder notification to the device, which may include a message such as "Your emotional state is declining. Please take a break."

[1357] 6. How to play relaxing music:

[1358] The server selects relaxing music along with the reminder notification. The selection process refers to the analysis results of the emotion engine and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[1359] 7. Ways to connect with social support groups:

[1360] When a user requests to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[1361] 8. How to record and analyze your emotional diary:

[1362] Users use a smartphone app to record their emotions in a diary. Records such as "I'm tired today" are sent to a server and stored in a database. The server analyzes this data and provides feedback to the user suggesting they take a rest if fatigue persists.

[1363] Specific actions

[1364] Specific examples of health measurements

[1365] While the user is wearing Google Glass, heart rate data is measured every second and sent to a server, which receives the data and records it in a database. If the data shows abnormal values, the user's stress level is evaluated.

[1366] Examples of emotion engines

[1367] When a user inputs a voice message using a smartphone app, the emotion engine analyzes the voice data and determines that the user is in a "stressed state." This analysis result is sent to the server, which then sends a reminder notification to the device along with an instruction to play relaxing music.

[1368] Prompt Sentence Examples

[1369] For example, by inputting to the generative AI model, "Please tell me the code to generate a specific music list for the user to relax and send the playback instructions to the server," an appropriate program can be generated.

[1370] This system allows users to manage their health in real time and receive psychological support when necessary. The emotional engine enables the system to accurately recognize the user's emotional state and provide effective feedback.

[1371] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1372] Step 1:

[1373] Users wear smart glasses to measure their health data.

[1374] The user wears the smart glasses. The smart glasses have a built-in heart rate sensor that measures heart rate data every second. The measured data is not stored directly, but is sent to the server in the next step.

[1375] Input: Smart glasses worn by the user, user's heart rate

[1376] Output: Heart rate data measured every second

[1377] Step 2:

[1378] The smart glasses send the data to the server.

[1379] The smart glasses transmit the measured heart rate data to a server in real time via Wi-Fi or Bluetooth, allowing the data to be analyzed immediately.

[1380] Input: Heart rate data measured every second

[1381] Output: Heart rate data sent to the server

[1382] Step 3:

[1383] The server stores health data in a database and periodically analyzes it.

[1384] The server stores the received heart rate data in a database. The data stored in the database (e.g., MySQL) is periodically analyzed using an analysis algorithm (e.g., an anomaly detection algorithm). If an abnormal value is detected, the user's health condition is evaluated.

[1385] Input: Heart rate data sent to the server

[1386] Output: Data stored in a database, analysis results (health status is evaluated)

[1387] Step 4:

[1388] The user generates emotion data using the emotion engine.

[1389] Users use a smartphone app to input voice messages or text data. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes the input voice or text data and recognizes the user's emotional state. The results are sent to the server.

[1390] Input: Voice messages and text data

[1391] Output: Parsed emotional state data

[1392] Step 5:

[1393] The server analyzes the emotion data and generates reminder notifications as needed.

[1394] The server analyzes the emotion data sent from the emotion engine. If the user's emotional state is recognized as "stressed," it generates a reminder notification and sends it to the device. The reminder notification may include a message such as "Your emotional state is declining. Please take a break."

[1395] Input: Parsed emotional state data

[1396] Output: Reminder notification

[1397] Step 6:

[1398] The server selects relaxing music along with the reminder notification and sends it to the device.

[1399] The server selects relaxing music based on the user's current emotional state and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[1400] Input: Reminders, emotional state data, historical data

[1401] Output: Relaxing music selection, music playback instructions

[1402] Step 7:

[1403] When a user wants to connect to a social support group, the server provides the information and helps them connect.

[1404] When a user requests to join a social support group through a smartphone app, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[1405] Input: User profile information, desired connection information

[1406] Output: Information on suitable social support groups

[1407] Step 8:

[1408] Users record their emotions in a diary, and the server analyzes the data and provides feedback.

[1409] Users use a smartphone app to record their emotions in a diary. The recorded data (e.g., "I feel tired today") is sent to a server and stored in a database. The server analyzes this data and analyzes the user's emotional patterns. If fatigue persists, the server provides feedback to the user suggesting that they take a rest.

[1410] Input: Emotion diary data

[1411] Output: Feedback notification based on analysis results

[1412] (Application example 2)

[1413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] Conventional technologies for supporting users' mental and physical health have struggled to accurately recognize the user's emotional state, preventing them from providing appropriate support. Furthermore, providing services in physical stores has also been problematic, as it has been difficult to provide individualized support based on the user's current health and emotional state, making it difficult to improve user satisfaction. Therefore, there is a need for the development of a system that can more accurately recognize the user's emotional state and provide appropriate feedback and support in real time.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1416] In this invention, the server includes means for measuring a user's physical health index in real time, means for storing the measured health index in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for providing appropriate services in a physical store based on the user's physical health index and emotional state, means for suggesting products according to the user's health state and emotional state, and means for playing relaxing music in a specific area when the user's stress state is determined to be high. This allows the user to manage their health in real time and receive appropriate psychological support and personalized services in a physical store as needed.

[1417] "User's physical health indicators" refers to physiological data such as the user's heart rate, steps, blood pressure, and body temperature.

[1418] "Means of measuring in real time" refers to technology that continuously acquires physical health indicators from the user's body and instantly converts them into data.

[1419] "Means for storing data in a database" refers to a system for systematically recording measured data and storing it in a format that can be managed and searched.

[1420] "Means for analyzing and assessing the user's health status" refers to algorithms and software that objectively assess the user's health status based on collected health indicator data.

[1421] "Means for sending reminder notifications" refers to a system that sends messages to users to encourage them to take a break or suggest ways to reduce stress.

[1422] "Means for playing relaxing music" refers to a function that plays music that relieves stress based on the user's emotional state.

[1423] "Means for connecting users to social support groups" refers to technology that allows users to access appropriate communities and groups to receive emotional support.

[1424] "A means for recording and analyzing a user's emotional diary" refers to a system that allows users to input the emotions they feel on a daily basis, and then records and analyzes that data.

[1425] "Means for providing appropriate services in physical stores" refers to a system for providing services in a physical store environment that take into account the user's current health and emotional state.

[1426] "Means for suggesting products based on health and emotional state" refers to technology that suggests optimal products and services based on the user's health and emotional data.

[1427] "Means for playing relaxing music in specific areas" refers to the function of playing music in specific areas within the store to help users relax.

[1428] The present invention is a system for supporting the mental and physical health of a user, and in particular incorporates an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. This system is realized mainly using the following hardware and software.

[1429] Hardware and software used:

[1430] Smart glasses (e.g., Google Glass, Vuzix Blade): measure the user's physical health indicators (heart rate, steps, etc.) in real time.

[1431] Smartphone / tablet (e.g. iPad, Android tablet): Displays and notifies the user's emotional state and health data.

[1432] Server (e.g. AWS EC2, Google Cloud VM): Analyzes and stores data, and generates notifications.

[1433] Emotion engine (e.g., Microsoft Azure Emotion API, IBM Watson Tone Analyzer): Evaluates the user's emotional state.

[1434] System features:

[1435] The system is equipped with the following functions, and analyzes the user's health and emotional state in real time and provides appropriate support based on that.

[1436] 1. Real-time health monitoring:

[1437] The smart glasses measure the user's heart rate, number of steps, and other data in real time and send it to a server, where it stores the data in a database and analyzes it, allowing the user's health status to be tracked.

[1438] 2. Recognition of emotional states:

[1439] The emotion engine analyzes the user's facial expressions and voice data to assess their emotional state. For example, it analyzes what the user says and their facial expressions using a smartphone app to determine their stress level. The data obtained by the emotion engine is also sent to the server and used as part of the analysis.

[1440] 3. Reminder Notification:

[1441] If the user's emotional state is determined to be declining, the server generates a reminder notification and sends it to the user's smartphone or tablet, which may include a message to take a break or specific stress reduction advice.

[1442] 4. Play relaxing music:

[1443] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends playback instructions to the smart glasses or smartphone, allowing the user to relax by listening to music such as the sound of a tranquil forest or the sound of waves.

[1444] 5. Connect with social support groups:

[1445] If a user wishes to join a social support group, the server will recommend and connect them to an appropriate group based on their profile information, allowing them to communicate with other members in real time and receive emotional support.

[1446] 6. Personalized services in-store:

[1447] When a user is in a physical store, real-time information is provided to store staff based on the user's health indicators and emotional state. For example, a tense user can be provided with a relaxing environment, and a stressed user can be suggested products that will have a relaxing effect.

[1448] Specific use cases:

[1449] 1. Health measurement and emotional state recognition:

[1450] When a user wears smart glasses and goes shopping, their heart rate is measured to be high, and the emotion engine detects a state of stress. In this case, the server issues a command to "play relaxing music," and nature music is played on the user's smartphone.

[1451] 2. Reminder Notification:

[1452] If the server analyzes health data and emotional state to be declining, a reminder notification such as "Take a break" will be sent to the user's device.

[1453] 3. Social support group connections:

[1454] When a user types "I need support" into the smartphone app, the server recommends appropriate social support groups and connects the user to them.

[1455] Example prompt sentence:

[1456] "Implement a system that recognizes customers' stress levels and suggests products that will help them relax. Use smart glasses and an emotion recognition engine to notify staff when a certain threshold is exceeded."

[1457] This allows users to receive appropriate mental and physical support in real time, making the shopping experience in physical stores more comfortable.

[1458] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1459] Step 1:

[1460] When a user wears smart glasses, physical health indicators (heart rate, number of steps, etc.) are measured in real time. The smart glasses instantly convert this data into digital data and send it to a server. The input data is health information such as heart rate and number of steps, and the output is the measurement data sent to the server. The smart glasses have built-in heart rate monitors and acceleration sensors, which are used to acquire data in real time.

[1461] Step 2:

[1462] The server receives the health index data sent from the smart glasses and stores it in a database. At this time, the server organizes the data so that it can be easily searched and analyzed later. The input data is the measurement data sent from the smart glasses, and the output is the health data stored in the database. The server receives the data using a data transfer protocol (e.g., HTTP, HTTPS) and stores it in a database (e.g., MySQL, PostgreSQL).

[1463] Step 3:

[1464] The server analyzes the stored data and evaluates the user's health condition. This analysis uses an algorithm that, for example, determines that an abnormally high heart rate indicates stress. The input data is the health data stored in the database, and the output is the evaluation result of the user's health condition. The server analyzes the data using a programming language such as Python and a data analysis library (e.g., Pandas, NumPy).

[1465] Step 4:

[1466] The emotion engine analyzes the user's facial expression and voice data to evaluate their emotional state. The emotion engine analyzes facial expression and voice data input via a smartphone or tablet. The input data is the user's facial expression and voice information, and the output is an evaluation of their emotional state. The emotion engine uses the Microsoft Azure Emotion API, IBM Watson Tone Analyzer, etc. to analyze facial expressions and voice emotions.

[1467] Step 5:

[1468] If the server determines that the user's health and emotional state is declining, it generates a reminder notification. The reminder notification includes a message encouraging the user to take a break and specific advice for reducing stress. The input data is the assessment result of the user's health and emotional state, and the output is the reminder notification message. The server creates the message using a notification generation algorithm and sends the notification using a notification service (e.g., Firebase Cloud Messaging).

[1469] Step 6:

[1470] The server sends a reminder notification and an instruction to play relaxing music to the smart glasses or smartphone. The relaxing music may include, for example, the sound of a quiet forest or the sound of waves. The input data is the instruction to play relaxing music, and the output is the relaxing music played from the smart glasses or smartphone. The server uses a music selection algorithm and sends the play instruction to the device.

[1471] Step 7:

[1472] When a user wishes to join a social support group, the server recommends and connects them to appropriate groups based on the user's profile information. The input data is the user's profile information, and the output is information about recommended social support groups. The server uses a recommendation algorithm and connects to communication platforms (e.g., Slack, Discord).

[1473] Step 8:

[1474] When a user is in a physical store, the server provides the user's health indicators and emotional state to store staff in real time. The staff obtains this information via tablets or smart devices and provides individualized support. The input data is the user's health indicators and emotional state, and the output is the information displayed to the staff. The server uses a data transfer protocol to send the data to the staff application.

[1475] Step 9:

[1476] In a physical store, the server makes product suggestions based on the user's health and emotional state. For example, a user experiencing high stress may be recommended a product with a relaxing effect. The input data is the user's health and emotional state, and the output is personalized product suggestions. The server uses a recommendation engine to generate the suggestions.

[1477] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1479] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1480] [Fourth embodiment]

[1481] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1482] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1484] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1488] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1489] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1490] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1491] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1492] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1493] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1494] The present invention is a system for supporting a user's mental and physical health. The system measures a user's physical health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group.

[1495] Overall system configuration

[1496] health measurement

[1497] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[1498] Data analysis

[1499] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[1500] Reminders

[1501] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[1502] Play relaxing music

[1503] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[1504] Connecting to social support groups

[1505] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[1506] Emotional diary recording

[1507] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[1508] Specific examples

[1509] Specific examples of health measurements

[1510] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[1511] Examples of reminder notifications

[1512] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[1513] Example of relaxing music playback

[1514] If the server determines that the user is feeling stressed, it sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music including the sounds of a quiet forest or waves can be played to help the user relax.

[1515] Examples of connecting with social support groups

[1516] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[1517] Examples of emotional diary entries

[1518] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[1519] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate psychological support as needed.

[1520] The processing flow will be explained below.

[1521] health measurement

[1522] Program processing steps

[1523] Step 1:

[1524] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[1525] Step 2:

[1526] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[1527] Step 3:

[1528] The device sends the measurement data to the server, which then sends the data to the server via API.

[1529] Step 4:

[1530] The server analyzes the received data and stores it in a real-time database.

[1531] Data analysis

[1532] Program processing steps

[1533] Step 1:

[1534] The server periodically queries the database for the latest heart rate data.

[1535] Step 2:

[1536] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[1537] Step 3:

[1538] The server stores the analysis results in a database and prepares notifications as needed.

[1539] Reminders

[1540] Program processing steps

[1541] Step 1:

[1542] The server monitors the user's health data and detects a decline in emotional state.

[1543] Step 2:

[1544] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[1545] Step 3:

[1546] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[1547] Play relaxing music

[1548] Program processing steps

[1549] Step 1:

[1550] The server analyzes the user's emotional state and determines that relaxing music is required.

[1551] Step 2:

[1552] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[1553] Step 3:

[1554] The server sends a music playback command to the device, including the music file information and playback command.

[1555] Step 4:

[1556] The device receives the instruction and plays the selected relaxing music.

[1557] Connecting to social support groups

[1558] Program processing steps

[1559] Step 1:

[1560] A user sends a request from the device to connect to a social support group.

[1561] Step 2:

[1562] The device sends the user's request to the server, which also includes the user's profile information.

[1563] Step 3:

[1564] The server selects appropriate social support groups based on the user's profile information.

[1565] Step 4:

[1566] The server transmits information about the selected social support group to the terminal.

[1567] Step 5:

[1568] The terminal displays the received group information and connects the user to the group.

[1569] Emotional diary recording

[1570] Program processing steps

[1571] Step 1:

[1572] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[1573] Step 2:

[1574] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[1575] Step 3:

[1576] The server stores the received data in a database for later analysis.

[1577] Step 4:

[1578] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[1579] Step 5:

[1580] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[1581] Example 1

[1582] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1583] In modern society, many people live with stress and fatigue. Conventional health management systems are primarily limited to measuring physical health indicators and lack the means to properly monitor changes in users' mental state and emotions. As a result, it is difficult for users to accurately understand their own mental health status and take prompt action. In addition, there are limited ways to provide relaxation and mental support when needed, making comprehensive health management difficult.

[1584] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1585] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for analyzing the user's emotional patterns based on the data recorded by the user in the emotional diary and providing feedback and suggestions, and means for analyzing the user's health data using a generative AI model and providing the user with optimal reminders and relaxing music, thereby enabling the user to comprehensively manage their health from both physical and mental perspectives.

[1586] "User physical health indicators" are data that indicate the user's physical condition or health in real time, such as heart rate, blood pressure, and body temperature.

[1587] "Real-time measurement means" refers to a mechanism that obtains a user's health indicators continuously or periodically and collects data instantly.

[1588] The "database" is a system that systematically stores and manages received data such as health indicators and emotional diaries.

[1589] "Means for storing data in a database" refers to the methods and technologies for efficiently storing collected data in a database.

[1590] "Means for analyzing and assessing the user's health status" means algorithms or methods for using the collected data to assess the user's physical and mental health status.

[1591] The "means for sending reminder notifications" is a mechanism that generates messages encouraging breaks or attention based on the user's health and emotional state and sends them to the user's device.

[1592] "Environmental adjustment means" is a mechanism that provides an appropriate music and audio environment for the purpose of user relaxation and stress relief.

[1593] "Means for playing relaxing music" refers to a system that allows users to select music that helps them relax and play it on their devices.

[1594] "Means of connecting users to social support groups" means mechanisms that allow users to join appropriate groups or communities to receive emotional support.

[1595] "User's Emotion Diary" is data that allows users to record their daily emotions and moods.

[1596] The "means for recording and analyzing an emotional diary" is a technology for saving an emotional diary entered by a user and analyzing the data to understand the user's emotional patterns.

[1597] A "generative AI model" is an artificial intelligence algorithm used to analyze collected data and provide optimal feedback and suggestions to users.

[1598] The present invention is a system for supporting a user's physical and mental health. The system measures a user's health indicators in real time, analyzes the data, and evaluates the user's health and emotional state. The system can then provide the user with reminders, relaxing music, and even connect them to social support groups.

[1599] Overall system configuration

[1600] health measurement

[1601] While the user is wearing the smart glasses, health indicators such as heart rate and blood pressure are measured in real time using the smart glasses' built-in sensors, and the data is sent to a server via Bluetooth or Wi-Fi.

[1602] Data analysis

[1603] The server stores the received health index data in a database and periodically evaluates the user's health status using a data analysis algorithm. For example, if the user's heart rate is higher than normal, the server determines that the user is feeling stressed.

[1604] Reminders

[1605] If the server determines that the user's emotional state is declining, it generates a reminder notification to encourage them to take a break and sends it to the user's device. Specifically, it sends a message saying, "Your emotional state is declining. Please take a break."

[1606] Play relaxing music

[1607] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends a command to play it to the device. The device then follows the command and plays music to help the user relax. For example, relaxing music such as the sounds of a quiet forest or the sound of waves can be played.

[1608] Connecting to social support groups

[1609] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information and sends the information to the device, where the user can join and communicate with other members in real time.

[1610] Emotional diary recording

[1611] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[1612] Using generative AI models

[1613] The server uses a generative AI model to analyze the received health index data and emotional diary data. This analysis allows it to provide optimal reminder notifications and relaxing music to the user. Specifically, it analyzes the user's stress level and suggests optimal measures accordingly.

[1614] Specific examples

[1615] Specific examples of health measurements

[1616] While the user is wearing the smart glasses, their heart rate is measured every second and sent to a server. The data is then recorded in a database and compared to their normal heart rate. For example, if their heart rate is higher than normal, the server will determine that the user's stress level is high.

[1617] Examples of reminder notifications

[1618] If the server determines that the user's emotional state is declining based on the user's emotional diary and health data, it generates a reminder notification saying, "Your emotional state is declining. Please take a break." The device receives this notification and displays it to the user.

[1619] Example of relaxing music playback

[1620] If the server determines that the user is feeling stressed, it sends a command to play relaxing music to the device. The device then follows this command and plays relaxing music such as the sound of a quiet forest or the sound of waves, allowing the user to relax.

[1621] Examples of connecting with social support groups

[1622] When a user requests to connect to a support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members in real time.

[1623] Examples of emotional diary entries

[1624] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[1625] Example prompts for generative AI models

[1626] Analyze the user's heart rate data and emotional diary to determine whether they are feeling stressed and suggest relaxing music to reduce stress.

[1627] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1628] Step 1:

[1629] User wear of the device and measurement of health indicators

[1630] The user wears smart glasses, which use built-in sensors to measure health indicators such as heart rate and blood pressure in real time. The input data is the user's health indicators, such as heart rate and blood pressure, which are collected by the smart glasses every second. The collected data is then sent to a server via Bluetooth or WiFi.

[1631] Step 2:

[1632] Sending and saving data to the server

[1633] The server receives the data sent from the smart glasses. The input data is the health index data sent from the smart glasses. The server receives this data and first checks the integrity of the data. Then, it stores it in the database. The output is the health index data whose integrity has been confirmed, which is then stored in the database.

[1634] Step 3:

[1635] Regular analysis of health data

[1636] The server periodically analyzes the health index data stored in the database. The input data is the past and current health index data stored in the database. The server uses a data analysis algorithm to evaluate the user's health status. Specifically, it analyzes fluctuations in heart rate and blood pressure to detect signs of stress and fatigue. The output is an evaluation result of the user's health status.

[1637] Step 4:

[1638] Evaluating emotional state and generating reminder notifications

[1639] The server evaluates the user's emotional state based on the analysis results. The input data is the analysis results of health data. For example, if the heart rate is continuously high, it is determined that the emotional state is declining. The server generates a reminder notification such as "Your emotional state is declining. Please take a break." The output is the generated reminder notification.

[1640] Step 5:

[1641] Sending reminders

[1642] The server sends the generated reminder notification to the user's device. The input data is the reminder notification generated by the server. The server sends the notification and the device receives it. The output is the reminder notification received by the user's device and displayed on the device screen.

[1643] Step 6:

[1644] Selecting relaxing music and sending playback instructions

[1645] The server selects relaxing music based on the reminder notification. The input data is the user's current health and emotional state, and the user's music preferences. The server uses a generative AI model to select the most suitable relaxing music for the user. It sends a play instruction for the selected music to the device. The output is the music play instruction sent to the device.

[1646] Step 7:

[1647] Playing relaxing music on your device

[1648] The terminal plays relaxing music based on the music playback instruction received from the server. The input data is the music playback instruction sent from the server. The terminal receives the instruction and plays the specified relaxing music to the user. The output is music that the user can listen to and feel relaxed.

[1649] Step 8:

[1650] Connecting to social support groups

[1651] When a user wants to connect to a social support group, the server selects an appropriate group based on the user's profile information. The input data is the user's profile information and a database of groups. The server selects an appropriate group and sends the information to the user's device. The output is the group information to connect to.

[1652] Step 9:

[1653] Record and send your emotional diary

[1654] Users record their emotion diary using smart glasses or a smartphone app. The input data is the diary of emotions recorded by the user. The recorded diary data is sent to a server and stored in a database. The output is the emotion diary data sent to and stored on the server.

[1655] Step 10:

[1656] Emotion diary analysis and feedback

[1657] The server analyzes the emotion diary data and provides feedback and suggestions. The input data is the emotion diary data stored in the database. The server analyzes it and generates feedback such as, "You seem to have been feeling tired lately. I recommend you take a rest." The output is the feedback and suggestions sent to the user.

[1658] Above are the specific processing steps and detailed explanation of this system, which allows users to comprehensively manage their physical and mental health.

[1659] (Application example 1)

[1660] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1661] In factory work environments, managing employee stress and health is important, but conventional methods make it difficult to grasp the situation of individual employees in real time and implement appropriate measures. Also, when employees need psychological support, there are limited ways to connect them to appropriate social support groups. To solve these issues, real-time analysis of data and automatic implementation of appropriate measures are required.

[1662] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1663] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for measuring the stress level and heart rate of factory workers and sending a health notification and playing relaxing music based on the analysis results, means for connecting the factory workers to an appropriate social support group, and means for sending notifications and playing music via a robot. This enables real-time management of the health and emotional states of employees in a factory work environment, and makes it possible to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[1664] "User" refers to an individual who uses this system.

[1665] "Physical health indicators" are data that indicate the user's physical condition, such as heart rate and blood pressure.

[1666] "Real-time measurement means" refers to technology that uses smart glasses or wearable devices to instantly collect health indicators.

[1667] The "means for storing in a database" refers to a technique for recording and storing the measured health indicators in a database.

[1668] "Means for assessing health status" refers to technology that analyzes stored data and determines the user's physical and mental status.

[1669] "Means for sending reminder notifications" refers to technology that sends notifications to users to warn them or suggest they take a break.

[1670] The "means for playing relaxing music as an environmental adjustment means" is a technology for selecting and playing appropriate relaxing music to improve the user's emotional state.

[1671] "Means for connecting users to social support groups" refers to technology that connects users to appropriate support groups when desired.

[1672] "Means for recording and analyzing emotional diaries" refers to a technology that allows users to record their own emotional state and then analyze that data.

[1673] "Means for measuring and analyzing stress levels and heart rates" refers to technology that measures and analyzes health indicators such as the heart rate of factory workers in real time.

[1674] The "means for sending health notifications and playing relaxing music" is a technology that sends notifications and plays relaxing music when stress or overwork is detected.

[1675] "Means for sending notifications and playing music via a robot" refers to a technology in which a robot sends reminder notifications to the user and plays relaxing music.

[1676] System Overview

[1677] The system of the present invention is designed to manage the physical and mental health of users and support stress management in factory work environments. Users wear smart glasses or wearable devices to measure health indicators such as heart rate in real time. The measured data is sent to a server, where it is recorded and analyzed in a database to evaluate the user's health. Monitoring the health of employees is particularly important in factory environments where workers are under heavy stress.

[1678] Hardware and software used

[1679] The system is implemented using the following hardware and software.

[1680] Wearable devices: Smart glasses and smart watches are used to measure data such as heart rate and stress levels.

[1681] Server: Stores the received health data in a database and analyzes it. Implemented in a programming language such as Python.

[1682] Database: A database system such as MySQL or PostgreSQL will be used to record and store health index data.

[1683] Robot: Use a robot to send reminder notifications to users and play relaxing music.

[1684] Data collection and analysis process

[1685] 1. Data collection: Users wear smart glasses or smartwatches, which measure their heart rate and stress levels in real time. The data is then sent to a server via Bluetooth or Wi-Fi.

[1686] 2. Data storage: The received data is stored on the server and recorded in a database.

[1687] 3. Data analysis: The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate is higher than normal, it determines that the user is feeling stressed.

[1688] 4. Reminder notification: If the user's emotional state is determined to be declining, the server generates a reminder notification and sends a message to the user encouraging them to take a break.

[1689] 5. Relaxing music playback: The server sends a reminder notification and a command to play relaxing music to the robot, allowing the user to relax while listening to the music.

[1690] Social support function

[1691] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[1692] Specific examples

[1693] As a specific example of operation, consider the case where a factory worker feels fatigued or stressed. If the factory worker wears a smartwatch and their heart rate exceeds the normal level, the server will send a notification via a robot saying, "High stress level detected. Please take a break." In addition, relaxing music will be played automatically.

[1694] Prompt Sentence Examples

[1695] "Write a program that notifies you and plays relaxing music if it determines that a factory worker is stressed."

[1696] In this way, this invention enables real-time management of employee health and emotional states in a factory environment, and is expected to improve employee health and productivity through appropriate reminder notifications, playing relaxing music, and providing social support.

[1697] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1698] Program processing steps

[1699] Step 1: Data collection

[1700] Health index data such as heart rate and stress level is collected in real time while the user is wearing the wearable device. Data is sent from the device to a server via Bluetooth or WiFi. Input: Measurement data of heart rate and stress level. Output: Health index data sent to the server.

[1701] Step 2: Save data

[1702] The server stores the received data in a database. The stored data is used for later analysis. Input: Health index data received from the device. Output: Stored data recorded in the database.

[1703] Step 3: Data analysis

[1704] The server analyzes the stored data and evaluates the user's health and emotional state. For example, if the heart rate exceeds normal values, it determines that the user is feeling stressed. Data analysis is performed using Python and other tools. Input: Health index data in the database. Output: Analysis results (user's stress and health state).

[1705] Step 4: Generate reminder notifications

[1706] If the data analysis determines that the user's emotional state is declining, the server generates a reminder notification. The notification may contain a text message such as "High stress detected. Please take a break." Input: Analysis results. Output: Reminder notification text message.

[1707] Step 5: Select and play relaxation music

[1708] The server selects appropriate relaxation music based on the user's emotional state and sends playback instructions to the robot. For example, it may select forest sounds or the sound of waves. The robot plays the music based on the instructions. Input: Reminder notification and emotional state data. Output: Relaxation music to be played.

[1709] Step 6: Social support connections

[1710] When a user requests a social support connection, the server selects an appropriate social support group based on the user's profile information and establishes a connection. Input: User's profile information and connection request. Output: Selected social support group information.

[1711] Specific actions

[1712] For example, when a factory worker wears a smartwatch, their heart rate is measured in real time. If their heart rate exceeds 110 and their stress level is high at 8, the server generates a notification saying "High stress detected. Please take a break" and sends it to the worker via a robot. The robot also automatically plays relaxation music.

[1713] Example prompt sentence:

[1714] "Write a program that notifies you and plays relaxation music if it determines that a factory worker is stressed."

[1715] In this way, the processing at each step is carried out specifically, and real-time management of the user's health condition is realized.

[1716] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1717] The present invention is a system for supporting a user's mental and physical health, and in particular, aims to incorporate an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine also recognizes the user's emotions and provides accurate feedback based on those emotions.

[1718] Overall system configuration

[1719] health measurement

[1720] While the user is wearing the smart glasses, health indicators such as heart rate are measured in real time, and the measured data is sent from the smart glasses to a server and stored in a database.

[1721] Data analysis

[1722] The server periodically analyzes the received health index data to evaluate the user's health status, and determines whether the user is experiencing stress or fatigue based on the analysis results.

[1723] Use of emotion engine

[1724] The emotion engine analyzes the user's facial expressions, voice, input text data, etc. to recognize the user's emotional state. The emotion data obtained by the emotion engine is sent to the server and used as part of the analysis.

[1725] Reminders

[1726] If the user's emotional state is judged to be declining, the server generates and sends a reminder notification to the device, which may include a message encouraging the user to take a break or specific advice on how to reduce stress.

[1727] Play relaxing music

[1728] Depending on the situation, relaxing music is played along with the reminder notification. The server selects appropriate music based on the user's emotional state and environment and sends a playback command to the device, allowing the user to relax while listening to music.

[1729] Connecting to social support groups

[1730] If the user so desires, the server will provide information about appropriate social support groups and connect the user to those groups, allowing the user to communicate with other members in real time and receive emotional support.

[1731] Emotional diary recording

[1732] Users can record their emotional diary using smart glasses or a smartphone app. This diary is sent to a server and stored in a database. The server uses this data to analyze the user's emotional patterns and provide appropriate feedback and suggestions.

[1733] Specific examples

[1734] Specific examples of health measurements

[1735] When a user wears the smart glasses, their heart rate data is measured every second and sent to a server, which receives the data and records it in a database. For example, if a user's heart rate is higher than normal, the server may determine that the user is experiencing stress.

[1736] Examples of emotion engines

[1737] When a user speaks using the smartphone app, the voice data is analyzed by the emotion engine. The emotion engine recognizes the user's emotional state from the tone of the voice and the way they speak, and determines that they are in a "stressed state." As a result, the server sends appropriate instructions, such as sending a reminder notification or playing relaxing music.

[1738] Examples of reminder notifications

[1739] The server determines that the user's emotional state is declining based on the user's emotional diary and health data. In this case, the server creates a reminder notification saying "Your emotional state is declining. Please take a break" and sends it to the device. The device receives this notification and displays it to the user.

[1740] Example of relaxing music playback

[1741] When the emotion engine recognizes that the user is feeling stressed, the server sends an instruction to the device to play relaxing music. The device follows this instruction and plays music to help the user relax. For example, relaxing music that includes the sounds of a quiet forest or waves can help the user relax.

[1742] Examples of connecting with social support groups

[1743] When a user wishes to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information is sent to the user's device, allowing the user to join the group and communicate with other members.

[1744] Examples of emotional diary entries

[1745] The user uses a smartphone app to record their emotion diary entry, such as "I'm tired today." The data is sent to a server and stored in a database. The server analyzes the data and, if it detects that the user's fatigue continues, it notifies the user with feedback suggesting that they take a rest.

[1746] The above is a specific embodiment of the present invention. This system allows users to manage their health in real time and receive appropriate mental support as needed. In addition, the emotion engine helps recognize emotional states and provides more accurate feedback, thereby more effectively supporting the user's mental health.

[1747] The processing flow will be explained below.

[1748] health measurement

[1749] Program processing steps

[1750] Step 1:

[1751] The device (smart glasses) starts up and starts the user's heart rate measurement module.

[1752] Step 2:

[1753] The device measures the user's heart rate every second, and once the measurement is obtained, it is temporarily stored in the smart glasses' memory.

[1754] Step 3:

[1755] The device sends the measurement data to the server, which then sends the data to the server via API.

[1756] Step 4:

[1757] The server analyzes the received data and stores it in a real-time database.

[1758] Data analysis

[1759] Program processing steps

[1760] Step 1:

[1761] The server periodically queries the database for the latest heart rate data.

[1762] Step 2:

[1763] The server analyzes the acquired data to assess the user's health status and generates an alert if stress or abnormal heart rate is detected.

[1764] Step 3:

[1765] The server stores the analysis results in a database and prepares notifications as needed.

[1766] Use of emotion engine

[1767] Program processing steps

[1768] Step 1:

[1769] The user starts inputting emotions through a smartphone app or smart glasses, and the user's facial, voice, and text data are sent to the emotion engine.

[1770] Step 2:

[1771] The emotion engine analyzes the received data and recognizes the user's emotional state. The recognized emotion data is sent to the server.

[1772] Step 3:

[1773] The server uses the data from the emotion engine as part of its analysis to assess the user's current emotional state.

[1774] Reminders

[1775] Program processing steps

[1776] Step 1:

[1777] The server monitors the user's health status data and the analysis results of the emotion engine to detect a decline in the emotional state.

[1778] Step 2:

[1779] The server generates a reminder notification and sends it to the device, which includes specific advice such as whether to take a break.

[1780] Step 3:

[1781] The device receives the reminder notification and displays it to the user, using a pop-up or audio alert to help the user acknowledge the notification.

[1782] Play relaxing music

[1783] Program processing steps

[1784] Step 1:

[1785] The server analyzes the user's emotional state and determines that relaxing music is required.

[1786] Step 2:

[1787] The server will select appropriate relaxing music (e.g. forest sounds, ocean sounds, etc.).

[1788] Step 3:

[1789] The server sends a music playback command to the device, including the music file information and playback command.

[1790] Step 4:

[1791] The device receives the instruction and plays the selected relaxing music.

[1792] Connecting to social support groups

[1793] Program processing steps

[1794] Step 1:

[1795] A user sends a request from the device to connect to a social support group.

[1796] Step 2:

[1797] The device sends the user's request to the server, which also includes the user's profile information.

[1798] Step 3:

[1799] The server selects appropriate social support groups based on the user's profile information.

[1800] Step 4:

[1801] The server transmits information about the selected social support group to the terminal.

[1802] Step 5:

[1803] The terminal displays the received group information and connects the user to the group.

[1804] Emotional diary recording

[1805] Program processing steps

[1806] Step 1:

[1807] Users record their emotional diary using a smartphone app or the voice input function of the smart glasses.

[1808] Step 2:

[1809] The device sends the recorded emotion data to the server, including a time stamp and the type of emotion.

[1810] Step 3:

[1811] The server stores the received data in a database for later analysis.

[1812] Step 4:

[1813] The server periodically analyzes the user's emotional patterns and generates feedback and suggestions as needed.

[1814] Step 5:

[1815] The server sends the generated feedback and suggestions to the device and notifies the user, who can then review the feedback and take necessary actions.

[1816] Example 2

[1817] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1818] In today's world, lifestyle habits and work stress can lead to a decline in users' mental and physical health. These health problems are particularly difficult to recognize, making it challenging to address them at the appropriate time. It is also difficult to provide users with appropriate feedback regarding a decline in mental health. Conventional systems have struggled to monitor a user's health and emotional state in real time and provide appropriate support. To address these issues, it is necessary to monitor a user's health and emotional state in real time and take appropriate measures.

[1819] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1820] In this invention, the server includes means for measuring a user's physical health indicators in real time, means for storing the measured health indicators in a database, means for analyzing the stored data to evaluate the user's health condition, means for analyzing the user's facial expressions, voice, and text data to recognize the user's emotional state, means for storing the recognized emotional state in a database, means for sending a reminder notification when the user's emotional state deteriorates, means for selecting and playing relaxing music together with the reminder notification, means for connecting the user to a social support group, and means for recording and analyzing the user's emotional diary, thereby making it possible to monitor the user's health condition and emotional state in real time and provide appropriate feedback and support.

[1821] A "user" is an individual or group that uses the system and provides data on health indicators and emotional states.

[1822] "Physical health indicators" are measurement data that indicate an individual's physical health status, such as heart rate, body temperature, blood pressure, and respiratory rate.

[1823] "Real-time" refers to the state in which data is processed immediately from the moment it is generated, without any delay.

[1824] The "database" is an electronic record system for systematically organizing and storing data on users' health indicators and emotional states.

[1825] "Analysis" is the process of extracting information from collected data using statistical methods and algorithms, and then evaluating and judging it.

[1826] "Emotional state" is information that indicates the user's psychological state and mood, and is obtained from facial expressions, voice, and text data.

[1827] A "reminder notification" is a notification that includes a message or advice that urges the user to take a break or take measures.

[1828] "Relaxing music" is music selected for the purpose of relieving the user's mental tension and has the role of promoting relaxation.

[1829] A "social support group" is an online or offline group that users can join to receive emotional support and communication from other members.

[1830] An "emotion diary" is an electronic diary that allows users to record their emotions, moods, physical conditions, etc.

[1831] The present invention provides a system for supporting a user's mental and physical health, particularly by incorporating an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. The system measures the user's physical health indicators in real time and analyzes the data to evaluate the user's health and emotional state. It also provides a reminder notification when the user's emotional state declines, an environmental adjustment function to play relaxing music, and a function to connect the user to a social support group. The emotion engine is also used to recognize the user's emotions and provide accurate feedback based on those emotions.

[1832] The components of this system are:

[1833] 1. A means of measuring the user's physical health indicators:

[1834] When a user wears smart glasses, health indicators such as heart rate are measured in real time. A specific example is a heart rate sensor built into smart glasses such as Google Glass.

[1835] 2. Means of storing health indicator data in the database:

[1836] The smart glasses transmit the measured heart rate data to a server, which stores the data in a database (e.g., MySQL). The data is transmitted via Wi-Fi or Bluetooth and recorded in real time.

[1837] 3. Means of analyzing health indicator data:

[1838] The server periodically analyzes the received health index data and evaluates the user's health condition. If an abnormal value (e.g., a higher heart rate than normal) is detected at this stage, an analysis result is generated indicating that the user may be experiencing stress.

[1839] 4. Means of recognizing emotional states:

[1840] The user inputs voice and facial expression data using a smartphone app. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes this data and recognizes the user's emotional state. The analysis results are sent to a server and stored in a database.

[1841] 5. Send reminder notifications by:

[1842] If the server determines that the user's emotional state is declining, it generates and sends a reminder notification to the device, which may include a message such as "Your emotional state is declining. Please take a break."

[1843] 6. How to play relaxing music:

[1844] The server selects relaxing music along with the reminder notification. The selection process refers to the analysis results of the emotion engine and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[1845] 7. Ways to connect with social support groups:

[1846] When a user requests to join a social support group, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[1847] 8. How to record and analyze your emotional diary:

[1848] Users use a smartphone app to record their emotions in a diary. Records such as "I'm tired today" are sent to a server and stored in a database. The server analyzes this data and provides feedback to the user suggesting they take a rest if fatigue persists.

[1849] Specific actions

[1850] Specific examples of health measurements

[1851] While the user is wearing Google Glass, heart rate data is measured every second and sent to a server, which receives the data and records it in a database. If the data shows abnormal values, the user's stress level is evaluated.

[1852] Examples of emotion engines

[1853] When a user inputs a voice message using a smartphone app, the emotion engine analyzes the voice data and determines that the user is in a "stressed state." This analysis result is sent to the server, which then sends a reminder notification to the device along with an instruction to play relaxing music.

[1854] Prompt Sentence Examples

[1855] For example, by inputting to the generative AI model, "Please tell me the code to generate a specific music list for the user to relax and send the playback instructions to the server," an appropriate program can be generated.

[1856] This system allows users to manage their health in real time and receive psychological support when necessary. The emotional engine enables the system to accurately recognize the user's emotional state and provide effective feedback.

[1857] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1858] Step 1:

[1859] Users wear smart glasses to measure their health data.

[1860] The user wears the smart glasses. The smart glasses have a built-in heart rate sensor that measures heart rate data every second. The measured data is not stored directly, but is sent to the server in the next step.

[1861] Input: Smart glasses worn by the user, user's heart rate

[1862] Output: Heart rate data measured every second

[1863] Step 2:

[1864] The smart glasses send the data to the server.

[1865] The smart glasses transmit the measured heart rate data to a server in real time via Wi-Fi or Bluetooth, allowing the data to be analyzed immediately.

[1866] Input: Heart rate data measured every second

[1867] Output: Heart rate data sent to the server

[1868] Step 3:

[1869] The server stores health data in a database and periodically analyzes it.

[1870] The server stores the received heart rate data in a database. The data stored in the database (e.g., MySQL) is periodically analyzed using an analysis algorithm (e.g., an anomaly detection algorithm). If an abnormal value is detected, the user's health condition is evaluated.

[1871] Input: Heart rate data sent to the server

[1872] Output: Data stored in a database, analysis results (health status is evaluated)

[1873] Step 4:

[1874] The user generates emotion data using the emotion engine.

[1875] Users use a smartphone app to input voice messages or text data. The emotion engine (e.g., Microsoft Azure Emotion API) analyzes the input voice or text data and recognizes the user's emotional state. The results are sent to the server.

[1876] Input: Voice messages and text data

[1877] Output: Parsed emotional state data

[1878] Step 5:

[1879] The server analyzes the emotion data and generates reminder notifications as needed.

[1880] The server analyzes the emotion data sent from the emotion engine. If the user's emotional state is recognized as "stressed," it generates a reminder notification and sends it to the device. The reminder notification may include a message such as "Your emotional state is declining. Please take a break."

[1881] Input: Parsed emotional state data

[1882] Output: Reminder notification

[1883] Step 6:

[1884] The server selects relaxing music along with the reminder notification and sends it to the device.

[1885] The server selects relaxing music based on the user's current emotional state and past data. The selected music (e.g., a loop containing the sounds of a quiet forest or waves) is sent to the device, which then plays it.

[1886] Input: Reminders, emotional state data, historical data

[1887] Output: Relaxing music selection, music playback instructions

[1888] Step 7:

[1889] When a user wants to connect to a social support group, the server provides the information and helps them connect.

[1890] When a user requests to join a social support group through a smartphone app, the server selects an appropriate group based on the user's profile information. The selected group information (e.g., group name, joining method) is sent to the user's device, and the user joins the group based on that information.

[1891] Input: User profile information, desired connection information

[1892] Output: Information on suitable social support groups

[1893] Step 8:

[1894] Users record their emotions in a diary, and the server analyzes the data and provides feedback.

[1895] Users use a smartphone app to record their emotions in a diary. The recorded data (e.g., "I feel tired today") is sent to a server and stored in a database. The server analyzes this data and analyzes the user's emotional patterns. If fatigue persists, the server provides feedback to the user suggesting that they take a rest.

[1896] Input: Emotion diary data

[1897] Output: Feedback notification based on analysis results

[1898] (Application example 2)

[1899] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1900] Conventional technologies for supporting users' mental and physical health have struggled to accurately recognize the user's emotional state, preventing them from providing appropriate support. Furthermore, providing services in physical stores has also been problematic, as it has been difficult to provide individualized support based on the user's current health and emotional state, making it difficult to improve user satisfaction. Therefore, there is a need for the development of a system that can more accurately recognize the user's emotional state and provide appropriate feedback and support in real time.

[1901] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1902] In this invention, the server includes means for measuring a user's physical health index in real time, means for storing the measured health index in a database, means for analyzing the stored data to evaluate the user's health state, means for sending a reminder notification when the user's emotional state declines, means for playing relaxing music as an environmental adjustment means, means for connecting the user to a social support group, means for recording and analyzing the user's emotional diary, means for providing appropriate services in a physical store based on the user's physical health index and emotional state, means for suggesting products according to the user's health state and emotional state, and means for playing relaxing music in a specific area when the user's stress state is determined to be high. This allows the user to manage their health in real time and receive appropriate psychological support and personalized services in a physical store as needed.

[1903] "User's physical health indicators" refers to physiological data such as the user's heart rate, steps, blood pressure, and body temperature.

[1904] "Means of measuring in real time" refers to technology that continuously acquires physical health indicators from the user's body and instantly converts them into data.

[1905] "Means for storing data in a database" refers to a system for systematically recording measured data and storing it in a format that can be managed and searched.

[1906] "Means for analyzing and assessing the user's health status" refers to algorithms and software that objectively assess the user's health status based on collected health indicator data.

[1907] "Means for sending reminder notifications" refers to a system that sends messages to users to encourage them to take a break or suggest ways to reduce stress.

[1908] "Means for playing relaxing music" refers to a function that plays music that relieves stress based on the user's emotional state.

[1909] "Means for connecting users to social support groups" refers to technology that allows users to access appropriate communities and groups to receive emotional support.

[1910] "A means for recording and analyzing a user's emotional diary" refers to a system that allows users to input the emotions they feel on a daily basis, and then records and analyzes that data.

[1911] "Means for providing appropriate services in physical stores" refers to a system for providing services in a physical store environment that take into account the user's current health and emotional state.

[1912] "Means for suggesting products based on health and emotional state" refers to technology that suggests optimal products and services based on the user's health and emotional data.

[1913] "Means for playing relaxing music in specific areas" refers to the function of playing music in specific areas within the store to help users relax.

[1914] The present invention is a system for supporting the mental and physical health of a user, and in particular incorporates an emotion engine to more accurately recognize the user's emotional state and provide appropriate support. This system is realized mainly using the following hardware and software.

[1915] Hardware and software used:

[1916] Smart glasses (e.g., Google Glass, Vuzix Blade): measure the user's physical health indicators (heart rate, steps, etc.) in real time.

[1917] Smartphone / tablet (e.g. iPad, Android tablet): Displays and notifies the user's emotional state and health data.

[1918] Server (e.g. AWS EC2, Google Cloud VM): Analyzes and stores data, and generates notifications.

[1919] Emotion engine (e.g., Microsoft Azure Emotion API, IBM Watson Tone Analyzer): Evaluates the user's emotional state.

[1920] System features:

[1921] The system is equipped with the following functions, and analyzes the user's health and emotional state in real time and provides appropriate support based on that.

[1922] 1. Real-time health monitoring:

[1923] The smart glasses measure the user's heart rate, number of steps, and other data in real time and send it to a server, where it stores the data in a database and analyzes it, allowing the user's health status to be tracked.

[1924] 2. Recognition of emotional states:

[1925] The emotion engine analyzes the user's facial expressions and voice data to assess their emotional state. For example, it analyzes what the user says and their facial expressions using a smartphone app to determine their stress level. The data obtained by the emotion engine is also sent to the server and used as part of the analysis.

[1926] 3. Reminder Notification:

[1927] If the user's emotional state is determined to be declining, the server generates a reminder notification and sends it to the user's smartphone or tablet, which may include a message to take a break or specific stress reduction advice.

[1928] 4. Play relaxing music:

[1929] The server selects appropriate relaxing music based on the user's emotional state and environment, and sends playback instructions to the smart glasses or smartphone, allowing the user to relax by listening to music such as the sound of a tranquil forest or the sound of waves.

[1930] 5. Connect with social support groups:

[1931] If a user wishes to join a social support group, the server will recommend and connect them to an appropriate group based on their profile information, allowing them to communicate with other members in real time and receive emotional support.

[1932] 6. Personalized services in-store:

[1933] When a user is in a physical store, real-time information is provided to store staff based on the user's health indicators and emotional state. For example, a tense user can be provided with a relaxing environment, and a stressed user can be suggested products that will have a relaxing effect.

[1934] Specific use cases:

[1935] 1. Health measurement and emotional state recognition:

[1936] When a user wears smart glasses and goes shopping, their heart rate is measured to be high, and the emotion engine detects a state of stress. In this case, the server issues a command to "play relaxing music," and nature music is played on the user's smartphone.

[1937] 2. Reminder Notification:

[1938] If the server analyzes health data and emotional state to be declining, a reminder notification such as "Take a break" will be sent to the user's device.

[1939] 3. Social support group connections:

[1940] When a user types "I need support" into the smartphone app, the server recommends appropriate social support groups and connects the user to them.

[1941] Example prompt sentence:

[1942] "Implement a system that recognizes customers' stress levels and suggests products that will help them relax. Use smart glasses and an emotion recognition engine to notify staff when a certain threshold is exceeded."

[1943] This allows users to receive appropriate mental and physical support in real time, making the shopping experience in physical stores more comfortable.

[1944] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1945] Step 1:

[1946] When a user wears smart glasses, physical health indicators (heart rate, number of steps, etc.) are measured in real time. The smart glasses instantly convert this data into digital data and send it to a server. The input data is health information such as heart rate and number of steps, and the output is the measurement data sent to the server. The smart glasses have built-in heart rate monitors and acceleration sensors, which are used to acquire data in real time.

[1947] Step 2:

[1948] The server receives the health index data sent from the smart glasses and stores it in a database. At this time, the server organizes the data so that it can be easily searched and analyzed later. The input data is the measurement data sent from the smart glasses, and the output is the health data stored in the database. The server receives the data using a data transfer protocol (e.g., HTTP, HTTPS) and stores it in a database (e.g., MySQL, PostgreSQL).

[1949] Step 3:

[1950] The server analyzes the stored data and evaluates the user's health condition. This analysis uses an algorithm that, for example, determines that an abnormally high heart rate indicates stress. The input data is the health data stored in the database, and the output is the evaluation result of the user's health condition. The server analyzes the data using a programming language such as Python and a data analysis library (e.g., Pandas, NumPy).

[1951] Step 4:

[1952] The emotion engine analyzes the user's facial expression and voice data to evaluate their emotional state. The emotion engine analyzes facial expression and voice data input via a smartphone or tablet. The input data is the user's facial expression and voice information, and the output is an evaluation of their emotional state. The emotion engine uses the Microsoft Azure Emotion API, IBM Watson Tone Analyzer, etc. to analyze facial expressions and voice emotions.

[1953] Step 5:

[1954] If the server determines that the user's health and emotional state is declining, it generates a reminder notification. The reminder notification includes a message encouraging the user to take a break and specific advice for reducing stress. The input data is the assessment result of the user's health and emotional state, and the output is the reminder notification message. The server creates the message using a notification generation algorithm and sends the notification using a notification service (e.g., Firebase Cloud Messaging).

[1955] Step 6:

[1956] The server sends a reminder notification and an instruction to play relaxing music to the smart glasses or smartphone. The relaxing music may include, for example, the sound of a quiet forest or the sound of waves. The input data is the instruction to play relaxing music, and the output is the relaxing music played from the smart glasses or smartphone. The server uses a music selection algorithm and sends the play instruction to the device.

[1957] Step 7:

[1958] When a user wishes to join a social support group, the server recommends and connects them to appropriate groups based on the user's profile information. The input data is the user's profile information, and the output is information about recommended social support groups. The server uses a recommendation algorithm and connects to communication platforms (e.g., Slack, Discord).

[1959] Step 8:

[1960] When a user is in a physical store, the server provides the user's health indicators and emotional state to store staff in real time. The staff obtains this information via tablets or smart devices and provides individualized support. The input data is the user's health indicators and emotional state, and the output is the information displayed to the staff. The server uses a data transfer protocol to send the data to the staff application.

[1961] Step 9:

[1962] In a physical store, the server makes product suggestions based on the user's health and emotional state. For example, a user experiencing high stress may be recommended a product with a relaxing effect. The input data is the user's health and emotional state, and the output is personalized product suggestions. The server uses a recommendation engine to generate the suggestions.

[1963] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1964] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indic...

Claims

1. a means of measuring the user's physical health indicators in real time; means for storing the measured health indicators in a database; a means for analyzing the stored data to assess the user's health status; means for sending a reminder notification when the user's emotional state drops; A means for playing relaxing music as an environmental adjustment means; a means of connecting users to social support groups; A system including means for recording and analyzing a user's emotion diary.

2. The system of claim 1 , further comprising means for playing music to encourage relaxation in the user if the user's emotional state is assessed as low.

3. The system of claim 1 further comprising means for selecting an appropriate social support group based on user input and connecting the user to that group.

Images