System

Abstract

A system is provided.SOLUTION: A system comprising: a data collection unit including a wearable device that can be worn by a user and is configured to acquire vital data; an analysis unit configured to analyze data collected by the data collection unit and evaluate a health condition and an emotional state of the user; and a notification unit configured to notify the user of a suggestion content generated by the analysis unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, it is extremely difficult for individual users to manage their health and emotions in a way that suits them. It often takes a great deal of time and effort to obtain timely advice and suggestions. Furthermore, there is a lack of appropriate tools to help users understand their own behavioral patterns and improve their quality of life. Given this background, there is a need to provide users with support tailored to their individual needs. [Means for solving the problem]

[0005] To solve this problem, we propose the following solution. We provide a system including: a data collection means including a wearable device that can be worn by a user to acquire vital data; an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state; and a notification means for notifying the user of suggestions generated by the analysis means. In particular, the notification means has an interactive function for accepting questions or inquiries from the user and notifying the user of responses generated based on the questions or inquiries. Furthermore, the data collection means includes a camera and a microphone for analyzing the user's facial expressions and voice to acquire emotional data. This makes it possible to efficiently and effectively support the user's health and emotional management.

[0006] A "data collection means" is a device that uses a wearable device worn by a user to acquire vital data such as heart rate, body temperature, and sleep patterns.

[0007] A "wearable device" is a device that can be worn by a user and includes sensors for measuring body movements and physiological indicators.

[0008] The "analysis means" is a combination of software and hardware for processing the vital data acquired by the data collection means and evaluating the user's health condition and emotional state.

[0009] The "notification means" is a device or function for notifying the user of the suggestions and responses generated by the analysis means, and includes output devices such as a display, voice synthesis, and a vibration motor.

[0010] The "interactive function" is a function that accepts questions or inquiries from users and generates responses based on those questions and communicates them to the users.

[0011] A "camera" is an image capture device that captures the user's facial expressions and uses them for data analysis.

[0012] A "microphone" is a voice capture device that captures the user's voice and analyzes it to assess their emotional state.

[0013] "Emotion data" is data that indicates the user's emotional state, analyzed from the user's facial expression, tone of voice, and the like.

[0014] "Vital data" refers to data that indicates physiological indicators such as the user's heart rate, body temperature, and sleep patterns. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and an embodiment thereof will be described below.

[0037] Data collection methods

[0038] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis.

[0039] Data transmission and analysis

[0040] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0041] Proposal generation and notification

[0042] The server then generates optimal exercise and dietary recommendations based on the analysis results. These recommendations are customized based on the user's current health and emotional state. For example, if the user's sleep quality is deteriorating, the server will suggest relaxing yoga and appropriate eating habits.

[0043] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions. This notification is done using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[0044] Interactive features

[0045] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0046] Specific examples

[0047] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0048] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0049] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[0050] The processing flow will be explained below.

[0051] Step 1:

[0052] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[0053] Step 2:

[0054] The device packages the collected vital and emotional data and periodically transmits it to a server via the internet in JSON format or similar, with the data being temporarily stored in memory.

[0055] Step 3:

[0056] The server analyzes the received data using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[0057] Step 4:

[0058] Based on the analysis results, the server generates optimal exercise and dietary suggestions for the user. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods.

[0059] Step 5:

[0060] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[0061] Step 6:

[0062] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[0063] Step 7:

[0064] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot recently" is sent.

[0065] Step 8:

[0066] The server analyzes the user's questions and concerns and generates appropriate advice and responses, such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data.

[0067] Step 9:

[0068] The server then sends the generated response to the device, which then notifies the user, who receives the suggestions and answers via voice or text and can incorporate them into their daily lives.

[0069] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[0070] Example 1

[0071] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0072] In recent years, there has been a growing emphasis on individual health and emotional management. However, conventional systems lack the ability to adequately collect and analyze data in real time, making it difficult to provide users with appropriate advice. Furthermore, when users have specific questions or concerns, it is difficult to obtain an immediate, appropriate response. Furthermore, there is a lack of means to accurately grasp a user's emotional state. To solve these issues, more comprehensive, real-time data analysis and individualized responses are required.

[0073] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0074] In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire health data, a transmission means for periodically transmitting the data collected by the data collection means to the server, an analysis means for analyzing the data transmitted to the server by the transmission means and evaluating the user's health condition and emotional state, and a notification means for notifying the user of suggestions generated by the analysis means. This enables real-time data acquisition and analysis, realizing a system that can instantly provide appropriate advice and responses to individual users. It is also possible to analyze the user's facial expressions and voice to grasp their emotional state, allowing for more accurate support.

[0075] A "wearable device" is an electronic device that can be worn by a user and is used to acquire health data and vital signs.

[0076] "Data collection means" refers to a mechanism for measuring the user's health and emotional state using a wearable device and collecting the necessary data.

[0077] The "transmission means" is a function for transmitting the data acquired by the data collection means to the server.

[0078] "Analysis means" refers to a program or algorithm that analyzes the data sent to the server and evaluates the user's health and emotional state.

[0079] The "notification means" is a mechanism for notifying the user of the suggestions and responses generated by the analysis means, and may include a display and a voice synthesis function.

[0080] A "server" is a computer system for storing and analyzing received data.

[0081] A "machine learning model" is an algorithm that learns patterns based on large amounts of data and assesses a user's health status.

[0082] A "database management system" is software that efficiently stores, searches, and manages collected data.

[0083] "Natural Language Generation (NLG)" is a technology that allows machines to generate human language and provide suggestions and responses to users.

[0084] The "voice synthesis function" is a technology for converting text data into voice and notifying the user.

[0085] A "prompt" is a textual instruction given to a generative AI model, serving as a guideline for the model to generate appropriate responses or suggestions.

[0086] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and embodiments thereof will be described below.

[0087] Data collection methods

[0088] The wearable device (terminal) worn by the user collects health data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors (heart rate sensor, thermometer, and accelerometer) that can accurately measure the user's physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis. This data collection method makes it possible to closely monitor both the user's daily health and emotional state.

[0089] Sending data

[0090] The device sends the collected data to the server at regular intervals. This transmission process is carried out using a secure communication protocol (e.g., HTTPS), so the data remains safe.

[0091] Data analysis

[0092] The server stores the transmitted data in a database management system (e.g., SQL database). It then uses machine learning models (e.g., TensorFlow or PyTorch) to analyze the data and evaluate the user's health and emotional state. This analysis uses data mining techniques that compare past health data with newly acquired data. For example, it analyzes heart rate variability patterns and sleep quality to evaluate the user's overall health.

[0093] Proposal generation and notification

[0094] The server generates optimal exercise and dietary recommendations for the user based on the analysis results. These recommendations are generated in a format that is easy for humans to understand using natural language generation (NLG) technology. The generated recommendations are sent from the server to the device, which notifies the user using a display or voice synthesis function. For example, a user whose sleep quality is declining may be shown suggestions for relaxing yoga or a meal.

[0095] Interactive features

[0096] When a user wants to ask a specific question (e.g., "Do you get tired easily?"), they use the device's dialogue function. When the user speaks, their voice is converted into text data and sent to the server. The server analyzes the text data and generates an appropriate response or advice. This response is sent back to the device and notified to the user via the voice synthesis function.

[0097] Specific examples

[0098] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0099] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0100] Prompt Sentence Examples

[0101] Prompt statement:

[0102] When a user says, "I feel like I get tired easily," the following prompt is input to the generative AI model:

[0103] "User input: I feel tired easily.

[0104] Current vital signs: Heart rate 80, body temperature 36.5°C, exercise volume in the past week: low.

[0105] Use this information to generate appropriate advice.

[0106] Generative AI response: "You haven't been exercising much lately. Try taking a light walk in the afternoon."

[0107] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0108] Step 1: Data collection

[0109] The wearable device collects real-time health data such as the user's heart rate, body temperature, and sleep patterns. It also uses a camera and microphone to collect the user's facial expressions and voice. All of this data is collected with high precision using sensors and analytical algorithms.

[0110] Input: User's biometric information (heart rate, body temperature, facial expression, voice, etc.)

[0111] Output: Collected data (biometric data and emotional data)

[0112] Step 2: Send data

[0113] The device sends the collected data to the server at regular intervals, and this process uses a secure communication protocol (e.g., HTTPS) to ensure the safety of the data.

[0114] Input: Collected data

[0115] Output: Data sent

[0116] Step 3: Save Data

[0117] The server receives the transmitted data and stores it in a database management system (e.g., an SQL database). This storage process ensures the consistency and integrity of the data.

[0118] Input: Data sent

[0119] Output: Data stored in the database

[0120] Step 4: Data analysis

[0121] The server analyzes the stored data using machine learning models (e.g., TensorFlow or PyTorch), and uses data mining techniques to compare past health data with newly acquired data and evaluate the user's health and emotional state.

[0122] Input: Data stored in a database, machine learning model

[0123] Output: Analysis results (evaluation of health and emotional state)

[0124] Step 5: Generate proposals

[0125] Based on the analysis results, the server generates optimal exercise and dietary recommendations for the user, using natural language generation (NLG) technology to create recommendations in a format that is easy for the user to understand.

[0126] Input: Analysis results

[0127] Output: Generated proposals

[0128] Step 6: Submit your proposal

[0129] The server then transmits the generated proposal to the device, and this transmission process is also performed using a secure communication protocol.

[0130] Input: Generated proposal

[0131] Output: Submitted proposal

[0132] Step 7: Notification of proposal

[0133] The device notifies the user of the received suggestions using a display or voice synthesis function, allowing the user to confirm and follow the suggestions.

[0134] Input: Submitted proposal

[0135] Output: The suggestion sent to the user

[0136] Step 8: Interactivity

[0137] The user speaks a question or request to the terminal, which converts the speech into text data and sends it to the server.

[0138] Input: User voice input

[0139] Output: Content sent to the server as text data

[0140] Step 9: Analyzing the dialogue

[0141] The server analyzes the received text data and generates appropriate responses and advice, and this analysis process is also carried out using machine learning models and natural language processing techniques.

[0142] Input: Text data

[0143] Output: The response or advice generated

[0144] Step 10: Sending a Response

[0145] The server then generates a response and sends it to the terminal, a process that is also performed securely.

[0146] Input: Generated responses and advice

[0147] Output: Response or advice sent

[0148] Step 11: Notification of response

[0149] The device notifies the user of the response received from the server through a voice synthesis function, allowing the user to receive appropriate support in real time.

[0150] Input: Responses and advice sent

[0151] Output: Response and advice given to the user

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Conventional store staff health management systems lack the functionality to monitor staff health and emotional state in real time and suggest optimal working styles and break times based on that state. This leads to staff fatigue and stress building up, leading to lower productivity and a worsening workplace atmosphere. Furthermore, because no customized suggestions are made based on the health and emotional state of each staff member, effective health management is not possible.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes a data collection means including a wearable device that can be worn by a user to acquire vital data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for notifying the user of suggestions generated by the analysis means, and a means for monitoring the health and emotional states of store staff in real time and suggesting optimal working styles and breaks. This allows for efficient and effective management of the health and emotional states of store staff, reducing staff fatigue and stress, and enabling increased productivity and an improved workplace atmosphere.

[0157] A "wearable device that can be worn by a user" is a device that can be worn directly on the wearer's body and has the function of acquiring vital data.

[0158] "Data collection means" includes wearable devices and is a means for collecting vital data and emotional data of a user.

[0159] The "analysis means" is a means for evaluating the health condition and emotional state of the user based on the data collected by the data collection means, and uses machine learning and data mining techniques.

[0160] The "notification means" is a means for notifying the user of the content of the proposal generated by the analysis means, and is carried out through a display or a voice synthesis function.

[0161] "Store staff" refers to employees who perform work in physical stores.

[0162] "Health status" refers to the physical condition of a user, assessed based on physiological data such as heart rate, body temperature, and sleep patterns.

[0163] "Emotional state" refers to the psychological state of the user that is evaluated based on information obtained from facial expressions and voice.

[0164] A "generative AI model" is an artificial intelligence model used to analyze collected data, assess the user's health and emotional state, and generate optimal suggestions.

[0165] A "prompt" is a sentence to be input to a generative AI model, containing input information that enables the model to generate appropriate suggestions.

[0166] MODE FOR CARRYING OUT THE INVENTION

[0167] The present invention relates to a system for managing the health and emotional states of store staff, and an embodiment thereof will be described below.

[0168] Data collection methods

[0169] The wearable device worn by the user collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable accurate measurement of these physiological indicators. It is also equipped with a camera and microphone, and can capture the user's emotional state through facial expression recognition and voice analysis.

[0170] Data transmission and analysis

[0171] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0172] Proposal generation and notification

[0173] Based on the analysis results, the server generates exercise and dietary suggestions that suggest optimal working styles and break times for the user. These suggestions are customized according to the user's current health and emotional state. For example, if a staff member's stress level is high, the server will suggest a short break or some light exercise to refresh themselves. The generated suggestions are sent from the server to the device. The device then notifies the user of these suggestions. This notification is made using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[0174] Interactive features

[0175] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0176] Specific examples

[0177] When the user wakes up in the morning, the device sends the overnight sleep data to the server. The server analyzes this data and notifies the device with, "Good morning. Let's have another healthy day today." If the user says to the device while at work, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been exercising less recently. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0178] Prompt Sentence Examples

[0179] Use the following data to assess the user's health and emotional state and generate optimal recommendations:

[0180] Heart rate: {heart_rate}

[0181] Body temperature: {body_temperature}

[0182] Sleep pattern: {sleep_pattern}

[0183] Emotional state: {emotion}

[0184] Proposal details:

[0185] The system is built using hardware and software such as wearable devices, smartphones, servers, and machine learning models, enabling real-time health management of store staff and the provision of effective support and suggestions.

[0186] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0187] Step 1:

[0188] The device collects the user's vital data and emotional data.

[0189] Input: User's heart rate, body temperature, sleep patterns, facial expressions, and voice

[0190] Specific operation: The device's built-in high-precision sensors, camera, and microphone collect data in real time.

[0191] Output: Collected vital and emotional data

[0192] Step 2:

[0193] The terminal transmits the collected data to the server.

[0194] Input: Collected vital and emotional data

[0195] Specific operation: Data is sent to the server at regular intervals. HTTP or HTTPS is used as the communication protocol.

[0196] Output: Vital and emotional data sent to the server

[0197] Step 3:

[0198] The server analyzes the received data using a machine learning model.

[0199] Input: Transmitted vital data and emotional data

[0200] How it works: The server uses machine learning models to assess the user's health and emotional state. The algorithm uses data mining techniques.

[0201] Output: Evaluation results on the user's health and emotional state

[0202] Step 4:

[0203] The server generates optimal proposals based on the evaluation results.

[0204] Input: Health and emotional state assessment results

[0205] Specific operation: The server uses a generative AI model to customize appropriate exercise and rest content and generate suggestions.

[0206] Output: Generated proposals

[0207] Step 5:

[0208] The server transmits the generated proposal to the terminal.

[0209] Input: Generated proposal

[0210] Specific operation: The proposals are sent from the server to the device via a communication protocol. The sending timing can be real-time or at regular intervals.

[0211] Output: Suggestions sent to the device

[0212] Step 6:

[0213] The terminal notifies the user of the content of the proposal.

[0214] Input: Proposal sent from the server

[0215] Specific operation: The device notifies the user of the suggestions using a display or voice synthesis.

[0216] Output: User-recognized suggestions

[0217] Step 7:

[0218] The user asks a question or asks for advice via the terminal.

[0219] Input: User's voice or text questions and inquiries

[0220] Specific action: The user speaks into the device or types text.

[0221] Output: Questions and inquiries entered into the device

[0222] Step 8:

[0223] The server receives and analyzes the user's question or inquiry and generates a response.

[0224] Input: Questions and inquiries sent from the device

[0225] Specific operation: The server analyzes the question or consultation content using a machine learning model and generates an appropriate response based on past data and algorithms.

[0226] Output: The generated response

[0227] Step 9:

[0228] The server sends the generated response to the terminal.

[0229] Input: Generated response content

[0230] Specific operation: The response content is sent from the server to the terminal in the same way as the proposal content.

[0231] Output: Response sent to the terminal

[0232] Step 10:

[0233] The terminal notifies the user of the response.

[0234] Input: Response sent from the server

[0235] Specific operation: The terminal notifies the user of the response content using a display or voice synthesis.

[0236] Output: The user's perceived response

[0237] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0238] The present invention relates to a system for efficiently and effectively supporting a user's health management and emotion management, and in particular to a form in which an emotion engine is combined. An embodiment of the system will be described in detail below.

[0239] Data collection methods

[0240] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone that captures the user's facial expressions and tone of voice to collect emotional data.

[0241] Emotion Engine

[0242] The device is equipped with an emotion engine to analyze the collected emotion data. The emotion engine uses an algorithm to classify the user's emotional state based on facial expression data and tone of voice data. Emotional states are classified into multiple categories, such as joy, sadness, anger, and surprise.

[0243] Data transmission and analysis

[0244] The device periodically transmits the collected and analyzed vital and emotional data to a server. The server then analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning algorithms and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0245] Proposal generation and notification

[0246] Based on the analysis results, the server generates recommendations for optimal exercises, dietary habits, and relaxation methods to soothe emotions. These recommendations are customized according to the user's current health and emotional state. For example, if the user's emotional state is classified as "sad," the server will suggest relaxation methods and fun entertainment.

[0247] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions using a display or speech synthesis function in a format that is easy for the user to understand.

[0248] Interactive features

[0249] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0250] Specific examples

[0251] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0252] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0253] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects the emotion of "sadness" from that expression, the server will also make suggestions such as "It would be good to listen to some relaxing music."

[0254] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[0255] The processing flow will be explained below.

[0256] Step 1:

[0257] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[0258] Step 2:

[0259] The device packages the collected vital and emotional data and transmits it to a server over the Internet at regular intervals in JSON format, etc. The data is then temporarily stored in memory.

[0260] Step 3:

[0261] The server analyzes the received data, using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[0262] Step 4:

[0263] The server then generates optimal exercise and diet suggestions for the user based on the analysis results. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods. If the emotion engine evaluates the emotional data, it will also suggest relaxation methods and entertainment options.

[0264] Step 5:

[0265] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[0266] Step 6:

[0267] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[0268] Step 7:

[0269] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot lately" is sent.

[0270] Step 8:

[0271] The server analyzes the user's questions and concerns and generates appropriate advice and responses. For example, it generates a response such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data. The emotion engine is also involved in this process, providing advice tailored to the user's emotional state.

[0272] Step 9:

[0273] The server then sends the generated response to the device, which then notifies the user. The user receives the suggestions and answers via voice or text and can incorporate them into their daily lives. For example, additional suggestions such as "Listen to relaxing music" can also be made.

[0274] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[0275] Example 2

[0276] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0277] Conventional health management and emotion management systems lacked efficient means for collecting and analyzing users' vital and emotional data in real time. Furthermore, they often failed to provide personalized recommendations based on the user's health and emotional state, preventing improvements in quality of life. Furthermore, they lacked a dialogue function that could provide appropriate responses to users' inquiries in real time. This made it difficult to provide comprehensive support for users' daily lives.

[0278] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0279] In this invention, the server includes: a data collection means including a sensor device wearable by the user for acquiring physiological data such as heart rate, body temperature, and sleep patterns; a means for acquiring emotional data by using a camera and a microphone to collect the user's facial expressions and tone of voice; an analysis means for analyzing the physiological data and emotional data collected by the data collection means and evaluating the user's health and emotional state; a means for performing a comprehensive analysis based on the collected data using a machine learning algorithm and data mining technology; and a means for notifying the user of suggestions generated by the analysis means. This makes it possible to efficiently manage the user's health and emotions in real time and provide customized suggestions.

[0280] The "data collection means" refers to a means for collecting physiological data and emotional data using a sensor device that can be worn by the user, as well as a camera and microphone.

[0281] A "sensor device" is a wearable device equipped with high-precision sensors that collect physiological data such as heart rate, body temperature, and sleep patterns.

[0282] A "camera" is an image capture device for capturing a user's facial expression and is used to collect emotional data.

[0283] A "microphone" is a voice capture device for capturing the tone of a user's voice and is used to collect emotion data.

[0284] "Emotion data" is data on the emotional state of the user analyzed from facial expressions and tone of voice obtained through a camera and microphone.

[0285] "Physiological data" refers to data on physiological indicators such as heart rate, body temperature, and sleep patterns collected by sensor devices.

[0286] The "analysis means" is a means for analyzing the collected physiological data and emotional data to evaluate the health condition and emotional state of the user.

[0287] A "machine learning algorithm" is an algorithm used to analyze, classify, and predict data based on accumulated data.

[0288] "Data mining technology" is a technique for extracting useful patterns and knowledge from large amounts of data.

[0289] The "suggested content" is information generated by the analysis means about optimal exercises, dietary habits, relaxation methods, etc. for the user.

[0290] The "notification means" is a means including a display, a voice synthesis function, etc., for notifying the user of the generated proposal content.

[0291] The "interactive function" is a function that accepts questions or inquiries from users and notifies the users of a response that is generated based on those questions or inquiries.

[0292] The present invention relates to a system for supporting a user's health management and emotion management, and particularly to a form in which an emotion engine is combined.

[0293] The system uses the following hardware and software:

[0294] Hardware

[0295] 1. Wearable devices (terminals)

[0296] Wearable by the user, it collects physiological data such as heart rate, body temperature, and sleep patterns in real time. High-precision sensors are built in to accurately measure these physiological indicators. It also has a camera and microphone to capture the user's facial expressions and tone of voice, collecting emotional data.

[0297] software

[0298] 1. Emotion Engine (Device)

[0299] The emotion engine analyzes facial expression and tone of voice data collected by the camera and microphone. It uses an algorithm to classify the user's emotional state based on the facial expression and tone of voice data. Emotional states are classified into categories such as joy, sadness, anger, and surprise.

[0300] 2. Data analysis system (server)

[0301] The server receives collected and analyzed data from the device and performs a comprehensive analysis using machine learning algorithms and data mining techniques. The server evaluates the user's health and emotional state and generates recommendations based on that.

[0302] Specific examples

[0303] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes the data and, if it determines that the user's sleep quality is good, generates a suggestion message such as "Good morning. Let's have a healthy day today" and notifies the device.

[0304] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, generates a specific suggestion such as, "You've been getting less exercise lately. Try taking a light walk this afternoon," and sends it to the device. The device then notifies the user of this message by voice.

[0305] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects "sadness" from the expression, the server will make suggestions such as "It would be good to listen to some relaxing music."

[0306] Prompt Sentence Examples

[0307] Analyze your sleep data and let you know the quality of your sleep.

[0308] Generate optimal exercise suggestions when the user reports feeling fatigued.

[0309] Please suggest relaxation methods if you detect "sadness" from the user's facial expression using the emotion engine.

[0310] This makes it possible to provide a system that efficiently and effectively supports users in managing their health and emotions, improving their quality of life.

[0311] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0312] Step 1:

[0313] Data collection

[0314] The device uses a wearable device worn by the user to collect physiological data such as heart rate, body temperature, and sleep patterns in real time, and also uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[0315] Input: User's heart rate, body temperature, sleep patterns, facial expression data, and tone of voice.

[0316] How it works: The wearable device's sensors measure the user's heart rate every minute, the camera captures facial expressions every 10 seconds, and the microphone records the tone of voice in real time.

[0317] Output: Physiological and emotional data.

[0318] Step 2:

[0319] Emotional Data Analysis

[0320] The device analyzes the collected facial expression data and tone of voice data using an emotion engine, which uses this data to classify the user's emotional state into categories such as "joy," "sadness," "anger," and "surprise."

[0321] Input: facial expression data, tone of voice data.

[0322] Specific operation: The emotion engine detects and classifies facial expression patterns of happiness from the collected facial expression data. If the voice tone is low, it is classified as "sad."

[0323] Output: Emotional state category (e.g., happy, sad).

[0324] Step 3:

[0325] Sending data

[0326] The device periodically transmits the collected and analyzed physiological and emotional data to a server, where the encrypted data is transferred over a secure network.

[0327] Input: physiological data, emotional state categories.

[0328] Specific operation: The device collects all data into packets every 30 minutes and sends them to the server.

[0329] Output: The data sent to the server.

[0330] Step 4:

[0331] Data analysis

[0332] The server analyzes the received physiological and emotional data and uses machine learning algorithms and data mining techniques to assess the user's health and emotional state, comparing past data with new data to perform a comprehensive analysis.

[0333] Input: Physiological and emotional data sent to the server.

[0334] What it does: The server compares the past week's data with the current data and flags any abnormal patterns (e.g., a sudden increase in heart rate).

[0335] Output: Assessment of the user's health and emotional state.

[0336] Step 5:

[0337] Proposal generation

[0338] The server generates recommendations for the user based on the results of the data analysis, which are customized according to the user's current health and emotional state.

[0339] Input: User's health and emotional state assessment results.

[0340] Specific behavior: If the user's emotional state is assessed as "sad," generate a list of relaxation techniques and fun entertainment. If data indicates a lack of exercise, suggest an afternoon walk.

[0341] Output: Suggestions (e.g. relaxation techniques, exercise suggestions).

[0342] Step 6:

[0343] Notification of proposal details

[0344] The server sends the generated suggestions to the device, which then notifies the user of the suggestions using a display or voice synthesis function.

[0345] Input: Proposal.

[0346] Specific operation: When the suggestion from the server arrives at the device, the device's display will show the message, "You've been getting less exercise recently. Try taking a light walk this afternoon." The speech synthesis function will read this message aloud.

[0347] Output: Notification to the user.

[0348] Step 7:

[0349] Use of interactive features

[0350] When a user wants to ask a question or seek advice about fatigue or stress, they use the device's dialogue function. What they say to the device is sent as text data to the server, and appropriate advice is generated.

[0351] Input: The user's question or inquiry.

[0352] Specific operation: When a user says to the device, "I get tired easily," the speech is converted into text and sent to the server. The server analyzes the content and generates advice such as, "You've been getting less exercise recently. Try taking a light walk in the afternoon," which is sent to the device. The device then notifies the user by voice.

[0353] Output: Advice or response to the user.

[0354] (Application example 2)

[0355] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] Conventional health management systems can assess a user's health and emotional state and make suggestions, but they cannot provide advice or relaxation tailored to the specific circumstances of users performing specific tasks, such as security work. In particular, there was no system that could immediately provide appropriate countermeasures when security guards felt stressed or fatigued. There was also a lack of systems that could provide specific security guidelines through real-time monitoring. This resulted in a decrease in the work efficiency of security guards and an inability to guarantee safety.

[0357] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire vital data and emotional data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for providing security guidelines based on the suggestions generated by the analysis means and the user's current emotional state, and an emergency notification means for recommending relaxation in an emergency. This enables efficient and effective health and emotional management of security guards and provides appropriate countermeasures immediately when they feel stressed or fatigued. Furthermore, providing specific security guidelines based on real-time monitoring improves the efficiency and safety of security operations.

[0358] The "data collection means" is a device that acquires vital data and emotional data using a wearable device that can be worn by the user.

[0359] The "analysis means" is a system for analyzing the data collected by the data collection means and evaluating the health condition and emotional state of the user.

[0360] The "notification means" is a device for providing security guidelines according to the proposal content generated by the analysis means and the current emotional state of the user.

[0361] The "emergency notification means" is a device that recommends appropriate relaxation in real time when the user falls into a state of stress or fatigue.

[0362] A "wearable device" is an electronic device that can acquire vital data and emotional data by being worn by a user.

[0363] "Vital data" refers to data related to physiological indicators such as a user's heart rate, body temperature, and sleep patterns.

[0364] "Emotion data" is data that indicates the user's emotional state based on facial expressions, tone of voice, and the like.

[0365] "Analysis" is the process of assessing the user's health and emotional state based on the collected data.

[0366] "Security guidelines" are specific instructions and advice on security tasks that are provided according to the user's current emotional state.

[0367] "Relaxation" refers to relaxation methods and activities that users should undertake when they feel stressed or tired.

[0368] This invention is a system related to smart glasses worn by security guards that monitors the user's health and emotional state in real time and provides appropriate security guidelines and relaxation methods depending on the situation.

[0369] 1. System Configuration

[0370] Data collection methods

[0371] The smart glasses worn by security guards are equipped with a wearable device for acquiring vital and emotional data. The wearable device is equipped with high-precision sensors, cameras, and microphones to collect real-time data such as heart rate, body temperature, sleep patterns, facial expressions, and tone of voice.

[0372] Analysis means

[0373] The device analyzes the collected data to assess the user's health and emotional state. The collected vital and emotional data is analyzed using machine learning algorithms and data mining techniques to assess the user's condition. Software such as Python and TensorFlow is used for the analysis.

[0374] Notification means

[0375] The device notifies the user of the analysis results. Based on the user's health and emotional state, security guidelines and relaxation techniques are provided. Notification methods include voice synthesis and a display. For example, if a security guard is detected as stressed, the device will display a message on the screen saying, "Take a deep breath."

[0376] Emergency notification means

[0377] The terminal also has an emergency notification function to recommend relaxation in case of an emergency. If a security guard is experiencing excessive stress or fatigue, it will immediately suggest relaxation methods.

[0378] 2. Data submission and analysis process

[0379] The collected data is periodically sent to a server, which analyzes the received data using advanced machine learning algorithms. Based on the results of the data analysis, appropriate advice and guidelines are generated for the user. The server uses cloud services such as AWS and Google Cloud.

[0380] 3. Working Example

[0381] Specific examples

[0382] The smart glasses worn by security guards while on duty use built-in sensors to collect data on heart rate and facial expressions. The collected data is sent to a server in real time for analysis. For example, if a high heart rate and "stress" are detected from facial expressions, the server will generate a guideline for the security guard to "take a deep breath" and display it on the smart glasses' display.

[0383] Input prompt example

[0384] Given the heart rate (90), facial expression (captured image data), and voice tone (neutral), analyze the user's emotional state and provide appropriate advice. For example, if stress is detected, suggest relaxation.

[0385] This method allows for efficient and effective health and emotional management of security guards, improving work efficiency and safety.

[0386] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0387] Step 1:

[0388] The device uses sensors, cameras, and microphones installed in the smart glasses to collect the user's vital data (heart rate, body temperature, sleep patterns, etc.) and emotional data (facial expressions, tone of voice).The input is the user's real-time biological information and environmental sounds, and the output is a form in which these data are temporarily stored.

[0389] Step 2:

[0390] The device sends the collected vital data and emotional data to the analysis means, which then analyzes this data using a machine learning algorithm. Specifically, it uses Python and TensorFlow to analyze changes in heart rate and facial expressions to evaluate the user's health and emotional state. The input is the data acquired in step 1, and the output is the evaluation results of the user's health and emotional state.

[0391] Step 3:

[0392] The server generates appropriate security guidelines and relaxation methods based on the evaluation results sent from the device. The input is the evaluation results, and the output is the generated guidelines and relaxation methods. Specifically, it compares past data with current data and executes an algorithm to make optimal suggestions.

[0393] Step 4:

[0394] The security guidelines and relaxation methods generated by the analysis means are sent to the notification means, which notifies the user. The notification means uses the smart glasses' display and voice synthesis function. The input is the generated guidelines and relaxation methods, and the output is a notification to the user. The specific operation is to convey the generated text and voice to the user visually and audibly.

[0395] Step 5:

[0396] If the user suddenly experiences stress or fatigue, the device will use emergency notification means to immediately suggest relaxation methods. The input is a sudden change in the user's vital signs, and the output is a relaxation suggestion. Specifically, it detects a sudden increase in heart rate or a change in facial expression in real time and immediately notifies the user by suggesting, for example, "Take a deep breath."

[0397] Step 6:

[0398] The results of data analysis and user feedback are stored and managed on a server and are used to improve the accuracy of future data analysis and proposals. The input is the analysis results and feedback data, and the output is an accumulated database. Specifically, the system uses cloud services (AWS or Google Cloud) to safely store the data and use it for the next analysis or proposal.

[0399] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0400] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0401] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0402] [Second embodiment]

[0403] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0404] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0405] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0406] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0407] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0408] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0409] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0410] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0411] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0412] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0413] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0414] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0415] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and an embodiment thereof will be described below.

[0416] Data collection methods

[0417] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis.

[0418] Data transmission and analysis

[0419] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0420] Proposal generation and notification

[0421] The server then generates optimal exercise and dietary recommendations based on the analysis results. These recommendations are customized based on the user's current health and emotional state. For example, if the user's sleep quality is deteriorating, the server will suggest relaxing yoga and appropriate eating habits.

[0422] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions. This notification is done using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[0423] Interactive features

[0424] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0425] Specific examples

[0426] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0427] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0428] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[0429] The processing flow will be explained below.

[0430] Step 1:

[0431] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[0432] Step 2:

[0433] The device packages the collected vital and emotional data and periodically transmits it to a server via the internet in JSON format or similar, with the data being temporarily stored in memory.

[0434] Step 3:

[0435] The server analyzes the received data using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[0436] Step 4:

[0437] Based on the analysis results, the server generates optimal exercise and dietary suggestions for the user. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods.

[0438] Step 5:

[0439] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[0440] Step 6:

[0441] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[0442] Step 7:

[0443] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot recently" is sent.

[0444] Step 8:

[0445] The server analyzes the user's questions and concerns and generates appropriate advice and responses, such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data.

[0446] Step 9:

[0447] The server then sends the generated response to the device, which then notifies the user, who receives the suggestions and answers via voice or text and can incorporate them into their daily lives.

[0448] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[0449] Example 1

[0450] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0451] In recent years, there has been a growing emphasis on individual health and emotional management. However, conventional systems lack the ability to adequately collect and analyze data in real time, making it difficult to provide users with appropriate advice. Furthermore, when users have specific questions or concerns, it is difficult to obtain an immediate, appropriate response. Furthermore, there is a lack of means to accurately grasp a user's emotional state. To solve these issues, more comprehensive, real-time data analysis and individualized responses are required.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0453] In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire health data, a transmission means for periodically transmitting the data collected by the data collection means to the server, an analysis means for analyzing the data transmitted to the server by the transmission means and evaluating the user's health condition and emotional state, and a notification means for notifying the user of suggestions generated by the analysis means. This enables real-time data acquisition and analysis, realizing a system that can instantly provide appropriate advice and responses to individual users. It is also possible to analyze the user's facial expressions and voice to grasp their emotional state, allowing for more accurate support.

[0454] A "wearable device" is an electronic device that can be worn by a user and is used to acquire health data and vital signs.

[0455] "Data collection means" refers to a mechanism for measuring the user's health and emotional state using a wearable device and collecting the necessary data.

[0456] The "transmission means" is a function for transmitting the data acquired by the data collection means to the server.

[0457] "Analysis means" refers to a program or algorithm that analyzes the data sent to the server and evaluates the user's health and emotional state.

[0458] The "notification means" is a mechanism for notifying the user of the suggestions and responses generated by the analysis means, and may include a display and a voice synthesis function.

[0459] A "server" is a computer system for storing and analyzing received data.

[0460] A "machine learning model" is an algorithm that learns patterns based on large amounts of data and assesses a user's health status.

[0461] A "database management system" is software that efficiently stores, searches, and manages collected data.

[0462] "Natural Language Generation (NLG)" is a technology that allows machines to generate human language and provide suggestions and responses to users.

[0463] The "voice synthesis function" is a technology for converting text data into voice and notifying the user.

[0464] A "prompt" is a textual instruction given to a generative AI model, serving as a guideline for the model to generate appropriate responses or suggestions.

[0465] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and embodiments thereof will be described below.

[0466] Data collection methods

[0467] The wearable device (terminal) worn by the user collects health data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors (heart rate sensor, thermometer, and accelerometer) that can accurately measure the user's physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis. This data collection method makes it possible to closely monitor both the user's daily health and emotional state.

[0468] Sending data

[0469] The device sends the collected data to the server at regular intervals. This transmission process is carried out using a secure communication protocol (e.g., HTTPS), so the data remains safe.

[0470] Data analysis

[0471] The server stores the transmitted data in a database management system (e.g., SQL database). It then uses machine learning models (e.g., TensorFlow or PyTorch) to analyze the data and evaluate the user's health and emotional state. This analysis uses data mining techniques that compare past health data with newly acquired data. For example, it analyzes heart rate variability patterns and sleep quality to evaluate the user's overall health.

[0472] Proposal generation and notification

[0473] The server generates optimal exercise and dietary recommendations for the user based on the analysis results. These recommendations are generated in a format that is easy for humans to understand using natural language generation (NLG) technology. The generated recommendations are sent from the server to the device, which notifies the user using a display or voice synthesis function. For example, a user whose sleep quality is declining may be shown suggestions for relaxing yoga or a meal.

[0474] Interactive features

[0475] When a user wants to ask a specific question (e.g., "Do you get tired easily?"), they use the device's dialogue function. When the user speaks, their voice is converted into text data and sent to the server. The server analyzes the text data and generates an appropriate response or advice. This response is sent back to the device and notified to the user via the voice synthesis function.

[0476] Specific examples

[0477] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0478] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0479] Prompt Sentence Examples

[0480] Prompt statement:

[0481] When a user says, "I feel like I get tired easily," the following prompt is input to the generative AI model:

[0482] "User input: I feel tired easily.

[0483] Current vital signs: Heart rate 80, body temperature 36.5°C, exercise volume in the past week: low.

[0484] Use this information to generate appropriate advice.

[0485] Generative AI response: "You haven't been exercising much lately. Try taking a light walk in the afternoon."

[0486] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0487] Step 1: Data collection

[0488] The wearable device collects real-time health data such as the user's heart rate, body temperature, and sleep patterns. It also uses a camera and microphone to collect the user's facial expressions and voice. All of this data is collected with high precision using sensors and analytical algorithms.

[0489] Input: User's biometric information (heart rate, body temperature, facial expression, voice, etc.)

[0490] Output: Collected data (biometric data and emotional data)

[0491] Step 2: Send data

[0492] The device sends the collected data to the server at regular intervals, and this process uses a secure communication protocol (e.g., HTTPS) to ensure the safety of the data.

[0493] Input: Collected data

[0494] Output: Data sent

[0495] Step 3: Save Data

[0496] The server receives the transmitted data and stores it in a database management system (e.g., an SQL database). This storage process ensures the consistency and integrity of the data.

[0497] Input: Data sent

[0498] Output: Data stored in the database

[0499] Step 4: Data analysis

[0500] The server analyzes the stored data using machine learning models (e.g., TensorFlow or PyTorch), and uses data mining techniques to compare past health data with newly acquired data and evaluate the user's health and emotional state.

[0501] Input: Data stored in a database, machine learning model

[0502] Output: Analysis results (evaluation of health and emotional state)

[0503] Step 5: Generate proposals

[0504] Based on the analysis results, the server generates optimal exercise and dietary recommendations for the user, using natural language generation (NLG) technology to create recommendations in a format that is easy for the user to understand.

[0505] Input: Analysis results

[0506] Output: Generated proposals

[0507] Step 6: Submit your proposal

[0508] The server then transmits the generated proposal to the device, and this transmission process is also performed using a secure communication protocol.

[0509] Input: Generated proposal

[0510] Output: Submitted proposal

[0511] Step 7: Notification of proposal

[0512] The device notifies the user of the received suggestions using a display or voice synthesis function, allowing the user to confirm and follow the suggestions.

[0513] Input: Submitted proposal

[0514] Output: The suggestion sent to the user

[0515] Step 8: Interactivity

[0516] The user speaks a question or request to the terminal, which converts the speech into text data and sends it to the server.

[0517] Input: User voice input

[0518] Output: Content sent to the server as text data

[0519] Step 9: Analyzing the dialogue

[0520] The server analyzes the received text data and generates appropriate responses and advice, and this analysis process is also carried out using machine learning models and natural language processing techniques.

[0521] Input: Text data

[0522] Output: The response or advice generated

[0523] Step 10: Sending a Response

[0524] The server then generates a response and sends it to the terminal, a process that is also performed securely.

[0525] Input: Generated responses and advice

[0526] Output: Response or advice sent

[0527] Step 11: Notification of response

[0528] The device notifies the user of the response received from the server through a voice synthesis function, allowing the user to receive appropriate support in real time.

[0529] Input: Responses and advice sent

[0530] Output: Response and advice given to the user

[0531] (Application example 1)

[0532] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0533] Conventional store staff health management systems lack the functionality to monitor staff health and emotional state in real time and suggest optimal working styles and break times based on that state. This leads to staff fatigue and stress building up, leading to lower productivity and a worsening workplace atmosphere. Furthermore, because no customized suggestions are made based on the health and emotional state of each staff member, effective health management is not possible.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0535] In this invention, the server includes a data collection means including a wearable device that can be worn by a user to acquire vital data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for notifying the user of suggestions generated by the analysis means, and a means for monitoring the health and emotional states of store staff in real time and suggesting optimal working styles and breaks. This allows for efficient and effective management of the health and emotional states of store staff, reducing staff fatigue and stress, and enabling increased productivity and an improved workplace atmosphere.

[0536] A "wearable device that can be worn by a user" is a device that can be worn directly on the wearer's body and has the function of acquiring vital data.

[0537] "Data collection means" includes wearable devices and is a means for collecting vital data and emotional data of a user.

[0538] The "analysis means" is a means for evaluating the health condition and emotional state of the user based on the data collected by the data collection means, and uses machine learning and data mining techniques.

[0539] The "notification means" is a means for notifying the user of the content of the proposal generated by the analysis means, and is carried out through a display or a voice synthesis function.

[0540] "Store staff" refers to employees who perform work in physical stores.

[0541] "Health status" refers to the physical condition of a user, assessed based on physiological data such as heart rate, body temperature, and sleep patterns.

[0542] "Emotional state" refers to the psychological state of the user that is evaluated based on information obtained from facial expressions and voice.

[0543] A "generative AI model" is an artificial intelligence model used to analyze collected data, assess the user's health and emotional state, and generate optimal suggestions.

[0544] A "prompt" is a sentence to be input to a generative AI model, containing input information that enables the model to generate appropriate suggestions.

[0545] MODE FOR CARRYING OUT THE INVENTION

[0546] The present invention relates to a system for managing the health and emotional states of store staff, and an embodiment thereof will be described below.

[0547] Data collection methods

[0548] The wearable device worn by the user collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable accurate measurement of these physiological indicators. It is also equipped with a camera and microphone, and can capture the user's emotional state through facial expression recognition and voice analysis.

[0549] Data transmission and analysis

[0550] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0551] Proposal generation and notification

[0552] Based on the analysis results, the server generates exercise and dietary suggestions that suggest optimal working styles and break times for the user. These suggestions are customized according to the user's current health and emotional state. For example, if a staff member's stress level is high, the server will suggest a short break or some light exercise to refresh themselves. The generated suggestions are sent from the server to the device. The device then notifies the user of these suggestions. This notification is made using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[0553] Interactive features

[0554] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0555] Specific examples

[0556] When the user wakes up in the morning, the device sends the overnight sleep data to the server. The server analyzes this data and notifies the device with, "Good morning. Let's have another healthy day today." If the user says to the device while at work, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been exercising less recently. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0557] Prompt Sentence Examples

[0558] Use the following data to assess the user's health and emotional state and generate optimal recommendations:

[0559] Heart rate: {heart_rate}

[0560] Body temperature: {body_temperature}

[0561] Sleep pattern: {sleep_pattern}

[0562] Emotional state: {emotion}

[0563] Proposal details:

[0564] The system is built using hardware and software such as wearable devices, smartphones, servers, and machine learning models, enabling real-time health management of store staff and the provision of effective support and suggestions.

[0565] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0566] Step 1:

[0567] The device collects the user's vital data and emotional data.

[0568] Input: User's heart rate, body temperature, sleep patterns, facial expressions, and voice

[0569] Specific operation: The device's built-in high-precision sensors, camera, and microphone collect data in real time.

[0570] Output: Collected vital and emotional data

[0571] Step 2:

[0572] The terminal transmits the collected data to the server.

[0573] Input: Collected vital and emotional data

[0574] Specific operation: Data is sent to the server at regular intervals. HTTP or HTTPS is used as the communication protocol.

[0575] Output: Vital and emotional data sent to the server

[0576] Step 3:

[0577] The server analyzes the received data using a machine learning model.

[0578] Input: Transmitted vital data and emotional data

[0579] How it works: The server uses machine learning models to assess the user's health and emotional state. The algorithm uses data mining techniques.

[0580] Output: Evaluation results on the user's health and emotional state

[0581] Step 4:

[0582] The server generates optimal proposals based on the evaluation results.

[0583] Input: Health and emotional state assessment results

[0584] Specific operation: The server uses a generative AI model to customize appropriate exercise and rest content and generate suggestions.

[0585] Output: Generated proposals

[0586] Step 5:

[0587] The server transmits the generated proposal to the terminal.

[0588] Input: Generated proposal

[0589] Specific operation: The proposals are sent from the server to the device via a communication protocol. The sending timing can be real-time or at regular intervals.

[0590] Output: Suggestions sent to the device

[0591] Step 6:

[0592] The terminal notifies the user of the content of the proposal.

[0593] Input: Proposal sent from the server

[0594] Specific operation: The device notifies the user of the suggestions using a display or voice synthesis.

[0595] Output: User-recognized suggestions

[0596] Step 7:

[0597] The user asks a question or asks for advice via the terminal.

[0598] Input: User's voice or text questions and inquiries

[0599] Specific action: The user speaks into the device or types text.

[0600] Output: Questions and inquiries entered into the device

[0601] Step 8:

[0602] The server receives and analyzes the user's question or inquiry and generates a response.

[0603] Input: Questions and inquiries sent from the device

[0604] Specific operation: The server analyzes the question or consultation content using a machine learning model and generates an appropriate response based on past data and algorithms.

[0605] Output: The generated response

[0606] Step 9:

[0607] The server sends the generated response to the terminal.

[0608] Input: Generated response content

[0609] Specific operation: The response content is sent from the server to the terminal in the same way as the proposal content.

[0610] Output: Response sent to the terminal

[0611] Step 10:

[0612] The terminal notifies the user of the response.

[0613] Input: Response sent from the server

[0614] Specific operation: The terminal notifies the user of the response content using a display or voice synthesis.

[0615] Output: The user's perceived response

[0616] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0617] The present invention relates to a system for efficiently and effectively supporting a user's health management and emotion management, and in particular to a form in which an emotion engine is combined. An embodiment of the system will be described in detail below.

[0618] Data collection methods

[0619] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone that captures the user's facial expressions and tone of voice to collect emotional data.

[0620] Emotion Engine

[0621] The device is equipped with an emotion engine to analyze the collected emotion data. The emotion engine uses an algorithm to classify the user's emotional state based on facial expression data and tone of voice data. Emotional states are classified into multiple categories, such as joy, sadness, anger, and surprise.

[0622] Data transmission and analysis

[0623] The device periodically transmits the collected and analyzed vital and emotional data to a server. The server then analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning algorithms and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0624] Proposal generation and notification

[0625] Based on the analysis results, the server generates recommendations for optimal exercises, dietary habits, and relaxation methods to soothe emotions. These recommendations are customized according to the user's current health and emotional state. For example, if the user's emotional state is classified as "sad," the server will suggest relaxation methods and fun entertainment.

[0626] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions using a display or speech synthesis function in a format that is easy for the user to understand.

[0627] Interactive features

[0628] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0629] Specific examples

[0630] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0631] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0632] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects the emotion of "sadness" from that expression, the server will also make suggestions such as "It would be good to listen to some relaxing music."

[0633] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[0634] The processing flow will be explained below.

[0635] Step 1:

[0636] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[0637] Step 2:

[0638] The device packages the collected vital and emotional data and transmits it to a server over the Internet at regular intervals in JSON format, etc. The data is then temporarily stored in memory.

[0639] Step 3:

[0640] The server analyzes the received data, using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[0641] Step 4:

[0642] The server then generates optimal exercise and diet suggestions for the user based on the analysis results. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods. If the emotion engine evaluates the emotional data, it will also suggest relaxation methods and entertainment options.

[0643] Step 5:

[0644] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[0645] Step 6:

[0646] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[0647] Step 7:

[0648] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot lately" is sent.

[0649] Step 8:

[0650] The server analyzes the user's questions and concerns and generates appropriate advice and responses. For example, it generates a response such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data. The emotion engine is also involved in this process, providing advice tailored to the user's emotional state.

[0651] Step 9:

[0652] The server then sends the generated response to the device, which then notifies the user. The user receives the suggestions and answers via voice or text and can incorporate them into their daily lives. For example, additional suggestions such as "Listen to relaxing music" can also be made.

[0653] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[0654] Example 2

[0655] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] Conventional health management and emotion management systems lacked efficient means for collecting and analyzing users' vital and emotional data in real time. Furthermore, they often failed to provide personalized recommendations based on the user's health and emotional state, preventing improvements in quality of life. Furthermore, they lacked a dialogue function that could provide appropriate responses to users' inquiries in real time. This made it difficult to provide comprehensive support for users' daily lives.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0658] In this invention, the server includes: a data collection means including a sensor device wearable by the user for acquiring physiological data such as heart rate, body temperature, and sleep patterns; a means for acquiring emotional data by using a camera and a microphone to collect the user's facial expressions and tone of voice; an analysis means for analyzing the physiological data and emotional data collected by the data collection means and evaluating the user's health and emotional state; a means for performing a comprehensive analysis based on the collected data using a machine learning algorithm and data mining technology; and a means for notifying the user of suggestions generated by the analysis means. This makes it possible to efficiently manage the user's health and emotions in real time and provide customized suggestions.

[0659] The "data collection means" refers to a means for collecting physiological data and emotional data using a sensor device that can be worn by the user, as well as a camera and microphone.

[0660] A "sensor device" is a wearable device equipped with high-precision sensors that collect physiological data such as heart rate, body temperature, and sleep patterns.

[0661] A "camera" is an image capture device for capturing a user's facial expression and is used to collect emotional data.

[0662] A "microphone" is a voice capture device for capturing the tone of a user's voice and is used to collect emotion data.

[0663] "Emotion data" is data on the emotional state of the user analyzed from facial expressions and tone of voice obtained through a camera and microphone.

[0664] "Physiological data" refers to data on physiological indicators such as heart rate, body temperature, and sleep patterns collected by sensor devices.

[0665] The "analysis means" is a means for analyzing the collected physiological data and emotional data to evaluate the health condition and emotional state of the user.

[0666] A "machine learning algorithm" is an algorithm used to analyze, classify, and predict data based on accumulated data.

[0667] "Data mining technology" is a technique for extracting useful patterns and knowledge from large amounts of data.

[0668] The "suggested content" is information generated by the analysis means about optimal exercises, dietary habits, relaxation methods, etc. for the user.

[0669] The "notification means" is a means including a display, a voice synthesis function, etc., for notifying the user of the generated proposal content.

[0670] The "interactive function" is a function that accepts questions or inquiries from users and notifies the users of a response that is generated based on those questions or inquiries.

[0671] The present invention relates to a system for supporting a user's health management and emotion management, and particularly to a form in which an emotion engine is combined.

[0672] The system uses the following hardware and software:

[0673] Hardware

[0674] 1. Wearable devices (terminals)

[0675] Wearable by the user, it collects physiological data such as heart rate, body temperature, and sleep patterns in real time. High-precision sensors are built in to accurately measure these physiological indicators. It also has a camera and microphone to capture the user's facial expressions and tone of voice, collecting emotional data.

[0676] software

[0677] 1. Emotion Engine (Device)

[0678] The emotion engine analyzes facial expression and tone of voice data collected by the camera and microphone. It uses an algorithm to classify the user's emotional state based on the facial expression and tone of voice data. Emotional states are classified into categories such as joy, sadness, anger, and surprise.

[0679] 2. Data analysis system (server)

[0680] The server receives collected and analyzed data from the device and performs a comprehensive analysis using machine learning algorithms and data mining techniques. The server evaluates the user's health and emotional state and generates recommendations based on that.

[0681] Specific examples

[0682] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes the data and, if it determines that the user's sleep quality is good, generates a suggestion message such as "Good morning. Let's have a healthy day today" and notifies the device.

[0683] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, generates a specific suggestion such as, "You've been getting less exercise lately. Try taking a light walk this afternoon," and sends it to the device. The device then notifies the user of this message by voice.

[0684] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects "sadness" from the expression, the server will make suggestions such as "It would be good to listen to some relaxing music."

[0685] Prompt Sentence Examples

[0686] Analyze your sleep data and let you know the quality of your sleep.

[0687] Generate optimal exercise suggestions when the user reports feeling fatigued.

[0688] Please suggest relaxation methods if you detect "sadness" from the user's facial expression using the emotion engine.

[0689] This makes it possible to provide a system that efficiently and effectively supports users in managing their health and emotions, improving their quality of life.

[0690] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0691] Step 1:

[0692] Data collection

[0693] The device uses a wearable device worn by the user to collect physiological data such as heart rate, body temperature, and sleep patterns in real time, and also uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[0694] Input: User's heart rate, body temperature, sleep patterns, facial expression data, and tone of voice.

[0695] How it works: The wearable device's sensors measure the user's heart rate every minute, the camera captures facial expressions every 10 seconds, and the microphone records the tone of voice in real time.

[0696] Output: Physiological and emotional data.

[0697] Step 2:

[0698] Emotional Data Analysis

[0699] The device analyzes the collected facial expression data and tone of voice data using an emotion engine, which uses this data to classify the user's emotional state into categories such as "joy," "sadness," "anger," and "surprise."

[0700] Input: facial expression data, tone of voice data.

[0701] Specific operation: The emotion engine detects and classifies facial expression patterns of happiness from the collected facial expression data. If the voice tone is low, it is classified as "sad."

[0702] Output: Emotional state category (e.g., happy, sad).

[0703] Step 3:

[0704] Sending data

[0705] The device periodically transmits the collected and analyzed physiological and emotional data to a server, where the encrypted data is transferred over a secure network.

[0706] Input: physiological data, emotional state categories.

[0707] Specific operation: The device collects all data into packets every 30 minutes and sends them to the server.

[0708] Output: The data sent to the server.

[0709] Step 4:

[0710] Data analysis

[0711] The server analyzes the received physiological and emotional data and uses machine learning algorithms and data mining techniques to assess the user's health and emotional state, comparing past data with new data to perform a comprehensive analysis.

[0712] Input: Physiological and emotional data sent to the server.

[0713] What it does: The server compares the past week's data with the current data and flags any abnormal patterns (e.g., a sudden increase in heart rate).

[0714] Output: Assessment of the user's health and emotional state.

[0715] Step 5:

[0716] Proposal generation

[0717] The server generates recommendations for the user based on the results of the data analysis, which are customized according to the user's current health and emotional state.

[0718] Input: User's health and emotional state assessment results.

[0719] Specific behavior: If the user's emotional state is assessed as "sad," generate a list of relaxation techniques and fun entertainment. If data indicates a lack of exercise, suggest an afternoon walk.

[0720] Output: Suggestions (e.g. relaxation techniques, exercise suggestions).

[0721] Step 6:

[0722] Notification of proposal details

[0723] The server sends the generated suggestions to the device, which then notifies the user of the suggestions using a display or voice synthesis function.

[0724] Input: Proposal.

[0725] Specific operation: When the suggestion from the server arrives at the device, the device's display will show the message, "You've been getting less exercise recently. Try taking a light walk this afternoon." The speech synthesis function will read this message aloud.

[0726] Output: Notification to the user.

[0727] Step 7:

[0728] Use of interactive features

[0729] When a user wants to ask a question or seek advice about fatigue or stress, they use the device's dialogue function. What they say to the device is sent as text data to the server, and appropriate advice is generated.

[0730] Input: The user's question or inquiry.

[0731] Specific operation: When a user says to the device, "I get tired easily," the speech is converted into text and sent to the server. The server analyzes the content and generates advice such as, "You've been getting less exercise recently. Try taking a light walk in the afternoon," which is sent to the device. The device then notifies the user by voice.

[0732] Output: Advice or response to the user.

[0733] (Application example 2)

[0734] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0735] Conventional health management systems can assess a user's health and emotional state and make suggestions, but they cannot provide advice or relaxation tailored to the specific circumstances of users performing specific tasks, such as security work. In particular, there was no system that could immediately provide appropriate countermeasures when security guards felt stressed or fatigued. There was also a lack of systems that could provide specific security guidelines through real-time monitoring. This resulted in a decrease in the work efficiency of security guards and an inability to guarantee safety.

[0736] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire vital data and emotional data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for providing security guidelines based on the suggestions generated by the analysis means and the user's current emotional state, and an emergency notification means for recommending relaxation in an emergency. This enables efficient and effective health and emotional management of security guards and provides appropriate countermeasures immediately when they feel stressed or fatigued. Furthermore, providing specific security guidelines based on real-time monitoring improves the efficiency and safety of security operations.

[0737] The "data collection means" is a device that acquires vital data and emotional data using a wearable device that can be worn by the user.

[0738] The "analysis means" is a system for analyzing the data collected by the data collection means and evaluating the health condition and emotional state of the user.

[0739] The "notification means" is a device for providing security guidelines according to the proposal content generated by the analysis means and the current emotional state of the user.

[0740] The "emergency notification means" is a device that recommends appropriate relaxation in real time when the user falls into a state of stress or fatigue.

[0741] A "wearable device" is an electronic device that can acquire vital data and emotional data by being worn by a user.

[0742] "Vital data" refers to data related to physiological indicators such as a user's heart rate, body temperature, and sleep patterns.

[0743] "Emotion data" is data that indicates the user's emotional state based on facial expressions, tone of voice, and the like.

[0744] "Analysis" is the process of assessing the user's health and emotional state based on the collected data.

[0745] "Security guidelines" are specific instructions and advice on security tasks that are provided according to the user's current emotional state.

[0746] "Relaxation" refers to relaxation methods and activities that users should undertake when they feel stressed or tired.

[0747] This invention is a system related to smart glasses worn by security guards that monitors the user's health and emotional state in real time and provides appropriate security guidelines and relaxation methods depending on the situation.

[0748] 1. System Configuration

[0749] Data collection methods

[0750] The smart glasses worn by security guards are equipped with a wearable device for acquiring vital and emotional data. The wearable device is equipped with high-precision sensors, cameras, and microphones to collect real-time data such as heart rate, body temperature, sleep patterns, facial expressions, and tone of voice.

[0751] Analysis means

[0752] The device analyzes the collected data to assess the user's health and emotional state. The collected vital and emotional data is analyzed using machine learning algorithms and data mining techniques to assess the user's condition. Software such as Python and TensorFlow is used for the analysis.

[0753] Notification means

[0754] The device notifies the user of the analysis results. Based on the user's health and emotional state, security guidelines and relaxation techniques are provided. Notification methods include voice synthesis and a display. For example, if a security guard is detected as stressed, the device will display a message on the screen saying, "Take a deep breath."

[0755] Emergency notification means

[0756] The terminal also has an emergency notification function to recommend relaxation in case of an emergency. If a security guard is experiencing excessive stress or fatigue, it will immediately suggest relaxation methods.

[0757] 2. Data submission and analysis process

[0758] The collected data is periodically sent to a server, which analyzes the received data using advanced machine learning algorithms. Based on the results of the data analysis, appropriate advice and guidelines are generated for the user. The server uses cloud services such as AWS and Google Cloud.

[0759] 3. Working Example

[0760] Specific examples

[0761] The smart glasses worn by security guards while on duty use built-in sensors to collect data on heart rate and facial expressions. The collected data is sent to a server in real time for analysis. For example, if a high heart rate and "stress" are detected from facial expressions, the server will generate a guideline for the security guard to "take a deep breath" and display it on the smart glasses' display.

[0762] Input prompt example

[0763] Given the heart rate (90), facial expression (captured image data), and voice tone (neutral), analyze the user's emotional state and provide appropriate advice. For example, if stress is detected, suggest relaxation.

[0764] This method allows for efficient and effective health and emotional management of security guards, improving work efficiency and safety.

[0765] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0766] Step 1:

[0767] The device uses sensors, cameras, and microphones installed in the smart glasses to collect the user's vital data (heart rate, body temperature, sleep patterns, etc.) and emotional data (facial expressions, tone of voice).The input is the user's real-time biological information and environmental sounds, and the output is a form in which these data are temporarily stored.

[0768] Step 2:

[0769] The device sends the collected vital data and emotional data to the analysis means, which then analyzes this data using a machine learning algorithm. Specifically, it uses Python and TensorFlow to analyze changes in heart rate and facial expressions to evaluate the user's health and emotional state. The input is the data acquired in step 1, and the output is the evaluation results of the user's health and emotional state.

[0770] Step 3:

[0771] The server generates appropriate security guidelines and relaxation methods based on the evaluation results sent from the device. The input is the evaluation results, and the output is the generated guidelines and relaxation methods. Specifically, it compares past data with current data and executes an algorithm to make optimal suggestions.

[0772] Step 4:

[0773] The security guidelines and relaxation methods generated by the analysis means are sent to the notification means, which notifies the user. The notification means uses the smart glasses' display and voice synthesis function. The input is the generated guidelines and relaxation methods, and the output is a notification to the user. The specific operation is to convey the generated text and voice to the user visually and audibly.

[0774] Step 5:

[0775] If the user suddenly experiences stress or fatigue, the device will use emergency notification means to immediately suggest relaxation methods. The input is a sudden change in the user's vital signs, and the output is a relaxation suggestion. Specifically, it detects a sudden increase in heart rate or a change in facial expression in real time and immediately notifies the user by suggesting, for example, "Take a deep breath."

[0776] Step 6:

[0777] The results of data analysis and user feedback are stored and managed on a server and are used to improve the accuracy of future data analysis and proposals. The input is the analysis results and feedback data, and the output is an accumulated database. Specifically, the system uses cloud services (AWS or Google Cloud) to safely store the data and use it for the next analysis or proposal.

[0778] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0779] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0780] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0781] [Third embodiment]

[0782] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0783] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0784] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0785] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0786] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0787] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0788] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0789] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0790] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0791] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0792] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0793] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0794] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and an embodiment thereof will be described below.

[0795] Data collection methods

[0796] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis.

[0797] Data transmission and analysis

[0798] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0799] Proposal generation and notification

[0800] The server then generates optimal exercise and dietary recommendations based on the analysis results. These recommendations are customized based on the user's current health and emotional state. For example, if the user's sleep quality is deteriorating, the server will suggest relaxing yoga and appropriate eating habits.

[0801] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions. This notification is done using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[0802] Interactive features

[0803] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0804] Specific examples

[0805] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0806] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0807] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[0808] The processing flow will be explained below.

[0809] Step 1:

[0810] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[0811] Step 2:

[0812] The device packages the collected vital and emotional data and periodically transmits it to a server via the internet in JSON format or similar, with the data being temporarily stored in memory.

[0813] Step 3:

[0814] The server analyzes the received data using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[0815] Step 4:

[0816] Based on the analysis results, the server generates optimal exercise and dietary suggestions for the user. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods.

[0817] Step 5:

[0818] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[0819] Step 6:

[0820] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[0821] Step 7:

[0822] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot recently" is sent.

[0823] Step 8:

[0824] The server analyzes the user's questions and concerns and generates appropriate advice and responses, such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data.

[0825] Step 9:

[0826] The server then sends the generated response to the device, which then notifies the user, who receives the suggestions and answers via voice or text and can incorporate them into their daily lives.

[0827] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[0828] Example 1

[0829] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0830] In recent years, there has been a growing emphasis on individual health and emotional management. However, conventional systems lack the ability to adequately collect and analyze data in real time, making it difficult to provide users with appropriate advice. Furthermore, when users have specific questions or concerns, it is difficult to obtain an immediate, appropriate response. Furthermore, there is a lack of means to accurately grasp a user's emotional state. To solve these issues, more comprehensive, real-time data analysis and individualized responses are required.

[0831] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0832] In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire health data, a transmission means for periodically transmitting the data collected by the data collection means to the server, an analysis means for analyzing the data transmitted to the server by the transmission means and evaluating the user's health condition and emotional state, and a notification means for notifying the user of suggestions generated by the analysis means. This enables real-time data acquisition and analysis, realizing a system that can instantly provide appropriate advice and responses to individual users. It is also possible to analyze the user's facial expressions and voice to grasp their emotional state, allowing for more accurate support.

[0833] A "wearable device" is an electronic device that can be worn by a user and is used to acquire health data and vital signs.

[0834] "Data collection means" refers to a mechanism for measuring the user's health and emotional state using a wearable device and collecting the necessary data.

[0835] The "transmission means" is a function for transmitting the data acquired by the data collection means to the server.

[0836] "Analysis means" refers to a program or algorithm that analyzes the data sent to the server and evaluates the user's health and emotional state.

[0837] The "notification means" is a mechanism for notifying the user of the suggestions and responses generated by the analysis means, and may include a display and a voice synthesis function.

[0838] A "server" is a computer system for storing and analyzing received data.

[0839] A "machine learning model" is an algorithm that learns patterns based on large amounts of data and assesses a user's health status.

[0840] A "database management system" is software that efficiently stores, searches, and manages collected data.

[0841] "Natural Language Generation (NLG)" is a technology that allows machines to generate human language and provide suggestions and responses to users.

[0842] The "voice synthesis function" is a technology for converting text data into voice and notifying the user.

[0843] A "prompt" is a textual instruction given to a generative AI model, serving as a guideline for the model to generate appropriate responses or suggestions.

[0844] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and embodiments thereof will be described below.

[0845] Data collection methods

[0846] The wearable device (terminal) worn by the user collects health data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors (heart rate sensor, thermometer, and accelerometer) that can accurately measure the user's physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis. This data collection method makes it possible to closely monitor both the user's daily health and emotional state.

[0847] Sending data

[0848] The device sends the collected data to the server at regular intervals. This transmission process is carried out using a secure communication protocol (e.g., HTTPS), so the data remains safe.

[0849] Data analysis

[0850] The server stores the transmitted data in a database management system (e.g., SQL database). It then uses machine learning models (e.g., TensorFlow or PyTorch) to analyze the data and evaluate the user's health and emotional state. This analysis uses data mining techniques that compare past health data with newly acquired data. For example, it analyzes heart rate variability patterns and sleep quality to evaluate the user's overall health.

[0851] Proposal generation and notification

[0852] The server generates optimal exercise and dietary recommendations for the user based on the analysis results. These recommendations are generated in a format that is easy for humans to understand using natural language generation (NLG) technology. The generated recommendations are sent from the server to the device, which notifies the user using a display or voice synthesis function. For example, a user whose sleep quality is declining may be shown suggestions for relaxing yoga or a meal.

[0853] Interactive features

[0854] When a user wants to ask a specific question (e.g., "Do you get tired easily?"), they use the device's dialogue function. When the user speaks, their voice is converted into text data and sent to the server. The server analyzes the text data and generates an appropriate response or advice. This response is sent back to the device and notified to the user via the voice synthesis function.

[0855] Specific examples

[0856] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[0857] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0858] Prompt Sentence Examples

[0859] Prompt statement:

[0860] When a user says, "I feel like I get tired easily," the following prompt is input to the generative AI model:

[0861] "User input: I feel tired easily.

[0862] Current vital signs: Heart rate 80, body temperature 36.5°C, exercise volume in the past week: low.

[0863] Use this information to generate appropriate advice.

[0864] Generative AI response: "You haven't been exercising much lately. Try taking a light walk in the afternoon."

[0865] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0866] Step 1: Data collection

[0867] The wearable device collects real-time health data such as the user's heart rate, body temperature, and sleep patterns. It also uses a camera and microphone to collect the user's facial expressions and voice. All of this data is collected with high precision using sensors and analytical algorithms.

[0868] Input: User's biometric information (heart rate, body temperature, facial expression, voice, etc.)

[0869] Output: Collected data (biometric data and emotional data)

[0870] Step 2: Send data

[0871] The device sends the collected data to the server at regular intervals, and this process uses a secure communication protocol (e.g., HTTPS) to ensure the safety of the data.

[0872] Input: Collected data

[0873] Output: Data sent

[0874] Step 3: Save Data

[0875] The server receives the transmitted data and stores it in a database management system (e.g., an SQL database). This storage process ensures the consistency and integrity of the data.

[0876] Input: Data sent

[0877] Output: Data stored in the database

[0878] Step 4: Data analysis

[0879] The server analyzes the stored data using machine learning models (e.g., TensorFlow or PyTorch), specifically by comparing past health data with newly acquired data and using data mining techniques to assess the user's health and emotional state.

[0880] Input: Data stored in a database, machine learning model

[0881] Output: Analysis results (evaluation of health and emotional state)

[0882] Step 5: Generate proposals

[0883] The server then uses the analysis results to generate optimal exercise and dietary recommendations for the user, using natural language generation (NLG) technology to create recommendations in a format that is easy for the user to understand.

[0884] Input: Analysis results

[0885] Output: Generated proposals

[0886] Step 6: Submit your proposal

[0887] The server then transmits the generated proposal to the device, and this transmission process is also performed using a secure communication protocol.

[0888] Input: Generated proposal

[0889] Output: Submitted proposal

[0890] Step 7: Notification of proposal

[0891] The device notifies the user of the received suggestions using a display or voice synthesis function, allowing the user to confirm and follow the suggestions.

[0892] Input: Submitted proposal

[0893] Output: The suggestion sent to the user

[0894] Step 8: Interactivity

[0895] The user speaks a question or request to the terminal, which converts the speech into text data and sends it to the server.

[0896] Input: User voice input

[0897] Output: Content sent to the server as text data

[0898] Step 9: Analyzing the dialogue

[0899] The server analyzes the received text data and generates appropriate responses and advice, and this analysis process is also carried out using machine learning models and natural language processing techniques.

[0900] Input: Text data

[0901] Output: The response or advice generated

[0902] Step 10: Sending a Response

[0903] The server then generates a response and sends it to the terminal, a process that is also performed securely.

[0904] Input: Generated responses and advice

[0905] Output: Response or advice sent

[0906] Step 11: Notification of response

[0907] The device notifies the user of the response received from the server through a voice synthesis function, allowing the user to receive appropriate support in real time.

[0908] Input: Responses and advice sent

[0909] Output: Response and advice given to the user

[0910] (Application example 1)

[0911] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0912] Conventional store staff health management systems lack the functionality to monitor staff health and emotional state in real time and suggest optimal working styles and break times based on that state. This leads to staff fatigue and stress building up, leading to lower productivity and a worsening workplace atmosphere. Furthermore, because no customized suggestions are made based on the health and emotional state of each staff member, effective health management is not possible.

[0913] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0914] In this invention, the server includes a data collection means including a wearable device that can be worn by a user to acquire vital data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for notifying the user of suggestions generated by the analysis means, and a means for monitoring the health and emotional states of store staff in real time and suggesting optimal working styles and breaks. This allows for efficient and effective management of the health and emotional states of store staff, reducing staff fatigue and stress, and enabling increased productivity and an improved workplace atmosphere.

[0915] A "wearable device that can be worn by a user" is a device that can be worn directly on the wearer's body and has the function of acquiring vital data.

[0916] "Data collection means" includes wearable devices and is a means for collecting vital data and emotional data of a user.

[0917] The "analysis means" is a means for evaluating the health condition and emotional state of the user based on the data collected by the data collection means, and uses machine learning and data mining techniques.

[0918] The "notification means" is a means for notifying the user of the content of the proposal generated by the analysis means, and is carried out through a display or a voice synthesis function.

[0919] "Store staff" refers to employees who perform work in physical stores.

[0920] "Health status" refers to the physical condition of a user, assessed based on physiological data such as heart rate, body temperature, and sleep patterns.

[0921] "Emotional state" refers to the psychological state of the user that is evaluated based on information obtained from facial expressions and voice.

[0922] A "generative AI model" is an artificial intelligence model used to analyze collected data, assess the user's health and emotional state, and generate optimal suggestions.

[0923] A "prompt" is a sentence to be input to a generative AI model, containing input information that enables the model to generate appropriate suggestions.

[0924] MODE FOR CARRYING OUT THE INVENTION

[0925] The present invention relates to a system for managing the health and emotional states of store staff, and an embodiment thereof will be described below.

[0926] Data collection methods

[0927] The wearable device worn by the user collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable accurate measurement of these physiological indicators. It is also equipped with a camera and microphone, and can capture the user's emotional state through facial expression recognition and voice analysis.

[0928] Data transmission and analysis

[0929] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[0930] Proposal generation and notification

[0931] Based on the analysis results, the server generates exercise and dietary suggestions that suggest optimal working styles and break times for the user. These suggestions are customized according to the user's current health and emotional state. For example, if a staff member's stress level is high, the server will suggest a short break or some light exercise to refresh themselves. The generated suggestions are sent from the server to the device. The device then notifies the user of these suggestions. This notification is made using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[0932] Interactive features

[0933] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[0934] Specific examples

[0935] When the user wakes up in the morning, the device sends the overnight sleep data to the server. The server analyzes this data and notifies the device with, "Good morning. Let's have another healthy day today." If the user says to the device while at work, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been exercising less recently. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[0936] Prompt Sentence Examples

[0937] Use the following data to assess the user's health and emotional state and generate optimal recommendations:

[0938] Heart rate: {heart_rate}

[0939] Body temperature: {body_temperature}

[0940] Sleep pattern: {sleep_pattern}

[0941] Emotional state: {emotion}

[0942] Proposal details:

[0943] The system is built using hardware and software such as wearable devices, smartphones, servers, and machine learning models, enabling real-time health management of store staff and the provision of effective support and suggestions.

[0944] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0945] Step 1:

[0946] The device collects the user's vital data and emotional data.

[0947] Input: User's heart rate, body temperature, sleep patterns, facial expressions, and voice

[0948] Specific operation: The device's built-in high-precision sensors, camera, and microphone collect data in real time.

[0949] Output: Collected vital and emotional data

[0950] Step 2:

[0951] The terminal transmits the collected data to the server.

[0952] Input: Collected vital and emotional data

[0953] Specific operation: Data is sent to the server at regular intervals. HTTP or HTTPS is used as the communication protocol.

[0954] Output: Vital and emotional data sent to the server

[0955] Step 3:

[0956] The server analyzes the received data using a machine learning model.

[0957] Input: Transmitted vital data and emotional data

[0958] How it works: The server uses machine learning models to assess the user's health and emotional state. The algorithm uses data mining techniques.

[0959] Output: Evaluation results on the user's health and emotional state

[0960] Step 4:

[0961] The server generates optimal proposals based on the evaluation results.

[0962] Input: Health and emotional state assessment results

[0963] Specific operation: The server uses a generative AI model to customize appropriate exercise and rest content and generate suggestions.

[0964] Output: Generated proposals

[0965] Step 5:

[0966] The server transmits the generated proposal to the terminal.

[0967] Input: Generated proposal

[0968] Specific operation: The proposals are sent from the server to the device via a communication protocol. The sending timing can be real-time or at regular intervals.

[0969] Output: Suggestions sent to the device

[0970] Step 6:

[0971] The terminal notifies the user of the content of the proposal.

[0972] Input: Proposal sent from the server

[0973] Specific operation: The device notifies the user of the suggestions using a display or voice synthesis.

[0974] Output: User-recognized suggestions

[0975] Step 7:

[0976] The user asks a question or asks for advice via the terminal.

[0977] Input: User's voice or text questions and inquiries

[0978] Specific action: The user speaks into the device or types text.

[0979] Output: Questions and inquiries entered into the device

[0980] Step 8:

[0981] The server receives and analyzes the user's question or inquiry and generates a response.

[0982] Input: Questions and inquiries sent from the device

[0983] Specific operation: The server analyzes the question or consultation content using a machine learning model and generates an appropriate response based on past data and algorithms.

[0984] Output: The generated response

[0985] Step 9:

[0986] The server sends the generated response to the terminal.

[0987] Input: Generated response content

[0988] Specific operation: The response content is sent from the server to the terminal in the same way as the proposal content.

[0989] Output: Response sent to the terminal

[0990] Step 10:

[0991] The terminal notifies the user of the response.

[0992] Input: Response sent from the server

[0993] Specific operation: The terminal notifies the user of the response content using a display or voice synthesis.

[0994] Output: The user's perceived response

[0995] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0996] The present invention relates to a system for efficiently and effectively supporting a user's health management and emotion management, and in particular to a form in which an emotion engine is combined. An embodiment of the system will be described in detail below.

[0997] Data collection methods

[0998] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone that captures the user's facial expressions and tone of voice to collect emotional data.

[0999] Emotion Engine

[1000] The device is equipped with an emotion engine to analyze the collected emotion data. The emotion engine uses an algorithm to classify the user's emotional state based on facial expression data and tone of voice data. Emotional states are classified into multiple categories, such as joy, sadness, anger, and surprise.

[1001] Data transmission and analysis

[1002] The device periodically transmits the collected and analyzed vital and emotional data to a server. The server then analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning algorithms and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[1003] Proposal generation and notification

[1004] Based on the analysis results, the server generates recommendations for optimal exercises, dietary habits, and relaxation methods to soothe emotions. These recommendations are customized according to the user's current health and emotional state. For example, if the user's emotional state is classified as "sad," the server will suggest relaxation methods and fun entertainment.

[1005] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions using a display or speech synthesis function in a format that is easy for the user to understand.

[1006] Interactive features

[1007] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[1008] Specific examples

[1009] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[1010] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[1011] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects the emotion of "sadness" from that expression, the server will also make suggestions such as "It would be good to listen to some relaxing music."

[1012] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[1013] The processing flow will be explained below.

[1014] Step 1:

[1015] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[1016] Step 2:

[1017] The device packages the collected vital and emotional data and transmits it to a server over the Internet at regular intervals in JSON format or similar, with the data being temporarily stored in memory.

[1018] Step 3:

[1019] The server analyzes the received data using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[1020] Step 4:

[1021] The server then generates optimal exercise and diet suggestions for the user based on the analysis results. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods. If the emotion engine evaluates the emotional data, it will also suggest relaxation methods and entertainment options.

[1022] Step 5:

[1023] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[1024] Step 6:

[1025] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[1026] Step 7:

[1027] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot lately" is sent.

[1028] Step 8:

[1029] The server analyzes the user's questions and concerns and generates appropriate advice and responses. For example, it generates a response such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data. The emotion engine is also involved in this process, providing advice tailored to the user's emotional state.

[1030] Step 9:

[1031] The server then sends the generated response to the device, which then notifies the user. The user receives the suggestions and answers via voice or text and can incorporate them into their daily lives. For example, additional suggestions such as "Listen to relaxing music" can also be made.

[1032] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[1033] Example 2

[1034] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1035] Conventional health management and emotion management systems lacked efficient means for collecting and analyzing users' vital and emotional data in real time. Furthermore, they often failed to provide personalized recommendations based on the user's health and emotional state, preventing improvements in quality of life. Furthermore, they lacked a dialogue function that could provide appropriate responses to users' inquiries in real time. This made it difficult to provide comprehensive support for users' daily lives.

[1036] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1037] In this invention, the server includes: a data collection means including a sensor device wearable by the user for acquiring physiological data such as heart rate, body temperature, and sleep patterns; a means for acquiring emotional data by using a camera and a microphone to collect the user's facial expressions and tone of voice; an analysis means for analyzing the physiological data and emotional data collected by the data collection means and evaluating the user's health and emotional state; a means for performing a comprehensive analysis based on the collected data using a machine learning algorithm and data mining technology; and a means for notifying the user of suggestions generated by the analysis means. This makes it possible to efficiently manage the user's health and emotions in real time and provide customized suggestions.

[1038] The "data collection means" refers to a means for collecting physiological data and emotional data using a sensor device that can be worn by the user, as well as a camera and microphone.

[1039] A "sensor device" is a wearable device equipped with high-precision sensors that collect physiological data such as heart rate, body temperature, and sleep patterns.

[1040] A "camera" is an image capture device for capturing a user's facial expression and is used to collect emotional data.

[1041] A "microphone" is a voice capture device for capturing the tone of a user's voice and is used to collect emotion data.

[1042] "Emotion data" is data on the emotional state of the user analyzed from facial expressions and tone of voice obtained through a camera and microphone.

[1043] "Physiological data" refers to data on physiological indicators such as heart rate, body temperature, and sleep patterns collected by sensor devices.

[1044] The "analysis means" is a means for analyzing the collected physiological data and emotional data to evaluate the health condition and emotional state of the user.

[1045] A "machine learning algorithm" is an algorithm used to analyze, classify, and predict data based on accumulated data.

[1046] "Data mining technology" is a technique for extracting useful patterns and knowledge from large amounts of data.

[1047] The "suggested content" is information generated by the analysis means about optimal exercises, dietary habits, relaxation methods, etc. for the user.

[1048] The "notification means" is a means including a display, a voice synthesis function, etc., for notifying the user of the generated proposal content.

[1049] The "interactive function" is a function that accepts questions or inquiries from users and notifies the users of a response that is generated based on those questions or inquiries.

[1050] The present invention relates to a system for supporting a user's health management and emotion management, and particularly to a form in which an emotion engine is combined.

[1051] The system uses the following hardware and software:

[1052] Hardware

[1053] 1. Wearable devices (terminals)

[1054] Wearable by the user, it collects physiological data such as heart rate, body temperature, and sleep patterns in real time. High-precision sensors are built in to accurately measure these physiological indicators. It also has a camera and microphone to capture the user's facial expressions and tone of voice, collecting emotional data.

[1055] software

[1056] 1. Emotion Engine (Device)

[1057] The emotion engine analyzes facial expression and tone of voice data collected by the camera and microphone. It uses an algorithm to classify the user's emotional state based on the facial expression and tone of voice data. Emotional states are classified into categories such as joy, sadness, anger, and surprise.

[1058] 2. Data analysis system (server)

[1059] The server receives collected and analyzed data from the device and performs a comprehensive analysis using machine learning algorithms and data mining techniques. The server evaluates the user's health and emotional state and generates recommendations based on that.

[1060] Specific examples

[1061] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes the data and, if it determines that the user's sleep quality is good, generates a suggestion message to the device saying, "Good morning. Let's have a healthy day today."

[1062] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, generates a specific suggestion such as, "You've been getting less exercise lately. Try taking a light walk this afternoon," and sends it to the device. The device then notifies the user of this message by voice.

[1063] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects "sadness" from the expression, the server will make suggestions such as "It would be good to listen to some relaxing music."

[1064] Prompt Sentence Examples

[1065] Analyze your sleep data and let you know the quality of your sleep.

[1066] Generate optimal exercise suggestions when the user reports feeling fatigued.

[1067] Please suggest relaxation methods if you detect "sadness" from the user's facial expression using the emotion engine.

[1068] This makes it possible to provide a system that efficiently and effectively supports users in managing their health and emotions, improving their quality of life.

[1069] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1070] Step 1:

[1071] Data collection

[1072] The device uses a wearable device worn by the user to collect physiological data such as heart rate, body temperature, and sleep patterns in real time, and also uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[1073] Input: User's heart rate, body temperature, sleep patterns, facial expression data, and tone of voice.

[1074] How it works: The wearable device's sensors measure the user's heart rate every minute, the camera captures facial expressions every 10 seconds, and the microphone records the tone of voice in real time.

[1075] Output: Physiological and emotional data.

[1076] Step 2:

[1077] Emotional Data Analysis

[1078] The device analyzes the collected facial expression data and tone of voice data using an emotion engine, which uses this data to classify the user's emotional state into categories such as "joy," "sadness," "anger," and "surprise."

[1079] Input: facial expression data, tone of voice data.

[1080] Specific operation: The emotion engine detects and classifies facial expression patterns of happiness from the collected facial expression data. If the voice tone is low, it is classified as "sad."

[1081] Output: Emotional state category (e.g., happy, sad).

[1082] Step 3:

[1083] Sending data

[1084] The device periodically transmits the collected and analyzed physiological and emotional data to a server, where the encrypted data is transferred over a secure network.

[1085] Input: physiological data, emotional state categories.

[1086] Specific operation: The device collects all data into packets every 30 minutes and sends them to the server.

[1087] Output: The data sent to the server.

[1088] Step 4:

[1089] Data analysis

[1090] The server analyzes the received physiological and emotional data and uses machine learning algorithms and data mining techniques to assess the user's health and emotional state, comparing past data with new data to perform a comprehensive analysis.

[1091] Input: Physiological and emotional data sent to the server.

[1092] What it does: The server compares the past week's data with the current data and flags any abnormal patterns (e.g., a sudden increase in heart rate).

[1093] Output: Assessment of the user's health and emotional state.

[1094] Step 5:

[1095] Proposal generation

[1096] The server generates recommendations for the user based on the results of the data analysis, which are customized according to the user's current health and emotional state.

[1097] Input: User's health and emotional state assessment results.

[1098] Specific behavior: If the user's emotional state is assessed as "sad," generate a list of relaxation techniques and fun entertainment. If data indicates a lack of exercise, suggest an afternoon walk.

[1099] Output: Suggestions (e.g. relaxation techniques, exercise suggestions).

[1100] Step 6:

[1101] Notification of proposal details

[1102] The server sends the generated suggestions to the device, which then notifies the user of the suggestions using a display or voice synthesis function.

[1103] Input: Proposal.

[1104] Specific operation: When the suggestion from the server arrives at the device, the device's display will show the message, "You've been getting less exercise recently. Try taking a light walk this afternoon." The speech synthesis function will read this message aloud.

[1105] Output: Notification to the user.

[1106] Step 7:

[1107] Use of interactive features

[1108] When a user wants to ask a question or seek advice about fatigue or stress, they use the device's dialogue function. What they say to the device is sent as text data to the server, and appropriate advice is generated.

[1109] Input: The user's question or inquiry.

[1110] Specific operation: When a user says to the device, "I get tired easily," the speech is converted into text and sent to the server. The server analyzes the content and generates advice such as, "You've been getting less exercise recently. Try taking a light walk in the afternoon," which is sent to the device. The device then notifies the user by voice.

[1111] Output: Advice or response to the user.

[1112] (Application example 2)

[1113] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1114] Conventional health management systems can assess a user's health and emotional state and make suggestions, but they cannot provide advice or relaxation tailored to the specific circumstances of users performing specific tasks, such as security work. In particular, there was no system that could immediately provide appropriate countermeasures when security guards felt stressed or fatigued. There was also a lack of systems that could provide specific security guidelines through real-time monitoring. This resulted in a decrease in the work efficiency of security guards and an inability to guarantee safety.

[1115] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire vital data and emotional data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for providing security guidelines based on the suggestions generated by the analysis means and the user's current emotional state, and an emergency notification means for recommending relaxation in an emergency. This enables efficient and effective health and emotional management of security guards and provides appropriate countermeasures immediately when they feel stressed or fatigued. Furthermore, providing specific security guidelines based on real-time monitoring improves the efficiency and safety of security operations.

[1116] The "data collection means" is a device that acquires vital data and emotional data using a wearable device that can be worn by the user.

[1117] The "analysis means" is a system for analyzing the data collected by the data collection means and evaluating the health condition and emotional state of the user.

[1118] The "notification means" is a device for providing security guidelines according to the proposal content generated by the analysis means and the current emotional state of the user.

[1119] The "emergency notification means" is a device that recommends appropriate relaxation in real time when the user falls into a state of stress or fatigue.

[1120] A "wearable device" is an electronic device that can acquire vital data and emotional data by being worn by a user.

[1121] "Vital data" refers to data related to physiological indicators such as a user's heart rate, body temperature, and sleep patterns.

[1122] "Emotion data" is data that indicates the user's emotional state based on facial expressions, tone of voice, and the like.

[1123] "Analysis" is the process of assessing the user's health and emotional state based on the collected data.

[1124] "Security guidelines" are specific instructions and advice on security tasks that are provided according to the user's current emotional state.

[1125] "Relaxation" refers to relaxation methods and activities that users should undertake when they feel stressed or tired.

[1126] This invention is a system related to smart glasses worn by security guards that monitors the user's health and emotional state in real time and provides appropriate security guidelines and relaxation methods depending on the situation.

[1127] 1. System Configuration

[1128] Data collection methods

[1129] The smart glasses worn by security guards are equipped with a wearable device for acquiring vital and emotional data. The wearable device is equipped with high-precision sensors, cameras, and microphones to collect real-time data such as heart rate, body temperature, sleep patterns, facial expressions, and tone of voice.

[1130] Analysis means

[1131] The device analyzes the collected data to assess the user's health and emotional state. The collected vital and emotional data is analyzed using machine learning algorithms and data mining techniques to assess the user's condition. Software such as Python and TensorFlow is used for the analysis.

[1132] Notification means

[1133] The device notifies the user of the analysis results. Based on the user's health and emotional state, security guidelines and relaxation techniques are provided. Notification methods include voice synthesis and a display. For example, if a security guard is detected as stressed, the device will display a message on the screen saying, "Take a deep breath."

[1134] Emergency notification means

[1135] The terminal also has an emergency notification function to recommend relaxation in case of an emergency. If a security guard is experiencing excessive stress or fatigue, it will immediately suggest relaxation methods.

[1136] 2. Data submission and analysis process

[1137] The collected data is periodically sent to a server, which analyzes the received data using advanced machine learning algorithms. Based on the results of the data analysis, appropriate advice and guidelines are generated for the user. The server uses cloud services such as AWS and Google Cloud.

[1138] 3. Working Example

[1139] Specific examples

[1140] The smart glasses worn by security guards while on duty use built-in sensors to collect data on heart rate and facial expressions. The collected data is sent to a server in real time for analysis. For example, if a high heart rate and "stress" are detected from facial expressions, the server will generate a guideline for the security guard to "take a deep breath" and display it on the smart glasses' display.

[1141] Input prompt example

[1142] Given the heart rate (90), facial expression (captured image data), and voice tone (neutral), analyze the user's emotional state and provide appropriate advice. For example, if stress is detected, suggest relaxation.

[1143] This method allows for efficient and effective health and emotional management of security guards, improving work efficiency and safety.

[1144] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1145] Step 1:

[1146] The device uses sensors, cameras, and microphones installed in the smart glasses to collect the user's vital data (heart rate, body temperature, sleep patterns, etc.) and emotional data (facial expressions, tone of voice).The input is the user's real-time biological information and environmental sounds, and the output is a form in which these data are temporarily stored.

[1147] Step 2:

[1148] The device sends the collected vital data and emotional data to the analysis means, which then analyzes this data using a machine learning algorithm. Specifically, it uses Python and TensorFlow to analyze changes in heart rate and facial expressions to evaluate the user's health and emotional state. The input is the data acquired in step 1, and the output is the evaluation results of the user's health and emotional state.

[1149] Step 3:

[1150] The server generates appropriate security guidelines and relaxation methods based on the evaluation results sent from the device. The input is the evaluation results, and the output is the generated guidelines and relaxation methods. Specifically, it compares past data with current data and executes an algorithm to make optimal suggestions.

[1151] Step 4:

[1152] The security guidelines and relaxation methods generated by the analysis means are sent to the notification means, which notifies the user. The notification means uses the smart glasses' display and voice synthesis function. The input is the generated guidelines and relaxation methods, and the output is a notification to the user. The specific operation is to convey the generated text and voice to the user visually and audibly.

[1153] Step 5:

[1154] If the user suddenly experiences stress or fatigue, the device will use emergency notification means to immediately suggest relaxation methods. The input is a sudden change in the user's vital signs, and the output is a relaxation suggestion. Specifically, it detects a sudden increase in heart rate or a change in facial expression in real time and immediately notifies the user by suggesting, for example, "Take a deep breath."

[1155] Step 6:

[1156] The results of data analysis and user feedback are stored and managed on a server and are used to improve the accuracy of future data analysis and proposals. The input is the analysis results and feedback data, and the output is an accumulated database. Specifically, the system uses cloud services (AWS or Google Cloud) to safely store the data and use it for the next analysis or proposal.

[1157] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1158] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1159] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1160] [Fourth embodiment]

[1161] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1162] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1163] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1165] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1166] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1167] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1168] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1169] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1170] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1171] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1172] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1173] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1174] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and an embodiment thereof will be described below.

[1175] Data collection methods

[1176] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis.

[1177] Data transmission and analysis

[1178] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[1179] Proposal generation and notification

[1180] The server then generates optimal exercise and dietary recommendations based on the analysis results. These recommendations are customized based on the user's current health and emotional state. For example, if the user's sleep quality is deteriorating, the server will suggest relaxing yoga and appropriate eating habits.

[1181] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions. This notification is done using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[1182] Interactive features

[1183] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[1184] Specific examples

[1185] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[1186] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[1187] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[1188] The processing flow will be explained below.

[1189] Step 1:

[1190] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[1191] Step 2:

[1192] The device packages the collected vital and emotional data and periodically transmits it to a server via the internet in JSON format or similar, with the data being temporarily stored in memory.

[1193] Step 3:

[1194] The server analyzes the received data using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[1195] Step 4:

[1196] Based on the analysis results, the server generates optimal exercise and dietary suggestions for the user. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods.

[1197] Step 5:

[1198] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[1199] Step 6:

[1200] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[1201] Step 7:

[1202] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot recently" is sent.

[1203] Step 8:

[1204] The server analyzes the user's questions and concerns and generates appropriate advice and responses, such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data.

[1205] Step 9:

[1206] The server then sends the generated response to the device, which then notifies the user, who receives the suggestions and answers via voice or text and can incorporate them into their daily lives.

[1207] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[1208] Example 1

[1209] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1210] In recent years, there has been a growing emphasis on individual health and emotional management. However, conventional systems lack the ability to adequately collect and analyze data in real time, making it difficult to provide users with appropriate advice. Furthermore, when users have specific questions or concerns, it is difficult to obtain an immediate, appropriate response. Furthermore, there is a lack of means to accurately grasp a user's emotional state. To solve these issues, more comprehensive, real-time data analysis and individualized responses are required.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1212] In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire health data, a transmission means for periodically transmitting the data collected by the data collection means to the server, an analysis means for analyzing the data transmitted to the server by the transmission means and evaluating the user's health condition and emotional state, and a notification means for notifying the user of suggestions generated by the analysis means. This enables real-time data acquisition and analysis, realizing a system that can instantly provide appropriate advice and responses to individual users. It is also possible to analyze the user's facial expressions and voice to grasp their emotional state, allowing for more accurate support.

[1213] A "wearable device" is an electronic device that can be worn by a user and is used to acquire health data and vital signs.

[1214] "Data collection means" refers to a mechanism for measuring the user's health and emotional state using a wearable device and collecting the necessary data.

[1215] The "transmission means" is a function for transmitting the data acquired by the data collection means to the server.

[1216] "Analysis means" refers to a program or algorithm that analyzes the data sent to the server and evaluates the user's health and emotional state.

[1217] The "notification means" is a mechanism for notifying the user of the suggestions and responses generated by the analysis means, and may include a display and a voice synthesis function.

[1218] A "server" is a computer system for storing and analyzing received data.

[1219] A "machine learning model" is an algorithm that learns patterns based on large amounts of data and assesses a user's health status.

[1220] A "database management system" is software that efficiently stores, searches, and manages collected data.

[1221] "Natural Language Generation (NLG)" is a technology that allows machines to generate human language and provide suggestions and responses to users.

[1222] The "voice synthesis function" is a technology for converting text data into voice and notifying the user.

[1223] A "prompt" is a textual instruction given to a generative AI model, serving as a guideline for the model to generate appropriate responses or suggestions.

[1224] The present invention relates to a system that efficiently and effectively supports a user's health management and emotional management, and embodiments thereof will be described below.

[1225] Data collection methods

[1226] The wearable device (terminal) worn by the user collects health data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors (heart rate sensor, thermometer, and accelerometer) that can accurately measure the user's physiological indicators. It is also equipped with a camera and microphone, and can obtain the user's emotional state through facial expression recognition and voice analysis. This data collection method makes it possible to closely monitor both the user's daily health and emotional state.

[1227] Sending data

[1228] The device sends the collected data to the server at regular intervals. This transmission process is carried out using a secure communication protocol (e.g., HTTPS), so the data remains safe.

[1229] Data analysis

[1230] The server stores the transmitted data in a database management system (e.g., SQL database). It then uses machine learning models (e.g., TensorFlow or PyTorch) to analyze the data and evaluate the user's health and emotional state. This analysis uses data mining techniques that compare past health data with newly acquired data. For example, it analyzes heart rate variability patterns and sleep quality to evaluate the user's overall health.

[1231] Proposal generation and notification

[1232] The server generates optimal exercise and dietary recommendations for the user based on the analysis results. These recommendations are generated in a format that is easy for humans to understand using natural language generation (NLG) technology. The generated recommendations are sent from the server to the device, which notifies the user using a display or voice synthesis function. For example, a user whose sleep quality is declining may be shown suggestions for relaxing yoga or a meal.

[1233] Interactive features

[1234] When a user wants to ask a specific question (e.g., "Do you get tired easily?"), they use the device's dialogue function. When the user speaks, their voice is converted into text data and sent to the server. The server analyzes the text data and generates an appropriate response or advice. This response is sent back to the device and notified to the user via the voice synthesis function.

[1235] Specific examples

[1236] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[1237] For example, if a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[1238] Prompt Sentence Examples

[1239] Prompt statement:

[1240] When a user says, "I feel like I get tired easily," the following prompt is input to the generative AI model:

[1241] "User input: I feel tired easily.

[1242] Current vital signs: Heart rate 80, body temperature 36.5°C, exercise volume in the past week: low.

[1243] Use this information to generate appropriate advice.

[1244] Generative AI response: "You haven't been exercising much lately. Try taking a light walk in the afternoon."

[1245] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1246] Step 1: Data collection

[1247] The wearable device collects real-time health data such as the user's heart rate, body temperature, and sleep patterns. It also uses a camera and microphone to collect the user's facial expressions and voice. All of this data is collected with high precision using sensors and analytical algorithms.

[1248] Input: User's biometric information (heart rate, body temperature, facial expression, voice, etc.)

[1249] Output: Collected data (biometric data and emotional data)

[1250] Step 2: Send data

[1251] The device sends the collected data to the server at regular intervals, and this process uses a secure communication protocol (e.g., HTTPS) to ensure the safety of the data.

[1252] Input: Collected data

[1253] Output: Data sent

[1254] Step 3: Save Data

[1255] The server receives the transmitted data and stores it in a database management system (e.g., an SQL database). This storage process ensures the consistency and integrity of the data.

[1256] Input: Data sent

[1257] Output: Data stored in the database

[1258] Step 4: Data analysis

[1259] The server analyzes the stored data using machine learning models (e.g., TensorFlow or PyTorch), specifically by comparing past health data with newly acquired data and using data mining techniques to assess the user's health and emotional state.

[1260] Input: Data stored in a database, machine learning model

[1261] Output: Analysis results (evaluation of health and emotional state)

[1262] Step 5: Generate proposals

[1263] The server then uses the analysis results to generate optimal exercise and dietary recommendations for the user, using natural language generation (NLG) technology to create recommendations in a format that is easy for the user to understand.

[1264] Input: Analysis results

[1265] Output: Generated proposals

[1266] Step 6: Submit your proposal

[1267] The server then transmits the generated proposal to the device, and this transmission process is also performed using a secure communication protocol.

[1268] Input: Generated proposal

[1269] Output: Submitted proposal

[1270] Step 7: Notification of proposal

[1271] The device notifies the user of the received suggestions using a display or voice synthesis function, allowing the user to confirm and follow the suggestions.

[1272] Input: Submitted proposal

[1273] Output: The suggestion sent to the user

[1274] Step 8: Interactivity

[1275] The user speaks a question or request to the terminal, which converts the speech into text data and sends it to the server.

[1276] Input: User voice input

[1277] Output: Content sent to the server as text data

[1278] Step 9: Analyzing the dialogue

[1279] The server analyzes the received text data and generates appropriate responses and advice, and this analysis process is also carried out using machine learning models and natural language processing techniques.

[1280] Input: Text data

[1281] Output: The response or advice generated

[1282] Step 10: Sending a Response

[1283] The server then generates a response and sends it to the terminal, a process that is also performed securely.

[1284] Input: Generated responses and advice

[1285] Output: Response or advice sent

[1286] Step 11: Notification of response

[1287] The device notifies the user of the response received from the server through a voice synthesis function, allowing the user to receive appropriate support in real time.

[1288] Input: Responses and advice sent

[1289] Output: Response and advice given to the user

[1290] (Application example 1)

[1291] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1292] Conventional store staff health management systems lack the functionality to monitor staff health and emotional state in real time and suggest optimal working styles and break times based on that state. This leads to staff fatigue and stress building up, leading to lower productivity and a worsening workplace atmosphere. Furthermore, because no customized suggestions are made based on the health and emotional state of each staff member, effective health management is not possible.

[1293] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1294] In this invention, the server includes a data collection means including a wearable device that can be worn by a user to acquire vital data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for notifying the user of suggestions generated by the analysis means, and a means for monitoring the health and emotional states of store staff in real time and suggesting optimal working styles and breaks. This allows for efficient and effective management of the health and emotional states of store staff, reducing staff fatigue and stress, and enabling increased productivity and an improved workplace atmosphere.

[1295] A "wearable device that can be worn by a user" is a device that can be worn directly on the wearer's body and has the function of acquiring vital data.

[1296] "Data collection means" includes wearable devices and is a means for collecting vital data and emotional data of a user.

[1297] The "analysis means" is a means for evaluating the health condition and emotional state of the user based on the data collected by the data collection means, and uses machine learning and data mining techniques.

[1298] The "notification means" is a means for notifying the user of the content of the proposal generated by the analysis means, and is carried out through a display or a voice synthesis function.

[1299] "Store staff" refers to employees who perform work in physical stores.

[1300] "Health status" refers to the physical condition of a user, assessed based on physiological data such as heart rate, body temperature, and sleep patterns.

[1301] "Emotional state" refers to the psychological state of the user that is evaluated based on information obtained from facial expressions and voice.

[1302] A "generative AI model" is an artificial intelligence model used to analyze collected data, assess the user's health and emotional state, and generate optimal suggestions.

[1303] A "prompt" is a sentence to be input to a generative AI model, containing input information that enables the model to generate appropriate suggestions.

[1304] MODE FOR CARRYING OUT THE INVENTION

[1305] The present invention relates to a system for managing the health and emotional states of store staff, and an embodiment thereof will be described below.

[1306] Data collection methods

[1307] The wearable device worn by the user collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable accurate measurement of these physiological indicators. It is also equipped with a camera and microphone, and can capture the user's emotional state through facial expression recognition and voice analysis.

[1308] Data transmission and analysis

[1309] The device periodically transmits the collected vital and emotional data to a server. The server analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[1310] Proposal generation and notification

[1311] Based on the analysis results, the server generates exercise and dietary suggestions that suggest optimal working styles and break times for the user. These suggestions are customized according to the user's current health and emotional state. For example, if a staff member's stress level is high, the server will suggest a short break or some light exercise to refresh themselves. The generated suggestions are sent from the server to the device. The device then notifies the user of these suggestions. This notification is made using a display or voice synthesis function, and is communicated in a way that is easy for the user to understand.

[1312] Interactive features

[1313] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[1314] Specific examples

[1315] When the user wakes up in the morning, the device sends the overnight sleep data to the server. The server analyzes this data and notifies the device with, "Good morning. Let's have another healthy day today." If the user says to the device while at work, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns and sends specific suggestions to the device, such as, "You've been exercising less recently. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[1316] Prompt Sentence Examples

[1317] Use the following data to assess the user's health and emotional state and generate optimal recommendations:

[1318] Heart rate: {heart_rate}

[1319] Body temperature: {body_temperature}

[1320] Sleep pattern: {sleep_pattern}

[1321] Emotional state: {emotion}

[1322] Proposal details:

[1323] The system is built using hardware and software such as wearable devices, smartphones, servers, and machine learning models, enabling real-time health management of store staff and the provision of effective support and suggestions.

[1324] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1325] Step 1:

[1326] The device collects the user's vital data and emotional data.

[1327] Input: User's heart rate, body temperature, sleep patterns, facial expressions, and voice

[1328] Specific operation: The device's built-in high-precision sensors, camera, and microphone collect data in real time.

[1329] Output: Collected vital and emotional data

[1330] Step 2:

[1331] The terminal transmits the collected data to the server.

[1332] Input: Collected vital and emotional data

[1333] Specific operation: Data is sent to the server at regular intervals. HTTP or HTTPS is used as the communication protocol.

[1334] Output: Vital and emotional data sent to the server

[1335] Step 3:

[1336] The server analyzes the received data using a machine learning model.

[1337] Input: Transmitted vital data and emotional data

[1338] How it works: The server uses machine learning models to assess the user's health and emotional state. The algorithm uses data mining techniques.

[1339] Output: Evaluation results on the user's health and emotional state

[1340] Step 4:

[1341] The server generates optimal proposals based on the evaluation results.

[1342] Input: Health and emotional state assessment results

[1343] Specific operation: The server uses a generative AI model to customize appropriate exercise and rest content and generate suggestions.

[1344] Output: Generated proposals

[1345] Step 5:

[1346] The server transmits the generated proposal to the terminal.

[1347] Input: Generated proposal

[1348] Specific operation: The proposals are sent from the server to the device via a communication protocol. The sending timing can be real-time or at regular intervals.

[1349] Output: Suggestions sent to the device

[1350] Step 6:

[1351] The terminal notifies the user of the content of the proposal.

[1352] Input: Proposal sent from the server

[1353] Specific operation: The device notifies the user of the suggestions using a display or voice synthesis.

[1354] Output: User-recognized suggestions

[1355] Step 7:

[1356] The user asks a question or asks for advice via the terminal.

[1357] Input: User's voice or text questions and inquiries

[1358] Specific action: The user speaks into the device or types text.

[1359] Output: Questions and inquiries entered into the device

[1360] Step 8:

[1361] The server receives and analyzes the user's question or inquiry and generates a response.

[1362] Input: Questions and inquiries sent from the device

[1363] Specific operation: The server analyzes the question or consultation content using a machine learning model and generates an appropriate response based on past data and algorithms.

[1364] Output: The generated response

[1365] Step 9:

[1366] The server sends the generated response to the terminal.

[1367] Input: Generated response content

[1368] Specific operation: The response content is sent from the server to the terminal in the same way as the proposal content.

[1369] Output: Response sent to the terminal

[1370] Step 10:

[1371] The terminal notifies the user of the response.

[1372] Input: Response sent from the server

[1373] Specific operation: The terminal notifies the user of the response content using a display or voice synthesis.

[1374] Output: The user's perceived response

[1375] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1376] The present invention relates to a system for efficiently and effectively supporting a user's health management and emotion management, and in particular to a form in which an emotion engine is combined. An embodiment of the system will be described in detail below.

[1377] Data collection methods

[1378] The device (a wearable device worn by the user) collects vital data such as heart rate, body temperature, and sleep patterns in real time. The device is equipped with high-precision sensors that enable it to accurately measure these physiological indicators. It is also equipped with a camera and microphone that captures the user's facial expressions and tone of voice to collect emotional data.

[1379] Emotion Engine

[1380] The device is equipped with an emotion engine to analyze the collected emotion data. The emotion engine uses an algorithm to classify the user's emotional state based on facial expression data and tone of voice data. Emotional states are classified into multiple categories, such as joy, sadness, anger, and surprise.

[1381] Data transmission and analysis

[1382] The device periodically transmits the collected and analyzed vital and emotional data to a server. The server then analyzes the received data and evaluates the user's health and emotional state. This analysis uses machine learning algorithms and data mining techniques, enabling comprehensive analysis by comparing past data with new data.

[1383] Proposal generation and notification

[1384] Based on the analysis results, the server generates recommendations for optimal exercises, dietary habits, and relaxation methods to soothe emotions. These recommendations are customized according to the user's current health and emotional state. For example, if the user's emotional state is classified as "sad," the server will suggest relaxation methods and fun entertainment.

[1385] The generated suggestions are sent from the server to the device, which then notifies the user of the suggestions using a display or speech synthesis function in a format that is easy for the user to understand.

[1386] Interactive features

[1387] If a user wants to seek specific advice about fatigue, stress, or other issues, they can use the device's dialogue function. When the user speaks a question or request into the device, it is sent as text data to the server. The server analyzes the content, generates appropriate advice or a response, and sends it back to the device. This allows the user to receive appropriate support in real time.

[1388] Specific examples

[1389] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes this data, determines that the user's sleep quality is good, and sends a message to the device saying, "Good morning. Let's have a healthy day today."

[1390] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, and sends specific suggestions to the device, such as, "You've been getting less exercise lately. Try taking a light walk in the afternoon." The device then notifies the user of this message by voice.

[1391] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects the emotion of "sadness" from that expression, the server will also make suggestions such as "It would be good to listen to some relaxing music."

[1392] By combining these steps, the present invention provides a system that comprehensively supports the user's daily life and improves the quality of life.

[1393] The processing flow will be explained below.

[1394] Step 1:

[1395] The device collects vital data such as heart rate, body temperature, and sleep patterns in real time through a wearable device worn by the user, and also uses a camera and microphone to capture the user's facial expressions and voice to collect emotional data.

[1396] Step 2:

[1397] The device packages the collected vital and emotional data and transmits it to a server over the Internet at regular intervals in JSON format or similar, with the data being temporarily stored in memory.

[1398] Step 3:

[1399] The server analyzes the received data using machine learning algorithms and data mining techniques to assess the user's health and emotional state, and compares it with past data to identify trends and patterns.

[1400] Step 4:

[1401] The server then generates optimal exercise and diet suggestions for the user based on the analysis results. For example, if the user's sleep quality is poor, it will recommend relaxing exercises or specific foods. If the emotion engine evaluates the emotional data, it will also suggest relaxation methods and entertainment options.

[1402] Step 5:

[1403] The server then converts the generated proposals back into JSON format and sends them to the device over the network. The resulting proposals include customized proposals that specifically reflect the analysis results.

[1404] Step 6:

[1405] The device will then notify the user of the received suggestions, using a display or voice synthesis to communicate the information in a way that is easy for the user to understand. For example, a message such as "Good morning. Have a healthy day today" will be displayed.

[1406] Step 7:

[1407] When a user uses the dialogue function to ask a question or ask for advice, their voice input is converted into text and sent from the device to the server. For example, a question such as "I feel like I've been getting tired a lot lately" is sent.

[1408] Step 8:

[1409] The server analyzes the user's questions and concerns and generates appropriate advice and responses. For example, it generates a response such as "Try taking a light walk in the afternoon" based on the user's behavioral patterns and health data. The emotion engine is also involved in this process, providing advice tailored to the user's emotional state.

[1410] Step 9:

[1411] The server then sends the generated response to the device, which then notifies the user. The user receives the suggestions and answers via voice or text and can incorporate them into their daily lives. For example, additional suggestions such as "Listen to relaxing music" can also be made.

[1412] Through this series of steps, users receive ongoing support and advice to improve their health and quality of life.

[1413] Example 2

[1414] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1415] Conventional health management and emotion management systems lacked efficient means for collecting and analyzing users' vital and emotional data in real time. Furthermore, they often failed to provide personalized recommendations based on the user's health and emotional state, preventing improvements in quality of life. Furthermore, they lacked a dialogue function that could provide appropriate responses to users' inquiries in real time. This made it difficult to provide comprehensive support for users' daily lives.

[1416] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1417] In this invention, the server includes: a data collection means including a sensor device wearable by the user for acquiring physiological data such as heart rate, body temperature, and sleep patterns; a means for acquiring emotional data by using a camera and a microphone to collect the user's facial expressions and tone of voice; an analysis means for analyzing the physiological data and emotional data collected by the data collection means and evaluating the user's health and emotional state; a means for performing a comprehensive analysis based on the collected data using a machine learning algorithm and data mining technology; and a means for notifying the user of suggestions generated by the analysis means. This makes it possible to efficiently manage the user's health and emotions in real time and provide customized suggestions.

[1418] The "data collection means" refers to a means for collecting physiological data and emotional data using a sensor device that can be worn by the user, as well as a camera and microphone.

[1419] A "sensor device" is a wearable device equipped with high-precision sensors that collect physiological data such as heart rate, body temperature, and sleep patterns.

[1420] A "camera" is an image capture device for capturing a user's facial expression and is used to collect emotional data.

[1421] A "microphone" is a voice capture device for capturing the tone of a user's voice and is used to collect emotion data.

[1422] "Emotion data" is data on the emotional state of the user analyzed from facial expressions and tone of voice obtained through a camera and microphone.

[1423] "Physiological data" refers to data on physiological indicators such as heart rate, body temperature, and sleep patterns collected by sensor devices.

[1424] The "analysis means" is a means for analyzing the collected physiological data and emotional data to evaluate the health condition and emotional state of the user.

[1425] A "machine learning algorithm" is an algorithm used to analyze, classify, and predict data based on accumulated data.

[1426] "Data mining technology" is a technique for extracting useful patterns and knowledge from large amounts of data.

[1427] The "suggested content" is information generated by the analysis means about optimal exercises, dietary habits, relaxation methods, etc. for the user.

[1428] The "notification means" is a means including a display, a voice synthesis function, etc., for notifying the user of the generated proposal content.

[1429] The "interactive function" is a function that accepts questions or inquiries from users and notifies the users of a response that is generated based on those questions or inquiries.

[1430] The present invention relates to a system for supporting a user's health management and emotion management, and particularly to a form in which an emotion engine is combined.

[1431] The system uses the following hardware and software:

[1432] Hardware

[1433] 1. Wearable devices (terminals)

[1434] Wearable by the user, it collects physiological data such as heart rate, body temperature, and sleep patterns in real time. High-precision sensors are built in to accurately measure these physiological indicators. It also has a camera and microphone to capture the user's facial expressions and tone of voice, collecting emotional data.

[1435] software

[1436] 1. Emotion Engine (Device)

[1437] The emotion engine analyzes facial expression and tone of voice data collected by the camera and microphone. It uses an algorithm to classify the user's emotional state based on the facial expression and tone of voice data. Emotional states are classified into categories such as joy, sadness, anger, and surprise.

[1438] 2. Data analysis system (server)

[1439] The server receives collected and analyzed data from the device and performs a comprehensive analysis using machine learning algorithms and data mining techniques. The server evaluates the user's health and emotional state and generates recommendations based on that.

[1440] Specific examples

[1441] When the user wakes up in the morning, the device sends the night's sleep data to the server. The server analyzes the data and, if it determines that the user's sleep quality is good, generates a suggestion message to the device saying, "Good morning. Let's have a healthy day today."

[1442] When a user says to the device in the afternoon, "I feel like I get tired easily," the device converts the speech into text and sends it to the server. The server analyzes the user's vital data and behavioral patterns, generates a specific suggestion such as, "You've been getting less exercise lately. Try taking a light walk this afternoon," and sends it to the device. The device then notifies the user of this message by voice.

[1443] Furthermore, if the user's facial expression is captured by a camera and the emotion engine detects "sadness" from the expression, the server will make suggestions such as "It would be good to listen to some relaxing music."

[1444] Prompt Sentence Examples

[1445] Analyze your sleep data and let you know the quality of your sleep.

[1446] Generate optimal exercise suggestions when the user reports feeling fatigued.

[1447] Please suggest relaxation methods if you detect "sadness" from the user's facial expression using the emotion engine.

[1448] This makes it possible to provide a system that efficiently and effectively supports users in managing their health and emotions, improving their quality of life.

[1449] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1450] Step 1:

[1451] Data collection

[1452] The device uses a wearable device worn by the user to collect physiological data such as heart rate, body temperature, and sleep patterns in real time, and also uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[1453] Input: User's heart rate, body temperature, sleep patterns, facial expression data, and tone of voice.

[1454] How it works: The wearable device's sensors measure the user's heart rate every minute, the camera captures facial expressions every 10 seconds, and the microphone records the tone of voice in real time.

[1455] Output: Physiological and emotional data.

[1456] Step 2:

[1457] Emotional Data Analysis

[1458] The device analyzes the collected facial expression data and tone of voice data using an emotion engine, which uses this data to classify the user's emotional state into categories such as "joy," "sadness," "anger," and "surprise."

[1459] Input: facial expression data, tone of voice data.

[1460] Specific operation: The emotion engine detects and classifies facial expression patterns of happiness from the collected facial expression data. If the voice tone is low, it is classified as "sad."

[1461] Output: Emotional state category (e.g., happy, sad).

[1462] Step 3:

[1463] Sending data

[1464] The device periodically transmits the collected and analyzed physiological and emotional data to a server, where the encrypted data is transferred over a secure network.

[1465] Input: physiological data, emotional state categories.

[1466] Specific operation: The device collects all data into packets every 30 minutes and sends them to the server.

[1467] Output: The data sent to the server.

[1468] Step 4:

[1469] Data analysis

[1470] The server analyzes the received physiological and emotional data and uses machine learning algorithms and data mining techniques to assess the user's health and emotional state, comparing past data with new data to perform a comprehensive analysis.

[1471] Input: Physiological and emotional data sent to the server.

[1472] What it does: The server compares the past week's data with the current data and flags any abnormal patterns (e.g., a sudden increase in heart rate).

[1473] Output: Assessment of the user's health and emotional state.

[1474] Step 5:

[1475] Proposal generation

[1476] The server generates recommendations for the user based on the results of the data analysis, which are customized according to the user's current health and emotional state.

[1477] Input: User's health and emotional state assessment results.

[1478] Specific behavior: If the user's emotional state is assessed as "sad," generate a list of relaxation techniques and fun entertainment. If data indicates a lack of exercise, suggest an afternoon walk.

[1479] Output: Suggestions (e.g. relaxation techniques, exercise suggestions).

[1480] Step 6:

[1481] Notification of proposal details

[1482] The server sends the generated suggestions to the device, which then notifies the user of the suggestions using a display or voice synthesis function.

[1483] Input: Proposal.

[1484] Specific operation: When the suggestion from the server arrives at the device, the device's display will show the message, "You've been getting less exercise recently. Try taking a light walk this afternoon." The speech synthesis function will read this message aloud.

[1485] Output: Notification to the user.

[1486] Step 7:

[1487] Use of interactive features

[1488] When a user wants to ask a question or seek advice about fatigue or stress, they use the device's dialogue function. What they say to the device is sent as text data to the server, and appropriate advice is generated.

[1489] Input: The user's question or inquiry.

[1490] Specific operation: When a user says to the device, "I get tired easily," the speech is converted into text and sent to the server. The server analyzes the content and generates advice such as, "You've been getting less exercise recently. Try taking a light walk in the afternoon," which is sent to the device. The device then notifies the user by voice.

[1491] Output: Advice or response to the user.

[1492] (Application example 2)

[1493] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1494] Conventional health management systems can assess a user's health and emotional state and make suggestions, but they cannot provide advice or relaxation tailored to the specific circumstances of users performing specific tasks, such as security work. In particular, there was no system that could immediately provide appropriate countermeasures when security guards felt stressed or fatigued. There was also a lack of systems that could provide specific security guidelines through real-time monitoring. This resulted in a decrease in the work efficiency of security guards and an inability to guarantee safety.

[1495] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means including a wearable device that can be worn by the user to acquire vital data and emotional data, an analysis means for analyzing the data collected by the data collection means and evaluating the user's health and emotional state, a notification means for providing security guidelines based on the suggestions generated by the analysis means and the user's current emotional state, and an emergency notification means for recommending relaxation in an emergency. This enables efficient and effective health and emotional management of security guards and provides appropriate countermeasures immediately when they feel stressed or fatigued. Furthermore, providing specific security guidelines based on real-time monitoring improves the efficiency and safety of security operations.

[1496] The "data collection means" is a device that acquires vital data and emotional data using a wearable device that can be worn by the user.

[1497] The "analysis means" is a system for analyzing the data collected by the data collection means and evaluating the health condition and emotional state of the user.

[1498] The "notification means" is a device for providing security guidelines according to the proposal content generated by the analysis means and the current emotional state of the user.

[1499] The "emergency notification means" is a device that recommends appropriate relaxation in real time when the user falls into a state of stress or fatigue.

[1500] A "wearable device" is an electronic device that can acquire vital data and emotional data by being worn by a user.

[1501] "Vital data" refers to data related to physiological indicators such as the user's heart rate, body temperature, and sleep patterns.

[1502] "Emotion data" is data that indicates the user's emotional state based on facial expressions, tone of voice, etc.

[1503] "Analysis" is the process of assessing the user's health and emotional state based on the collected data.

[1504] "Security guidelines" are specific instructions and advice on security tasks that are provided according to the user's current emotional state.

[1505] "Relaxation" refers to relaxation methods and activities that users should undertake when they feel stressed or tired.

[1506] This invention is a system related to smart glasses worn by security guards that monitors the user's health and emotional state in real time and provides appropriate security guidelines and relaxation methods depending on the situation.

[1507] 1. System Configuration

[1508] Data collection methods

[1509] The smart glasses worn by security guards are equipped with a wearable device for acquiring vital and emotional data. The wearable device is equipped with high-precision sensors, cameras, and microphones to collect real-time data such as heart rate, body temperature, sleep patterns, facial expressions, and tone of voice.

[1510] Analysis means

[1511] The device analyzes the collected data to assess the user's health and emotional state. The collected vital and emotional data is analyzed using machine learning algorithms and data mining techniques to assess the user's condition. Software such as Python and TensorFlow is used for the analysis.

[1512] Notification means

[1513] The device notifies the user of the analysis results. Based on the user's health and emotional state, security guidelines and relaxation techniques are provided. Notification methods include voice synthesis and a display. For example, if a security guard is detected as stressed, the device will display a message on the screen saying, "Take a deep breath."

[1514] Emergency notification means

[1515] The terminal also has an emergency notification function to recommend relaxation in case of an emergency. If a security guard is experiencing excessive stress or fatigue, it will immediately suggest relaxation methods.

[1516] 2. Data submission and analysis process

[1517] The collected data is periodically sent to a server, which analyzes the received data using advanced machine learning algorithms. Based on the results of the data analysis, appropriate advice and guidelines are generated for the user. The server uses cloud services such as AWS and Google Cloud.

[1518] 3. Working Example

[1519] Specific examples

[1520] The smart glasses worn by security guards while on duty use built-in sensors to collect data on heart rate and facial expressions. The collected data is sent to a server in real time for analysis. For example, if a high heart rate and "stress" are detected from facial expressions, the server will generate a guideline for the security guard to "take a deep breath" and display it on the smart glasses' display.

[1521] Input prompt statement example

[1522] Given the heart rate (90), facial expression (captured image data), and voice tone (neutral), analyze the user's emotional state and provide appropriate advice. For example, if stress is detected, suggest relaxation.

[1523] This method allows for efficient and effective health and emotional management of security guards, improving work efficiency and safety.

[1524] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1525] Step 1:

[1526] The device uses sensors, cameras, and microphones installed in the smart glasses to collect the user's vital data (heart rate, body temperature, sleep patterns, etc.) and emotional data (facial expressions, tone of voice).The input is the user's real-time biological information and environmental sounds, and the output is a form in which these data are temporarily stored.

[1527] Step 2:

[1528] The device sends the collected vital data and emotional data to the analysis means, which then analyzes this data using a machine learning algorithm. Specifically, it uses Python and TensorFlow to analyze changes in heart rate and facial expressions to evaluate the user's health and emotional state. The input is the data acquired in step 1, and the output is the evaluation results of the user's health and emotional state.

[1529] Step 3:

[1530] The server generates appropriate security guidelines and relaxation methods based on the evaluation results sent from the device. The input is the evaluation results, and the output is the generated guidelines and relaxation methods. Specifically, it compares past data with current data and executes an algorithm to make optimal suggestions.

[1531] Step 4:

[1532] The security guidelines and relaxation methods generated by the analysis means are sent to the notification means, which notifies the user. The notification means uses the smart glasses' display and voice synthesis function. The input is the generated guidelines and relaxation methods, and the output is a notification to the user. The specific operation is to convey the generated text and voice to the user visually and audibly.

[1533] Step 5:

[1534] If the user suddenly experiences stress or fatigue, the device will use emergency notification means to immediately suggest relaxation methods. The input is a sudden change in the user's vital signs, and the output is a relaxation suggestion. Specifically, it detects a sudden increase in heart rate or a change in facial expression in real time and immediately notifies the user by suggesting, for example, "Take a deep breath."

[1535] Step 6:

[1536] The results of data analysis and user feedback are stored and managed on a server and are used to improve the accuracy of future data analysis and proposals. The input is the analysis results and feedback data, and the output is an accumulated database. Specifically, the system uses cloud services (AWS or Google Cloud) to safely store the data and use it for the next analysis or proposal.

[1537] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1538] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1539] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1540] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1541] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1542] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1543] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1544] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1545] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1546] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1547] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1548] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1549] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1550] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1551] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1552] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1553] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1554] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1555] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1556] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1557] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1558] The following is further disclosed regarding the above embodiment.

[1559] (Claim 1)

[1560] a data collection means including a wearable device that can be worn by a user and that collects vital data;

[1561] analysis means for analyzing the data collected by the data collection means and evaluating the health and emotional state of the user;

[1562] notification means for notifying the user of the proposal content generated by the analysis means;

[1563] A system including:

[1564] (Claim 2)

[1565] 2. The system according to claim 1, wherein said notification means has an interactive function of accepting a question or inquiry from a user and notifying the user of a response generated based on the question or inquiry.

[1566] (Claim 3)

[1567] 2. The system of claim 1, wherein the data collection means includes a camera and a microphone for analyzing the user's facial expressions and voice to obtain emotion data.

[1568] "Example 1"

[1569] (Claim 1)

[1570] a data collection means including a wearable device that can be worn by a user to acquire health data;

[1571] a transmitting means for transmitting the data collected by the data collecting means to a server at regular intervals;

[1572] analysis means for analyzing the data transmitted to the server by the transmission means and evaluating the health condition and emotional state of the user;

[1573] notification means for notifying the user of the proposal content generated by the analysis means;

[1574] A system including:

[1575] (Claim 2)

[1576] 2. The system according to claim 1, wherein said notification means has an interactive function of accepting a question or inquiry from a user and notifying the user of a response generated based on the question or inquiry.

[1577] (Claim 3)

[1578] 2. The system of claim 1, wherein the data collection means includes a camera and a microphone for analyzing the user's facial expressions and voice to obtain emotion data.

[1579] (Claim 4)

[1580] 2. The system according to claim 1, wherein the analysis means has a function of storing data in cooperation with a database management system and analyzing the data using a machine learning model.

[1581] (Claim 5)

[1582] 2. The system according to claim 1, wherein the notification means has a function of notifying the user of the suggestion content using a display and a voice synthesis function.

[1583] "Application Example 1"

[1584] (Claim 1)

[1585] a data collection means including a wearable device that can be worn by a user and that collects vital data;

[1586] analysis means for analyzing the data collected by the data collection means and evaluating the health and emotional state of the user;

[1587] notification means for notifying the user of the proposal content generated by the analysis means;

[1588] A means to monitor the health and emotional state of store staff in real time and suggest optimal working styles and breaks,

[1589] A system including:

[1590] (Claim 2)

[1591] the notification means has an interactive function of accepting a question or inquiry from a user and notifying the user of a response generated based on the question or inquiry,

[1592] Providing optimal break times for store staff when they feel fatigued or stressed during work hours,

[1593] 10. The system of claim 1.

[1594] (Claim 3)

[1595] the data collection means includes a camera and a microphone for analyzing the user's facial expressions and voice to acquire emotion data;

[1596] This includes a generative AI model that assesses health and emotional states using prompts generated from collected data.

[1597] 10. The system of claim 1.

[1598] "Example 2: Combining Emotion Engines"

[1599] (Claim 1)

[1600] a data collection means including a sensor device wearable by a user for acquiring physiological data such as heart rate, body temperature, and sleep patterns;

[1601] means for collecting facial expressions and tone of voice of a user using a camera and a microphone to obtain emotion data;

[1602] analysis means for analyzing the physiological data and emotional data collected by the data collection means and evaluating the health condition and emotional state of the user;

[1603] A means of conducting comprehensive analysis based on the collected data using machine learning algorithms and data mining techniques;

[1604] means for notifying a user of the content of the proposal generated by the analysis means;

[1605] A system including:

[1606] (Claim 2)

[1607] 2. The system according to claim 1, wherein the notification means has an interactive function of accepting a question or inquiry from a user and notifying the user of a response generated based on the question or inquiry.

[1608] (Claim 3)

[1609] 10. The system of claim 1, comprising a camera and a microphone for analyzing a user's facial expressions and voice to obtain emotion data.

[1610] "Application example 2 when combining emotion engines"

[1611] (Claim 1)

[1612] a data collection means including a wearable device that can be worn by a user to acquire vital data and emotional data;

[1613] analysis means for analyzing the data collected by the data collection means and evaluating the health and emotional state of the user;

[1614] notification means for providing security guidelines according to the proposal content generated by the analysis means and the current emotional state of the user;

[1615] an emergency notification means for advising relaxation in an emergency;

[1616] A system including:

[1617] (Claim 2)

[1618] 2. The system according to claim 1, wherein said notification means has an interactive function of accepting a question or inquiry from a user and notifying the user of a response generated based on the question or inquiry.

[1619] (Claim 3)

[1620] 2. The system according to claim 1, wherein the data collection means includes a photographing device and a voice collection device for analyzing the user's facial expressions and voice to obtain emotion data. [Explanation of symbols]

[1621] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a data collection means including a wearable device that can be worn by a user and that collects vital data; analysis means for analyzing the data collected by the data collection means and evaluating the health and emotional state of the user; notification means for notifying the user of the proposal content generated by the analysis means; A system including:

2. 2. The system according to claim 1, wherein said notification means has an interactive function of accepting a question or inquiry from a user and notifying the user of a response generated based on the question or inquiry.

3. 2. The system of claim 1, wherein the data collection means includes a camera and a microphone for analyzing the user's facial expressions and voice to obtain emotion data.

Images