system

Abstract

We provide the system. [Solution] A sensor means for acquiring biological information, A means of acquiring communication information, An analysis means for analyzing the aforementioned biological information and communication information and determining the emotional state, A generation means for generating an alert based on the aforementioned emotional state, A notification means for outputting the aforementioned alert, A system that includes this.

Description

Technical Field

[0004] , , ,

[0005] , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Due to recent workstyle reforms and the spread of remote work, it has become difficult to grasp the emotional states and stress levels of individual employees within a company. In particular, if a high-stress state continues, it may cause a decrease in employee productivity and an increase in resignations, which may result in significant losses for the company. Additionally, there is also a problem in that employees themselves lack support for self-awareness and coping with their own emotional states. As a result, there is a need for a new method to improve work efficiency while maintaining the health of employees.

Means for Solving the Problems

[0005] To solve this problem, the present invention enables the collection of employees' biometric information and daily communication data by using sensor means for acquiring biometric information and acquisition means for acquiring communication information. Furthermore, it includes generation means that determines the emotional state via analysis means that analyzes the biometric information and communication information in real time and generates alerts according to the results. By notifying employees of the generated alerts, the system provides immediate feedback and enables them to receive timely support as needed. This allows companies to appropriately monitor the emotional state of their employees and take prompt action.

[0006] "Biometric information" refers to physiological data obtained from the human body, such as heart rate, body temperature, and activity level.

[0007] "Sensor means" refers to a device or apparatus used to detect and acquire biological information.

[0008] "Communication information" refers to information related to individual communications, such as text data and audio data obtained through email, business chat, and voice calls.

[0009] "Means of acquisition" refers to the technology or system for collecting communication information and storing it in an analyzable format.

[0010] "Analysis means" refers to algorithms and technologies used to process collected biometric and communication information and evaluate the user's emotional state.

[0011] "Generation means" refers to a system or module that has the function of creating alerts and notifications based on analysis results.

[0012] "Notification means" refers to a method or device for communicating generated alerts to the user.

[0013] "Emotional state" refers to the user's psychological state as determined by analytical methods, and can be expressed numerically or as an indicator, for example, as the level of stress or happiness. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a RAM (Random Access Memory) with a reference number is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a storage with a reference number is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] To implement the present invention, the following configuration is conceivable. First, the terminal periodically acquires biometric information through a wearable device worn by the user. This includes data such as heart rate, body temperature, and activity level. This information is transmitted to the server in real time.

[0036] Simultaneously, users communicate daily through email, business chat tools, and online meetings. The device acquires text and audio data generated from these communication activities. This acquired data is also sent to the server.

[0037] The server analyzes the user's physical health status and activity patterns based on the received biometric information. Specifically, it refers to historical biometric data and detects values ​​and patterns that deviate from the normal range. The server also uses natural language processing (NLP) and speech analysis techniques to analyze communication data and determine the user's emotional state. This takes into account factors such as word choice, tone of voice, and frequency of speech in communication, and is expressed as a positive, negative, or neutral emotional score.

[0038] Based on the analysis results, the server generates an alert if the emotional state exceeds a certain threshold or if high stress levels are suspected based on biometric information. Specifically, this alert is a message summarizing cautionary advice and recommended actions, and is sent to the terminal via a notification system.

[0039] The terminal displays alerts sent from the server to the user. Ideally, the system should provide an interface that allows the user to access additional support and counseling services as needed. This enables the user to recognize their own emotions and stress levels and take appropriate action.

[0040] As a concrete example, suppose a user is under pressure to meet a project deadline and starts making more negative statements such as "I'm tired" or "I can't do this anymore." At this time, the heart rate transmitted from the wearable device tends to be higher than usual. The server quickly analyzes this data, determines that stress levels are high, and sends an alert to the device saying, "You may be experiencing high stress levels recently. Take a rest and seek counseling if necessary." The user receives this notification, becomes aware of their condition, and can take appropriate action.

[0041] Thus, the system of the present invention makes a significant contribution to understanding the user's health status and providing appropriate support.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The device acquires biometric information such as heart rate, body temperature, and activity level from the wearable device worn by the user at regular intervals. This data is transmitted in real time to the server via the cloud.

[0045] Step 2:

[0046] Users utilize email, business chat tools, and online meetings in their daily work. Their devices record text and audio data generated from these communication activities and send it to a server.

[0047] Step 3:

[0048] The server receives biometric information transmitted from the terminal and identifies anomalies by comparing it to a database with a set normal range. In this process, it detects rapid changes in heart rate and abnormal patterns in activity levels.

[0049] Step 4:

[0050] The server uses natural language processing (NLP) to analyze the user's communication data. Based on the content and tone of their speech, it calculates a sentiment score and determines whether the user is positive, negative, or neutral.

[0051] Step 5:

[0052] The server integrates biometric data and analyzed emotion scores to evaluate the user's overall emotional state. If the emotion score exceeds a set threshold and abnormalities are detected in the biometric data, the server generates a warning.

[0053] Step 6:

[0054] The server sends the generated warning as a message to the terminal. The message includes a warning about increased stress and recommended actions (e.g., taking a break, seeking counseling).

[0055] Step 7:

[0056] The terminal displays received messages to the user. Through these messages, the user can understand their current situation and, if necessary, access support services provided through the system.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] In modern society, there are concerns about the impact of daily stress and emotional fluctuations on health. However, conventional technologies have lacked the means to monitor an individual's biometric data and emotional state in real time and provide appropriate support based on that data. As a result, there is a challenge in that it is difficult for users to accurately understand their own health status and quickly access the support they need.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes a sensor device for acquiring biosignals, a data acquisition device for acquiring communication data, and a data analysis device for integrating the biosignals and communication data and identifying the emotional state. This makes it possible to accurately grasp the user's health and emotional state in real time and provide alerts at the appropriate time.

[0062] A "sensor device" is a device that detects biological signals and records and transmits that information.

[0063] A "data collection device" is a device that acquires communication data and stores it for analysis.

[0064] A "data analysis device" is a device that integrates and analyzes biological signals and communication data to identify emotional states.

[0065] An "alert generator" is a device that generates warnings and recommendations to notify users based on their emotional state, as identified by a data analysis device.

[0066] A "notification device" is a device that has an interface for communicating generated alerts to the user.

[0067] A "device that provides an interface" is a device that has the functionality to allow users to access additional support services.

[0068] "Biosignals" refer to physiological parameters of the body, such as heart rate and body temperature, that are acquired through wearable devices.

[0069] "Communication data" refers to data including text and audio information obtained from users' emails, chats, and voice calls.

[0070] To implement the invention, it is necessary to configure a system that links a server, a terminal, and a wearable device. First, the terminal periodically acquires biometric signals such as heart rate, body temperature, and activity level from the wearable device worn by the user. To do this, communication technologies such as Bluetooth and Wi-Fi are used to stably transfer the biometric signals to the server.

[0071] When the server receives biometric signals, it uses a data analysis device to compare and analyze these signals with historical data. Time series analysis and peak detection algorithms are used to identify patterns that deviate from the normal range.

[0072] Furthermore, the device collects user communication data, emails, chats, and voice recordings through a data collection device and sends this data to a server. The server uses a generative AI model equipped with natural language processing and speech analysis technologies to analyze the text and voice data and calculate sentiment scores.

[0073] Based on the analysis results, the server generates a message using an alert generator if it detects a high-stress state or a specific emotional state. The alert is sent to the terminal via a notification device, and the user receives it. Furthermore, an interface is displayed on the terminal that allows the user to easily access support services and counseling services if needed.

[0074] As a concrete example, suppose a user is overwhelmed with a project deadline and starts using negative expressions such as "I'm tired" and "I can't do this anymore." In this case, the server quickly analyzes the increased heart rate pattern obtained from the wearable device and determines that stress is increasing. The server sends an alert to the device saying, "A high level of stress has been detected recently. Take a rest and seek support if necessary." The user receives this notification, understands their situation, and can take appropriate action.

[0075] An example of a prompt message might be a request like, "Please describe a system that determines the user's stress level based on data and communication content acquired from a wearable device and issues an alert as needed."

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The terminal acquires biometric signals from the wearable device worn by the user. It receives heart rate, body temperature, and activity level data as input, collecting this data at approximately one-minute intervals. The terminal formats this data and outputs it to the server via a communication protocol. Specifically, the terminal periodically communicates with the device to acquire real-time data and perform format conversion.

[0079] Step 2:

[0080] The server receives biometric signals transmitted from the terminal. Using the input data, the data analysis device performs comparative analysis with historical data. For example, it uses time series analysis techniques and machine learning algorithms to detect anomalies and generate output that evaluates the user's health status. The server performs specific operations such as data cleansing, normalization, and input into the analysis model.

[0081] Step 3:

[0082] The device acquires communication data from the user's daily communication activities. It collects inputs such as email and chat text, and audio data from online meetings. The collected data is appropriately formatted, and output is generated that is transferred to the server using encryption protocols. Specifically, the device works in conjunction with the application being used to manage data capture and storage.

[0083] Step 4:

[0084] The server receives the collected communication data and analyzes it using natural language processing and speech analysis technologies. It performs sentiment analysis on the input text and audio data and generates output that calculates positive, negative, and neutral sentiment scores. Specifically, it uses a generative AI model to calculate the sentiment value of language and analyzes the tone of the audio data.

[0085] Step 5:

[0086] The server integrates the analysis results of biosignals and communication data to evaluate the user's emotional state and stress level. Based on this, if a high-stress state or emotional patterns requiring attention are detected, an alert is generated. The generated alert message is constructed and output for transmission to the terminal via a notification device.

[0087] Step 6:

[0088] The device receives alerts sent from the server and displays them to the user immediately. Specifically, it displays pop-up notifications on the screen or plays a notification sound. It also provides an interface that allows the user to access additional support and counseling services depending on the content of the alert. This helps the user take appropriate action based on the information.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] In managing the health of the elderly, there is a need to detect physical abnormalities and emotional stress early and respond quickly. However, conventional systems have difficulty integrating and handling this information, and in particular, capturing emotional changes has been challenging.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes a device for acquiring biometric information, a device for acquiring communication information, and an analysis device for analyzing the biometric information and communication information and determining the emotional state. This makes it possible to comprehensively monitor changes in biometric information and emotional changes in real time and generate appropriate warnings when an abnormality is detected.

[0094] "Biometric information" is a general term for data that indicates an individual's physical condition, such as heart rate, body temperature, and activity level.

[0095] "Device" refers to equipment or systems used to acquire or analyze biological information or communication information.

[0096] "Communication information" refers to text and audio data obtained from communication activities that users use on a daily basis, such as email, chat, and online meetings.

[0097] "Emotional state" refers to a user's psychological state expressed as a score, such as positive, negative, or neutral.

[0098] An "analysis device" refers to a device or software that determines a user's health status and emotions based on biological and communication information.

[0099] A "generator" is part of a system that creates warnings and notifications based on emotional states determined through analysis.

[0100] A "notification device" refers to a device or interface used as a means of informing a user of generated warnings or notifications.

[0101] A "remote monitoring device" is part of a system that allows the health status and emotions of specific individuals, such as the elderly, to be monitored from a distance.

[0102] This invention provides a system for monitoring the health and emotional state of elderly individuals in real time. The server acquires biometric information from wearable devices and other sensors to understand the user's activity status. It also acquires text and voice data generated from the communication methods the user uses daily and analyzes the emotional state based on this data.

[0103] Specifically, the server executes an analysis program using Python to detect trends and anomalies in biometric data. Google Cloud Natural Language API is used as a natural language processing technology for analyzing communication data, and Amazon Transcribe is used for analyzing voice data. The resulting sentiment score is expressed using three indicators: positive, negative, and neutral.

[0104] When signs of abnormality or stress are detected, the generator immediately generates a warning and sends an alert to caregivers or family members via a notification device. This alert allows for appropriate intervention.

[0105] For example, if a user is using words like "tired" more frequently than usual, and their heart rate is elevated, the system will quickly identify the stress and issue a warning. This allows for early intervention.

[0106] An example of a prompt using a generative AI model might be: "We want to monitor the health status of elderly people in real time and perform sentiment analysis. How can we implement anomaly detection and stress assessment using wearable devices and voice data?"

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The device acquires biometric information from wearable devices and sensors. Inputs include heart rate, body temperature, and activity level, while output is a dataset containing this data. The device collects data from devices using Bluetooth or similar technologies and periodically transmits this data to a server.

[0110] Step 2:

[0111] The device acquires communication information. The input consists of text and audio data from emails, chats, online meetings, etc., used by the user, and the output is a dataset containing this data. The device extracts data from communication applications and uploads it to the server.

[0112] Step 3:

[0113] The server analyzes the received biometric information. The input is the biometric data set obtained in step 1, and the output is the analyzed health indicators and anomaly detection results. The server uses Python to perform statistical processing and anomaly detection algorithms to evaluate the health status.

[0114] Step 4:

[0115] The server analyzes the communication information and determines the emotional state. The input is the communication information dataset obtained in step 2, and the output is the emotional score. The server uses the Google Cloud Natural Language API for natural language processing, converts speech to text using Amazon Transcribe, and performs emotional analysis.

[0116] Step 5:

[0117] The server generates alerts based on emotion scores and health indicators. The input is the results of steps 3 and 4, and the output is the alert message. The server sets thresholds and immediately generates an alert and compiles the necessary information if an anomaly is detected.

[0118] Step 6:

[0119] The device receives warnings sent from the server and notifies the user or caregiver. The input is the warning message generated in step 5, and the output is the notification displayed on the device. The device uses push notification functionality to display the warning message on the screen and deploys an interface to provide additional support as needed.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] To implement this invention, the terminal first acquires biometric information through the user's wearable device and transmits it to a server. This biometric information includes heart rate, body temperature, activity data, etc. Meanwhile, communication data from emails, business chats, and online meetings used by the user are also transmitted to the server via the terminal.

[0122] The server inputs this biometric and communication data into the emotion engine. The emotion engine uses natural language processing (NLP) techniques to analyze the communication data and calculate an emotion score. Furthermore, it detects abnormalities such as heart rate and body temperature from the biometric information and combines this data to comprehensively determine the user's emotional state.

[0123] The emotion engine uses machine learning algorithms to learn individual patterns from past data and predict changes in each user's emotions and stress levels. Based on these predictions, the emotion engine evaluates the user's emotional state in real time and generates alerts as needed.

[0124] Specifically, if a user frequently uses negative words such as "tough" or "anxious" due to work-related stress, the emotion engine records this as a negative score based on sentiment analysis. At the same time, if a wearable device detects a higher-than-normal heart rate, the server determines that the user may be experiencing increased stress.

[0125] Based on the results, the server generates an alert and sends it to the terminal. This alert might say something like, "Your stress level is high. Take a break and consider counseling if necessary." The terminal displays this message to the user, who can then access support services provided through the system as needed.

[0126] Thus, the system of the present invention, which combines an emotion engine, improves the user's health and work efficiency by closely monitoring the user's emotional state and stress level and providing appropriate support.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The device acquires biometric information such as heart rate, body temperature, and activity level from the user's wearable device at regular intervals. This information is transmitted to the server in real time.

[0130] Step 2:

[0131] The device acquires communication data from email, business chat, and online meeting systems that the user uses on a daily basis. This data includes text messages and call content, which are also sent to the server.

[0132] Step 3:

[0133] The server receives biometric information transmitted from the terminal and evaluates the current health status by comparing it with past data. It can also determine the possibility of stress by detecting sudden changes in heart rate or abnormal body temperature.

[0134] Step 4:

[0135] The server inputs the transmitted communication data into the emotion engine. The emotion engine uses natural language processing to analyze the text and speech and calculates the user's emotion score. This score quantitatively evaluates positive, negative, and neutral emotions.

[0136] Step 5:

[0137] The emotion engine integrates the obtained emotion score with biometric information and uses machine learning algorithms to comprehensively determine the user's emotional state. Based on patterns learned from past data, it predicts stress levels and emotional changes.

[0138] Step 6:

[0139] Based on the results of the emotion engine, the server generates an alert if the user's emotional state or stress level exceeds a set threshold. This alert includes specific countermeasures and recommended actions.

[0140] Step 7:

[0141] The server sends the generated alert to the terminal. The terminal notifies and displays this alert to the user.

[0142] Step 8:

[0143] Users check alerts displayed on their devices to recognize their emotional state and stress levels. If necessary, they can access counseling or other support services from their devices and take appropriate action.

[0144] (Example 2)

[0145] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0146] With the increasing complexity of modern living and work environments, it is crucial to appropriately monitor and proactively address individual mental stress and emotional fluctuations. However, conventional systems have struggled to comprehensively analyze biometric and communication data to individually assess stress levels. This has resulted in a challenge where users are unable to manage their stress at the appropriate time.

[0147] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0148] In this invention, the server includes a device for acquiring biometric information, means for acquiring communication information, and a device for collecting and storing the biometric information and communication information. This enables the accurate monitoring of changes in emotions and stress based on the user's biological state and communication content, and allows the user to appropriately manage stress by issuing alerts in advance.

[0149] "Biometric information" refers to information about the user's physical condition, such as heart rate, body temperature, and activity data.

[0150] "Communication information" refers to information related to digital communication, such as emails, chat messages, and online meeting content sent and received by users.

[0151] A "device" is a system of hardware and software designed to perform a specific function.

[0152] "Means" refer to the methods or techniques used to achieve a specific objective.

[0153] "Analytical function" refers to the ability or technology to process data, extract specific information, and analyze it.

[0154] "Machine learning techniques" refer to algorithms and processes that use large amounts of data to build models and perform predictions and pattern recognition.

[0155] An "alert" is a notification or warning intended to draw attention to a specific situation or condition.

[0156] "Wearable technology" refers to devices and technologies designed to be worn on the body, primarily used for monitoring health and fitness information.

[0157] To implement this invention, the terminal must first acquire biometric information from the user's wearable device. This terminal connects to the device using Bluetooth technology and collects heart rate, body temperature, and activity data in real time. Next, the terminal transmits the collected data to a server. This data is securely transferred to the server via a communication line.

[0158] The server inputs the received biometric information and communication information (emails, chats, and online meeting logs) obtained from the user's device into an emotion analysis engine. This emotion analysis engine operates on a machine learning platform such as TENSORFLOW®, and uses natural language processing techniques to analyze the communication information and calculate an emotion score. It also analyzes the biometric information to detect abnormalities in heart rate and body temperature.

[0159] Based on the analysis, the server comprehensively assesses the user's emotional state and generates an alert if the stress level exceeds a certain threshold. This alert is sent to the terminal and displayed to the user as a message such as, "Your stress level is high. Take a break and consider counseling if necessary."

[0160] Specifically, if a user uses negative expressions such as "tired" or "troubled" in a business chat, and the system detects that their heart rate is higher than normal, the system will determine that they are under stress and notify the user at an appropriate time.

[0161] An example of a prompt for the generating AI model would be, "What alert message should be generated when a user has a high heart rate and sends a negative chat message?" By utilizing this system in this way, it becomes possible to monitor the user's emotional state in detail and improve their health and work efficiency.

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1:

[0164] The terminal acquires biometric information from the wearable device. Specifically, the terminal connects to the user's wearable device via Bluetooth and collects heart rate, body temperature, and activity data in real time. At this point, the input is sensor data from the wearable device, and the output is biometric data formatted for analysis.

[0165] Step 2:

[0166] The device transmits biometric information to the server. The device transfers the collected biometric information to the server via the internet. The data is encrypted before transmission to enhance security. The input here is formatted biometric data, and the output is storage in an analysis database on the server.

[0167] Step 3:

[0168] The user's device collects communication information. When a user sends an email or engages in chat, the content of that communication is recorded. The input is the user's text communication data, and the output is communication information data for later analysis.

[0169] Step 4:

[0170] The terminal sends communication information to the server. The terminal encrypts the collected communication information using SSL and securely sends it to the server. In this case, the input is the communication information data, and the output is its storage in the server's analysis database.

[0171] Step 5:

[0172] The server inputs biometric and communication data into the sentiment analysis engine. The server inputs this data into the sentiment analysis engine, which uses TensorFlow, and starts the analysis. The input is a dataset of biometric and communication data, and the output is the intermediate analysis result.

[0173] Step 6:

[0174] The server analyzes the data and calculates an emotion score. It uses natural language processing techniques to analyze communication information and quantify the user's emotional state. It also analyzes biometric information and detects anomalies. The input is the intermediate analysis result, and the output is the emotion score and anomaly detection results.

[0175] Step 7:

[0176] The server comprehensively assesses the emotional state and generates alerts as needed. It integrates the emotional score and biometric data analysis results to evaluate the user's emotional state. If the evaluation exceeds a predetermined threshold, an alert is generated. The input is the emotional score and anomaly detection results, and the output is alert data.

[0177] Step 8:

[0178] The server generates alerts and sends them to the terminal. The server then sends the generated alerts to the user's terminal as push notifications. The input is the alert data, and the output is the notification sent to the user's terminal.

[0179] Step 9:

[0180] The terminal displays an alert to the user. The terminal displays the received alert on the screen to inform the user. The input is the alert notification from the server, and the output is the visual alert display to the user.

[0181] (Application Example 2)

[0182] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0183] Traditionally, stress management and improvement of working conditions for care staff have been crucial issues in caregiving settings. However, the provision of concrete stress assessments and support measures using biometric information and communication data has been insufficient. This could negatively impact the health and work efficiency of care staff. Therefore, there is a need for a system that can assess the stress levels of care staff in real time and provide appropriate support measures.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0185] In this invention, the server includes detection means, acquisition means, analysis means, generation means, notification means, and evaluation means. This makes it possible to evaluate the stress level of care staff using their biometric and communication information and provide support measures as needed.

[0186] "Biometric information" refers to data that indicates an individual's physical condition, including information such as heart rate, body temperature, and activity level.

[0187] "Communication information" refers to data obtained from emails, business chats, online meetings, and other communications that individuals use on a daily basis.

[0188] "Emotional state" is an indicator that shows an individual's emotional response and stress level, obtained by analyzing biometric and communication information.

[0189] A "wearable device" is an electronic device that is attached to the body and can continuously measure biometric information.

[0190] An "emotion score" is a numerical measure of emotion calculated by analyzing the language data contained in communication information using natural language processing technology.

[0191] "Evaluation methods" refer to functions or processes that analyze the stress levels of care staff based on biometric information and emotional scores, and provide support measures based on the results.

[0192] A "generative AI model" is a form of artificial intelligence that uses machine learning algorithms to learn patterns from data and derive stress predictions and support measures.

[0193] To implement this invention, the server first acquires biometric information from a wearable device worn by the care staff. The wearable device continuously measures data such as heart rate, body temperature, and activity level, and transmits this information to the server. In addition, work-related communication information is acquired via the smartphone or tablet used by the care staff. This communication information includes emails, business chats, and records of work-related communications.

[0194] The server uses natural language processing (NLP) techniques to analyze communication information and calculate sentiment scores. This includes analyzing emotional responses to specific keywords and sentences. Furthermore, by combining biometric information with sentiment scores and utilizing a generative AI model, it predicts the stress levels of care staff.

[0195] Based on this information, the server uses assessment tools to generate an alert if it determines that the stress level is high. This alert sends a notification to the care staff's terminal, such as, "Your stress level is high. We recommend you take a short break." This allows the care staff to access support services provided through the system as needed.

[0196] As a concrete example, consider a situation where a caregiver is working long hours without taking breaks. In this case, a wearable device detects an abnormally high heart rate, and a negative emotion score is calculated from the communication data. The generative AI model analyzes this data and determines that the person is in a high-stress state. The server immediately generates an alert and sends a notification to the caregiver urging them to take a break.

[0197] Example prompt: "What kind of breaks and relaxation methods would you suggest to reduce stress for care staff?"

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] The server acquires biometric information from wearable devices. The input consists of biometric data such as heart rate, body temperature, and activity level, which are stored in a database on the server. The data is configured to be updated periodically.

[0201] Step 2:

[0202] The server acquires communication information related to the work through terminals used by care staff. The input consists of email and business chat messages, which are analyzed using natural language processing technology. As a result, emotion-related keywords and phrases are extracted and output as emotion scores.

[0203] Step 3:

[0204] The server combines acquired biometric and communication information and inputs it into a generating AI model. The inputs are heart rate, body temperature, activity level, and emotion score, which the AI ​​model uses to predict the stress level of care staff. The output is an evaluation of the stress level, which is returned to the server as a numerical index.

[0205] Step 4:

[0206] The server analyzes the condition of care staff using an evaluation method based on the stress levels output by the generated AI model. The input here is the evaluation result of the stress level, and the output is an evaluation result such as "high stress" or "moderate stress".

[0207] Step 5:

[0208] The server generates an alert if it determines that the stress level is high. The alert content is something like, "We recommend taking a short break," and is output as data to notify the user.

[0209] Step 6:

[0210] The terminal receives notifications from the server and displays alert messages to care staff. The input is alert information from the server, and the output is the message displayed on the terminal's screen. This allows care staff to take the instructed action immediately.

[0211] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0212] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0213] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0214] [Second Embodiment]

[0215] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0216] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0217] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0218] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0219] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0220] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0221] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0222] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0223] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0224] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0225] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0226] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0227] To implement the present invention, the following configuration is conceivable. First, the terminal periodically acquires biometric information through a wearable device worn by the user. This includes data such as heart rate, body temperature, and activity level. This information is transmitted to the server in real time.

[0228] Simultaneously, users communicate daily through email, business chat tools, and online meetings. The device acquires text and audio data generated from these communication activities. This acquired data is also sent to the server.

[0229] The server analyzes the user's physical health status and activity patterns based on the received biometric information. Specifically, it refers to historical biometric data and detects values ​​and patterns that deviate from the normal range. The server also uses natural language processing (NLP) and speech analysis techniques to analyze communication data and determine the user's emotional state. This takes into account factors such as word choice, tone of voice, and frequency of speech in communication, and is expressed as a positive, negative, or neutral emotional score.

[0230] Based on the analysis results, the server generates an alert if the emotional state exceeds a certain threshold or if high stress levels are suspected based on biometric information. Specifically, this alert is a message summarizing cautionary advice and recommended actions, and is sent to the terminal via a notification system.

[0231] The terminal displays alerts sent from the server to the user. Ideally, the system should provide an interface that allows the user to access additional support and counseling services as needed. This enables the user to recognize their own emotions and stress levels and take appropriate action.

[0232] As a concrete example, suppose a user is under pressure to meet a project deadline and starts making more negative statements such as "I'm tired" or "I can't do this anymore." At this time, the heart rate transmitted from the wearable device tends to be higher than usual. The server quickly analyzes this data, determines that stress levels are high, and sends an alert to the device saying, "You may be experiencing high stress levels recently. Take a rest and seek counseling if necessary." The user receives this notification, becomes aware of their condition, and can take appropriate action.

[0233] Thus, the system of the present invention makes a significant contribution to understanding the user's health status and providing appropriate support.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The device acquires biometric information such as heart rate, body temperature, and activity level from the wearable device worn by the user at regular intervals. This data is transmitted in real time to the server via the cloud.

[0237] Step 2:

[0238] Users utilize email, business chat tools, and online meetings in their daily work. Their devices record text and audio data generated from these communication activities and send it to a server.

[0239] Step 3:

[0240] The server receives biometric information transmitted from the terminal and identifies anomalies by comparing it to a database with a set normal range. In this process, it detects rapid changes in heart rate and abnormal patterns in activity levels.

[0241] Step 4:

[0242] The server uses natural language processing (NLP) to analyze the user's communication data. Based on the content and tone of their speech, it calculates a sentiment score and determines whether the user is positive, negative, or neutral.

[0243] Step 5:

[0244] The server integrates biometric data and analyzed emotion scores to evaluate the user's overall emotional state. If the emotion score exceeds a set threshold and abnormalities are detected in the biometric data, the server generates a warning.

[0245] Step 6:

[0246] The server sends the generated warning as a message to the terminal. The message includes a warning about increased stress and recommended actions (e.g., taking a break, seeking counseling).

[0247] Step 7:

[0248] The terminal displays received messages to the user. Through these messages, the user can understand their current situation and, if necessary, access support services provided through the system.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] In modern society, there are concerns about the impact of daily stress and emotional fluctuations on health. However, conventional technologies have lacked the means to monitor an individual's biometric data and emotional state in real time and provide appropriate support based on that data. As a result, there is a challenge in that it is difficult for users to accurately understand their own health status and quickly access the support they need.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes a sensor device for acquiring biosignals, a data acquisition device for acquiring communication data, and a data analysis device for integrating the biosignals and communication data and identifying the emotional state. This makes it possible to accurately grasp the user's health and emotional state in real time and provide alerts at the appropriate time.

[0254] A "sensor device" is a device that detects biological signals and records and transmits that information.

[0255] A "data collection device" is a device that acquires communication data and stores it for analysis.

[0256] A "data analysis device" is a device that integrates and analyzes biological signals and communication data to identify emotional states.

[0257] An "alert generator" is a device that generates warnings and recommendations to notify users based on their emotional state, as identified by a data analysis device.

[0258] A "notification device" is a device that has an interface for communicating generated alerts to the user.

[0259] A "device that provides an interface" is a device that has the functionality to allow users to access additional support services.

[0260] "Biosignals" refer to physiological parameters of the body, such as heart rate and body temperature, that are acquired through wearable devices.

[0261] "Communication data" refers to data including text and audio information obtained from users' emails, chats, and voice calls.

[0262] To implement the invention, it is necessary to configure a system that links a server, a terminal, and a wearable device. First, the terminal periodically acquires biometric signals such as heart rate, body temperature, and activity level from the wearable device worn by the user. To do this, communication technologies such as Bluetooth and Wi-Fi are used to stably transfer the biometric signals to the server.

[0263] When the server receives biometric signals, it uses a data analysis device to compare and analyze these signals with historical data. Time series analysis and peak detection algorithms are used to identify patterns that deviate from the normal range.

[0264] Furthermore, the device collects user communication data, emails, chats, and voice recordings through a data collection device and sends this data to a server. The server uses a generative AI model equipped with natural language processing and speech analysis technologies to analyze the text and voice data and calculate sentiment scores.

[0265] Based on the analysis results, the server generates a message using an alert generator if it detects a high-stress state or a specific emotional state. The alert is sent to the terminal via a notification device, and the user receives it. Furthermore, an interface is displayed on the terminal that allows the user to easily access support services and counseling services if needed.

[0266] As a concrete example, suppose a user is overwhelmed with a project deadline and starts using negative expressions such as "I'm tired" and "I can't do this anymore." In this case, the server quickly analyzes the increased heart rate pattern obtained from the wearable device and determines that stress is increasing. The server sends an alert to the device saying, "A high level of stress has been detected recently. Take a rest and seek support if necessary." The user receives this notification, understands their situation, and can take appropriate action.

[0267] An example of a prompt message might be a request like, "Please describe a system that determines the user's stress level based on data and communication content acquired from a wearable device and issues an alert as needed."

[0268] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0269] Step 1:

[0270] The terminal acquires biometric signals from the wearable device worn by the user. It receives heart rate, body temperature, and activity level data as input, collecting this data at approximately one-minute intervals. The terminal formats this data and outputs it to the server via a communication protocol. Specifically, the terminal periodically communicates with the device to acquire real-time data and perform format conversion.

[0271] Step 2:

[0272] The server receives biometric signals transmitted from the terminal. Using the input data, the data analysis device performs comparative analysis with historical data. For example, it uses time series analysis techniques and machine learning algorithms to detect anomalies and generate output that evaluates the user's health status. The server performs specific operations such as data cleansing, normalization, and input into the analysis model.

[0273] Step 3:

[0274] The device acquires communication data from the user's daily communication activities. It collects inputs such as email and chat text, and audio data from online meetings. The collected data is appropriately formatted, and output is generated that is transferred to the server using encryption protocols. Specifically, the device works in conjunction with the application being used to manage data capture and storage.

[0275] Step 4:

[0276] The server receives the collected communication data and analyzes it using natural language processing and speech analysis technologies. It performs sentiment analysis on the input text and audio data and generates output that calculates positive, negative, and neutral sentiment scores. Specifically, it uses a generative AI model to calculate the sentiment value of language and analyzes the tone of the audio data.

[0277] Step 5:

[0278] The server integrates the analysis results of biometric signals and communication data, and evaluates the user's emotional state and stress level. Based on this, when a high-stress state or an emotional pattern requiring attention is detected, an alert is generated. Output is made to construct the generated alert message and transfer it to the terminal through the notification device.

[0279] Step 6:

[0280] The terminal receives the alert sent from the server and immediately displays it to the user. Specifically, a pop-up notification is issued on the screen or a notification sound is played. Also, an interface is provided that allows the user to access additional support or counseling services according to the alert content. This provides support for the user to take appropriate actions based on the information.

[0281] (Application Example 1)

[0282] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0283] In the health management of the elderly, it is required to detect physical abnormalities and emotional stress at an early stage and respond promptly. However, in conventional systems, it has been difficult to handle these types of information integratively, and in particular, it has been difficult to capture changes in the emotional aspect.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0285] In this invention, the server includes a device for acquiring biometric information, a device for acquiring communication information, and an analysis device for analyzing the biometric information and communication information and determining the emotional state. Thereby, it becomes possible to comprehensively monitor in real time the changes in biometric information and emotional changes, and generate an appropriate warning when an abnormality is detected.

[0286] "Biological information" is a general term for data indicating an individual's physical condition, such as heart rate, body temperature, activity level, etc.

[0287] "Device" refers to a device or system used to acquire or analyze biological information or communication information.

[0288] "Communication information" refers to text data and voice data obtained from communication activities such as emails, chats, online meetings, etc. that users use daily.

[0289] "Emotional state" is what represents the user's psychological state with scores such as positive, negative, and neutral.

[0290] "Analysis device" refers to a device or software for determining the user's health status and emotions based on biological information and communication information.

[0291] "Generation device" is part of a system for creating warnings and notifications based on the emotional state determined by analysis.

[0292] "Notification device" refers to a device or interface used as a means to inform the user of the generated warnings and notifications.

[0293] "Remote monitoring device" is part of a system that can monitor the health status and emotions of specific people such as the elderly from a remote location.

[0294] This invention provides a system for monitoring the health status and emotional state of the elderly in real time. The server acquires biological information from wearable devices and other sensors to grasp the user's activity situation. It also acquires text data and voice data generated from the communication means that the user uses daily, and analyzes the emotional state based on this.

[0295] Specifically, the server executes an analysis program using Python to detect trends and anomalies in biometric data. Google Cloud Natural Language API is used for analyzing communication data as a natural language processing technique, and Amazon Transcribe is used for analyzing voice data. The resulting sentiment score is expressed using three indicators: positive, negative, and neutral.

[0296] When signs of abnormality or stress are detected, the generator immediately generates a warning and sends an alert to caregivers or family members via a notification device. This alert allows for appropriate intervention.

[0297] For example, if a user is using words like "tired" more frequently than usual, and their heart rate is elevated, the system will quickly identify the stress and issue a warning. This allows for early intervention.

[0298] An example of a prompt using a generative AI model might be: "We want to monitor the health status of elderly people in real time and perform sentiment analysis. How can we implement anomaly detection and stress assessment using wearable devices and voice data?"

[0299] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0300] Step 1:

[0301] The device acquires biometric information from wearable devices and sensors. Inputs include heart rate, body temperature, and activity level, while output is a dataset containing this data. The device collects data from devices using Bluetooth or similar technologies and periodically transmits this data to a server.

[0302] Step 2:

[0303] The terminal acquires communication information. The input is text data and voice data such as emails, chats, and online meetings used by the user, and the output is a dataset obtained by organizing these data. The terminal extracts data from the communication application and uploads it to the server.

[0304] Step 3:

[0305] The server analyzes the acquired biometric information. The input is the biometric information dataset obtained in Step 1, and the output is the analyzed health indicators and anomaly detection results. The server uses Python to execute statistical processing and outlier detection algorithms to evaluate the health status.

[0306] Step 4:

[0307] The server analyzes the communication information and determines the emotional state. The input is the communication information dataset obtained in Step 2, and the output is the emotion score. The server performs natural language processing using the Google Cloud Natural Language API, converts the voice to text using Amazon Transcribe, and conducts sentiment analysis.

[0308] Step 5:

[0309] The server generates a warning based on the emotion score and health indicators. The input is the results of Step 3 and Step 4, and the output is a warning message. The server sets a threshold value, generates a warning immediately when an anomaly is detected, and summarizes the necessary information.

[0310] Step 6:

[0311] The terminal receives the warning sent from the server and notifies the user or caregiver. The input is the warning message generated in Step 5, and the output is a notification displayed on the terminal. The terminal uses the push notification function to display the warning message on the screen and expands an interface for providing additional support if necessary.

[0312] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0313] To implement this invention, the terminal first acquires biometric information through the user's wearable device and transmits it to a server. This biometric information includes heart rate, body temperature, activity data, etc. Meanwhile, communication data from emails, business chats, and online meetings used by the user are also transmitted to the server via the terminal.

[0314] The server inputs this biometric and communication data into the emotion engine. The emotion engine uses natural language processing (NLP) techniques to analyze the communication data and calculate an emotion score. Furthermore, it detects abnormalities such as heart rate and body temperature from the biometric information and combines this data to comprehensively determine the user's emotional state.

[0315] The emotion engine uses machine learning algorithms to learn individual patterns from past data and predict changes in each user's emotions and stress levels. Based on these predictions, the emotion engine evaluates the user's emotional state in real time and generates alerts as needed.

[0316] Specifically, if a user frequently uses negative words such as "tough" or "anxious" due to work-related stress, the emotion engine records this as a negative score based on sentiment analysis. At the same time, if a wearable device detects a higher-than-normal heart rate, the server determines that the user may be experiencing increased stress.

[0317] Based on the results, the server generates an alert and sends it to the terminal. This alert might say something like, "Your stress level is high. Take a break and consider counseling if necessary." The terminal displays this message to the user, who can then access support services provided through the system as needed.

[0318] Thus, the system of the present invention, which combines an emotion engine, improves the user's health and work efficiency by closely monitoring the user's emotional state and stress level and providing appropriate support.

[0319] The following describes the processing flow.

[0320] Step 1:

[0321] The device acquires biometric information such as heart rate, body temperature, and activity level from the user's wearable device at regular intervals. This information is transmitted to the server in real time.

[0322] Step 2:

[0323] The device acquires communication data from email, business chat, and online meeting systems that the user uses on a daily basis. This data includes text messages and call content, which are also sent to the server.

[0324] Step 3:

[0325] The server receives biometric information transmitted from the terminal and evaluates the current health status by comparing it with past data. It can also determine the possibility of stress by detecting sudden changes in heart rate or abnormal body temperature.

[0326] Step 4:

[0327] The server inputs the transmitted communication data into the emotion engine. The emotion engine uses natural language processing to analyze the text and speech and calculates the user's emotion score. This score quantitatively evaluates positive, negative, and neutral emotions.

[0328] Step 5:

[0329] The emotion engine integrates the obtained emotion score with biometric information and uses machine learning algorithms to comprehensively determine the user's emotional state. Based on patterns learned from past data, it predicts stress levels and emotional changes.

[0330] Step 6:

[0331] Based on the results of the emotion engine, the server generates an alert if the user's emotional state or stress level exceeds a set threshold. This alert includes specific countermeasures and recommended actions.

[0332] Step 7:

[0333] The server sends the generated alert to the terminal. The terminal notifies and displays this alert to the user.

[0334] Step 8:

[0335] Users check alerts displayed on their devices to recognize their emotional state and stress levels. If necessary, they can access counseling or other support services from their devices and take appropriate action.

[0336] (Example 2)

[0337] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0338] With the increasing complexity of modern living and work environments, it is crucial to appropriately monitor and proactively address individual mental stress and emotional fluctuations. However, conventional systems have struggled to comprehensively analyze biometric and communication data to individually assess stress levels. This has resulted in a challenge where users are unable to manage their stress at the appropriate time.

[0339] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0340] In this invention, the server includes a device for acquiring biometric information, means for acquiring communication information, and a device for collecting and storing the biometric information and communication information. This enables the accurate monitoring of changes in emotions and stress based on the user's biological state and communication content, and allows the user to appropriately manage stress by issuing alerts in advance.

[0341] "Biometric information" refers to information about the user's physical condition, such as heart rate, body temperature, and activity data.

[0342] "Communication information" refers to information related to digital communication, such as emails, chat messages, and online meeting content sent and received by users.

[0343] A "device" is a system of hardware and software designed to perform a specific function.

[0344] "Means" refer to the methods or techniques used to achieve a specific objective.

[0345] "Analytical function" refers to the ability or technology to process data, extract specific information, and analyze it.

[0346] "Machine learning techniques" refer to algorithms and processes that use large amounts of data to build models and perform predictions and pattern recognition.

[0347] An "alert" is a notification or warning intended to draw attention to a specific situation or condition.

[0348] "Wearable technology" refers to devices and technologies designed to be worn on the body, primarily used for monitoring health and fitness information.

[0349] To implement this invention, the terminal must first acquire biometric information from the user's wearable device. This terminal connects to the device using Bluetooth technology and collects heart rate, body temperature, and activity data in real time. Next, the terminal transmits the collected data to a server. This data is securely transferred to the server via a communication line.

[0350] The server inputs the received biometric information and communication information (emails, chats, and online meeting logs) obtained from the user's device into an emotion analysis engine. This emotion analysis engine runs on a machine learning platform such as TensorFlow, and uses natural language processing techniques to analyze the communication information and calculate an emotion score. It also analyzes the biometric information to detect abnormalities in heart rate and body temperature.

[0351] Based on the analysis, the server comprehensively assesses the user's emotional state and generates an alert if the stress level exceeds a certain threshold. This alert is sent to the terminal and displayed to the user as a message such as, "Your stress level is high. Take a break and consider counseling if necessary."

[0352] Specifically, if a user uses negative expressions such as "tired" or "troubled" in a business chat, and the system detects that their heart rate is higher than normal, the system will determine that they are under stress and notify the user at an appropriate time.

[0353] An example of a prompt for the generating AI model would be, "What alert message should be generated when a user has a high heart rate and sends a negative chat message?" By utilizing this system in this way, it becomes possible to monitor the user's emotional state in detail and improve their health and work efficiency.

[0354] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0355] Step 1:

[0356] The terminal acquires biometric information from the wearable device. Specifically, the terminal connects to the user's wearable device via Bluetooth and collects heart rate, body temperature, and activity data in real time. At this point, the input is sensor data from the wearable device, and the output is biometric data formatted for analysis.

[0357] Step 2:

[0358] The device transmits biometric information to the server. The device transfers the collected biometric information to the server via the internet. The data is encrypted before transmission to enhance security. The input here is formatted biometric data, and the output is storage in an analysis database on the server.

[0359] Step 3:

[0360] The user's device collects communication information. When a user sends an email or engages in chat, the content of that communication is recorded. The input is the user's text communication data, and the output is communication information data for later analysis.

[0361] Step 4:

[0362] The terminal sends communication information to the server. The terminal encrypts the collected communication information using SSL and securely sends it to the server. In this case, the input is the communication information data, and the output is its storage in the server's analysis database.

[0363] Step 5:

[0364] The server inputs biometric and communication data into the sentiment analysis engine. The server inputs this data into the sentiment analysis engine, which uses TensorFlow, and starts the analysis. The input is a dataset of biometric and communication data, and the output is the intermediate analysis result.

[0365] Step 6:

[0366] The server analyzes the data and calculates an emotion score. It uses natural language processing techniques to analyze communication information and quantify the user's emotional state. It also analyzes biometric information and detects anomalies. The input is the intermediate analysis result, and the output is the emotion score and anomaly detection results.

[0367] Step 7:

[0368] The server comprehensively assesses the emotional state and generates alerts as needed. It integrates the emotional score and biometric data analysis results to evaluate the user's emotional state. If the evaluation exceeds a predetermined threshold, an alert is generated. The input is the emotional score and anomaly detection results, and the output is alert data.

[0369] Step 8:

[0370] The server generates alerts and sends them to the terminal. The server then sends the generated alerts to the user's terminal as push notifications. The input is the alert data, and the output is the notification sent to the user's terminal.

[0371] Step 9:

[0372] The terminal displays an alert to the user. The terminal displays the received alert on the screen to inform the user. The input is the alert notification from the server, and the output is the visual alert display to the user.

[0373] (Application Example 2)

[0374] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0375] Traditionally, stress management and improvement of working conditions for care staff have been crucial issues in caregiving settings. However, the provision of concrete stress assessments and support measures using biometric information and communication data has been insufficient. This could negatively impact the health and work efficiency of care staff. Therefore, there is a need for a system that can assess the stress levels of care staff in real time and provide appropriate support measures.

[0376] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0377] In this invention, the server includes detection means, acquisition means, analysis means, generation means, notification means, and evaluation means. This makes it possible to evaluate the stress level of care staff using their biometric and communication information and provide support measures as needed.

[0378] "Biometric information" refers to data that indicates an individual's physical condition, including information such as heart rate, body temperature, and activity level.

[0379] "Communication information" refers to data obtained from emails, business chats, online meetings, and other communications that individuals use on a daily basis.

[0380] "Emotional state" is an indicator that shows an individual's emotional response and stress level, obtained by analyzing biometric and communication information.

[0381] A "wearable device" is an electronic device that is attached to the body and can continuously measure biometric information.

[0382] An "emotion score" is a numerical measure of emotion calculated by analyzing the language data contained in communication information using natural language processing technology.

[0383] "Evaluation methods" refer to functions or processes that analyze the stress levels of care staff based on biometric information and emotional scores, and provide support measures based on the results.

[0384] A "generative AI model" is a form of artificial intelligence that uses machine learning algorithms to learn patterns from data and derive stress predictions and support measures.

[0385] To implement this invention, the server first acquires biometric information from a wearable device worn by the care staff. The wearable device continuously measures data such as heart rate, body temperature, and activity level, and transmits this information to the server. In addition, work-related communication information is acquired via the smartphone or tablet used by the care staff. This communication information includes emails, business chats, and records of work-related communications.

[0386] The server uses natural language processing (NLP) techniques to analyze communication information and calculate sentiment scores. This includes analyzing emotional responses to specific keywords and sentences. Furthermore, by combining biometric information with sentiment scores and utilizing a generative AI model, it predicts the stress levels of care staff.

[0387] Based on this information, the server uses assessment tools to generate an alert if it determines that the stress level is high. This alert sends a notification to the care staff's terminal, such as, "Your stress level is high. We recommend you take a short break." This allows the care staff to access support services provided through the system as needed.

[0388] As a concrete example, consider a situation where a caregiver is working long hours without taking breaks. In this case, a wearable device detects an abnormally high heart rate, and a negative emotion score is calculated from the communication data. The generative AI model analyzes this data and determines that the person is in a high-stress state. The server immediately generates an alert and sends a notification to the caregiver urging them to take a break.

[0389] Example prompt: "What kind of breaks and relaxation methods would you suggest to reduce stress for care staff?"

[0390] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0391] Step 1:

[0392] The server acquires biometric information from wearable devices. The input consists of biometric data such as heart rate, body temperature, and activity level, which are stored in a database on the server. The data is configured to be updated periodically.

[0393] Step 2:

[0394] The server acquires communication information related to the work through terminals used by care staff. The input consists of email and business chat messages, which are analyzed using natural language processing technology. As a result, emotion-related keywords and phrases are extracted and output as emotion scores.

[0395] Step 3:

[0396] The server combines acquired biometric and communication information and inputs it into a generating AI model. The inputs are heart rate, body temperature, activity level, and emotion score, which the AI ​​model uses to predict the stress level of care staff. The output is an evaluation of the stress level, which is returned to the server as a numerical index.

[0397] Step 4:

[0398] The server analyzes the condition of care staff using an evaluation method based on the stress levels output by the generated AI model. The input here is the evaluation result of the stress level, and the output is an evaluation result such as "high stress" or "moderate stress".

[0399] Step 5:

[0400] The server generates an alert if it determines that the stress level is high. The alert content is something like, "We recommend taking a short break," and is output as data to notify the user.

[0401] Step 6:

[0402] The terminal receives notifications from the server and displays alert messages to care staff. The input is alert information from the server, and the output is the message displayed on the terminal's screen. This allows care staff to take the instructed action immediately.

[0403] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0404] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0405] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0406] [Third Embodiment]

[0407] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0408] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0409] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0410] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0411] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0412] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0413] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0414] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0415] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0416] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0417] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0418] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0419] To implement the present invention, the following configuration is conceivable. First, the terminal periodically acquires biometric information through a wearable device worn by the user. This includes data such as heart rate, body temperature, and activity level. This information is transmitted to the server in real time.

[0420] Simultaneously, users communicate daily through email, business chat tools, and online meetings. The device acquires text and audio data generated from these communication activities. This acquired data is also sent to the server.

[0421] The server analyzes the user's physical health status and activity patterns based on the received biometric information. Specifically, it refers to historical biometric data and detects values ​​and patterns that deviate from the normal range. The server also uses natural language processing (NLP) and speech analysis techniques to analyze communication data and determine the user's emotional state. This takes into account factors such as word choice, tone of voice, and frequency of speech in communication, and is expressed as a positive, negative, or neutral emotional score.

[0422] Based on the analysis results, the server generates an alert if the emotional state exceeds a certain threshold or if high stress levels are suspected based on biometric information. Specifically, this alert is a message summarizing cautionary advice and recommended actions, and is sent to the terminal via a notification system.

[0423] The terminal displays alerts sent from the server to the user. Ideally, the system should provide an interface that allows the user to access additional support and counseling services as needed. This enables the user to recognize their own emotions and stress levels and take appropriate action.

[0424] As a concrete example, suppose a user is under pressure to meet a project deadline and starts making more negative statements such as "I'm tired" or "I can't do this anymore." At this time, the heart rate transmitted from the wearable device tends to be higher than usual. The server quickly analyzes this data, determines that stress levels are high, and sends an alert to the device saying, "You may be experiencing high stress levels recently. Take a rest and seek counseling if necessary." The user receives this notification, becomes aware of their condition, and can take appropriate action.

[0425] Thus, the system of the present invention makes a significant contribution to understanding the user's health status and providing appropriate support.

[0426] The following describes the processing flow.

[0427] Step 1:

[0428] The device acquires biometric information such as heart rate, body temperature, and activity level from the wearable device worn by the user at regular intervals. This data is transmitted in real time to the server via the cloud.

[0429] Step 2:

[0430] Users utilize email, business chat tools, and online meetings in their daily work. Their devices record text and audio data generated from these communication activities and send it to a server.

[0431] Step 3:

[0432] The server receives biometric information transmitted from the terminal and identifies anomalies by comparing it to a database with a set normal range. In this process, it detects rapid changes in heart rate and abnormal patterns in activity levels.

[0433] Step 4:

[0434] The server uses natural language processing (NLP) to analyze the user's communication data. Based on the content and tone of their speech, it calculates a sentiment score and determines whether the user is positive, negative, or neutral.

[0435] Step 5:

[0436] The server integrates biometric data and analyzed emotion scores to evaluate the user's overall emotional state. If the emotion score exceeds a set threshold and abnormalities are detected in the biometric data, the server generates a warning.

[0437] Step 6:

[0438] The server sends the generated warning as a message to the terminal. The message includes a warning about increased stress and recommended actions (e.g., taking a break, seeking counseling).

[0439] Step 7:

[0440] The terminal displays received messages to the user. Through these messages, the user can understand their current situation and, if necessary, access support services provided through the system.

[0441] (Example 1)

[0442] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0443] In modern society, there are concerns about the impact of daily stress and emotional fluctuations on health. However, conventional technologies have lacked the means to monitor an individual's biometric data and emotional state in real time and provide appropriate support based on that data. As a result, there is a challenge in that it is difficult for users to accurately understand their own health status and quickly access the support they need.

[0444] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0445] In this invention, the server includes a sensor device for acquiring biosignals, a data acquisition device for acquiring communication data, and a data analysis device for integrating the biosignals and communication data and identifying the emotional state. This makes it possible to accurately grasp the user's health and emotional state in real time and provide alerts at the appropriate time.

[0446] A "sensor device" is a device that detects biological signals and records and transmits that information.

[0447] A "data collection device" is a device that acquires communication data and stores it for analysis.

[0448] A "data analysis device" is a device that integrates and analyzes biological signals and communication data to identify emotional states.

[0449] An "alert generator" is a device that generates warnings and recommendations to notify users based on their emotional state, as identified by a data analysis device.

[0450] A "notification device" is a device that has an interface for communicating generated alerts to the user.

[0451] A "device that provides an interface" is a device that has the functionality to allow users to access additional support services.

[0452] "Biosignals" refer to physiological parameters of the body, such as heart rate and body temperature, that are acquired through wearable devices.

[0453] "Communication data" refers to data including text and audio information obtained from users' emails, chats, and voice calls.

[0454] To implement the invention, it is necessary to configure a system that links a server, a terminal, and a wearable device. First, the terminal periodically acquires biometric signals such as heart rate, body temperature, and activity level from the wearable device worn by the user. To do this, communication technologies such as Bluetooth and Wi-Fi are used to stably transfer the biometric signals to the server.

[0455] When the server receives biometric signals, it uses a data analysis device to compare and analyze these signals with historical data. Time series analysis and peak detection algorithms are used to identify patterns that deviate from the normal range.

[0456] Furthermore, the device collects user communication data, emails, chats, and voice recordings through a data collection device and sends this data to a server. The server uses a generative AI model equipped with natural language processing and speech analysis technologies to analyze the text and voice data and calculate sentiment scores.

[0457] Based on the analysis results, the server generates a message using an alert generator if it detects a high-stress state or a specific emotional state. The alert is sent to the terminal via a notification device, and the user receives it. Furthermore, an interface is displayed on the terminal that allows the user to easily access support services and counseling services if needed.

[0458] As a concrete example, suppose a user is overwhelmed with a project deadline and starts using negative expressions such as "I'm tired" and "I can't do this anymore." In this case, the server quickly analyzes the increased heart rate pattern obtained from the wearable device and determines that stress is increasing. The server sends an alert to the device saying, "A high level of stress has been detected recently. Take a rest and seek support if necessary." The user receives this notification, understands their situation, and can take appropriate action.

[0459] An example of a prompt message might be a request like, "Please describe a system that determines the user's stress level based on data and communication content acquired from a wearable device and issues an alert as needed."

[0460] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0461] Step 1:

[0462] The terminal acquires biometric signals from the wearable device worn by the user. It receives heart rate, body temperature, and activity level data as input, collecting this data at approximately one-minute intervals. The terminal formats this data and outputs it to the server via a communication protocol. Specifically, the terminal periodically communicates with the device to acquire real-time data and perform format conversion.

[0463] Step 2:

[0464] The server receives biometric signals transmitted from the terminal. Using the input data, the data analysis device performs comparative analysis with historical data. For example, it uses time series analysis techniques and machine learning algorithms to detect anomalies and generate output that evaluates the user's health status. The server performs specific operations such as data cleansing, normalization, and input into the analysis model.

[0465] Step 3:

[0466] The device acquires communication data from the user's daily communication activities. It collects inputs such as email and chat text, and audio data from online meetings. The collected data is appropriately formatted, and output is generated that is transferred to the server using encryption protocols. Specifically, the device works in conjunction with the application being used to manage data capture and storage.

[0467] Step 4:

[0468] The server receives the collected communication data and analyzes it using natural language processing and speech analysis technologies. It performs sentiment analysis on the input text and audio data and generates output that calculates positive, negative, and neutral sentiment scores. Specifically, it uses a generative AI model to calculate the sentiment value of language and analyzes the tone of the audio data.

[0469] Step 5:

[0470] The server integrates the analysis results of biosignals and communication data to evaluate the user's emotional state and stress level. Based on this, if a high-stress state or emotional patterns requiring attention are detected, an alert is generated. The generated alert message is constructed and output for transmission to the terminal via a notification device.

[0471] Step 6:

[0472] The device receives alerts sent from the server and displays them to the user immediately. Specifically, it displays pop-up notifications on the screen or plays a notification sound. It also provides an interface that allows the user to access additional support and counseling services depending on the content of the alert. This helps the user take appropriate action based on the information.

[0473] (Application Example 1)

[0474] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0475] In managing the health of the elderly, there is a need to detect physical abnormalities and emotional stress early and respond quickly. However, conventional systems have difficulty integrating and handling this information, and in particular, capturing emotional changes has been challenging.

[0476] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0477] In this invention, the server includes a device for acquiring biometric information, a device for acquiring communication information, and an analysis device for analyzing the biometric information and communication information and determining the emotional state. This makes it possible to comprehensively monitor changes in biometric information and emotional changes in real time and generate appropriate warnings when an abnormality is detected.

[0478] "Biometric information" is a general term for data that indicates an individual's physical condition, such as heart rate, body temperature, and activity level.

[0479] "Device" refers to equipment or systems used to acquire or analyze biological information or communication information.

[0480] "Communication information" refers to text and audio data obtained from communication activities that users use on a daily basis, such as email, chat, and online meetings.

[0481] "Emotional state" refers to a user's psychological state expressed as a score, such as positive, negative, or neutral.

[0482] An "analysis device" refers to a device or software that determines a user's health status and emotions based on biological and communication information.

[0483] A "generator" is part of a system that creates warnings and notifications based on emotional states determined through analysis.

[0484] A "notification device" refers to a device or interface used as a means of informing a user of generated warnings or notifications.

[0485] A "remote monitoring device" is part of a system that allows the health status and emotions of specific individuals, such as the elderly, to be monitored from a distance.

[0486] This invention provides a system for monitoring the health and emotional state of elderly individuals in real time. The server acquires biometric information from wearable devices and other sensors to understand the user's activity status. It also acquires text and voice data generated from the communication methods the user uses daily and analyzes the emotional state based on this data.

[0487] Specifically, the server executes an analysis program using Python to detect trends and anomalies in biometric data. Google Cloud Natural Language API is used for analyzing communication data as a natural language processing technique, and Amazon Transcribe is used for analyzing voice data. The resulting sentiment score is expressed using three indicators: positive, negative, and neutral.

[0488] When signs of abnormality or stress are detected, the generator immediately generates a warning and sends an alert to caregivers or family members via a notification device. This alert allows for appropriate intervention.

[0489] For example, if a user is using words like "tired" more frequently than usual, and their heart rate is elevated, the system will quickly identify the stress and issue a warning. This allows for early intervention.

[0490] An example of a prompt using a generative AI model might be: "We want to monitor the health status of elderly people in real time and perform sentiment analysis. How can we implement anomaly detection and stress assessment using wearable devices and voice data?"

[0491] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0492] Step 1:

[0493] The device acquires biometric information from wearable devices and sensors. Inputs include heart rate, body temperature, and activity level, while output is a dataset containing this data. The device collects data from devices using Bluetooth or similar technologies and periodically transmits this data to a server.

[0494] Step 2:

[0495] The device acquires communication information. The input consists of text and audio data from emails, chats, online meetings, etc., used by the user, and the output is a dataset containing this data. The device extracts data from communication applications and uploads it to the server.

[0496] Step 3:

[0497] The server analyzes the received biometric information. The input is the biometric data set obtained in step 1, and the output is the analyzed health indicators and anomaly detection results. The server uses Python to perform statistical processing and anomaly detection algorithms to evaluate the health status.

[0498] Step 4:

[0499] The server analyzes the communication information and determines the emotional state. The input is the communication information dataset obtained in step 2, and the output is the emotional score. The server uses the Google Cloud Natural Language API for natural language processing, converts speech to text using Amazon Transcribe, and performs emotional analysis.

[0500] Step 5:

[0501] The server generates alerts based on emotion scores and health indicators. The input is the results of steps 3 and 4, and the output is the alert message. The server sets thresholds and immediately generates an alert and compiles the necessary information if an anomaly is detected.

[0502] Step 6:

[0503] The device receives warnings sent from the server and notifies the user or caregiver. The input is the warning message generated in step 5, and the output is the notification displayed on the device. The device uses push notification functionality to display the warning message on the screen and deploys an interface to provide additional support as needed.

[0504] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0505] To implement this invention, the terminal first acquires biometric information through the user's wearable device and transmits it to a server. This biometric information includes heart rate, body temperature, activity data, etc. Meanwhile, communication data from emails, business chats, and online meetings used by the user are also transmitted to the server via the terminal.

[0506] The server inputs this biometric and communication data into the emotion engine. The emotion engine uses natural language processing (NLP) techniques to analyze the communication data and calculate an emotion score. Furthermore, it detects abnormalities such as heart rate and body temperature from the biometric information and combines this data to comprehensively determine the user's emotional state.

[0507] The emotion engine uses machine learning algorithms to learn individual patterns from past data and predict changes in each user's emotions and stress levels. Based on these predictions, the emotion engine evaluates the user's emotional state in real time and generates alerts as needed.

[0508] Specifically, if a user frequently uses negative words such as "tough" or "anxious" due to work-related stress, the emotion engine records this as a negative score based on sentiment analysis. At the same time, if a wearable device detects a higher-than-normal heart rate, the server determines that the user may be experiencing increased stress.

[0509] Based on the results, the server generates an alert and sends it to the terminal. This alert might say something like, "Your stress level is high. Take a break and consider counseling if necessary." The terminal displays this message to the user, who can then access support services provided through the system as needed.

[0510] Thus, the system of the present invention, which combines an emotion engine, improves the user's health and work efficiency by closely monitoring the user's emotional state and stress level and providing appropriate support.

[0511] The following describes the processing flow.

[0512] Step 1:

[0513] The device acquires biometric information such as heart rate, body temperature, and activity level from the user's wearable device at regular intervals. This information is transmitted to the server in real time.

[0514] Step 2:

[0515] The device acquires communication data from email, business chat, and online meeting systems that the user uses on a daily basis. This data includes text messages and call content, which are also sent to the server.

[0516] Step 3:

[0517] The server receives biometric information transmitted from the terminal and evaluates the current health status by comparing it with past data. It can also determine the possibility of stress by detecting sudden changes in heart rate or abnormal body temperature.

[0518] Step 4:

[0519] The server inputs the transmitted communication data into the emotion engine. The emotion engine uses natural language processing to analyze the text and speech and calculates the user's emotion score. This score quantitatively evaluates positive, negative, and neutral emotions.

[0520] Step 5:

[0521] The emotion engine integrates the obtained emotion score with biometric information and uses machine learning algorithms to comprehensively determine the user's emotional state. Based on patterns learned from past data, it predicts stress levels and emotional changes.

[0522] Step 6:

[0523] Based on the results of the emotion engine, the server generates an alert if the user's emotional state or stress level exceeds a set threshold. This alert includes specific countermeasures and recommended actions.

[0524] Step 7:

[0525] The server sends the generated alert to the terminal. The terminal notifies and displays this alert to the user.

[0526] Step 8:

[0527] Users check alerts displayed on their devices to recognize their emotional state and stress levels. If necessary, they can access counseling or other support services from their devices and take appropriate action.

[0528] (Example 2)

[0529] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0530] With the increasing complexity of modern living and work environments, it is crucial to appropriately monitor and proactively address individual mental stress and emotional fluctuations. However, conventional systems have struggled to comprehensively analyze biometric and communication data to individually assess stress levels. This has resulted in a challenge where users are unable to manage their stress at the appropriate time.

[0531] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0532] In this invention, the server includes a device for acquiring biometric information, means for acquiring communication information, and a device for collecting and storing the biometric information and communication information. This enables the accurate monitoring of changes in emotions and stress based on the user's biological state and communication content, and allows the user to appropriately manage stress by issuing alerts in advance.

[0533] "Biometric information" refers to information about the user's physical condition, such as heart rate, body temperature, and activity data.

[0534] "Communication information" refers to information related to digital communication, such as emails, chat messages, and online meeting content sent and received by users.

[0535] A "device" is a system of hardware and software designed to perform a specific function.

[0536] "Means" refer to the methods or techniques used to achieve a specific objective.

[0537] "Analytical function" refers to the ability or technology to process data, extract specific information, and analyze it.

[0538] "Machine learning techniques" refer to algorithms and processes that use large amounts of data to build models and perform predictions and pattern recognition.

[0539] An "alert" is a notification or warning intended to draw attention to a specific situation or condition.

[0540] "Wearable technology" refers to devices and technologies designed to be worn on the body, primarily used for monitoring health and fitness information.

[0541] To implement this invention, the terminal must first acquire biometric information from the user's wearable device. This terminal connects to the device using Bluetooth technology and collects heart rate, body temperature, and activity data in real time. Next, the terminal transmits the collected data to a server. This data is securely transferred to the server via a communication line.

[0542] The server inputs the received biometric information and communication information (emails, chats, and online meeting logs) obtained from the user's device into an emotion analysis engine. This emotion analysis engine runs on a machine learning platform such as TensorFlow, and uses natural language processing techniques to analyze the communication information and calculate an emotion score. It also analyzes the biometric information to detect abnormalities in heart rate and body temperature.

[0543] Based on the analysis, the server comprehensively assesses the user's emotional state and generates an alert if the stress level exceeds a certain threshold. This alert is sent to the terminal and displayed to the user as a message such as, "Your stress level is high. Take a break and consider counseling if necessary."

[0544] Specifically, if a user uses negative expressions such as "tired" or "troubled" in a business chat, and the system detects that their heart rate is higher than normal, the system will determine that they are under stress and notify the user at an appropriate time.

[0545] An example of a prompt for the generating AI model would be, "What alert message should be generated when a user has a high heart rate and sends a negative chat message?" By utilizing this system in this way, it becomes possible to monitor the user's emotional state in detail and improve their health and work efficiency.

[0546] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0547] Step 1:

[0548] The terminal acquires biometric information from the wearable device. Specifically, the terminal connects to the user's wearable device via Bluetooth and collects heart rate, body temperature, and activity data in real time. At this point, the input is sensor data from the wearable device, and the output is biometric data formatted for analysis.

[0549] Step 2:

[0550] The device transmits biometric information to the server. The device transfers the collected biometric information to the server via the internet. The data is encrypted before transmission to enhance security. The input here is formatted biometric data, and the output is storage in an analysis database on the server.

[0551] Step 3:

[0552] The user's device collects communication information. When a user sends an email or engages in chat, the content of that communication is recorded. The input is the user's text communication data, and the output is communication information data for later analysis.

[0553] Step 4:

[0554] The terminal sends communication information to the server. The terminal encrypts the collected communication information using SSL and securely sends it to the server. In this case, the input is the communication information data, and the output is its storage in the server's analysis database.

[0555] Step 5:

[0556] The server inputs biometric and communication data into the sentiment analysis engine. The server inputs this data into the sentiment analysis engine, which uses TensorFlow, and starts the analysis. The input is a dataset of biometric and communication data, and the output is the intermediate analysis result.

[0557] Step 6:

[0558] The server analyzes the data and calculates an emotion score. It uses natural language processing techniques to analyze communication information and quantify the user's emotional state. It also analyzes biometric information and detects anomalies. The input is the intermediate analysis result, and the output is the emotion score and anomaly detection results.

[0559] Step 7:

[0560] The server comprehensively assesses the emotional state and generates alerts as needed. It integrates the emotional score and biometric data analysis results to evaluate the user's emotional state. If the evaluation exceeds a predetermined threshold, an alert is generated. The input is the emotional score and anomaly detection results, and the output is alert data.

[0561] Step 8:

[0562] The server generates alerts and sends them to the terminal. The server then sends the generated alerts to the user's terminal as push notifications. The input is the alert data, and the output is the notification sent to the user's terminal.

[0563] Step 9:

[0564] The terminal displays an alert to the user. The terminal displays the received alert on the screen to inform the user. The input is the alert notification from the server, and the output is the visual alert display to the user.

[0565] (Application Example 2)

[0566] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0567] Traditionally, stress management and improvement of working conditions for care staff have been crucial issues in caregiving settings. However, the provision of concrete stress assessments and support measures using biometric information and communication data has been insufficient. This could negatively impact the health and work efficiency of care staff. Therefore, there is a need for a system that can assess the stress levels of care staff in real time and provide appropriate support measures.

[0568] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0569] In this invention, the server includes detection means, acquisition means, analysis means, generation means, notification means, and evaluation means. This makes it possible to evaluate the stress level of care staff using their biometric and communication information and provide support measures as needed.

[0570] "Biometric information" refers to data that indicates an individual's physical condition, including information such as heart rate, body temperature, and activity level.

[0571] "Communication information" refers to data obtained from emails, business chats, online meetings, and other communications that individuals use on a daily basis.

[0572] "Emotional state" is an indicator that shows an individual's emotional response and stress level, obtained by analyzing biometric and communication information.

[0573] A "wearable device" is an electronic device that is attached to the body and can continuously measure biometric information.

[0574] An "emotion score" is a numerical measure of emotion calculated by analyzing the language data contained in communication information using natural language processing technology.

[0575] "Evaluation methods" refer to functions or processes that analyze the stress levels of care staff based on biometric information and emotional scores, and provide support measures based on the results.

[0576] A "generative AI model" is a form of artificial intelligence that uses machine learning algorithms to learn patterns from data and derive stress predictions and support measures.

[0577] To implement this invention, the server first acquires biometric information from a wearable device worn by the care staff. The wearable device continuously measures data such as heart rate, body temperature, and activity level, and transmits this information to the server. In addition, work-related communication information is acquired via the smartphone or tablet used by the care staff. This communication information includes emails, business chats, and records of work-related communications.

[0578] The server uses natural language processing (NLP) techniques to analyze communication information and calculate sentiment scores. This includes analyzing emotional responses to specific keywords and sentences. Furthermore, by combining biometric information with sentiment scores and utilizing a generative AI model, it predicts the stress levels of care staff.

[0579] Based on this information, the server uses assessment tools to generate an alert if it determines that the stress level is high. This alert sends a notification to the care staff's terminal, such as, "Your stress level is high. We recommend you take a short break." This allows the care staff to access support services provided through the system as needed.

[0580] As a concrete example, consider a situation where a caregiver is working long hours without taking breaks. In this case, a wearable device detects an abnormally high heart rate, and a negative emotion score is calculated from the communication data. The generative AI model analyzes this data and determines that the person is in a high-stress state. The server immediately generates an alert and sends a notification to the caregiver urging them to take a break.

[0581] Example prompt: "What kind of breaks and relaxation methods would you suggest to reduce stress for care staff?"

[0582] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0583] Step 1:

[0584] The server acquires biometric information from wearable devices. The input consists of biometric data such as heart rate, body temperature, and activity level, which are stored in a database on the server. The data is configured to be updated periodically.

[0585] Step 2:

[0586] The server acquires communication information related to the work through terminals used by care staff. The input consists of email and business chat messages, which are analyzed using natural language processing technology. As a result, emotion-related keywords and phrases are extracted and output as emotion scores.

[0587] Step 3:

[0588] The server combines acquired biometric and communication information and inputs it into a generating AI model. The inputs are heart rate, body temperature, activity level, and emotion score, which the AI ​​model uses to predict the stress level of care staff. The output is an evaluation of the stress level, which is returned to the server as a numerical index.

[0589] Step 4:

[0590] The server analyzes the condition of care staff using an evaluation method based on the stress levels output by the generated AI model. The input here is the evaluation result of the stress level, and the output is an evaluation result such as "high stress" or "moderate stress".

[0591] Step 5:

[0592] The server generates an alert if it determines that the stress level is high. The alert content is something like, "We recommend taking a short break," and is output as data to notify the user.

[0593] Step 6:

[0594] The terminal receives notifications from the server and displays alert messages to care staff. The input is alert information from the server, and the output is the message displayed on the terminal's screen. This allows care staff to take the instructed action immediately.

[0595] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0596] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0597] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0598] [Fourth Embodiment]

[0599] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0600] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0601] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0602] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0603] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0604] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0605] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0606] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0607] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0608] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0609] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0610] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0611] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0612] To implement the present invention, the following configuration is conceivable. First, the terminal periodically acquires biometric information through a wearable device worn by the user. This includes data such as heart rate, body temperature, and activity level. This information is transmitted to the server in real time.

[0613] Simultaneously, users communicate daily through email, business chat tools, and online meetings. The device acquires text and audio data generated from these communication activities. This acquired data is also sent to the server.

[0614] The server analyzes the user's physical health status and activity patterns based on the received biometric information. Specifically, it refers to historical biometric data and detects values ​​and patterns that deviate from the normal range. The server also uses natural language processing (NLP) and speech analysis techniques to analyze communication data and determine the user's emotional state. This takes into account factors such as word choice, tone of voice, and frequency of speech in communication, and is expressed as a positive, negative, or neutral emotional score.

[0615] Based on the analysis results, the server generates an alert if the emotional state exceeds a certain threshold or if high stress levels are suspected based on biometric information. Specifically, this alert is a message summarizing cautionary advice and recommended actions, and is sent to the terminal via a notification system.

[0616] The terminal displays alerts sent from the server to the user. Ideally, the system should provide an interface that allows the user to access additional support and counseling services as needed. This enables the user to recognize their own emotions and stress levels and take appropriate action.

[0617] As a concrete example, suppose a user is under pressure to meet a project deadline and starts making more negative statements such as "I'm tired" or "I can't do this anymore." At this time, the heart rate transmitted from the wearable device tends to be higher than usual. The server quickly analyzes this data, determines that stress levels are high, and sends an alert to the device saying, "You may be experiencing high stress levels recently. Take a rest and seek counseling if necessary." The user receives this notification, becomes aware of their condition, and can take appropriate action.

[0618] Thus, the system of the present invention makes a significant contribution to understanding the user's health status and providing appropriate support.

[0619] The following describes the processing flow.

[0620] Step 1:

[0621] The device acquires biometric information such as heart rate, body temperature, and activity level from the wearable device worn by the user at regular intervals. This data is transmitted in real time to the server via the cloud.

[0622] Step 2:

[0623] Users utilize email, business chat tools, and online meetings in their daily work. Their devices record text and audio data generated from these communication activities and send it to a server.

[0624] Step 3:

[0625] The server receives biometric information transmitted from the terminal and identifies anomalies by comparing it to a database with a set normal range. In this process, it detects rapid changes in heart rate and abnormal patterns in activity levels.

[0626] Step 4:

[0627] The server uses natural language processing (NLP) to analyze the user's communication data. Based on the content and tone of their speech, it calculates a sentiment score and determines whether the user is positive, negative, or neutral.

[0628] Step 5:

[0629] The server integrates biometric data and analyzed emotion scores to evaluate the user's overall emotional state. If the emotion score exceeds a set threshold and abnormalities are detected in the biometric data, the server generates a warning.

[0630] Step 6:

[0631] The server sends the generated warning as a message to the terminal. The message includes a warning about increased stress and recommended actions (e.g., taking a break, seeking counseling).

[0632] Step 7:

[0633] The terminal displays received messages to the user. Through these messages, the user can understand their current situation and, if necessary, access support services provided through the system.

[0634] (Example 1)

[0635] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0636] In modern society, there are concerns about the impact of daily stress and emotional fluctuations on health. However, conventional technologies have lacked the means to monitor an individual's biometric data and emotional state in real time and provide appropriate support based on that data. As a result, there is a challenge in that it is difficult for users to accurately understand their own health status and quickly access the support they need.

[0637] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0638] In this invention, the server includes a sensor device for acquiring biosignals, a data acquisition device for acquiring communication data, and a data analysis device for integrating the biosignals and communication data and identifying the emotional state. This makes it possible to accurately grasp the user's health and emotional state in real time and provide alerts at the appropriate time.

[0639] A "sensor device" is a device that detects biological signals and records and transmits that information.

[0640] A "data collection device" is a device that acquires communication data and stores it for analysis.

[0641] A "data analysis device" is a device that integrates and analyzes biological signals and communication data to identify emotional states.

[0642] An "alert generator" is a device that generates warnings and recommendations to notify users based on their emotional state, as identified by a data analysis device.

[0643] A "notification device" is a device that has an interface for communicating generated alerts to the user.

[0644] A "device that provides an interface" is a device that has the functionality to allow users to access additional support services.

[0645] "Biosignals" refer to physiological parameters of the body, such as heart rate and body temperature, that are acquired through wearable devices.

[0646] "Communication data" refers to data including text and audio information obtained from users' emails, chats, and voice calls.

[0647] To implement the invention, it is necessary to configure a system that links a server, a terminal, and a wearable device. First, the terminal periodically acquires biometric signals such as heart rate, body temperature, and activity level from the wearable device worn by the user. To do this, communication technologies such as Bluetooth and Wi-Fi are used to stably transfer the biometric signals to the server.

[0648] When the server receives biometric signals, it uses a data analysis device to compare and analyze these signals with historical data. Time series analysis and peak detection algorithms are used to identify patterns that deviate from the normal range.

[0649] Furthermore, the device collects user communication data, emails, chats, and voice recordings through a data collection device and sends this data to a server. The server uses a generative AI model equipped with natural language processing and speech analysis technologies to analyze the text and voice data and calculate sentiment scores.

[0650] Based on the analysis results, the server generates a message using an alert generator if it detects a high-stress state or a specific emotional state. The alert is sent to the terminal via a notification device, and the user receives it. Furthermore, an interface is displayed on the terminal that allows the user to easily access support services and counseling services if needed.

[0651] As a concrete example, suppose a user is overwhelmed with a project deadline and starts using negative expressions such as "I'm tired" and "I can't do this anymore." In this case, the server quickly analyzes the increased heart rate pattern obtained from the wearable device and determines that stress is increasing. The server sends an alert to the device saying, "A high level of stress has been detected recently. Take a rest and seek support if necessary." The user receives this notification, understands their situation, and can take appropriate action.

[0652] An example of a prompt message might be a request like, "Please describe a system that determines the user's stress level based on data and communication content acquired from a wearable device and issues an alert as needed."

[0653] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0654] Step 1:

[0655] The terminal acquires biometric signals from the wearable device worn by the user. It receives heart rate, body temperature, and activity level data as input, collecting this data at approximately one-minute intervals. The terminal formats this data and outputs it to the server via a communication protocol. Specifically, the terminal periodically communicates with the device to acquire real-time data and perform format conversion.

[0656] Step 2:

[0657] The server receives biometric signals transmitted from the terminal. Using the input data, the data analysis device performs comparative analysis with historical data. For example, it uses time series analysis techniques and machine learning algorithms to detect anomalies and generate output that evaluates the user's health status. The server performs specific operations such as data cleansing, normalization, and input into the analysis model.

[0658] Step 3:

[0659] The device acquires communication data from the user's daily communication activities. It collects inputs such as email and chat text, and audio data from online meetings. The collected data is appropriately formatted, and output is generated that is transferred to the server using encryption protocols. Specifically, the device works in conjunction with the application being used to manage data capture and storage.

[0660] Step 4:

[0661] The server receives the collected communication data and analyzes it using natural language processing and speech analysis technologies. It performs sentiment analysis on the input text and audio data and generates output that calculates positive, negative, and neutral sentiment scores. Specifically, it uses a generative AI model to calculate the sentiment value of language and analyzes the tone of the audio data.

[0662] Step 5:

[0663] The server integrates the analysis results of biosignals and communication data to evaluate the user's emotional state and stress level. Based on this, if a high-stress state or emotional patterns requiring attention are detected, an alert is generated. The generated alert message is constructed and output for transmission to the terminal via a notification device.

[0664] Step 6:

[0665] The device receives alerts sent from the server and displays them to the user immediately. Specifically, it displays pop-up notifications on the screen or plays a notification sound. It also provides an interface that allows the user to access additional support and counseling services depending on the content of the alert. This helps the user take appropriate action based on the information.

[0666] (Application Example 1)

[0667] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] In managing the health of the elderly, there is a need to detect physical abnormalities and emotional stress early and respond quickly. However, conventional systems have difficulty integrating and handling this information, and in particular, capturing emotional changes has been challenging.

[0669] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0670] In this invention, the server includes a device for acquiring biometric information, a device for acquiring communication information, and an analysis device for analyzing the biometric information and communication information and determining the emotional state. This makes it possible to comprehensively monitor changes in biometric information and emotional changes in real time and generate appropriate warnings when an abnormality is detected.

[0671] "Biometric information" is a general term for data that indicates an individual's physical condition, such as heart rate, body temperature, and activity level.

[0672] "Device" refers to equipment or systems used to acquire or analyze biological information or communication information.

[0673] "Communication information" refers to text and audio data obtained from communication activities that users use on a daily basis, such as email, chat, and online meetings.

[0674] "Emotional state" refers to a user's psychological state expressed as a score, such as positive, negative, or neutral.

[0675] An "analysis device" refers to a device or software that determines a user's health status and emotions based on biological and communication information.

[0676] A "generator" is part of a system that creates warnings and notifications based on emotional states determined through analysis.

[0677] A "notification device" refers to a device or interface used as a means of informing a user of generated warnings or notifications.

[0678] A "remote monitoring device" is part of a system that allows the health status and emotions of specific individuals, such as the elderly, to be monitored from a distance.

[0679] This invention provides a system for monitoring the health and emotional state of elderly individuals in real time. The server acquires biometric information from wearable devices and other sensors to understand the user's activity status. It also acquires text and voice data generated from the communication methods the user uses daily and analyzes the emotional state based on this data.

[0680] Specifically, the server executes an analysis program using Python to detect trends and anomalies in biometric data. Google Cloud Natural Language API is used for analyzing communication data as a natural language processing technique, and Amazon Transcribe is used for analyzing voice data. The resulting sentiment score is expressed using three indicators: positive, negative, and neutral.

[0681] When signs of abnormality or stress are detected, the generator immediately generates a warning and sends an alert to caregivers or family members via a notification device. This alert allows for appropriate intervention.

[0682] For example, if a user is using words like "tired" more frequently than usual, and their heart rate is elevated, the system will quickly identify the stress and issue a warning. This allows for early intervention.

[0683] An example of a prompt using a generative AI model might be: "We want to monitor the health status of elderly people in real time and perform sentiment analysis. How can we implement anomaly detection and stress assessment using wearable devices and voice data?"

[0684] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0685] Step 1:

[0686] The device acquires biometric information from wearable devices and sensors. Inputs include heart rate, body temperature, and activity level, while output is a dataset containing this data. The device collects data from devices using Bluetooth or similar technologies and periodically transmits this data to a server.

[0687] Step 2:

[0688] The device acquires communication information. The input consists of text and audio data from emails, chats, online meetings, etc., used by the user, and the output is a dataset containing this data. The device extracts data from communication applications and uploads it to the server.

[0689] Step 3:

[0690] The server analyzes the received biometric information. The input is the biometric data set obtained in step 1, and the output is the analyzed health indicators and anomaly detection results. The server uses Python to perform statistical processing and anomaly detection algorithms to evaluate the health status.

[0691] Step 4:

[0692] The server analyzes the communication information and determines the emotional state. The input is the communication information dataset obtained in step 2, and the output is the emotional score. The server uses the Google Cloud Natural Language API for natural language processing, converts speech to text using Amazon Transcribe, and performs emotional analysis.

[0693] Step 5:

[0694] The server generates alerts based on emotion scores and health indicators. The input is the results of steps 3 and 4, and the output is the alert message. The server sets thresholds and immediately generates an alert and compiles the necessary information if an anomaly is detected.

[0695] Step 6:

[0696] The device receives warnings sent from the server and notifies the user or caregiver. The input is the warning message generated in step 5, and the output is the notification displayed on the device. The device uses push notification functionality to display the warning message on the screen and deploys an interface to provide additional support as needed.

[0697] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0698] To implement this invention, the terminal first acquires biometric information through the user's wearable device and transmits it to a server. This biometric information includes heart rate, body temperature, activity data, etc. Meanwhile, communication data from emails, business chats, and online meetings used by the user are also transmitted to the server via the terminal.

[0699] The server inputs this biometric and communication data into the emotion engine. The emotion engine uses natural language processing (NLP) techniques to analyze the communication data and calculate an emotion score. Furthermore, it detects abnormalities such as heart rate and body temperature from the biometric information and combines this data to comprehensively determine the user's emotional state.

[0700] The emotion engine uses machine learning algorithms to learn individual patterns from past data and predict changes in each user's emotions and stress levels. Based on these predictions, the emotion engine evaluates the user's emotional state in real time and generates alerts as needed.

[0701] Specifically, if a user frequently uses negative words such as "tough" or "anxious" due to work-related stress, the emotion engine records this as a negative score based on sentiment analysis. At the same time, if a wearable device detects a higher-than-normal heart rate, the server determines that the user may be experiencing increased stress.

[0702] Based on the results, the server generates an alert and sends it to the terminal. This alert might say something like, "Your stress level is high. Take a break and consider counseling if necessary." The terminal displays this message to the user, who can then access support services provided through the system as needed.

[0703] Thus, the system of the present invention, which combines an emotion engine, improves the user's health and work efficiency by closely monitoring the user's emotional state and stress level and providing appropriate support.

[0704] The following describes the processing flow.

[0705] Step 1:

[0706] The device acquires biometric information such as heart rate, body temperature, and activity level from the user's wearable device at regular intervals. This information is transmitted to the server in real time.

[0707] Step 2:

[0708] The device acquires communication data from email, business chat, and online meeting systems that the user uses on a daily basis. This data includes text messages and call content, which are also sent to the server.

[0709] Step 3:

[0710] The server receives biometric information transmitted from the terminal and evaluates the current health status by comparing it with past data. It can also determine the possibility of stress by detecting sudden changes in heart rate or abnormal body temperature.

[0711] Step 4:

[0712] The server inputs the transmitted communication data into the emotion engine. The emotion engine uses natural language processing to analyze the text and speech and calculates the user's emotion score. This score quantitatively evaluates positive, negative, and neutral emotions.

[0713] Step 5:

[0714] The emotion engine integrates the obtained emotion score with biometric information and uses machine learning algorithms to comprehensively determine the user's emotional state. Based on patterns learned from past data, it predicts stress levels and emotional changes.

[0715] Step 6:

[0716] Based on the results of the emotion engine, the server generates an alert if the user's emotional state or stress level exceeds a set threshold. This alert includes specific countermeasures and recommended actions.

[0717] Step 7:

[0718] The server sends the generated alert to the terminal. The terminal notifies and displays this alert to the user.

[0719] Step 8:

[0720] Users check alerts displayed on their devices to recognize their emotional state and stress levels. If necessary, they can access counseling or other support services from their devices and take appropriate action.

[0721] (Example 2)

[0722] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0723] With the increasing complexity of modern living and work environments, it is crucial to appropriately monitor and proactively address individual mental stress and emotional fluctuations. However, conventional systems have struggled to comprehensively analyze biometric and communication data to individually assess stress levels. This has resulted in a challenge where users are unable to manage their stress at the appropriate time.

[0724] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0725] In this invention, the server includes a device for acquiring biometric information, means for acquiring communication information, and a device for collecting and storing the biometric information and communication information. This enables the accurate monitoring of changes in emotions and stress based on the user's biological state and communication content, and allows the user to appropriately manage stress by issuing alerts in advance.

[0726] "Biometric information" refers to information about the user's physical condition, such as heart rate, body temperature, and activity data.

[0727] "Communication information" refers to information related to digital communication, such as emails, chat messages, and online meeting content sent and received by users.

[0728] A "device" is a system of hardware and software designed to perform a specific function.

[0729] "Means" refer to the methods or techniques used to achieve a specific objective.

[0730] "Analytical function" refers to the ability or technology to process data, extract specific information, and analyze it.

[0731] "Machine learning techniques" refer to algorithms and processes that use large amounts of data to build models and perform predictions and pattern recognition.

[0732] An "alert" is a notification or warning intended to draw attention to a specific situation or condition.

[0733] "Wearable technology" refers to devices and technologies designed to be worn on the body, primarily used for monitoring health and fitness information.

[0734] To implement this invention, the terminal must first acquire biometric information from the user's wearable device. This terminal connects to the device using Bluetooth technology and collects heart rate, body temperature, and activity data in real time. Next, the terminal transmits the collected data to a server. This data is securely transferred to the server via a communication line.

[0735] The server inputs the received biometric information and communication information (emails, chats, and online meeting logs) obtained from the user's device into an emotion analysis engine. This emotion analysis engine runs on a machine learning platform such as TensorFlow, and uses natural language processing techniques to analyze the communication information and calculate an emotion score. It also analyzes the biometric information to detect abnormalities in heart rate and body temperature.

[0736] Based on the analysis, the server comprehensively assesses the user's emotional state and generates an alert if the stress level exceeds a certain threshold. This alert is sent to the terminal and displayed to the user as a message such as, "Your stress level is high. Take a break and consider counseling if necessary."

[0737] Specifically, if a user uses negative expressions such as "tired" or "troubled" in a business chat, and the system detects that their heart rate is higher than normal, the system will determine that they are under stress and notify the user at an appropriate time.

[0738] An example of a prompt for the generating AI model would be, "What alert message should be generated when a user has a high heart rate and sends a negative chat message?" By utilizing this system in this way, it becomes possible to monitor the user's emotional state in detail and improve their health and work efficiency.

[0739] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0740] Step 1:

[0741] The terminal acquires biometric information from the wearable device. Specifically, the terminal connects to the user's wearable device via Bluetooth and collects heart rate, body temperature, and activity data in real time. At this point, the input is sensor data from the wearable device, and the output is biometric data formatted for analysis.

[0742] Step 2:

[0743] The device transmits biometric information to the server. The device transfers the collected biometric information to the server via the internet. The data is encrypted before transmission to enhance security. The input here is formatted biometric data, and the output is storage in an analysis database on the server.

[0744] Step 3:

[0745] The user's device collects communication information. When a user sends an email or engages in chat, the content of that communication is recorded. The input is the user's text communication data, and the output is communication information data for later analysis.

[0746] Step 4:

[0747] The terminal sends communication information to the server. The terminal encrypts the collected communication information using SSL and securely sends it to the server. In this case, the input is the communication information data, and the output is its storage in the server's analysis database.

[0748] Step 5:

[0749] The server inputs biometric and communication data into the sentiment analysis engine. The server inputs this data into the sentiment analysis engine, which uses TensorFlow, and starts the analysis. The input is a dataset of biometric and communication data, and the output is the intermediate analysis result.

[0750] Step 6:

[0751] The server analyzes the data and calculates an emotion score. It uses natural language processing techniques to analyze communication information and quantify the user's emotional state. It also analyzes biometric information and detects anomalies. The input is the intermediate analysis result, and the output is the emotion score and anomaly detection results.

[0752] Step 7:

[0753] The server comprehensively assesses the emotional state and generates alerts as needed. It integrates the emotional score and biometric data analysis results to evaluate the user's emotional state. If the evaluation exceeds a predetermined threshold, an alert is generated. The input is the emotional score and anomaly detection results, and the output is alert data.

[0754] Step 8:

[0755] The server generates alerts and sends them to the terminal. The server then sends the generated alerts to the user's terminal as push notifications. The input is the alert data, and the output is the notification sent to the user's terminal.

[0756] Step 9:

[0757] The terminal displays an alert to the user. The terminal displays the received alert on the screen to inform the user. The input is the alert notification from the server, and the output is the visual alert display to the user.

[0758] (Application Example 2)

[0759] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0760] Traditionally, stress management and improvement of working conditions for care staff have been crucial issues in caregiving settings. However, the provision of concrete stress assessments and support measures using biometric information and communication data has been insufficient. This could negatively impact the health and work efficiency of care staff. Therefore, there is a need for a system that can assess the stress levels of care staff in real time and provide appropriate support measures.

[0761] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0762] In this invention, the server includes detection means, acquisition means, analysis means, generation means, notification means, and evaluation means. This makes it possible to evaluate the stress level of care staff using their biometric and communication information and provide support measures as needed.

[0763] "Biometric information" refers to data that indicates an individual's physical condition, including information such as heart rate, body temperature, and activity level.

[0764] "Communication information" refers to data obtained from emails, business chats, online meetings, and other communications that individuals use on a daily basis.

[0765] "Emotional state" is an indicator that shows an individual's emotional response and stress level, obtained by analyzing biometric and communication information.

[0766] A "wearable device" is an electronic device that is attached to the body and can continuously measure biometric information.

[0767] An "emotion score" is a numerical measure of emotion calculated by analyzing the language data contained in communication information using natural language processing technology.

[0768] "Evaluation methods" refer to functions or processes that analyze the stress levels of care staff based on biometric information and emotional scores, and provide support measures based on the results.

[0769] A "generative AI model" is a form of artificial intelligence that uses machine learning algorithms to learn patterns from data and derive stress predictions and support measures.

[0770] To implement this invention, the server first acquires biometric information from a wearable device worn by the care staff. The wearable device continuously measures data such as heart rate, body temperature, and activity level, and transmits this information to the server. In addition, work-related communication information is acquired via the smartphone or tablet used by the care staff. This communication information includes emails, business chats, and records of work-related communications.

[0771] The server uses natural language processing (NLP) techniques to analyze communication information and calculate sentiment scores. This includes analyzing emotional responses to specific keywords and sentences. Furthermore, by combining biometric information with sentiment scores and utilizing a generative AI model, it predicts the stress levels of care staff.

[0772] Based on this information, the server uses assessment tools to generate an alert if it determines that the stress level is high. This alert sends a notification to the care staff's terminal, such as, "Your stress level is high. We recommend you take a short break." This allows the care staff to access support services provided through the system as needed.

[0773] As a concrete example, consider a situation where a caregiver is working long hours without taking breaks. In this case, a wearable device detects an abnormally high heart rate, and a negative emotion score is calculated from the communication data. The generative AI model analyzes this data and determines that the person is in a high-stress state. The server immediately generates an alert and sends a notification to the caregiver urging them to take a break.

[0774] Example prompt: "What kind of breaks and relaxation methods would you suggest to reduce stress for care staff?"

[0775] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0776] Step 1:

[0777] The server acquires biometric information from wearable devices. The input consists of biometric data such as heart rate, body temperature, and activity level, which are stored in a database on the server. The data is configured to be updated periodically.

[0778] Step 2:

[0779] The server acquires communication information related to the work through terminals used by care staff. The input consists of email and business chat messages, which are analyzed using natural language processing technology. As a result, emotion-related keywords and phrases are extracted and output as emotion scores.

[0780] Step 3:

[0781] The server combines acquired biometric and communication information and inputs it into a generating AI model. The inputs are heart rate, body temperature, activity level, and emotion score, which the AI ​​model uses to predict the stress level of care staff. The output is an evaluation of the stress level, which is returned to the server as a numerical index.

[0782] Step 4:

[0783] The server analyzes the condition of care staff using an evaluation method based on the stress levels output by the generated AI model. The input here is the evaluation result of the stress level, and the output is an evaluation result such as "high stress" or "moderate stress".

[0784] Step 5:

[0785] The server generates an alert if it determines that the stress level is high. The alert content is something like, "We recommend taking a short break," and is output as data to notify the user.

[0786] Step 6:

[0787] The terminal receives notifications from the server and displays alert messages to care staff. The input is alert information from the server, and the output is the message displayed on the terminal's screen. This allows care staff to take the instructed action immediately.

[0788] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0789] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0790] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0791] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0792] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0793] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0794] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0795] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0796] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0797] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0798] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0799] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0800] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0801] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0802] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0803] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0804] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0805] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0806] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0807] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0808] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0809] The following is further disclosed regarding the embodiments described above.

[0810] (Claim 1)

[0811] A sensor means for acquiring biological information,

[0812] A means of acquiring communication information,

[0813] An analysis means for analyzing the aforementioned biological information and communication information and determining the emotional state,

[0814] A generation means for generating an alert based on the aforementioned emotional state,

[0815] A notification means for outputting the aforementioned alert,

[0816] A system that includes this.

[0817] (Claim 2)

[0818] The system according to claim 1, characterized in that the analysis means analyzes the communication information using natural language processing technology and calculates an emotion score.

[0819] (Claim 3)

[0820] The system according to claim 1, characterized in that the biometric information includes data related to physical activity measured by a wearable device.

[0821] "Example 1"

[0822] (Claim 1)

[0823] A sensor device that acquires biological signals,

[0824] A data collection device that acquires communication data,

[0825] A data analysis device that integrates the aforementioned biosignals and communication data to identify emotional states,

[0826] An alert generation device that detects high-stress states and emotional fluctuations based on the aforementioned analysis and generates an alert,

[0827] A notification device that presents the aforementioned alert to the user,

[0828] A device that provides an interface that allows access to support services through the aforementioned notification device,

[0829] A system that includes this.

[0830] (Claim 2)

[0831] The system according to claim 1, wherein the data analysis device analyzes the communication data using natural language processing technology and speech analysis technology and calculates an emotion score.

[0832] (Claim 3)

[0833] The system according to claim 1, which includes biological data detected by a physical activity monitoring device.

[0834] "Application Example 1"

[0835] (Claim 1)

[0836] A device for acquiring biological information,

[0837] A device for acquiring communication information,

[0838] An analytical device that analyzes biometric and communication information to determine emotional state,

[0839] A generator that generates warnings based on emotional states,

[0840] A notification device that outputs a warning,

[0841] A remote monitoring device for monitoring the health status of the elderly,

[0842] A system that includes this.

[0843] (Claim 2)

[0844] The system according to claim 1, characterized in that the analysis device analyzes communication information using natural language processing technology and calculates an emotion score.

[0845] (Claim 3)

[0846] The system according to claim 1, characterized in that it includes data related to physical activity measured by a wearable device that provides biometric information.

[0847] "Example 2 of combining an emotion engine"

[0848] (Claim 1)

[0849] A device for acquiring biological information,

[0850] Means for obtaining communication information,

[0851] A device for collecting and storing the aforementioned biological information and communication information,

[0852] The function analyzes the aforementioned biological information and communication information to determine the emotional state,

[0853] A device that uses machine learning technology to predict anomalies based on emotional states and generate alerts,

[0854] A means of outputting the aforementioned alert and notifying the user,

[0855] An information processing system that includes this.

[0856] (Claim 2)

[0857] The information processing system according to claim 1, characterized in that the analysis function analyzes the communication information using natural language processing technology and calculates an emotion score.

[0858] (Claim 3)

[0859] The information processing system according to claim 1, characterized in that the aforementioned biometric information includes data related to physical activity measured by wearable technology.

[0860] "Application example 2 when combining with an emotional engine"

[0861] (Claim 1)

[0862] A detection means for acquiring biological information,

[0863] A means of acquiring communication information,

[0864] An analysis means for analyzing the aforementioned biological information and communication information and determining the emotional state,

[0865] A generation means for generating an alert based on the aforementioned emotional state,

[0866] A notification means for outputting the aforementioned alert,

[0867] An evaluation tool for assessing the stress levels of care workers and providing support measures as needed,

[0868] A system that includes this.

[0869] (Claim 2)

[0870] The analysis means analyzes the communication information using natural language processing technology and calculates an emotion score.

[0871] The system according to claim 1, characterized in that the evaluation means predicts the stress level using biometric information and an emotional score.

[0872] (Claim 3)

[0873] The biometric information includes data related to physical activity measured by a wearable device.

[0874] The system according to claim 1, characterized in that the evaluation means proposes stress reduction support measures using a generated AI model. [Explanation of symbols]

[0875] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A sensor means for acquiring biological information, A means of acquiring communication information, An analysis means for analyzing the aforementioned biological information and communication information and determining the emotional state, A generation means for generating an alert based on the aforementioned emotional state, A notification means for outputting the aforementioned alert, A system that includes this.

2. The system according to claim 1, characterized in that the analysis means analyzes the communication information using natural language processing technology and calculates an emotion score.

3. The system according to claim 1, characterized in that the biometric information includes data related to physical activity measured by a wearable device.

Images