system

Abstract

Provide a system. [Solution] means for collecting vital data of a user; means for collecting audio data; means for analyzing the vital data and voice data to assess the emotional state of the user; means for generating a notification when the user's anger emotion exceeds a certain threshold; The system includes means for presenting the notification to a user and assisting the user in managing their emotions.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to a system for effectively managing anger felt by users in their daily lives and at work. It has traditionally been difficult to properly manage human emotions, particularly anger, and excessive anger can have a negative impact on health and interpersonal relationships, especially in stressful environments. The present invention proposes a system that collects a user's vital signs and voice data in real time and provides the user with appropriate anger management techniques when anger builds up, thereby suppressing impulsive speech and behavior and achieving healthy emotional management. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means.

[0006] First, it provides a means for collecting the user's vital data (e.g., pulse data). Next, it provides a means for collecting the user's voice data. It then provides a means for sending this data to a server, where an AI model deployed on the server analyzes the data and evaluates the user's emotional state. If the user's anger exceeds a certain threshold, it provides a means for generating an alert and displaying a message encouraging the user to take deep breaths or to leave the area. It also provides a means for collecting user feedback and updating the AI ​​model to improve the accuracy of the system. In this way, it becomes possible to support the user in managing their emotions and take appropriate measures even in stressful environments.

[0007] "Vital data" refers to the user's biological information, and specifically includes data such as pulse rate, respiratory rate, and body temperature.

[0008] "Audio data" refers to data that captures the user's voice and the surrounding acoustic environment.

[0009] "Emotional state" refers to the level of a particular emotion felt by the user, and in the present invention, the emotion of anger is particularly evaluated.

[0010] A "threshold" refers to a reference value above which a user's emotional state is determined to exceed a particular level.

[0011] "Alert" refers to a warning message that is sent to the user when their emotional state reaches a dangerous level.

[0012] "AI model" refers to a computational model that uses machine learning algorithms to analyze a user's vital data and voice data and assess their emotional state.

[0013] "Feedback" refers to information that a user inputs or reports as a result of their reaction to the system's instructions or advice.

[0014] A "deep breath prompt" refers to a notification that instructs a user to take a deep breath for a specific period of time.

[0015] "Notification" refers to information, including messages and advice, that is displayed to the user.

[0016] "Server" refers to a central computing device that manages the entire system of the present invention, analyzes data, and generates notifications. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The system of the present invention comprises an apparatus and a method for effectively managing anger felt by users in their daily lives or at work. Specifically, the system collects vital data and voice data of users in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary.

[0039] Program processing

[0040] 1. Data Collection

[0041] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[0042] The device recognizes the audio environment around the user in real time and collects audio data, focusing in particular on the user's voice, capturing the tone and volume of the voice.

[0043] The terminal periodically transmits the collected pulse data and voice data to the server.

[0044] 2. Data Analysis

[0045] The server receives the vital data (pulse data) and voice data sent from the terminal.

[0046] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[0047] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[0048] 3. Emotional assessment

[0049] The server combines vital data and voice data to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[0050] The server quantifies the user's anger level and determines whether it exceeds a certain threshold.

[0051] 4. Alert Generation and Notification

[0052] If the server evaluates the emotion and finds that the anger level exceeds a threshold, it generates an alert.

[0053] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[0054] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[0055] 5. Gather feedback and update the model

[0056] After the user follows the instructions of the system, they input feedback into the device, such as how their emotions changed and whether the instructions were helpful.

[0057] The device sends the collected feedback to the server.

[0058] The server updates the AI ​​model based on the feedback, improving the accuracy of the next emotion assessment.

[0059] Specific examples

[0060] Example 1: Frustration at work

[0061] 1. Data Collection

[0062] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[0063] The device collects the user's voice in real time and transmits it to the server.

[0064] 2. Data analysis and emotion assessment

[0065] The server detects a spike in pulse rate and an angry tone.

[0066] The server quantifies the anger level and determines that it has reached an alert level.

[0067] 3. Display of notifications

[0068] The device displays the message "Take a deep breath for 6 seconds" to the user.

[0069] 4. Gather feedback and update the model

[0070] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[0071] The server receives the feedback and fine-tunes the AI ​​model.

[0072] Example 2: Stress during child-rearing

[0073] 1. Data Collection

[0074] The device measures the user's pulse every 30 seconds.

[0075] The device collects conversations with family members and sends them to a server.

[0076] 2. Data analysis and emotion assessment

[0077] The server detects pulse spikes and vocal anger.

[0078] The server determines the anger level and assesses the user's stress state.

[0079] 3. Display of notifications

[0080] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[0081] 4. Gather feedback and update the model

[0082] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[0083] The server receives the feedback and updates the AI ​​model.

[0084] This allows the AI-powered anger management system to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[0085] The processing flow will be explained below.

[0086] Step 1:

[0087] The device (smartwatch) measures the user's pulse at regular intervals. Specifically, it uses a sensor to record the user's heart rate in real time.

[0088] Step 2:

[0089] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[0090] Step 3:

[0091] The device periodically transmits the collected pulse and voice data to a server via wireless communication.

[0092] Step 4:

[0093] The server receives vital data (pulse data) and voice data sent from the device and stores this data for analysis.

[0094] Step 5:

[0095] The AI ​​model on the server analyzes the received vital data, detects sudden spikes in pulse rate and abnormal fluctuations, and evaluates the heart rate change pattern to estimate the user's emotional state.

[0096] Step 6:

[0097] The server analyzes the voice data and recognizes the tone and vocabulary of the user's voice. It then uses a voice analysis algorithm to evaluate the expression and intensity of anger.

[0098] Step 7:

[0099] The server combines vital and voice data to assess the user's emotional state, and an AI model uses this data to quantify the user's anger level.

[0100] Step 8:

[0101] The server generates an alert when the user's anger level exceeds a certain threshold, and determines the composition of the warning message and instructions for next steps.

[0102] Step 9:

[0103] The server sends the generated alert to the terminal, and the alert data is transferred to the terminal via wireless communication.

[0104] Step 10:

[0105] The device will then prompt the user with appropriate anger management techniques, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you take a break from the situation."

[0106] Step 11:

[0107] The user acts according to the system's instructions, taking specific actions such as taking a deep breath or leaving the area.

[0108] Step 12:

[0109] After the user follows the system's instructions, they enter their feedback into a terminal, where they are provided with input fields to report how their emotions changed and whether the technique was helpful.

[0110] Step 13:

[0111] The terminal sends the user's feedback to the server. The input feedback data is periodically transferred to the server.

[0112] Step 14:

[0113] The server receives the feedback data and updates the AI ​​model, training the model based on the feedback and using it for the next emotion assessment.

[0114] Through these steps, the system supports users in managing their emotions and provides real-time information to promote a healthy lifestyle.

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] Conventional emotion management systems lack the functionality to integrate a user's biometric and voice data to assess their emotional state in real time and provide appropriate measures. Furthermore, they lack the functionality to improve the system based on user feedback, making it difficult to continuously provide users with appropriate advice.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes means for collecting biometric data of a user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotion management, means for collecting post-action feedback from the user, and means for analyzing the feedback and updating the evaluation model. This makes it possible to continuously monitor the user's emotional state in real time, provide an appropriate management method, and automatically improve the system based on the feedback.

[0120] "Biometric data" refers to information related to the user's bodily functions, and primarily includes vital signs such as pulse data.

[0121] "Voice data" refers to information related to the user's voice, including, among other things, tone and volume of speech.

[0122] "Analysis" refers to the process of detecting patterns and anomalies from collected data and extracting meaningful information.

[0123] The "emotional state" is an index that indicates the type and intensity of the emotion that the user is feeling.

[0124] "Notification" refers to a warning or suggestion message generated by the system to the user.

[0125] "Feedback" refers to the evaluation and opinion information that a user provides to a system after taking an action.

[0126] "Assessment Model" refers to a mathematical or statistical model, including artificial intelligence, used to analyze and assess a user's emotional state.

[0127] "Collection methods" refers to the devices, software, and methods used to obtain data.

[0128] "Analysis means" refers to devices, software, and methods for analyzing collected data and extracting information.

[0129] "Notification means" refers to a device or software for conveying the generated message to the user.

[0130] "Feedback collection means" refers to a device, software, or method for obtaining feedback from users.

[0131] "Evaluation model update means" refers to devices, software, and methods for improving the model based on collected feedback.

[0132] The system of the present invention comprises an apparatus and a method for collecting and analyzing a user's biometric and voice data to evaluate the user's emotional state and provide the user with appropriate anger management techniques as needed. Specifically, the system is implemented using the following hardware and software.

[0133] Hardware and software used

[0134] 1. Hardware: Smartwatches and wearable devices

[0135] 2. Software: AI models, servers, speech recognition software

[0136] Implementation details

[0137] 1. Data Collection

[0138] The device (smartwatch) collects the user's biometric data. Specifically, the device's optical sensor measures the user's pulse every 60 seconds and records heart rate fluctuations.

[0139] The device recognizes the user's surrounding sounds in real time, particularly capturing the tone and volume of the user's speech, by collecting voice data using the device's built-in microphone and temporarily storing it in local memory.

[0140] The collected pulse and voice data is periodically sent to a server via Wi-Fi or Bluetooth.

[0141] 2. Data Analysis

[0142] The server receives the biometric data and voice data sent from the terminal and stores them in a database.

[0143] The AI ​​model deployed on the server analyzes biometric data and detects sudden increases or abnormal fluctuations in pulse rate over a certain period of time. Specifically, if the heart rate rises by more than 30% from a normal resting level, an abnormality is detected.

[0144] The server uses voice recognition software to analyze the audio data, analyzing changes in tone and volume of the user's speech and identifying patterns that indicate anger, such as a sudden increase in volume or high-pitched sounds.

[0145] 3. Emotional assessment

[0146] The server combines the analysis results of biometric data and voice data and uses an AI model to assess the user's emotional state. If a sudden increase in pulse rate and an increase in high-pitched voice are observed simultaneously, it is determined to be angry.

[0147] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10).

[0148] 4. Alert Generation and Notification

[0149] The server generates an immediate alert if the emotion assessment reveals that the anger level exceeds a threshold.

[0150] The server generates a message suggesting appropriate anger management techniques to the user (e.g., "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend that you leave the situation for a while") and sends it to the device.

[0151] The device displays messages from the server on its display and prompts the user to take action, including text messages and audio alerts.

[0152] 5. Gather feedback and update the model

[0153] After following the system's suggestions, the user inputs feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they use the device's interface to input information such as "I felt calmer" or "The suggestions were helpful."

[0154] The device sends the collected feedback data to a server. Data transfer is secure and uses communication infrastructure.

[0155] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model, thereby improving the accuracy of the next emotion assessment.

[0156] Specific examples

[0157] Example 1: Frustration at work

[0158] 1. Data Collection

[0159] The device measures the user's pulse every 60 seconds and detects any sudden increases in heart rate.

[0160] The device collects the user's voice in real time and transmits it to the server.

[0161] 2. Data analysis and emotion assessment

[0162] The server detects a spike in pulse rate and an angry tone.

[0163] The server quantifies the anger level and assesses it as reaching an alert level.

[0164] 3. Display of notifications

[0165] The device displays the message "Take a deep breath for six seconds."

[0166] 4. Gather feedback and update the model

[0167] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[0168] The server receives the feedback and fine-tunes the AI ​​model.

[0169] Example 2: Stress during child-rearing

[0170] 1. Data Collection

[0171] The device measures the user's pulse every 30 seconds.

[0172] The device collects surrounding interactions and sends them to the server.

[0173] 2. Data analysis and emotion assessment

[0174] The server detects pulse spikes and vocal anger.

[0175] The server evaluates the anger level and determines the user's stress state.

[0176] 3. Display of notifications

[0177] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[0178] 4. Gather feedback and update the model

[0179] The user leaves the scene and inputs into the terminal as feedback that their emotions have calmed down, and this is sent to the server.

[0180] The server receives the feedback and updates the AI ​​model.

[0181] Prompt Sentence Examples

[0182] "Suggest appropriate actions when the user's pulse rate spikes and indicates anger."

[0183] The above is an embodiment of the present invention. This system makes it possible to monitor a user's emotions in real time and to effectively control their emotions by recommending management techniques.

[0184] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0185] Step 1: Data collection

[0186] The device (smartwatch) measures the user's pulse every 60 seconds. Specifically, the device's optical sensor uses light reflection to detect blood flow and calculate the pulse rate. The input is light reflection data from the skin on the user's wrist, and the output is the user's pulse rate.

[0187] The device recognizes the user's surrounding sounds in real time. It uses a built-in microphone to capture the tone and volume of the user's speech and generates audio data. The input is ambient sound, and the output is digital audio data.

[0188] The collected pulse and voice data are periodically sent to a server via Wi-Fi or Bluetooth. The input is the pulse and voice data collected on the device, and the output is the data sent to the server.

[0189] Step 2: Data analysis

[0190] The server receives the biometric data and voice data sent from the device and stores them in a database. The input is the pulse data and voice data sent from the device, and the output is the data stored in the database.

[0191] An AI model on the server analyzes biometric data to detect pulse rate spikes and abnormal fluctuations. For example, if the heart rate spikes by more than 30% from a normal resting level, an alert is issued. The input is pulse rate data retrieved from a database, and the output is a flag indicating the spike or abnormality.

[0192] The server uses speech recognition software to analyze the voice data, detect tones and patterns of anger in the user's voice, and assess their emotional state. The input is the voice data retrieved from the database, and the output is the analysis results showing patterns of anger.

[0193] Step 3: Emotional assessment

[0194] The server combines the results of biometric and voice data analysis to assess the user's emotional state. This process is performed using an AI model, which quantifies emotions based on the combined data. The input is the pulse data analysis results and voice data analysis results, and the output is a number that indicates the user's emotional state.

[0195] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10). The input is a number indicating the emotional state, and the output is a judgment result indicating whether the threshold has been exceeded.

[0196] Step 4: Alerting and Notification

[0197] If the server determines that the anger level exceeds a threshold as a result of emotion evaluation, it immediately generates an alert. The input is the judgment result indicating that the threshold has been exceeded, and the output is the alert data.

[0198] The server generates a message suggesting appropriate anger management techniques to the user. Specifically, it creates a notification with the following content: "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. It is recommended that you leave the area for a while." The input is the alert data, and the output is a text message.

[0199] The device displays a message from the server on the display, prompting the user to take action. Display methods include text messages and audio alerts. The input is a text message from the server, and the output is a notification through the device's display or speaker.

[0200] Step 5: Gather feedback and update the model

[0201] After the user acts according to the system's suggestions, they input feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they input information such as "I feel calmer" or "The suggestions were helpful." The input is information about changes in the user's emotions, and the output is the feedback data recorded on the device.

[0202] The feedback data collected by the device is sent to the server. The data is securely transferred using the communication infrastructure. The input is the feedback data, and the output is the data sent to the server.

[0203] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model. Specifically, it adjusts the model according to the individual characteristics of each user. The input is the feedback data, and the output is an updated AI model.

[0204] (Application example 1)

[0205] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0206] Conventional emotion management systems have difficulty detecting anger in real time during a user's daily life or work and taking effective measures. Furthermore, they only analyze vital and voice data when assessing emotional states, making it difficult to provide appropriate feedback based on the user's behavior and actions. This can leave users unsure of what actions to take, leading to inadequate stress management.

[0207] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0208] In this invention, the server includes means for collecting a user's vital data, means for collecting voice data, means for analyzing the vital data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotional management, means for an activity detection device to monitor the user's movements and suggest specific actions, means for collecting feedback from the user after the suggestion and updating the analytical model, and means for contactlessly collecting vital data. This allows the user to understand their own emotional state in real time and take appropriate action. Furthermore, the feedback improves the accuracy of the system, enabling more effective emotional management.

[0209] The "means for collecting user's vital data" refers to a device or system for collecting the user's biological information such as heart rate, blood pressure, and body temperature in real time.

[0210] A "means for collecting voice data" is a device or system for recording a user's voice and analyzing the tone and volume of the voice.

[0211] The "means for assessing the user's emotional state" is a device or system for analyzing the collected vital data and voice data to identify the user's emotions, such as stress or anger.

[0212] A "means for generating a notification" is a device or system that generates a warning or alert to notify a user when their emotional state exceeds a certain threshold.

[0213] A "means for supporting a user's emotional management" is a device or system for providing the user with advice and behavioral instructions necessary to control their emotional state.

[0214] An "activity detection device" is a device or system that monitors a user's activity or behavior and grasps the situation.

[0215] The "means for suggesting a specific action" is a device or system for presenting recommended actions or measures to the user based on the detected emotional state or behavior.

[0216] A "means for collecting feedback" is a device or system that allows a user to input and collect information about actions taken and the results of those actions.

[0217] The "means for updating the analytical model" is a device or system that continuously improves the AI ​​model to more accurately assess emotional states based on collected feedback data.

[0218] "Means for collecting vital data non-contact" refers to technology or devices for collecting vital data without requiring physical contact with the user.

[0219] This invention relates to a system that evaluates a user's emotional state in real time and provides appropriate management techniques. Specifically, it proposes a method for appropriately managing the user's anger and stress by collecting and analyzing the user's vital signs and voice data.

[0220] The system includes the following main components:

[0221] 1. Data Collection

[0222] The server collects the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data using the user's contactless pulse sensor and voice recognition microphone. Specific devices used include contactless pulse sensors and voice recognition microphones (e.g., TENSORFLOW® voice recognition models). This allows the user's biological state and emotional tone to be grasped in real time.

[0223] 2. Data Analysis

[0224] An analysis engine (e.g., SciKit-Learn + Keras) deployed on the server analyzes the collected vital signs and voice data. The data is stored and analyzed using a cloud-based platform (e.g., AWS (registered trademark) Lambda + S3 + SageMaker). The system detects sudden increases in pulse rate and changes in voice tone to assess the user's emotional state. Based on this analysis, the system quantifies the user's anger and stress levels.

[0225] 3. Notices and Suggestions

[0226] If the user's emotional state exceeds a certain threshold, the server generates a warning or alert, which is displayed on the task-sensing device (such as a factory robot's display). The notification includes specific actions the user should take, such as "Take a deep breath for 6 seconds" or "Stress level: 7 / 10. We recommend taking a short break."

[0227] 4. Gather feedback and update the model

[0228] The user performs the suggested action and then inputs feedback to the server. The feedback includes the change in the user's emotions after the action and the effectiveness of the suggestion. The analysis model is updated based on this feedback, improving the accuracy of the next evaluation and suggestion.

[0229] As a specific example, if a worker in a factory experiences a sudden increase in pulse rate due to being in a loud environment, the work detection device may suggest, "Take a deep breath." If the worker follows this suggestion, they can input feedback that "stress has been alleviated," which will improve the analysis model.

[0230] Below are some example prompts to input to a generative AI model:

[0231] "Implement a system that monitors the pulse and voice of workers in factories to assess their stress levels in real time. If a rapid rise in pulse or angry tone is detected, provide appropriate anger management techniques."

[0232] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0233] Step 1:

[0234] The device uses the user's contactless pulse sensor and voice recognition microphone to obtain the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data in real time. Specifically, the pulse sensor measures the user's pulse every 30 seconds, and the microphone collects the user's speech. The input is vital data and voice data, and the output is a real-time record of these data.

[0235] Step 2:

[0236] The device periodically transmits the collected vital and voice data to a server, where the data is uploaded to a cloud-based platform. Specifically, the data is securely stored using a storage service such as AWS S3. The input is the recorded data from the device, and the output is the data transferred to the server.

[0237] Step 3:

[0238] The server analyzes the received vital signs and voice data. Specifically, it uses SciKit-Learn and Keras to detect sudden increases in pulse rate and changes in voice tone. The analysis model evaluates the user's emotional state based on the data and quantifies levels of anger and stress. The input is the collected data, and the output is the evaluation of the emotional state.

[0239] Step 4:

[0240] The server generates an alert when the user's emotional state exceeds a certain threshold. The generated alert includes a specific action suggestion, such as "Take a deep breath for six seconds." The input is the evaluation result of the emotional state, and the output is the alert and the action suggestion.

[0241] Step 5:

[0242] The terminal presents the alerts and action suggestions from the server to the user. Specifically, it displays messages on the display and helps the user act according to the suggestions. The input is the server's alert, and the output is the message presented to the user.

[0243] Step 6:

[0244] The user inputs feedback on the results of following the suggestions into the terminal. Specifically, a feedback form is used to record changes in emotions and the effectiveness of the suggestions. The input is feedback after the user's actions, and the output is feedback data.

[0245] Step 7:

[0246] The device sends user feedback to the server, which then updates the AI ​​analysis model based on the feedback. This improves the accuracy of future emotion assessments. The input is the user feedback, and the output is the updated analysis model.

[0247] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0248] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects a user's vital signs and voice data in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[0249] Program processing

[0250] 1. Data Collection

[0251] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[0252] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[0253] The terminal periodically transmits the collected pulse data and voice data to the server.

[0254] 2. Data Analysis

[0255] The server receives the vital data (pulse data) and voice data sent from the terminal.

[0256] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[0257] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[0258] 3. Emotional assessment

[0259] The server integrates vital data and voice data and uses an emotion engine to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[0260] The emotion engine recognizes the user's multiple emotions (anger, stress, anxiety, etc.) and quantifies their overall emotional state.

[0261] 4. Alert Generation and Notification

[0262] The server generates an alert if the emotion assessment indicates that the anger level exceeds a certain threshold.

[0263] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[0264] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[0265] 5. Gather feedback and update the model

[0266] After the user follows the instructions of the system, they input feedback into the device, providing information such as how their emotions changed and whether the instructions were helpful.

[0267] The device sends the collected feedback to the server.

[0268] The server updates the AI ​​model and emotion engine based on the feedback, improving the accuracy of the next emotion assessment.

[0269] Specific examples

[0270] Example 1: Frustration at work

[0271] 1. Data Collection

[0272] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[0273] The device collects the user's voice in real time and transmits it to the server.

[0274] 2. Data analysis and emotion assessment

[0275] The server detects a spike in pulse rate and an angry tone.

[0276] The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[0277] 3. Display of notifications

[0278] The device displays the message "Take a deep breath for 6 seconds" to the user.

[0279] 4. Gather feedback and update the model

[0280] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[0281] The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[0282] Example 2: Stress during child-rearing

[0283] 1. Data Collection

[0284] The device measures the user's pulse every 30 seconds.

[0285] The device collects conversations with family members and sends them to a server.

[0286] 2. Data analysis and emotion assessment

[0287] The server detects pulse spikes and angry tones in the voice.

[0288] The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[0289] 3. Display of notifications

[0290] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[0291] 4. Gather feedback and update the model

[0292] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[0293] The server receives the feedback and updates the AI ​​model and emotion engine.

[0294] This allows the anger management system, which utilizes AI and an emotion engine, to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[0295] The processing flow will be explained below.

[0296] Step 1:

[0297] The device (smartwatch) measures the user's pulse at regular intervals, records the pulse data in real time using a sensor, and stores it in a data logger.

[0298] Step 2:

[0299] The device recognizes surrounding sounds and collects user voice data. It uses a microphone to capture the user's speech and environmental sounds, which are also stored in a data logger.

[0300] Step 3:

[0301] The terminal transmits the collected pulse data and voice data to the server at regular intervals. The data is transferred via wireless communication using the communication module.

[0302] Step 4:

[0303] The server receives vital data (pulse data) and voice data sent from the device, and stores this data in a database for analysis.

[0304] Step 5:

[0305] The AI ​​model on the server analyzes the received vital signs and detects sudden spikes or abnormal fluctuations in the pulse rate. The model evaluates the pulse rate change pattern and identifies abnormal values.

[0306] Step 6:

[0307] The server analyzes the voice data and recognizes angry tones and language in the user's voice. It uses a voice analysis algorithm to detect changes in voice intensity and tone.

[0308] Step 7:

[0309] The server integrates vital data and voice data and evaluates the user's emotional state using an emotion engine, which comprehensively analyzes the data and quantifies emotions such as anger, stress, and anxiety.

[0310] Step 8:

[0311] The server evaluates the user's emotional state and generates an alert if the anger level exceeds a certain threshold, with tailored countermeasures for the user.

[0312] Step 9:

[0313] The server generates an alert and sends it to the terminal. The alert message is wirelessly transferred to the terminal via the communication module.

[0314] Step 10:

[0315] The device will display notifications to the user, such as "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend you step away from the situation."

[0316] Step 11:

[0317] The user acts according to the system's instructions, such as taking a deep breath or leaving the area.

[0318] Step 12:

[0319] After the user follows the system's instructions, they input their feedback into the terminal, describing how their emotions changed and the effectiveness of the system's instructions in an input form.

[0320] Step 13:

[0321] The device sends the user's feedback to the server, and the collected feedback data is sent to the server via wireless communication.

[0322] Step 14:

[0323] The server receives the feedback data and updates the emotion engine and AI model, using the feedback to retrain the model and improve the accuracy of emotion assessment.

[0324] Through these steps, the system evaluates the user's emotional state in real time and provides appropriate anger management techniques at the appropriate time, helping the user to better control their emotions and respond appropriately even in stressful situations.

[0325] Example 2

[0326] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] Conventional emotion management systems lack the means to analyze users' biometric and voice data in real time, making it difficult to properly and timely evaluate the user's emotional state and prompt appropriate responses. Furthermore, they lack a mechanism for improving the accuracy of the system based on feedback, making it impossible to provide emotion management methods customized for individual users. This has led to the issue of being unable to properly manage users' stress and anger.

[0328] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data of the user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's emotional state exceeds a specific threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting feedback from the user and updating the analysis means, and means for integrating the biometric data and voice data to evaluate the user's emotional state in detail. This makes it possible to evaluate the user's emotional state in real time and provide an appropriate management method. Furthermore, by improving the accuracy of the system based on the feedback, it is possible to provide an emotion management method customized for each individual user.

[0329] "Biometric data" refers to data that indicates the physiological state of the user, and mainly includes heart rate and biorhythms.

[0330] "Audio data" is data that collects the sounds of the user and the surrounding area, and includes conversations and environmental sounds acquired through a microphone.

[0331] "Analysis means" refers to a means for analyzing collected biometric data and voice data to evaluate the user's emotional state, and is primarily a means that uses AI models and natural language processing technology.

[0332] An "emotional state" is a state that indicates the user's current psychological or emotional state, and includes multiple emotional elements such as anger, stress, and anxiety.

[0333] "Notifications" are warning and advice messages generated by the system for the user, and are messages that suggest appropriate emotion management techniques.

[0334] "Feedback" refers to data on reactions and evaluations provided by users, and indicates how appropriate the system's instructions were and how their emotions have changed.

[0335] "Update methods" are methods for improving and correcting analytical methods and models based on collected feedback, and are methods for improving the accuracy and effectiveness of the system.

[0336] "Integration" refers to the technique of combining different types of data (biometric data and audio data) and processing them as a single analysis target.

[0337] A "threshold" is a reference value set by the system when assessing an emotional state, which, if exceeded, triggers a particular action (such as generating a notification).

[0338] The "emotion engine" is a mechanism within the system that evaluates the user's emotional state in detail and analyzes it from multiple angles, and is an engine that is primarily built using AI technology.

[0339] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects biometric and voice data from users in real time, analyzes this data to evaluate the user's emotional state, and provides appropriate anger management techniques to the user when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[0340] Data collection

[0341] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[0342] The device collects the user's voice data in real time through a microphone, and is equipped with a noise-canceling function to improve sound quality.

[0343] The device collects pulse and voice data and periodically transmits it to a server via Wi-Fi or cellular networks.

[0344] Data analysis

[0345] The server stores the biometric data and voice data received from the terminal in a database.

[0346] The server is home to an AI model built using TensorFlow that analyzes biometric data and identifies sudden spikes and abnormal fluctuations in heart rate.

[0347] The server's natural language processing (NLP) model analyzes the audio data to detect angry tones and specific patterns in the user's voice, using pre-trained models such as BERT and WaveNet.

[0348] Emotional evaluation

[0349] The server integrates vital data and voice data and synchronizes them based on timestamps.

[0350] The emotion engine on the server analyzes this integrated data and evaluates the user's emotional state. The emotion engine generates multiple numerical values ​​(anger, stress, anxiety, etc.) to evaluate the overall emotional state.

[0351] Alerting and Notification

[0352] The server generates an alert if the emotional state exceeds a certain threshold as a result of the emotion assessment.

[0353] The server sends a message to the device suggesting appropriate anger management techniques to the user, such as "Your heart rate is increasing. Please take a deep breath."

[0354] Notifications received by the device are immediately displayed to the user, either as a vibration or a pop-up message.

[0355] Gathering feedback and updating the model

[0356] After the user follows the system's instructions, they input feedback from the device, such as how their emotions changed and whether the instructions were helpful.

[0357] The terminal transmits feedback data from the user to the server.

[0358] The server analyzes the collected feedback data and updates the AI ​​model and emotion engine, which improves the accuracy of the next emotion evaluation.

[0359] Specific examples

[0360] Example 1: Frustration at work

[0361] 1. The device measures the user's pulse every 60 seconds and detects any fluctuations.

[0362] 2. The device collects the user's voice in real time and sends it to the server.

[0363] 3. The server detects a spike in pulse rate and an angry tone.

[0364] 4. The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[0365] 5. The device displays the message "Take a deep breath for 6 seconds" to the user.

[0366] 6. After taking a deep breath, the user enters "I feel calm" into the device and sends it to the server.

[0367] 7. The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[0368] Example 2: Stress during child-rearing

[0369] 1. The device measures the user's pulse every 30 seconds.

[0370] 2. The device collects conversations with family members and sends them to the server.

[0371] 3. The server detects pulse spikes and angry tones in the voice.

[0372] 4. The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[0373] 5. The device displays the message "Anger level: 7 / 10. We recommend you take some time away."

[0374] 6. The user actually leaves the situation and inputs feedback that their emotions have calmed down.

[0375] 7. The server receives the feedback and updates the AI ​​model and emotion engine.

[0376] Prompt Sentence Examples

[0377] "Your heart rate is increasing. Take six deep breaths."

[0378] "Your anger levels are high. I suggest you step away from the area for a bit."

[0379] The system and method of the present invention allows for real-time monitoring of a user's emotional state and provides appropriate management techniques, thereby enabling users to manage their emotions in a healthy way and respond appropriately to stressful situations.

[0380] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0381] Step 1: Data collection

[0382] Input: Real-time biometric data (pulse) and voice data of the user.

[0383] Operation:

[0384] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[0385] The pulse data measured by the device is stored in the internal memory.

[0386] The device uses a built-in microphone to collect the user's voice data in real time, and the voice is captured clearly through noise cancellation.

[0387] Output: Collected biometric data (pulse data) and audio data.

[0388] Step 2: Send data

[0389] Input: Collected biometric data (pulse data) and audio data.

[0390] Operation:

[0391] The device compresses the biometric and audio data it collects and periodically transmits it to a server via Wi-Fi or cellular networks.

[0392] Output: Pulse data and audio data sent to the server.

[0393] Step 3: Analyzing vital data

[0394] Input: Pulse data sent to the server.

[0395] Operation:

[0396] The server stores the received pulse data in a database.

[0397] The received pulse data is analyzed using a server-based AI model (built using TensorFlow), specifically to identify sudden spikes and abnormal fluctuations in heart rate.

[0398] Output: Analyzed heart rate data (anomaly detection results).

[0399] Step 4: Analyzing the audio data

[0400] Input: The audio data sent to the server.

[0401] Operation:

[0402] The server stores the received audio data in cloud storage.

[0403] A server-based natural language processing (NLP) model (using BERT or WaveNet) analyzes the audio data and detects angry tones and patterns in the user's voice.

[0404] Output: Analyzed audio data (angry tone detection results).

[0405] Step 5: Data Integration

[0406] Input: Analyzed heart rate data and audio data (anomaly detection results and angry tone detection results).

[0407] Operation:

[0408] The server synchronizes and integrates the pulse data and voice data based on timestamps.

[0409] Output: A consolidated dataset (consolidated data based on a uniform timestamp).

[0410] Step 6: Emotional assessment

[0411] Input: The integrated dataset.

[0412] Operation:

[0413] The emotion engine on the server analyzes the integrated data and evaluates the user's emotional state, which is expressed as a numerical value such as anger, stress, or anxiety.

[0414] Output: User's emotional state assessment result (quantified emotional level).

[0415] Step 7: Alert Generation

[0416] Input: User's emotional state assessment results.

[0417] Operation:

[0418] The server generates an alert based on the emotion assessment results if the emotional state exceeds a certain threshold.

[0419] Output: Alert information (alert message when threshold is exceeded).

[0420] Step 8: Send and display notifications

[0421] Input: Alert information.

[0422] Operation:

[0423] The server sends an alert message to the terminal.

[0424] The device will receive notifications and display them to the user as a vibration or a pop-up message, such as "Your heart rate is increasing. Please take a deep breath."

[0425] Output: The notification message presented to the user.

[0426] Step 9: Gather feedback

[0427] Input: User feedback information (e.g., reaction to notifications, emotional changes, etc.).

[0428] Operation:

[0429] After the user acts according to the system's instructions, they input feedback into the device about changes in their emotions and the effectiveness of the notification.

[0430] The terminal transmits feedback data from the user to the server.

[0431] Output: Feedback data sent to the server.

[0432] Step 10: Model Update

[0433] Input: Feedback data.

[0434] Operation:

[0435] The server analyzes the collected feedback data.

[0436] The AI ​​model and emotion engine on the server are updated to improve the accuracy of the next emotion evaluation.

[0437] Output: Updated AI model and emotion engine.

[0438] (Application example 2)

[0439] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0440] In conventional food delivery work, delivery workers often experience various stressful situations, such as congested roads and complaints from customers, which can lead to problems such as reduced work efficiency and delivery quality. In particular, if delivery workers do not manage their emotions properly, the risk of accidents and trouble increases. Therefore, there is a need for a system that can immediately detect the stress and anger that delivery workers feel while working and provide appropriate management methods.

[0441] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0442] In this invention, the server includes means for collecting biometric data of a user, means for collecting acoustic data, means for analyzing the biometric data and acoustic data and evaluating the user's emotional state, means for generating a notification when the user's anger exceeds a certain threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting user behavior based on the notification as feedback to support the delivery person's stress management and using the feedback to update the AI ​​model, and means for analyzing events that cause the delivery person to feel stressed and presenting countermeasures. This makes it possible to detect the stress and anger that delivery people feel during their work in real time and provide appropriate management methods.

[0443] "Biometric data" refers to data obtained from the user's body, including, for example, heart rate, pulse, body temperature, and the like.

[0444] "Acoustic data" refers to audio information of the user and the surroundings, including the user's voice and surrounding environmental sounds.

[0445] "Emotional state" indicates the user's psychological state, and is a numerical representation of emotions such as anger, stress, and anxiety.

[0446] A "notification" is information, including warnings and instructions, sent to a user, such as a message displayed on the user's device.

[0447] "Emotion management" refers to techniques that allow users to appropriately control their emotions and reduce stress and anger.

[0448] An "AI model" is an algorithm that uses artificial intelligence to analyze user data and assess their emotional state.

[0449] "Feedback" refers to information that a user inputs after following instructions from the system and taking action, and is used to improve the system.

[0450] "Delivery person" refers to an employee of a food delivery service, specifically someone who delivers meals.

[0451] "Stress management" means that the system provides a way to reduce the mental burden on delivery personnel and improve their work efficiency.

[0452] "Countermeasures" means specific measures or instructions to help delivery personnel deal with stress or problems they face in the course of their work.

[0453] This invention is a system that manages the stress and anger felt by food delivery workers in real time and provides appropriate measures. The configuration and operation for realizing this system are described in detail below.

[0454] Hardware Configuration

[0455] 1. Smartwatch

[0456] Collects biometric data such as heart rate and pulse in real time.

[0457] 2. Smartphone

[0458] It has a built-in microphone for collecting acoustic data (voice data and environmental sounds).

[0459] A dedicated app is installed to collect and transmit data.

[0460] 3. Server

[0461] An AI model is deployed to perform data analysis.

[0462] Save the information to the database and update the model.

[0463] Software Configuration

[0464] 1. Data Collection App

[0465] Pulse data is collected from the smartwatch and transferred to a smartphone.

[0466] Audio data is collected using the smartphone's microphone.

[0467] 2. Server AI model

[0468] It is built using machine learning libraries such as TensorFlow and PyTorch.

[0469] Biometric and acoustic data is analyzed to assess the user's emotional state.

[0470] 3. Cloud Messaging System

[0471] Send real-time notifications to users using something like Firebase Cloud Messaging.

[0472] Data flow and processing overview

[0473] 1. Data Collection

[0474] The smartwatch measures the delivery person's heart rate every 10 seconds and transfers the data to a smartphone.

[0475] The smartphone's microphone collects ambient sounds and the delivery person's voice, and transmits them along with biometric data to a server.

[0476] 2. Data Analysis

[0477] The server analyzes the data it receives using an AI model to detect sudden increases in heart rate and angry tones in the audio data.

[0478] The emotion engine uses this data to quantify and evaluate the delivery person's emotional state.

[0479] 3. Notification Generation and Delivery

[0480] If the emotion assessment reveals that the anger or stress level exceeds a certain threshold, the server generates a notification with appropriate measures (e.g., "Take a deep breath" or "Take a 5-minute break") and sends it to the smartphone.

[0481] A notification will appear on the smartphone, prompting the delivery person to take specific action.

[0482] 4. Gather feedback and update the model

[0483] After the delivery person follows the system's instructions, they enter feedback on the effectiveness of their actions.

[0484] This feedback is received by the server and used to improve the accuracy of the AI ​​model.

[0485] Specific examples

[0486] A delivery driver received a complaint call from a customer while driving on a busy road. At this time, the smartwatch detected a sudden increase in the driver's heart rate, and the smartphone recognized an angry tone from the surrounding audio. The server's emotion engine analyzed this data and assessed the driver's anger level as 8 / 10, so a notification "Take a 5-minute break" was displayed on the driver's smartphone. The delivery driver followed the instructions, took a break, and entered feedback that his emotions had calmed down. The feedback data was sent to the server and used to improve the accuracy of the next assessment.

[0487] Prompt Sentence Examples

[0488] Generate a description for the following application: A food delivery app that manages the emotions of delivery drivers. It collects vital data and voice data, analyzes them using an emotion engine, and suggests appropriate anger management techniques in real time.

[0489] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0490] Step 1:

[0491] Data collection

[0492] The devices (smartwatches and smartphones) collect the delivery person's heart rate and voice data.

[0493] Specifically, the smartwatch measures the driver's heart rate every 10 seconds and transmits the data to the smartphone, whose microphone captures surrounding sounds and the voice of the delivery person.

[0494] Input: Delivery person's heart rate data, voice data

[0495] Output: Heart rate data and audio data are sent to a smartphone

[0496] Step 2:

[0497] Data Transfer

[0498] The terminal transmits the collected biometric data and acoustic data to a server.

[0499] Specifically, the smartphone uses Wi-Fi or mobile data communication to upload the collected data to a server in real time.

[0500] Input: Heart rate data, audio data

[0501] Output: Data is saved to the server

[0502] Step 3:

[0503] Data analysis

[0504] The server parses the received data.

[0505] Specifically, the AI ​​model on the server analyzes fluctuations in heart rate data to detect sudden changes, and analyzes voice data to recognize angry tones and specific voice patterns.

[0506] Input: Heart rate data, audio data

[0507] Output: Analysis results (heart rate fluctuations, anger tone detection)

[0508] Step 4:

[0509] Emotional evaluation

[0510] Based on the analysis results, the server's emotion engine quantifies and evaluates the delivery person's emotional state.

[0511] Specifically, the AI ​​model integrates heart rate variability and the tone of the voice data to calculate a score for anger and stress levels.

[0512] Input: Analysis results (heart rate fluctuations, anger tone detection)

[0513] Output: Emotional state evaluation result (e.g., anger level 8 / 10)

[0514] Step 5:

[0515] Notification generation

[0516] Based on the emotion evaluation results, the server generates a notification with specific measures to be taken by the delivery person.

[0517] Specifically, if the server detects an emotion score above a certain threshold, it generates messages such as "Take a deep breath" or "Take a five-minute break."

[0518] Input: Emotional state assessment results

[0519] Output: Notification message

[0520] Step 6:

[0521] Send notifications

[0522] The server generates a notification message and sends it to the device (smartphone).

[0523] Specifically, it uses cloud messaging services such as Firebase Cloud Messaging to send notifications to devices in real time.

[0524] Input: Notification message

[0525] Output: A notification appears on your smartphone

[0526] Step 7:

[0527] Feedback collection

[0528] After the user (delivery person) takes the action instructed in accordance with the notification, they input feedback about the effect of the action.

[0529] Specifically, delivery workers enter into a smartphone app whether they feel calmer after a break.

[0530] Input: Delivery person feedback information

[0531] Output: Feedback information is sent to the server

[0532] Step 8:

[0533] Model Update

[0534] Based on the feedback information received by the server, the AI ​​model is updated to improve its accuracy.

[0535] Specifically, the collected feedback data is used to fine-tune emotion evaluation algorithms and thresholds.

[0536] Input: Feedback information

[0537] Output: Updated AI model

[0538] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0540] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0541] [Second embodiment]

[0542] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0543] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0544] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0545] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0546] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0547] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0548] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0549] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0550] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0551] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0552] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0553] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0554] The system of the present invention comprises an apparatus and a method for effectively managing anger felt by users in their daily lives or at work. Specifically, the system collects vital data and voice data of users in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary.

[0555] Program processing

[0556] 1. Data Collection

[0557] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[0558] The device recognizes the audio environment around the user in real time and collects audio data, focusing in particular on the user's voice, capturing the tone and volume of the voice.

[0559] The terminal periodically transmits the collected pulse data and voice data to the server.

[0560] 2. Data Analysis

[0561] The server receives the vital data (pulse data) and voice data sent from the terminal.

[0562] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[0563] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[0564] 3. Emotional assessment

[0565] The server combines vital data and voice data to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[0566] The server quantifies the user's anger level and determines whether it exceeds a certain threshold.

[0567] 4. Alert Generation and Notification

[0568] If the server evaluates the emotion and finds that the anger level exceeds a threshold, it generates an alert.

[0569] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[0570] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[0571] 5. Gather feedback and update the model

[0572] After the user follows the instructions of the system, they input feedback into the device, such as how their emotions changed and whether the instructions were helpful.

[0573] The device sends the collected feedback to the server.

[0574] The server updates the AI ​​model based on the feedback, improving the accuracy of the next emotion assessment.

[0575] Specific examples

[0576] Example 1: Frustration at work

[0577] 1. Data Collection

[0578] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[0579] The device collects the user's voice in real time and transmits it to the server.

[0580] 2. Data analysis and emotion assessment

[0581] The server detects a spike in pulse rate and an angry tone.

[0582] The server quantifies the anger level and determines that it has reached an alert level.

[0583] 3. Display of notifications

[0584] The device displays the message "Take a deep breath for 6 seconds" to the user.

[0585] 4. Gather feedback and update the model

[0586] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[0587] The server receives the feedback and fine-tunes the AI ​​model.

[0588] Example 2: Stress during child-rearing

[0589] 1. Data Collection

[0590] The device measures the user's pulse every 30 seconds.

[0591] The device collects conversations with family members and sends them to a server.

[0592] 2. Data analysis and emotion assessment

[0593] The server detects pulse spikes and vocal anger.

[0594] The server determines the anger level and assesses the user's stress state.

[0595] 3. Display of notifications

[0596] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[0597] 4. Gather feedback and update the model

[0598] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[0599] The server receives the feedback and updates the AI ​​model.

[0600] This allows the AI-powered anger management system to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[0601] The processing flow will be explained below.

[0602] Step 1:

[0603] The device (smartwatch) measures the user's pulse at regular intervals. Specifically, it uses a sensor to record the user's heart rate in real time.

[0604] Step 2:

[0605] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[0606] Step 3:

[0607] The device periodically transmits the collected pulse and voice data to a server via wireless communication.

[0608] Step 4:

[0609] The server receives vital data (pulse data) and voice data sent from the device and stores this data for analysis.

[0610] Step 5:

[0611] The AI ​​model on the server analyzes the received vital data, detects sudden spikes in pulse rate and abnormal fluctuations, and evaluates the heart rate change pattern to estimate the user's emotional state.

[0612] Step 6:

[0613] The server analyzes the voice data and recognizes the tone and vocabulary of the user's voice. It then uses a voice analysis algorithm to evaluate the expression and intensity of anger.

[0614] Step 7:

[0615] The server combines vital and voice data to assess the user's emotional state, and an AI model uses this data to quantify the user's anger level.

[0616] Step 8:

[0617] The server generates an alert when the user's anger level exceeds a certain threshold, and determines the composition of the warning message and instructions for next steps.

[0618] Step 9:

[0619] The server sends the generated alert to the terminal, and the alert data is transferred to the terminal via wireless communication.

[0620] Step 10:

[0621] The device will then prompt the user with appropriate anger management techniques, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you take a break from the situation."

[0622] Step 11:

[0623] The user acts according to the system's instructions, taking specific actions such as taking a deep breath or leaving the area.

[0624] Step 12:

[0625] After the user follows the system's instructions, they enter their feedback into a terminal, where they are provided with input fields to report how their emotions changed and whether the technique was helpful.

[0626] Step 13:

[0627] The terminal sends the user's feedback to the server. The input feedback data is periodically transferred to the server.

[0628] Step 14:

[0629] The server receives the feedback data and updates the AI ​​model, training the model based on the feedback and using it for the next emotion assessment.

[0630] Through these steps, the system supports users in managing their emotions and provides real-time information to promote a healthy lifestyle.

[0631] Example 1

[0632] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0633] Conventional emotion management systems lack the functionality to integrate a user's biometric and voice data to assess their emotional state in real time and provide appropriate measures. Furthermore, they lack the functionality to improve the system based on user feedback, making it difficult to continuously provide users with appropriate advice.

[0634] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0635] In this invention, the server includes means for collecting biometric data of a user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotion management, means for collecting post-action feedback from the user, and means for analyzing the feedback and updating the evaluation model. This makes it possible to continuously monitor the user's emotional state in real time, provide an appropriate management method, and automatically improve the system based on the feedback.

[0636] "Biometric data" refers to information related to the user's bodily functions, and primarily includes vital signs such as pulse data.

[0637] "Voice data" refers to information related to the user's voice, including, among other things, tone and volume of speech.

[0638] "Analysis" refers to the process of detecting patterns and anomalies from collected data and extracting meaningful information.

[0639] The "emotional state" is an index that indicates the type and intensity of the emotion that the user is feeling.

[0640] "Notification" refers to a warning or suggestion message generated by the system to the user.

[0641] "Feedback" refers to the evaluation and opinion information that a user provides to a system after taking an action.

[0642] "Assessment Model" refers to a mathematical or statistical model, including artificial intelligence, used to analyze and assess a user's emotional state.

[0643] "Collection methods" refers to the devices, software, and methods used to obtain data.

[0644] "Analysis means" refers to devices, software, and methods for analyzing collected data and extracting information.

[0645] "Notification means" refers to a device or software for conveying the generated message to the user.

[0646] "Feedback collection means" refers to a device, software, or method for obtaining feedback from users.

[0647] "Evaluation model update means" refers to devices, software, and methods for improving the model based on collected feedback.

[0648] The system of the present invention comprises an apparatus and a method for collecting and analyzing a user's biometric and voice data to evaluate the user's emotional state and provide the user with appropriate anger management techniques as needed. Specifically, the system is implemented using the following hardware and software.

[0649] Hardware and software used

[0650] 1. Hardware: Smartwatches and wearable devices

[0651] 2. Software: AI models, servers, speech recognition software

[0652] Implementation details

[0653] 1. Data Collection

[0654] The device (smartwatch) collects the user's biometric data. Specifically, the device's optical sensor measures the user's pulse every 60 seconds and records heart rate fluctuations.

[0655] The device recognizes the user's surrounding sounds in real time, particularly capturing the tone and volume of the user's speech, by collecting voice data using the device's built-in microphone and temporarily storing it in local memory.

[0656] The collected pulse and voice data is periodically sent to a server via Wi-Fi or Bluetooth.

[0657] 2. Data Analysis

[0658] The server receives the biometric data and voice data sent from the terminal and stores them in a database.

[0659] The AI ​​model deployed on the server analyzes biometric data and detects sudden increases or abnormal fluctuations in pulse rate over a certain period of time. Specifically, if the heart rate rises by more than 30% from a normal resting level, an abnormality is detected.

[0660] The server uses voice recognition software to analyze the audio data, analyzing changes in tone and volume of the user's speech and identifying patterns that indicate anger, such as a sudden increase in volume or high-pitched sounds.

[0661] 3. Emotional assessment

[0662] The server combines the analysis results of biometric data and voice data and uses an AI model to assess the user's emotional state. If a sudden increase in pulse rate and an increase in high-pitched voice are observed simultaneously, it is determined to be angry.

[0663] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10).

[0664] 4. Alert Generation and Notification

[0665] The server generates an immediate alert if the emotion assessment reveals that the anger level exceeds a threshold.

[0666] The server generates a message suggesting appropriate anger management techniques to the user (e.g., "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend that you leave the situation for a while") and sends it to the device.

[0667] The device displays messages from the server on its display and prompts the user to take action, including text messages and audio alerts.

[0668] 5. Gather feedback and update the model

[0669] After following the system's suggestions, the user inputs feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they use the device's interface to input information such as "I felt calmer" or "The suggestions were helpful."

[0670] The device sends the collected feedback data to a server. Data transfer is secure and uses communication infrastructure.

[0671] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model, thereby improving the accuracy of the next emotion assessment.

[0672] Specific examples

[0673] Example 1: Frustration at work

[0674] 1. Data Collection

[0675] The device measures the user's pulse every 60 seconds and detects any sudden increases in heart rate.

[0676] The device collects the user's voice in real time and transmits it to the server.

[0677] 2. Data analysis and emotion assessment

[0678] The server detects a spike in pulse rate and an angry tone.

[0679] The server quantifies the anger level and assesses it as reaching an alert level.

[0680] 3. Display of notifications

[0681] The device displays the message "Take a deep breath for six seconds."

[0682] 4. Gather feedback and update the model

[0683] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[0684] The server receives the feedback and fine-tunes the AI ​​model.

[0685] Example 2: Stress during child-rearing

[0686] 1. Data Collection

[0687] The device measures the user's pulse every 30 seconds.

[0688] The device collects surrounding interactions and sends them to the server.

[0689] 2. Data analysis and emotion assessment

[0690] The server detects pulse spikes and vocal anger.

[0691] The server evaluates the anger level and determines the user's stress state.

[0692] 3. Display of notifications

[0693] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[0694] 4. Gather feedback and update the model

[0695] The user leaves the scene and inputs into the terminal as feedback that their emotions have calmed down, and this is sent to the server.

[0696] The server receives the feedback and updates the AI ​​model.

[0697] Prompt Sentence Examples

[0698] "Suggest appropriate actions when the user's pulse rate spikes and indicates anger."

[0699] The above is an embodiment of the present invention. This system makes it possible to monitor a user's emotions in real time and to effectively control their emotions by recommending management techniques.

[0700] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0701] Step 1: Data collection

[0702] The device (smartwatch) measures the user's pulse every 60 seconds. Specifically, the device's optical sensor uses light reflection to detect blood flow and calculate the pulse rate. The input is light reflection data from the skin on the user's wrist, and the output is the user's pulse rate.

[0703] The device recognizes the user's surrounding sounds in real time. It uses a built-in microphone to capture the tone and volume of the user's speech and generates audio data. The input is ambient sound, and the output is digital audio data.

[0704] The collected pulse and voice data are periodically sent to a server via Wi-Fi or Bluetooth. The input is the pulse and voice data collected on the device, and the output is the data sent to the server.

[0705] Step 2: Data analysis

[0706] The server receives the biometric data and voice data sent from the device and stores them in a database. The input is the pulse data and voice data sent from the device, and the output is the data stored in the database.

[0707] An AI model on the server analyzes biometric data to detect pulse rate spikes and abnormal fluctuations. For example, if the heart rate spikes by more than 30% from a normal resting level, an alert is issued. The input is pulse rate data retrieved from a database, and the output is a flag indicating the spike or abnormality.

[0708] The server uses speech recognition software to analyze the voice data, detect tones and patterns of anger in the user's voice, and assess their emotional state. The input is the voice data retrieved from the database, and the output is the analysis results showing patterns of anger.

[0709] Step 3: Emotional assessment

[0710] The server combines the results of biometric and voice data analysis to assess the user's emotional state. This process is performed using an AI model, which quantifies emotions based on the combined data. The input is the pulse data analysis results and voice data analysis results, and the output is a number that indicates the user's emotional state.

[0711] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10). The input is a number indicating the emotional state, and the output is a judgment result indicating whether the threshold has been exceeded.

[0712] Step 4: Alerting and Notification

[0713] If the server determines that the anger level exceeds a threshold as a result of emotion evaluation, it immediately generates an alert. The input is the judgment result indicating that the threshold has been exceeded, and the output is the alert data.

[0714] The server generates a message suggesting appropriate anger management techniques to the user. Specifically, it creates a notification with the following content: "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. It is recommended that you leave the area for a while." The input is the alert data, and the output is a text message.

[0715] The device displays a message from the server on the display, prompting the user to take action. Display methods include text messages and audio alerts. The input is a text message from the server, and the output is a notification through the device's display or speaker.

[0716] Step 5: Gather feedback and update the model

[0717] After the user acts according to the system's suggestions, they input feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they input information such as "I feel calmer" or "The suggestions were helpful." The input is information about changes in the user's emotions, and the output is the feedback data recorded on the device.

[0718] The feedback data collected by the device is sent to the server. The data is securely transferred using the communication infrastructure. The input is the feedback data, and the output is the data sent to the server.

[0719] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model. Specifically, it adjusts the model according to the individual characteristics of each user. The input is the feedback data, and the output is an updated AI model.

[0720] (Application example 1)

[0721] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] Conventional emotion management systems have difficulty detecting anger in real time during a user's daily life or work and taking effective measures. Furthermore, they only analyze vital and voice data when assessing emotional states, making it difficult to provide appropriate feedback based on the user's behavior and actions. This can leave users unsure of what actions to take, leading to inadequate stress management.

[0723] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0724] In this invention, the server includes means for collecting a user's vital data, means for collecting voice data, means for analyzing the vital data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotional management, means for an activity detection device to monitor the user's movements and suggest specific actions, means for collecting feedback from the user after the suggestion and updating the analytical model, and means for contactlessly collecting vital data. This allows the user to understand their own emotional state in real time and take appropriate action. Furthermore, the feedback improves the accuracy of the system, enabling more effective emotional management.

[0725] The "means for collecting user's vital data" refers to a device or system for collecting the user's biological information such as heart rate, blood pressure, and body temperature in real time.

[0726] A "means for collecting voice data" is a device or system for recording a user's voice and analyzing the tone and volume of the voice.

[0727] The "means for assessing the user's emotional state" is a device or system for analyzing the collected vital data and voice data to identify the user's emotions, such as stress or anger.

[0728] A "means for generating a notification" is a device or system that generates a warning or alert to notify a user when their emotional state exceeds a certain threshold.

[0729] A "means for supporting a user's emotional management" is a device or system for providing the user with advice and behavioral instructions necessary to control their emotional state.

[0730] An "activity detection device" is a device or system that monitors a user's activity or behavior and grasps the situation.

[0731] The "means for suggesting a specific action" is a device or system for presenting recommended actions or measures to the user based on the detected emotional state or behavior.

[0732] A "means for collecting feedback" is a device or system that allows a user to input and collect information about actions taken and the results of those actions.

[0733] The "means for updating the analytical model" is a device or system that continuously improves the AI ​​model to more accurately assess emotional states based on collected feedback data.

[0734] "Means for collecting vital data non-contact" refers to technology or devices for collecting vital data without requiring physical contact with the user.

[0735] This invention relates to a system that evaluates a user's emotional state in real time and provides appropriate management techniques. Specifically, it proposes a method for appropriately managing the user's anger and stress by collecting and analyzing the user's vital signs and voice data.

[0736] The system includes the following main components:

[0737] 1. Data Collection

[0738] The server collects the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data using the user's contactless pulse sensor and voice recognition microphone (e.g., TensorFlow voice recognition model). This allows the user's biological state and emotional tone to be understood in real time.

[0739] 2. Data Analysis

[0740] An analysis engine (e.g., SciKit-Learn + Keras) deployed on the server analyzes the collected vital signs and voice data. The data is stored and analyzed using a cloud-based platform (e.g., AWS Lambda + S3 + SageMaker). The system detects sudden increases in pulse rate and changes in voice tone to assess the user's emotional state. Based on this analysis, the system quantifies the user's anger and stress levels.

[0741] 3. Notices and Suggestions

[0742] If the user's emotional state exceeds a certain threshold, the server generates a warning or alert, which is displayed on the task-sensing device (such as a factory robot's display). The notification includes specific actions the user should take, such as "Take a deep breath for 6 seconds" or "Stress level: 7 / 10. We recommend taking a short break."

[0743] 4. Gather feedback and update the model

[0744] The user performs the suggested action and then inputs feedback to the server. The feedback includes the change in the user's emotions after the action and the effectiveness of the suggestion. The analysis model is updated based on this feedback, improving the accuracy of the next evaluation and suggestion.

[0745] As a specific example, if a worker in a factory experiences a sudden increase in pulse rate due to being in a loud environment, the work detection device may suggest, "Take a deep breath." If the worker follows this suggestion, they can input feedback that "stress has been alleviated," which will improve the analysis model.

[0746] Below are some example prompts to input to a generative AI model:

[0747] "Implement a system that monitors the pulse and voice of workers in factories to assess their stress levels in real time. If a rapid rise in pulse or angry tone is detected, provide appropriate anger management techniques."

[0748] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0749] Step 1:

[0750] The device uses the user's contactless pulse sensor and voice recognition microphone to obtain the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data in real time. Specifically, the pulse sensor measures the user's pulse every 30 seconds, and the microphone collects the user's speech. The input is vital data and voice data, and the output is a real-time record of these data.

[0751] Step 2:

[0752] The device periodically transmits the collected vital and voice data to a server, where the data is uploaded to a cloud-based platform. Specifically, the data is securely stored using a storage service such as AWS S3. The input is the recorded data from the device, and the output is the data transferred to the server.

[0753] Step 3:

[0754] The server analyzes the received vital signs and voice data. Specifically, it uses SciKit-Learn and Keras to detect sudden increases in pulse rate and changes in voice tone. The analysis model evaluates the user's emotional state based on the data and quantifies levels of anger and stress. The input is the collected data, and the output is the evaluation of the emotional state.

[0755] Step 4:

[0756] The server generates an alert when the user's emotional state exceeds a certain threshold. The generated alert includes a specific action suggestion, such as "Take a deep breath for six seconds." The input is the evaluation result of the emotional state, and the output is the alert and the action suggestion.

[0757] Step 5:

[0758] The terminal presents the alerts and action suggestions from the server to the user. Specifically, it displays messages on the display and helps the user act according to the suggestions. The input is the server's alert, and the output is the message presented to the user.

[0759] Step 6:

[0760] The user inputs feedback on the results of following the suggestions into the terminal. Specifically, a feedback form is used to record changes in emotions and the effectiveness of the suggestions. The input is feedback after the user's actions, and the output is feedback data.

[0761] Step 7:

[0762] The device sends user feedback to the server, which then updates the AI ​​analysis model based on the feedback. This improves the accuracy of future emotion assessments. The input is the user feedback, and the output is the updated analysis model.

[0763] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0764] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects a user's vital signs and voice data in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[0765] Program processing

[0766] 1. Data Collection

[0767] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[0768] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[0769] The terminal periodically transmits the collected pulse data and voice data to the server.

[0770] 2. Data Analysis

[0771] The server receives the vital data (pulse data) and voice data sent from the terminal.

[0772] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[0773] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[0774] 3. Emotional assessment

[0775] The server integrates vital data and voice data and uses an emotion engine to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[0776] The emotion engine recognizes the user's multiple emotions (anger, stress, anxiety, etc.) and quantifies their overall emotional state.

[0777] 4. Alert Generation and Notification

[0778] The server generates an alert if the emotion assessment indicates that the anger level exceeds a certain threshold.

[0779] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[0780] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[0781] 5. Gather feedback and update the model

[0782] After the user follows the instructions of the system, they input feedback into the device, providing information such as how their emotions changed and whether the instructions were helpful.

[0783] The device sends the collected feedback to the server.

[0784] The server updates the AI ​​model and emotion engine based on the feedback, improving the accuracy of the next emotion assessment.

[0785] Specific examples

[0786] Example 1: Frustration at work

[0787] 1. Data Collection

[0788] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[0789] The device collects the user's voice in real time and transmits it to the server.

[0790] 2. Data analysis and emotion assessment

[0791] The server detects a spike in pulse rate and an angry tone.

[0792] The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[0793] 3. Display of notifications

[0794] The device displays the message "Take a deep breath for 6 seconds" to the user.

[0795] 4. Gather feedback and update the model

[0796] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[0797] The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[0798] Example 2: Stress during child-rearing

[0799] 1. Data Collection

[0800] The device measures the user's pulse every 30 seconds.

[0801] The device collects conversations with family members and sends them to a server.

[0802] 2. Data analysis and emotion assessment

[0803] The server detects pulse spikes and angry tones in the voice.

[0804] The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[0805] 3. Display of notifications

[0806] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[0807] 4. Gather feedback and update the model

[0808] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[0809] The server receives the feedback and updates the AI ​​model and emotion engine.

[0810] This allows the anger management system, which utilizes AI and an emotion engine, to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[0811] The processing flow will be explained below.

[0812] Step 1:

[0813] The device (smartwatch) measures the user's pulse at regular intervals, records the pulse data in real time using a sensor, and stores it in a data logger.

[0814] Step 2:

[0815] The device recognizes surrounding sounds and collects user voice data. It uses a microphone to capture the user's speech and environmental sounds, which are also stored in a data logger.

[0816] Step 3:

[0817] The terminal transmits the collected pulse data and voice data to the server at regular intervals. The data is transferred via wireless communication using the communication module.

[0818] Step 4:

[0819] The server receives vital data (pulse data) and voice data sent from the device, and stores this data in a database for analysis.

[0820] Step 5:

[0821] The AI ​​model on the server analyzes the received vital signs and detects sudden spikes or abnormal fluctuations in the pulse rate. The model evaluates the pulse rate change pattern and identifies abnormal values.

[0822] Step 6:

[0823] The server analyzes the voice data and recognizes angry tones and language in the user's voice. It uses a voice analysis algorithm to detect changes in voice intensity and tone.

[0824] Step 7:

[0825] The server integrates vital data and voice data and evaluates the user's emotional state using an emotion engine, which comprehensively analyzes the data and quantifies emotions such as anger, stress, and anxiety.

[0826] Step 8:

[0827] The server evaluates the user's emotional state and generates an alert if the anger level exceeds a certain threshold, with tailored countermeasures for the user.

[0828] Step 9:

[0829] The server generates an alert and sends it to the terminal. The alert message is wirelessly transferred to the terminal via the communication module.

[0830] Step 10:

[0831] The device will display notifications to the user, such as "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend you step away from the situation."

[0832] Step 11:

[0833] The user acts according to the system's instructions, such as taking a deep breath or leaving the area.

[0834] Step 12:

[0835] After the user follows the system's instructions, they input their feedback into the terminal, describing how their emotions changed and the effectiveness of the system's instructions in an input form.

[0836] Step 13:

[0837] The device sends the user's feedback to the server, and the collected feedback data is sent to the server via wireless communication.

[0838] Step 14:

[0839] The server receives the feedback data and updates the emotion engine and AI model, using the feedback to retrain the model and improve the accuracy of emotion assessment.

[0840] Through these steps, the system evaluates the user's emotional state in real time and provides appropriate anger management techniques at the appropriate time, helping the user to better control their emotions and respond appropriately even in stressful situations.

[0841] Example 2

[0842] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0843] Conventional emotion management systems lack the means to analyze users' biometric and voice data in real time, making it difficult to properly and timely evaluate the user's emotional state and prompt appropriate responses. Furthermore, they lack a mechanism for improving the accuracy of the system based on feedback, making it impossible to provide emotion management methods customized for individual users. This has led to the issue of being unable to properly manage users' stress and anger.

[0844] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data of the user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's emotional state exceeds a specific threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting feedback from the user and updating the analysis means, and means for integrating the biometric data and voice data to evaluate the user's emotional state in detail. This makes it possible to evaluate the user's emotional state in real time and provide an appropriate management method. Furthermore, by improving the accuracy of the system based on the feedback, it is possible to provide an emotion management method customized for each individual user.

[0845] "Biometric data" refers to data that indicates the physiological state of the user, and mainly includes heart rate and biorhythms.

[0846] "Audio data" is data that collects the sounds of the user and the surrounding area, and includes conversations and environmental sounds acquired through a microphone.

[0847] "Analysis means" refers to a means for analyzing collected biometric data and voice data to evaluate the user's emotional state, and is primarily a means that uses AI models and natural language processing technology.

[0848] An "emotional state" is a state that indicates the user's current psychological or emotional state, and includes multiple emotional elements such as anger, stress, and anxiety.

[0849] "Notifications" are warning and advice messages generated by the system for the user, and are messages that suggest appropriate emotion management techniques.

[0850] "Feedback" refers to data on reactions and evaluations provided by users, and indicates how appropriate the system's instructions were and how their emotions have changed.

[0851] "Update methods" are methods for improving and correcting analytical methods and models based on collected feedback, and are methods for improving the accuracy and effectiveness of the system.

[0852] "Integration" refers to the technique of combining different types of data (biometric data and audio data) and processing them as a single analysis target.

[0853] A "threshold" is a reference value set by the system when assessing an emotional state, which, if exceeded, triggers a particular action (such as generating a notification).

[0854] The "emotion engine" is a mechanism within the system that evaluates the user's emotional state in detail and analyzes it from multiple angles, and is an engine that is primarily built using AI technology.

[0855] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects biometric and voice data from users in real time, analyzes this data to evaluate the user's emotional state, and provides appropriate anger management techniques to the user when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[0856] Data collection

[0857] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[0858] The device collects the user's voice data in real time through a microphone, and is equipped with a noise-canceling function to improve sound quality.

[0859] The device collects pulse and voice data and periodically transmits it to a server via Wi-Fi or cellular networks.

[0860] Data analysis

[0861] The server stores the biometric data and voice data received from the terminal in a database.

[0862] The server is home to an AI model built using TensorFlow that analyzes biometric data and identifies sudden spikes and abnormal fluctuations in heart rate.

[0863] The server's natural language processing (NLP) model analyzes the audio data to detect angry tones and specific patterns in the user's voice, using pre-trained models such as BERT and WaveNet.

[0864] Emotional evaluation

[0865] The server integrates vital data and voice data and synchronizes them based on timestamps.

[0866] The emotion engine on the server analyzes this integrated data and evaluates the user's emotional state. The emotion engine generates multiple numerical values ​​(anger, stress, anxiety, etc.) to evaluate the overall emotional state.

[0867] Alerting and Notification

[0868] The server generates an alert if the emotional state exceeds a certain threshold as a result of the emotion assessment.

[0869] The server sends a message to the device suggesting appropriate anger management techniques to the user, such as "Your heart rate is increasing. Please take a deep breath."

[0870] Notifications received by the device are immediately displayed to the user, either as a vibration or a pop-up message.

[0871] Gathering feedback and updating the model

[0872] After the user follows the system's instructions, they input feedback from the device, such as how their emotions changed and whether the instructions were helpful.

[0873] The terminal transmits feedback data from the user to the server.

[0874] The server analyzes the collected feedback data and updates the AI ​​model and emotion engine, which improves the accuracy of the next emotion evaluation.

[0875] Specific examples

[0876] Example 1: Frustration at work

[0877] 1. The device measures the user's pulse every 60 seconds and detects any fluctuations.

[0878] 2. The device collects the user's voice in real time and sends it to the server.

[0879] 3. The server detects a spike in pulse rate and an angry tone.

[0880] 4. The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[0881] 5. The device displays the message "Take a deep breath for 6 seconds" to the user.

[0882] 6. After taking a deep breath, the user enters "I feel calm" into the device and sends it to the server.

[0883] 7. The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[0884] Example 2: Stress during child-rearing

[0885] 1. The device measures the user's pulse every 30 seconds.

[0886] 2. The device collects conversations with family members and sends them to the server.

[0887] 3. The server detects pulse spikes and angry tones in the voice.

[0888] 4. The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[0889] 5. The device displays the message "Anger level: 7 / 10. We recommend you take some time away."

[0890] 6. The user actually leaves the situation and inputs feedback that their emotions have calmed down.

[0891] 7. The server receives the feedback and updates the AI ​​model and emotion engine.

[0892] Prompt Sentence Examples

[0893] "Your heart rate is increasing. Take six deep breaths."

[0894] "Your anger levels are high. I suggest you step away from the area for a bit."

[0895] The system and method of the present invention allows for real-time monitoring of a user's emotional state and provides appropriate management techniques, thereby enabling users to manage their emotions in a healthy way and respond appropriately to stressful situations.

[0896] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0897] Step 1: Data collection

[0898] Input: Real-time biometric data (pulse) and voice data of the user.

[0899] Operation:

[0900] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[0901] The pulse data measured by the device is stored in the internal memory.

[0902] The device uses a built-in microphone to collect the user's voice data in real time, and the voice is captured clearly through noise cancellation.

[0903] Output: Collected biometric data (pulse data) and audio data.

[0904] Step 2: Send data

[0905] Input: Collected biometric data (pulse data) and audio data.

[0906] Operation:

[0907] The device compresses the biometric and audio data it collects and periodically transmits it to a server via Wi-Fi or cellular networks.

[0908] Output: Pulse data and audio data sent to the server.

[0909] Step 3: Analyzing vital data

[0910] Input: Pulse data sent to the server.

[0911] Operation:

[0912] The server stores the received pulse data in a database.

[0913] The received pulse data is analyzed using a server-based AI model (built using TensorFlow), specifically to identify sudden spikes and abnormal fluctuations in heart rate.

[0914] Output: Analyzed heart rate data (anomaly detection results).

[0915] Step 4: Analyzing the audio data

[0916] Input: The audio data sent to the server.

[0917] Operation:

[0918] The server stores the received audio data in cloud storage.

[0919] A server-based natural language processing (NLP) model (using BERT or WaveNet) analyzes the audio data and detects angry tones and patterns in the user's voice.

[0920] Output: Analyzed audio data (angry tone detection results).

[0921] Step 5: Data Integration

[0922] Input: Analyzed heart rate data and audio data (anomaly detection results and angry tone detection results).

[0923] Operation:

[0924] The server synchronizes and integrates the pulse data and voice data based on timestamps.

[0925] Output: A consolidated dataset (consolidated data based on a uniform timestamp).

[0926] Step 6: Emotional assessment

[0927] Input: The integrated dataset.

[0928] Operation:

[0929] The emotion engine on the server analyzes the integrated data and evaluates the user's emotional state, which is expressed as a numerical value such as anger, stress, or anxiety.

[0930] Output: User's emotional state assessment result (quantified emotional level).

[0931] Step 7: Alert Generation

[0932] Input: User's emotional state assessment results.

[0933] Operation:

[0934] The server generates an alert based on the emotion assessment results if the emotional state exceeds a certain threshold.

[0935] Output: Alert information (alert message when threshold is exceeded).

[0936] Step 8: Send and display notifications

[0937] Input: Alert information.

[0938] Operation:

[0939] The server sends an alert message to the terminal.

[0940] The device will receive notifications and display them to the user as a vibration or a pop-up message, such as "Your heart rate is increasing. Please take a deep breath."

[0941] Output: The notification message presented to the user.

[0942] Step 9: Gather feedback

[0943] Input: User feedback information (e.g., reaction to notifications, emotional changes, etc.).

[0944] Operation:

[0945] After the user acts according to the system's instructions, they input feedback into the device about changes in their emotions and the effectiveness of the notification.

[0946] The terminal transmits feedback data from the user to the server.

[0947] Output: Feedback data sent to the server.

[0948] Step 10: Model Update

[0949] Input: Feedback data.

[0950] Operation:

[0951] The server analyzes the collected feedback data.

[0952] The AI ​​model and emotion engine on the server are updated to improve the accuracy of the next emotion evaluation.

[0953] Output: Updated AI model and emotion engine.

[0954] (Application example 2)

[0955] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0956] In conventional food delivery work, delivery workers often experience various stressful situations, such as congested roads and complaints from customers, which can lead to problems such as reduced work efficiency and delivery quality. In particular, if delivery workers do not manage their emotions properly, the risk of accidents and trouble increases. Therefore, there is a need for a system that can immediately detect the stress and anger that delivery workers feel while working and provide appropriate management methods.

[0957] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0958] In this invention, the server includes means for collecting biometric data of a user, means for collecting acoustic data, means for analyzing the biometric data and acoustic data and evaluating the user's emotional state, means for generating a notification when the user's anger exceeds a certain threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting user behavior based on the notification as feedback to support the delivery person's stress management and using the feedback to update the AI ​​model, and means for analyzing events that cause the delivery person to feel stressed and presenting countermeasures. This makes it possible to detect the stress and anger that delivery people feel during their work in real time and provide appropriate management methods.

[0959] "Biometric data" refers to data obtained from the user's body, including, for example, heart rate, pulse, body temperature, and the like.

[0960] "Acoustic data" refers to audio information of the user and the surroundings, including the user's voice and surrounding environmental sounds.

[0961] "Emotional state" indicates the user's psychological state, and is a numerical representation of emotions such as anger, stress, and anxiety.

[0962] A "notification" is information, including warnings and instructions, sent to a user, such as a message displayed on the user's device.

[0963] "Emotion management" refers to techniques that allow users to appropriately control their emotions and reduce stress and anger.

[0964] An "AI model" is an algorithm that uses artificial intelligence to analyze user data and assess their emotional state.

[0965] "Feedback" refers to information that a user inputs after following instructions from the system and taking action, and is used to improve the system.

[0966] "Delivery person" refers to an employee of a food delivery service, specifically someone who delivers meals.

[0967] "Stress management" means that the system provides a way to reduce the mental burden on delivery personnel and improve their work efficiency.

[0968] "Countermeasures" means specific measures or instructions to help delivery personnel deal with stress or problems they face in the course of their work.

[0969] This invention is a system that manages the stress and anger felt by food delivery workers in real time and provides appropriate measures. The configuration and operation for realizing this system are described in detail below.

[0970] Hardware Configuration

[0971] 1. Smartwatch

[0972] Collects biometric data such as heart rate and pulse in real time.

[0973] 2. Smartphone

[0974] It has a built-in microphone for collecting acoustic data (voice data and environmental sounds).

[0975] A dedicated app is installed to collect and transmit data.

[0976] 3. Server

[0977] An AI model is deployed to perform data analysis.

[0978] Save the information to the database and update the model.

[0979] Software Configuration

[0980] 1. Data Collection App

[0981] Pulse data is collected from the smartwatch and transferred to a smartphone.

[0982] Audio data is collected using the smartphone's microphone.

[0983] 2. Server AI model

[0984] It is built using machine learning libraries such as TensorFlow and PyTorch.

[0985] Biometric and acoustic data is analyzed to assess the user's emotional state.

[0986] 3. Cloud Messaging System

[0987] Send real-time notifications to users using something like Firebase Cloud Messaging.

[0988] Data flow and processing overview

[0989] 1. Data Collection

[0990] The smartwatch measures the delivery person's heart rate every 10 seconds and transfers the data to a smartphone.

[0991] The smartphone's microphone collects ambient sounds and the delivery person's voice, and transmits them along with biometric data to a server.

[0992] 2. Data Analysis

[0993] The server analyzes the data it receives using an AI model to detect sudden increases in heart rate and angry tones in the audio data.

[0994] The emotion engine uses this data to quantify and evaluate the delivery person's emotional state.

[0995] 3. Notification Generation and Delivery

[0996] If the emotion assessment reveals that the anger or stress level exceeds a certain threshold, the server generates a notification with appropriate measures (e.g., "Take a deep breath" or "Take a 5-minute break") and sends it to the smartphone.

[0997] A notification will appear on the smartphone, prompting the delivery person to take specific action.

[0998] 4. Gather feedback and update the model

[0999] After the delivery person follows the system's instructions, they enter feedback on the effectiveness of their actions.

[1000] This feedback is received by the server and used to improve the accuracy of the AI ​​model.

[1001] Specific examples

[1002] A delivery driver received a complaint call from a customer while driving on a busy road. At this time, the smartwatch detected a sudden increase in the driver's heart rate, and the smartphone recognized an angry tone from the surrounding audio. The server's emotion engine analyzed this data and assessed the driver's anger level as 8 / 10, so a notification "Take a 5-minute break" was displayed on the driver's smartphone. The delivery driver followed the instructions, took a break, and entered feedback that his emotions had calmed down. The feedback data was sent to the server and used to improve the accuracy of the next assessment.

[1003] Prompt Sentence Examples

[1004] Generate a description for the following application: A food delivery app that manages the emotions of delivery drivers. It collects vital data and voice data, analyzes them using an emotion engine, and suggests appropriate anger management techniques in real time.

[1005] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1006] Step 1:

[1007] Data collection

[1008] The devices (smartwatches and smartphones) collect the delivery person's heart rate and voice data.

[1009] Specifically, the smartwatch measures the driver's heart rate every 10 seconds and transmits the data to the smartphone, whose microphone captures surrounding sounds and the voice of the delivery person.

[1010] Input: Delivery person's heart rate data, voice data

[1011] Output: Heart rate data and audio data are sent to a smartphone

[1012] Step 2:

[1013] Data Transfer

[1014] The terminal transmits the collected biometric data and acoustic data to a server.

[1015] Specifically, the smartphone uses Wi-Fi or mobile data communication to upload the collected data to a server in real time.

[1016] Input: Heart rate data, audio data

[1017] Output: Data is saved to the server

[1018] Step 3:

[1019] Data analysis

[1020] The server parses the received data.

[1021] Specifically, the AI ​​model on the server analyzes fluctuations in heart rate data to detect sudden changes, and analyzes voice data to recognize angry tones and specific voice patterns.

[1022] Input: Heart rate data, audio data

[1023] Output: Analysis results (heart rate fluctuations, anger tone detection)

[1024] Step 4:

[1025] Emotional evaluation

[1026] Based on the analysis results, the server's emotion engine quantifies and evaluates the delivery person's emotional state.

[1027] Specifically, the AI ​​model integrates heart rate variability and the tone of the voice data to calculate a score for anger and stress levels.

[1028] Input: Analysis results (heart rate fluctuations, anger tone detection)

[1029] Output: Emotional state evaluation result (e.g., anger level 8 / 10)

[1030] Step 5:

[1031] Notification generation

[1032] Based on the emotion evaluation results, the server generates a notification with specific measures to be taken by the delivery person.

[1033] Specifically, if the server detects an emotion score above a certain threshold, it generates messages such as "Take a deep breath" or "Take a five-minute break."

[1034] Input: Emotional state assessment results

[1035] Output: Notification message

[1036] Step 6:

[1037] Send notifications

[1038] The server generates a notification message and sends it to the device (smartphone).

[1039] Specifically, it uses cloud messaging services such as Firebase Cloud Messaging to send notifications to devices in real time.

[1040] Input: Notification message

[1041] Output: A notification appears on your smartphone

[1042] Step 7:

[1043] Feedback collection

[1044] After the user (delivery person) takes the action instructed in accordance with the notification, they input feedback about the effect of the action.

[1045] Specifically, delivery workers enter into a smartphone app whether they feel calmer after a break.

[1046] Input: Delivery person feedback information

[1047] Output: Feedback information is sent to the server

[1048] Step 8:

[1049] Model Update

[1050] Based on the feedback information received by the server, the AI ​​model is updated to improve its accuracy.

[1051] Specifically, the collected feedback data is used to fine-tune emotion evaluation algorithms and thresholds.

[1052] Input: Feedback information

[1053] Output: Updated AI model

[1054] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1055] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1056] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1057] [Third embodiment]

[1058] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1059] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1060] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1061] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1062] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1063] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1064] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1065] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1066] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1067] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1068] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1069] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1070] The system of the present invention comprises an apparatus and a method for effectively managing anger felt by users in their daily lives or at work. Specifically, the system collects vital data and voice data of users in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary.

[1071] Program processing

[1072] 1. Data Collection

[1073] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[1074] The device recognizes the audio environment around the user in real time and collects audio data, focusing in particular on the user's voice, capturing the tone and volume of the voice.

[1075] The terminal periodically transmits the collected pulse data and voice data to the server.

[1076] 2. Data Analysis

[1077] The server receives the vital data (pulse data) and voice data sent from the terminal.

[1078] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[1079] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[1080] 3. Emotional assessment

[1081] The server combines vital data and voice data to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[1082] The server quantifies the user's anger level and determines whether it exceeds a certain threshold.

[1083] 4. Alert Generation and Notification

[1084] If the server evaluates the emotion and finds that the anger level exceeds a threshold, it generates an alert.

[1085] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[1086] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[1087] 5. Gather feedback and update the model

[1088] After the user follows the instructions of the system, they input feedback into the device, such as how their emotions changed and whether the instructions were helpful.

[1089] The device sends the collected feedback to the server.

[1090] The server updates the AI ​​model based on the feedback, improving the accuracy of the next emotion assessment.

[1091] Specific examples

[1092] Example 1: Frustration at work

[1093] 1. Data Collection

[1094] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[1095] The device collects the user's voice in real time and transmits it to the server.

[1096] 2. Data analysis and emotion assessment

[1097] The server detects a spike in pulse rate and an angry tone.

[1098] The server quantifies the anger level and determines that it has reached an alert level.

[1099] 3. Display of notifications

[1100] The device displays the message "Take a deep breath for 6 seconds" to the user.

[1101] 4. Gather feedback and update the model

[1102] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[1103] The server receives the feedback and fine-tunes the AI ​​model.

[1104] Example 2: Stress during child-rearing

[1105] 1. Data Collection

[1106] The device measures the user's pulse every 30 seconds.

[1107] The device collects conversations with family members and sends them to a server.

[1108] 2. Data analysis and emotion assessment

[1109] The server detects pulse spikes and vocal anger.

[1110] The server determines the anger level and assesses the user's stress state.

[1111] 3. Display of notifications

[1112] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[1113] 4. Gather feedback and update the model

[1114] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[1115] The server receives the feedback and updates the AI ​​model.

[1116] This allows the AI-powered anger management system to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[1117] The processing flow will be explained below.

[1118] Step 1:

[1119] The device (smartwatch) measures the user's pulse at regular intervals. Specifically, it uses a sensor to record the user's heart rate in real time.

[1120] Step 2:

[1121] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[1122] Step 3:

[1123] The device periodically transmits the collected pulse and voice data to a server via wireless communication.

[1124] Step 4:

[1125] The server receives vital data (pulse data) and voice data sent from the device and stores this data for analysis.

[1126] Step 5:

[1127] The AI ​​model on the server analyzes the received vital data, detects sudden spikes in pulse rate and abnormal fluctuations, and evaluates the heart rate change pattern to estimate the user's emotional state.

[1128] Step 6:

[1129] The server analyzes the voice data and recognizes the tone and vocabulary of the user's voice. It then uses a voice analysis algorithm to evaluate the expression and intensity of anger.

[1130] Step 7:

[1131] The server combines vital and voice data to assess the user's emotional state, and an AI model uses this data to quantify the user's anger level.

[1132] Step 8:

[1133] The server generates an alert when the user's anger level exceeds a certain threshold, and determines the composition of the warning message and instructions for next steps.

[1134] Step 9:

[1135] The server sends the generated alert to the terminal, and the alert data is transferred to the terminal via wireless communication.

[1136] Step 10:

[1137] The device will then prompt the user with appropriate anger management techniques, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you take a break from the situation."

[1138] Step 11:

[1139] The user acts according to the system's instructions, taking specific actions such as taking a deep breath or leaving the area.

[1140] Step 12:

[1141] After the user follows the system's instructions, they enter their feedback into a terminal, where they are provided with input fields to report how their emotions changed and whether the technique was helpful.

[1142] Step 13:

[1143] The terminal sends the user's feedback to the server. The input feedback data is periodically transferred to the server.

[1144] Step 14:

[1145] The server receives the feedback data and updates the AI ​​model, training the model based on the feedback and using it for the next emotion assessment.

[1146] Through these steps, the system supports users in managing their emotions and provides real-time information to promote a healthy lifestyle.

[1147] Example 1

[1148] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1149] Conventional emotion management systems lack the functionality to integrate a user's biometric and voice data to assess their emotional state in real time and provide appropriate measures. Furthermore, they lack the functionality to improve the system based on user feedback, making it difficult to continuously provide users with appropriate advice.

[1150] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1151] In this invention, the server includes means for collecting biometric data of a user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotion management, means for collecting post-action feedback from the user, and means for analyzing the feedback and updating the evaluation model. This makes it possible to continuously monitor the user's emotional state in real time, provide an appropriate management method, and automatically improve the system based on the feedback.

[1152] "Biometric data" refers to information related to the user's bodily functions, and primarily includes vital signs such as pulse data.

[1153] "Voice data" refers to information related to the user's voice, including, among other things, tone and volume of speech.

[1154] "Analysis" refers to the process of detecting patterns and anomalies from collected data and extracting meaningful information.

[1155] The "emotional state" is an index that indicates the type and intensity of the emotion that the user is feeling.

[1156] "Notification" refers to a warning or suggestion message generated by the system to the user.

[1157] "Feedback" refers to the evaluation and opinion information that a user provides to a system after taking an action.

[1158] "Assessment Model" refers to a mathematical or statistical model, including artificial intelligence, used to analyze and assess a user's emotional state.

[1159] "Collection methods" refers to the devices, software, and methods used to obtain data.

[1160] "Analysis means" refers to devices, software, and methods for analyzing collected data and extracting information.

[1161] "Notification means" refers to a device or software for conveying the generated message to the user.

[1162] "Feedback collection means" refers to a device, software, or method for obtaining feedback from users.

[1163] "Evaluation model update means" refers to devices, software, and methods for improving the model based on collected feedback.

[1164] The system of the present invention comprises an apparatus and a method for collecting and analyzing a user's biometric and voice data to evaluate the user's emotional state and provide the user with appropriate anger management techniques as needed. Specifically, the system is implemented using the following hardware and software.

[1165] Hardware and software used

[1166] 1. Hardware: Smartwatches and wearable devices

[1167] 2. Software: AI models, servers, speech recognition software

[1168] Implementation details

[1169] 1. Data Collection

[1170] The device (smartwatch) collects the user's biometric data. Specifically, the device's optical sensor measures the user's pulse every 60 seconds and records heart rate fluctuations.

[1171] The device recognizes the user's surrounding sounds in real time, particularly capturing the tone and volume of the user's speech, by collecting voice data using the device's built-in microphone and temporarily storing it in local memory.

[1172] The collected pulse and voice data is periodically sent to a server via Wi-Fi or Bluetooth.

[1173] 2. Data Analysis

[1174] The server receives the biometric data and voice data sent from the terminal and stores them in a database.

[1175] The AI ​​model deployed on the server analyzes biometric data and detects sudden increases or abnormal fluctuations in pulse rate over a certain period of time. Specifically, if the heart rate rises by more than 30% from a normal resting level, an abnormality is detected.

[1176] The server uses voice recognition software to analyze the audio data, analyzing changes in tone and volume of the user's speech and identifying patterns that indicate anger, such as a sudden increase in volume or high-pitched sounds.

[1177] 3. Emotional assessment

[1178] The server combines the analysis results of biometric data and voice data and uses an AI model to assess the user's emotional state. If a sudden increase in pulse rate and an increase in high-pitched voice are observed simultaneously, it is determined to be angry.

[1179] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10).

[1180] 4. Alert Generation and Notification

[1181] The server generates an immediate alert if the emotion assessment reveals that the anger level exceeds a threshold.

[1182] The server generates a message suggesting appropriate anger management techniques to the user (e.g., "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend that you leave the situation for a while") and sends it to the device.

[1183] The device displays messages from the server on its display and prompts the user to take action, including text messages and audio alerts.

[1184] 5. Gather feedback and update the model

[1185] After following the system's suggestions, the user inputs feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they use the device's interface to input information such as "I felt calmer" or "The suggestions were helpful."

[1186] The device sends the collected feedback data to a server. Data transfer is secure and uses communication infrastructure.

[1187] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model, thereby improving the accuracy of the next emotion assessment.

[1188] Specific examples

[1189] Example 1: Frustration at work

[1190] 1. Data Collection

[1191] The device measures the user's pulse every 60 seconds and detects any sudden increases in heart rate.

[1192] The device collects the user's voice in real time and transmits it to the server.

[1193] 2. Data analysis and emotion assessment

[1194] The server detects a spike in pulse rate and an angry tone.

[1195] The server quantifies the anger level and assesses it as reaching an alert level.

[1196] 3. Display of notifications

[1197] The device displays the message "Take a deep breath for six seconds."

[1198] 4. Gather feedback and update the model

[1199] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[1200] The server receives the feedback and fine-tunes the AI ​​model.

[1201] Example 2: Stress during child-rearing

[1202] 1. Data Collection

[1203] The device measures the user's pulse every 30 seconds.

[1204] The device collects surrounding interactions and sends them to the server.

[1205] 2. Data analysis and emotion assessment

[1206] The server detects pulse spikes and vocal anger.

[1207] The server evaluates the anger level and determines the user's stress state.

[1208] 3. Display of notifications

[1209] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[1210] 4. Gather feedback and update the model

[1211] The user leaves the scene and inputs into the terminal as feedback that their emotions have calmed down, and this is sent to the server.

[1212] The server receives the feedback and updates the AI ​​model.

[1213] Prompt Sentence Examples

[1214] "Suggest appropriate actions when the user's pulse rate spikes and indicates anger."

[1215] The above is an embodiment of the present invention. This system makes it possible to monitor a user's emotions in real time and to effectively control their emotions by recommending management techniques.

[1216] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1217] Step 1: Data collection

[1218] The device (smartwatch) measures the user's pulse every 60 seconds. Specifically, the device's optical sensor uses light reflection to detect blood flow and calculate the pulse rate. The input is light reflection data from the skin on the user's wrist, and the output is the user's pulse rate.

[1219] The device recognizes the user's surrounding sounds in real time. It uses a built-in microphone to capture the tone and volume of the user's speech and generates audio data. The input is ambient sound, and the output is digital audio data.

[1220] The collected pulse and voice data are periodically sent to a server via Wi-Fi or Bluetooth. The input is the pulse and voice data collected on the device, and the output is the data sent to the server.

[1221] Step 2: Data analysis

[1222] The server receives the biometric data and voice data sent from the device and stores them in a database. The input is the pulse data and voice data sent from the device, and the output is the data stored in the database.

[1223] An AI model on the server analyzes biometric data to detect pulse rate spikes and abnormal fluctuations. For example, if the heart rate spikes by more than 30% from a normal resting level, an alert is issued. The input is pulse rate data retrieved from a database, and the output is a flag indicating the spike or abnormality.

[1224] The server uses speech recognition software to analyze the voice data, detect tones and patterns of anger in the user's voice, and assess their emotional state. The input is the voice data retrieved from the database, and the output is the analysis results showing patterns of anger.

[1225] Step 3: Emotional assessment

[1226] The server combines the results of biometric and voice data analysis to assess the user's emotional state. This process is performed using an AI model, which quantifies emotions based on the combined data. The input is the pulse data analysis results and voice data analysis results, and the output is a number that indicates the user's emotional state.

[1227] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10). The input is a number indicating the emotional state, and the output is a judgment result indicating whether the threshold has been exceeded.

[1228] Step 4: Alerting and Notification

[1229] If the server determines that the anger level exceeds a threshold as a result of emotion evaluation, it immediately generates an alert. The input is the judgment result indicating that the threshold has been exceeded, and the output is the alert data.

[1230] The server generates a message suggesting appropriate anger management techniques to the user. Specifically, it creates a notification with the following content: "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. It is recommended that you leave the area for a while." The input is the alert data, and the output is a text message.

[1231] The device displays a message from the server on the display, prompting the user to take action. Display methods include text messages and audio alerts. The input is a text message from the server, and the output is a notification through the device's display or speaker.

[1232] Step 5: Gather feedback and update the model

[1233] After the user acts according to the system's suggestions, they input feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they input information such as "I feel calmer" or "The suggestions were helpful." The input is information about changes in the user's emotions, and the output is the feedback data recorded on the device.

[1234] The feedback data collected by the device is sent to the server. The data is securely transferred using the communication infrastructure. The input is the feedback data, and the output is the data sent to the server.

[1235] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model. Specifically, it adjusts the model according to the individual characteristics of each user. The input is the feedback data, and the output is an updated AI model.

[1236] (Application example 1)

[1237] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1238] Conventional emotion management systems have difficulty detecting anger in real time during a user's daily life or work and taking effective measures. Furthermore, they only analyze vital and voice data when assessing emotional states, making it difficult to provide appropriate feedback based on the user's behavior and actions. This can leave users unsure of what actions to take, leading to inadequate stress management.

[1239] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1240] In this invention, the server includes means for collecting a user's vital data, means for collecting voice data, means for analyzing the vital data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotional management, means for an activity detection device to monitor the user's movements and suggest specific actions, means for collecting feedback from the user after the suggestion and updating the analytical model, and means for contactlessly collecting vital data. This allows the user to understand their own emotional state in real time and take appropriate action. Furthermore, the feedback improves the accuracy of the system, enabling more effective emotional management.

[1241] The "means for collecting user's vital data" refers to a device or system for collecting the user's biological information such as heart rate, blood pressure, and body temperature in real time.

[1242] A "means for collecting voice data" is a device or system for recording a user's voice and analyzing the tone and volume of the voice.

[1243] The "means for assessing the user's emotional state" is a device or system for analyzing the collected vital data and voice data to identify the user's emotions, such as stress or anger.

[1244] A "means for generating a notification" is a device or system that generates a warning or alert to notify a user when their emotional state exceeds a certain threshold.

[1245] A "means for supporting a user's emotional management" is a device or system for providing the user with advice and behavioral instructions necessary to control their emotional state.

[1246] An "activity detection device" is a device or system that monitors a user's activity or behavior and grasps the situation.

[1247] The "means for suggesting a specific action" is a device or system for presenting recommended actions or measures to the user based on the detected emotional state or behavior.

[1248] A "means for collecting feedback" is a device or system that allows a user to input and collect information about actions taken and the results of those actions.

[1249] The "means for updating the analytical model" is a device or system that continuously improves the AI ​​model to more accurately assess emotional states based on collected feedback data.

[1250] "Means for collecting vital data non-contact" refers to technology or devices for collecting vital data without requiring physical contact with the user.

[1251] This invention relates to a system that evaluates a user's emotional state in real time and provides appropriate management techniques. Specifically, it proposes a method for appropriately managing the user's anger and stress by collecting and analyzing the user's vital signs and voice data.

[1252] The system includes the following main components:

[1253] 1. Data Collection

[1254] The server collects the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data using the user's contactless pulse sensor and voice recognition microphone (e.g., TensorFlow voice recognition model). This allows the user's biological state and emotional tone to be understood in real time.

[1255] 2. Data Analysis

[1256] An analysis engine (e.g., SciKit-Learn + Keras) deployed on the server analyzes the collected vital signs and voice data. The data is stored and analyzed using a cloud-based platform (e.g., AWS Lambda + S3 + SageMaker). The system detects sudden increases in pulse rate and changes in voice tone to assess the user's emotional state. Based on this analysis, the system quantifies the user's anger and stress levels.

[1257] 3. Notices and Suggestions

[1258] If the user's emotional state exceeds a certain threshold, the server generates a warning or alert, which is displayed on the task-sensing device (such as a factory robot's display). The notification includes specific actions the user should take, such as "Take a deep breath for 6 seconds" or "Stress level: 7 / 10. We recommend taking a short break."

[1259] 4. Gather feedback and update the model

[1260] The user performs the suggested action and then inputs feedback to the server. The feedback includes the change in the user's emotions after the action and the effectiveness of the suggestion. The analysis model is updated based on this feedback, improving the accuracy of the next evaluation and suggestion.

[1261] As a specific example, if a worker in a factory experiences a sudden increase in pulse rate due to being in a loud environment, the work detection device may suggest, "Take a deep breath." If the worker follows this suggestion, they can input feedback that "stress has been alleviated," which will improve the analysis model.

[1262] Below are some example prompts to input to a generative AI model:

[1263] "Implement a system that monitors the pulse and voice of workers in factories to assess their stress levels in real time. If a rapid rise in pulse or angry tone is detected, provide appropriate anger management techniques."

[1264] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1265] Step 1:

[1266] The device uses the user's contactless pulse sensor and voice recognition microphone to obtain the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data in real time. Specifically, the pulse sensor measures the user's pulse every 30 seconds, and the microphone collects the user's speech. The input is vital data and voice data, and the output is a real-time record of these data.

[1267] Step 2:

[1268] The device periodically transmits the collected vital and voice data to a server, where the data is uploaded to a cloud-based platform. Specifically, the data is securely stored using a storage service such as AWS S3. The input is the recorded data from the device, and the output is the data transferred to the server.

[1269] Step 3:

[1270] The server analyzes the received vital signs and voice data. Specifically, it uses SciKit-Learn and Keras to detect sudden increases in pulse rate and changes in voice tone. The analysis model evaluates the user's emotional state based on the data and quantifies levels of anger and stress. The input is the collected data, and the output is the evaluation of the emotional state.

[1271] Step 4:

[1272] The server generates an alert when the user's emotional state exceeds a certain threshold. The generated alert includes a specific action suggestion, such as "Take a deep breath for six seconds." The input is the evaluation result of the emotional state, and the output is the alert and the action suggestion.

[1273] Step 5:

[1274] The terminal presents the alerts and action suggestions from the server to the user. Specifically, it displays messages on the display and helps the user act according to the suggestions. The input is the server's alert, and the output is the message presented to the user.

[1275] Step 6:

[1276] The user inputs feedback on the results of following the suggestions into the terminal. Specifically, a feedback form is used to record changes in emotions and the effectiveness of the suggestions. The input is feedback after the user's actions, and the output is feedback data.

[1277] Step 7:

[1278] The device sends user feedback to the server, which then updates the AI ​​analysis model based on the feedback. This improves the accuracy of future emotion assessments. The input is the user feedback, and the output is the updated analysis model.

[1279] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1280] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects a user's vital signs and voice data in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[1281] Program processing

[1282] 1. Data Collection

[1283] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[1284] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[1285] The terminal periodically transmits the collected pulse data and voice data to the server.

[1286] 2. Data Analysis

[1287] The server receives the vital data (pulse data) and voice data sent from the terminal.

[1288] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[1289] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[1290] 3. Emotional assessment

[1291] The server integrates vital data and voice data and uses an emotion engine to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[1292] The emotion engine recognizes the user's multiple emotions (anger, stress, anxiety, etc.) and quantifies their overall emotional state.

[1293] 4. Alert Generation and Notification

[1294] The server generates an alert if the emotion assessment indicates that the anger level exceeds a certain threshold.

[1295] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[1296] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[1297] 5. Gather feedback and update the model

[1298] After the user follows the instructions of the system, they input feedback into the device, providing information such as how their emotions changed and whether the instructions were helpful.

[1299] The device sends the collected feedback to the server.

[1300] The server updates the AI ​​model and emotion engine based on the feedback, improving the accuracy of the next emotion assessment.

[1301] Specific examples

[1302] Example 1: Frustration at work

[1303] 1. Data Collection

[1304] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[1305] The device collects the user's voice in real time and transmits it to the server.

[1306] 2. Data analysis and emotion assessment

[1307] The server detects a spike in pulse rate and an angry tone.

[1308] The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[1309] 3. Display of notifications

[1310] The device displays the message "Take a deep breath for 6 seconds" to the user.

[1311] 4. Gather feedback and update the model

[1312] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[1313] The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[1314] Example 2: Stress during child-rearing

[1315] 1. Data Collection

[1316] The device measures the user's pulse every 30 seconds.

[1317] The device collects conversations with family members and sends them to a server.

[1318] 2. Data analysis and emotion assessment

[1319] The server detects pulse spikes and angry tones in the voice.

[1320] The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[1321] 3. Display of notifications

[1322] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[1323] 4. Gather feedback and update the model

[1324] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[1325] The server receives the feedback and updates the AI ​​model and emotion engine.

[1326] This allows the anger management system, which utilizes AI and an emotion engine, to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[1327] The processing flow will be explained below.

[1328] Step 1:

[1329] The device (smartwatch) measures the user's pulse at regular intervals, records the pulse data in real time using a sensor, and stores it in a data logger.

[1330] Step 2:

[1331] The device recognizes surrounding sounds and collects user voice data. It uses a microphone to capture the user's speech and environmental sounds, which are also stored in a data logger.

[1332] Step 3:

[1333] The terminal transmits the collected pulse data and voice data to the server at regular intervals. The data is transferred via wireless communication using the communication module.

[1334] Step 4:

[1335] The server receives vital data (pulse data) and voice data sent from the device, and stores this data in a database for analysis.

[1336] Step 5:

[1337] The AI ​​model on the server analyzes the received vital signs and detects sudden spikes or abnormal fluctuations in the pulse rate. The model evaluates the pulse rate change pattern and identifies abnormal values.

[1338] Step 6:

[1339] The server analyzes the voice data and recognizes angry tones and language in the user's voice. It uses a voice analysis algorithm to detect changes in voice intensity and tone.

[1340] Step 7:

[1341] The server integrates vital data and voice data and evaluates the user's emotional state using an emotion engine, which comprehensively analyzes the data and quantifies emotions such as anger, stress, and anxiety.

[1342] Step 8:

[1343] The server evaluates the user's emotional state and generates an alert if the anger level exceeds a certain threshold, with tailored countermeasures for the user.

[1344] Step 9:

[1345] The server generates an alert and sends it to the terminal. The alert message is wirelessly transferred to the terminal via the communication module.

[1346] Step 10:

[1347] The device will display notifications to the user, such as "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend you step away from the situation."

[1348] Step 11:

[1349] The user acts according to the system's instructions, such as taking a deep breath or leaving the area.

[1350] Step 12:

[1351] After the user follows the system's instructions, they input their feedback into the terminal, describing how their emotions changed and the effectiveness of the system's instructions in an input form.

[1352] Step 13:

[1353] The device sends the user's feedback to the server, and the collected feedback data is sent to the server via wireless communication.

[1354] Step 14:

[1355] The server receives the feedback data and updates the emotion engine and AI model, using the feedback to retrain the model and improve the accuracy of emotion assessment.

[1356] Through these steps, the system evaluates the user's emotional state in real time and provides appropriate anger management techniques at the appropriate time, helping the user to better control their emotions and respond appropriately even in stressful situations.

[1357] Example 2

[1358] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1359] Conventional emotion management systems lack the means to analyze users' biometric and voice data in real time, making it difficult to properly and timely evaluate the user's emotional state and prompt appropriate responses. Furthermore, they lack a mechanism for improving the accuracy of the system based on feedback, making it impossible to provide emotion management methods customized for individual users. This has led to the issue of being unable to properly manage users' stress and anger.

[1360] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data of the user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's emotional state exceeds a specific threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting feedback from the user and updating the analysis means, and means for integrating the biometric data and voice data to evaluate the user's emotional state in detail. This makes it possible to evaluate the user's emotional state in real time and provide an appropriate management method. Furthermore, by improving the accuracy of the system based on the feedback, it is possible to provide an emotion management method customized for each individual user.

[1361] "Biometric data" refers to data that indicates the physiological state of the user, and mainly includes heart rate and biorhythms.

[1362] "Audio data" is data that collects the sounds of the user and the surrounding area, and includes conversations and environmental sounds acquired through a microphone.

[1363] "Analysis means" refers to a means for analyzing collected biometric data and voice data to evaluate the user's emotional state, and is primarily a means that uses AI models and natural language processing technology.

[1364] An "emotional state" is a state that indicates the user's current psychological or emotional state, and includes multiple emotional elements such as anger, stress, and anxiety.

[1365] "Notifications" are warning and advice messages generated by the system for the user, and are messages that suggest appropriate emotion management techniques.

[1366] "Feedback" refers to data on reactions and evaluations provided by users, and indicates how appropriate the system's instructions were and how their emotions have changed.

[1367] "Update methods" are methods for improving and correcting analytical methods and models based on collected feedback, and are methods for improving the accuracy and effectiveness of the system.

[1368] "Integration" refers to the technique of combining different types of data (biometric data and audio data) and processing them as a single analysis target.

[1369] A "threshold" is a reference value set by the system when assessing an emotional state, which, if exceeded, triggers a particular action (such as generating a notification).

[1370] The "emotion engine" is a mechanism within the system that evaluates the user's emotional state in detail and analyzes it from multiple angles, and is an engine that is primarily built using AI technology.

[1371] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects biometric and voice data from users in real time, analyzes this data to evaluate the user's emotional state, and provides appropriate anger management techniques to the user when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[1372] Data collection

[1373] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[1374] The device collects the user's voice data in real time through a microphone, and is equipped with a noise-canceling function to improve sound quality.

[1375] The device collects pulse and voice data and periodically transmits it to a server via Wi-Fi or cellular networks.

[1376] Data analysis

[1377] The server stores the biometric data and voice data received from the terminal in a database.

[1378] The server is home to an AI model built using TensorFlow that analyzes biometric data and identifies sudden spikes and abnormal fluctuations in heart rate.

[1379] The server's natural language processing (NLP) model analyzes the audio data to detect angry tones and specific patterns in the user's voice, using pre-trained models such as BERT and WaveNet.

[1380] Emotional evaluation

[1381] The server integrates vital data and voice data and synchronizes them based on timestamps.

[1382] The emotion engine on the server analyzes this integrated data and evaluates the user's emotional state. The emotion engine generates multiple numerical values ​​(anger, stress, anxiety, etc.) to evaluate the overall emotional state.

[1383] Alerting and Notification

[1384] The server generates an alert if the emotional state exceeds a certain threshold as a result of the emotion assessment.

[1385] The server sends a message to the device suggesting appropriate anger management techniques to the user, such as "Your heart rate is increasing. Please take a deep breath."

[1386] Notifications received by the device are immediately displayed to the user, either as a vibration or a pop-up message.

[1387] Gathering feedback and updating the model

[1388] After the user follows the system's instructions, they input feedback from the device, such as how their emotions changed and whether the instructions were helpful.

[1389] The terminal transmits feedback data from the user to the server.

[1390] The server analyzes the collected feedback data and updates the AI ​​model and emotion engine, which improves the accuracy of the next emotion evaluation.

[1391] Specific examples

[1392] Example 1: Frustration at work

[1393] 1. The device measures the user's pulse every 60 seconds and detects any fluctuations.

[1394] 2. The device collects the user's voice in real time and sends it to the server.

[1395] 3. The server detects a spike in pulse rate and an angry tone.

[1396] 4. The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[1397] 5. The device displays the message "Take a deep breath for 6 seconds" to the user.

[1398] 6. After taking a deep breath, the user enters "I feel calm" into the device and sends it to the server.

[1399] 7. The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[1400] Example 2: Stress during child-rearing

[1401] 1. The device measures the user's pulse every 30 seconds.

[1402] 2. The device collects conversations with family members and sends them to the server.

[1403] 3. The server detects pulse spikes and angry tones in the voice.

[1404] 4. The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[1405] 5. The device displays the message "Anger level: 7 / 10. We recommend you take some time away."

[1406] 6. The user actually leaves the situation and inputs feedback that their emotions have calmed down.

[1407] 7. The server receives the feedback and updates the AI ​​model and emotion engine.

[1408] Prompt Sentence Examples

[1409] "Your heart rate is increasing. Take six deep breaths."

[1410] "Your anger levels are high. I suggest you step away from the area for a bit."

[1411] The system and method of the present invention allows for real-time monitoring of a user's emotional state and provides appropriate management techniques, thereby enabling users to manage their emotions in a healthy way and respond appropriately to stressful situations.

[1412] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1413] Step 1: Data collection

[1414] Input: Real-time biometric data (pulse) and voice data of the user.

[1415] Operation:

[1416] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[1417] The pulse data measured by the device is stored in the internal memory.

[1418] The device uses a built-in microphone to collect the user's voice data in real time, and the voice is captured clearly through noise cancellation.

[1419] Output: Collected biometric data (pulse data) and audio data.

[1420] Step 2: Send data

[1421] Input: Collected biometric data (pulse data) and audio data.

[1422] Operation:

[1423] The device compresses the biometric and audio data it collects and periodically transmits it to a server via Wi-Fi or cellular networks.

[1424] Output: Pulse data and audio data sent to the server.

[1425] Step 3: Analyzing vital data

[1426] Input: Pulse data sent to the server.

[1427] Operation:

[1428] The server stores the received pulse data in a database.

[1429] The received pulse data is analyzed using a server-based AI model (built using TensorFlow), specifically to identify sudden spikes and abnormal fluctuations in heart rate.

[1430] Output: Analyzed heart rate data (anomaly detection results).

[1431] Step 4: Analyzing the audio data

[1432] Input: The audio data sent to the server.

[1433] Operation:

[1434] The server stores the received audio data in cloud storage.

[1435] A server-based natural language processing (NLP) model (using BERT or WaveNet) analyzes the audio data and detects angry tones and patterns in the user's voice.

[1436] Output: Analyzed audio data (angry tone detection results).

[1437] Step 5: Data Integration

[1438] Input: Analyzed heart rate data and audio data (anomaly detection results and angry tone detection results).

[1439] Operation:

[1440] The server synchronizes and integrates the pulse data and voice data based on timestamps.

[1441] Output: A consolidated dataset (consolidated data based on a uniform timestamp).

[1442] Step 6: Emotional assessment

[1443] Input: The integrated dataset.

[1444] Operation:

[1445] The emotion engine on the server analyzes the integrated data and evaluates the user's emotional state, which is expressed as a numerical value such as anger, stress, or anxiety.

[1446] Output: User's emotional state assessment result (quantified emotional level).

[1447] Step 7: Alert Generation

[1448] Input: User's emotional state assessment results.

[1449] Operation:

[1450] The server generates an alert based on the emotion assessment results if the emotional state exceeds a certain threshold.

[1451] Output: Alert information (alert message when threshold is exceeded).

[1452] Step 8: Send and display notifications

[1453] Input: Alert information.

[1454] Operation:

[1455] The server sends an alert message to the terminal.

[1456] The device will receive notifications and display them to the user as a vibration or a pop-up message, such as "Your heart rate is increasing. Please take a deep breath."

[1457] Output: The notification message presented to the user.

[1458] Step 9: Gather feedback

[1459] Input: User feedback information (e.g., reaction to notifications, emotional changes, etc.).

[1460] Operation:

[1461] After the user acts according to the system's instructions, they input feedback into the device about changes in their emotions and the effectiveness of the notification.

[1462] The terminal transmits feedback data from the user to the server.

[1463] Output: Feedback data sent to the server.

[1464] Step 10: Model Update

[1465] Input: Feedback data.

[1466] Operation:

[1467] The server analyzes the collected feedback data.

[1468] The AI ​​model and emotion engine on the server are updated to improve the accuracy of the next emotion evaluation.

[1469] Output: Updated AI model and emotion engine.

[1470] (Application example 2)

[1471] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1472] In conventional food delivery work, delivery workers often experience various stressful situations, such as congested roads and complaints from customers, which can lead to problems such as reduced work efficiency and delivery quality. In particular, if delivery workers do not manage their emotions properly, the risk of accidents and trouble increases. Therefore, there is a need for a system that can immediately detect the stress and anger that delivery workers feel while working and provide appropriate management methods.

[1473] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1474] In this invention, the server includes means for collecting biometric data of a user, means for collecting acoustic data, means for analyzing the biometric data and acoustic data and evaluating the user's emotional state, means for generating a notification when the user's anger exceeds a certain threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting user behavior based on the notification as feedback to support the delivery person's stress management and using the feedback to update the AI ​​model, and means for analyzing events that cause the delivery person to feel stressed and presenting countermeasures. This makes it possible to detect the stress and anger that delivery people feel during their work in real time and provide appropriate management methods.

[1475] "Biometric data" refers to data obtained from the user's body, including, for example, heart rate, pulse, body temperature, and the like.

[1476] "Acoustic data" refers to audio information of the user and the surroundings, including the user's voice and surrounding environmental sounds.

[1477] "Emotional state" indicates the user's psychological state, and is a numerical representation of emotions such as anger, stress, and anxiety.

[1478] A "notification" is information, including warnings and instructions, sent to a user, such as a message displayed on the user's device.

[1479] "Emotion management" refers to techniques that allow users to appropriately control their emotions and reduce stress and anger.

[1480] An "AI model" is an algorithm that uses artificial intelligence to analyze user data and assess their emotional state.

[1481] "Feedback" refers to information that a user inputs after following instructions from the system and taking action, and is used to improve the system.

[1482] "Delivery person" refers to an employee of a food delivery service, specifically someone who delivers meals.

[1483] "Stress management" means that the system provides a way to reduce the mental burden on delivery personnel and improve their work efficiency.

[1484] "Countermeasures" means specific measures or instructions to help delivery personnel deal with stress or problems they face in the course of their work.

[1485] This invention is a system that manages the stress and anger felt by food delivery workers in real time and provides appropriate measures. The configuration and operation for realizing this system are described in detail below.

[1486] Hardware Configuration

[1487] 1. Smartwatch

[1488] Collects biometric data such as heart rate and pulse in real time.

[1489] 2. Smartphone

[1490] It has a built-in microphone for collecting acoustic data (voice data and environmental sounds).

[1491] A dedicated app is installed to collect and transmit data.

[1492] 3. Server

[1493] An AI model is deployed to perform data analysis.

[1494] Save the information to the database and update the model.

[1495] Software Configuration

[1496] 1. Data Collection App

[1497] Pulse data is collected from the smartwatch and transferred to a smartphone.

[1498] Audio data is collected using the smartphone's microphone.

[1499] 2. Server AI model

[1500] It is built using machine learning libraries such as TensorFlow and PyTorch.

[1501] Biometric and acoustic data is analyzed to assess the user's emotional state.

[1502] 3. Cloud Messaging System

[1503] Send real-time notifications to users using something like Firebase Cloud Messaging.

[1504] Data flow and processing overview

[1505] 1. Data Collection

[1506] The smartwatch measures the delivery person's heart rate every 10 seconds and transfers the data to a smartphone.

[1507] The smartphone's microphone collects ambient sounds and the delivery person's voice, and transmits them along with biometric data to a server.

[1508] 2. Data Analysis

[1509] The server analyzes the data it receives using an AI model to detect sudden increases in heart rate and angry tones in the audio data.

[1510] The emotion engine uses this data to quantify and evaluate the delivery person's emotional state.

[1511] 3. Notification Generation and Delivery

[1512] If the emotion assessment reveals that the anger or stress level exceeds a certain threshold, the server generates a notification with appropriate measures (e.g., "Take a deep breath" or "Take a 5-minute break") and sends it to the smartphone.

[1513] A notification will appear on the smartphone, prompting the delivery person to take specific action.

[1514] 4. Gather feedback and update the model

[1515] After the delivery person follows the system's instructions, they enter feedback on the effectiveness of their actions.

[1516] This feedback is received by the server and used to improve the accuracy of the AI ​​model.

[1517] Specific examples

[1518] A delivery driver received a complaint call from a customer while driving on a busy road. At this time, the smartwatch detected a sudden increase in the driver's heart rate, and the smartphone recognized an angry tone from the surrounding audio. The server's emotion engine analyzed this data and assessed the driver's anger level as 8 / 10, so a notification "Take a 5-minute break" was displayed on the driver's smartphone. The delivery driver followed the instructions, took a break, and entered feedback that his emotions had calmed down. The feedback data was sent to the server and used to improve the accuracy of the next assessment.

[1519] Prompt Sentence Examples

[1520] Generate a description for the following application: A food delivery app that manages the emotions of delivery drivers. It collects vital data and voice data, analyzes them using an emotion engine, and suggests appropriate anger management techniques in real time.

[1521] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1522] Step 1:

[1523] Data collection

[1524] The devices (smartwatches and smartphones) collect the delivery person's heart rate and voice data.

[1525] Specifically, the smartwatch measures the driver's heart rate every 10 seconds and transmits the data to the smartphone, whose microphone captures surrounding sounds and the voice of the delivery person.

[1526] Input: Delivery person's heart rate data, voice data

[1527] Output: Heart rate data and audio data are sent to a smartphone

[1528] Step 2:

[1529] Data Transfer

[1530] The terminal transmits the collected biometric data and acoustic data to a server.

[1531] Specifically, the smartphone uses Wi-Fi or mobile data communication to upload the collected data to a server in real time.

[1532] Input: Heart rate data, audio data

[1533] Output: Data is saved to the server

[1534] Step 3:

[1535] Data analysis

[1536] The server parses the received data.

[1537] Specifically, the AI ​​model on the server analyzes fluctuations in heart rate data to detect sudden changes, and analyzes voice data to recognize angry tones and specific voice patterns.

[1538] Input: Heart rate data, audio data

[1539] Output: Analysis results (heart rate fluctuations, anger tone detection)

[1540] Step 4:

[1541] Emotional evaluation

[1542] Based on the analysis results, the server's emotion engine quantifies and evaluates the delivery person's emotional state.

[1543] Specifically, the AI ​​model integrates heart rate variability and the tone of the voice data to calculate a score for anger and stress levels.

[1544] Input: Analysis results (heart rate fluctuations, anger tone detection)

[1545] Output: Emotional state evaluation result (e.g., anger level 8 / 10)

[1546] Step 5:

[1547] Notification generation

[1548] Based on the emotion evaluation results, the server generates a notification with specific measures to be taken by the delivery person.

[1549] Specifically, if the server detects an emotion score above a certain threshold, it generates messages such as "Take a deep breath" or "Take a five-minute break."

[1550] Input: Emotional state assessment results

[1551] Output: Notification message

[1552] Step 6:

[1553] Send notifications

[1554] The server generates a notification message and sends it to the device (smartphone).

[1555] Specifically, it uses cloud messaging services such as Firebase Cloud Messaging to send notifications to devices in real time.

[1556] Input: Notification message

[1557] Output: A notification appears on your smartphone

[1558] Step 7:

[1559] Feedback collection

[1560] After the user (delivery person) takes the action instructed in accordance with the notification, they input feedback about the effect of the action.

[1561] Specifically, delivery workers enter into a smartphone app whether they feel calmer after a break.

[1562] Input: Delivery person feedback information

[1563] Output: Feedback information is sent to the server

[1564] Step 8:

[1565] Model Update

[1566] Based on the feedback information received by the server, the AI ​​model is updated to improve its accuracy.

[1567] Specifically, the collected feedback data is used to fine-tune emotion evaluation algorithms and thresholds.

[1568] Input: Feedback information

[1569] Output: Updated AI model

[1570] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1571] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1572] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1573] [Fourth embodiment]

[1574] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1575] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1576] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1577] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1578] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1579] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1580] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1581] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1582] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1583] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1584] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1585] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1586] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1587] The system of the present invention comprises an apparatus and a method for effectively managing anger felt by users in their daily lives or at work. Specifically, the system collects vital data and voice data of users in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary.

[1588] Program processing

[1589] 1. Data Collection

[1590] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[1591] The device recognizes the audio environment around the user in real time and collects audio data, focusing in particular on the user's voice, capturing the tone and volume of the voice.

[1592] The terminal periodically transmits the collected pulse data and voice data to the server.

[1593] 2. Data Analysis

[1594] The server receives the vital data (pulse data) and voice data sent from the terminal.

[1595] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[1596] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[1597] 3. Emotional assessment

[1598] The server combines vital data and voice data to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[1599] The server quantifies the user's anger level and determines whether it exceeds a certain threshold.

[1600] 4. Alert Generation and Notification

[1601] If the server evaluates the emotion and finds that the anger level exceeds a threshold, it generates an alert.

[1602] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[1603] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[1604] 5. Gather feedback and update the model

[1605] After the user follows the instructions of the system, they input feedback into the device, such as how their emotions changed and whether the instructions were helpful.

[1606] The device sends the collected feedback to the server.

[1607] The server updates the AI ​​model based on the feedback, improving the accuracy of the next emotion assessment.

[1608] Specific examples

[1609] Example 1: Frustration at work

[1610] 1. Data Collection

[1611] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[1612] The device collects the user's voice in real time and transmits it to the server.

[1613] 2. Data analysis and emotion assessment

[1614] The server detects a spike in pulse rate and an angry tone.

[1615] The server quantifies the anger level and determines that it has reached an alert level.

[1616] 3. Display of notifications

[1617] The device displays the message "Take a deep breath for 6 seconds" to the user.

[1618] 4. Gather feedback and update the model

[1619] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[1620] The server receives the feedback and fine-tunes the AI ​​model.

[1621] Example 2: Stress during child-rearing

[1622] 1. Data Collection

[1623] The device measures the user's pulse every 30 seconds.

[1624] The device collects conversations with family members and sends them to a server.

[1625] 2. Data analysis and emotion assessment

[1626] The server detects pulse spikes and vocal anger.

[1627] The server determines the anger level and assesses the user's stress state.

[1628] 3. Display of notifications

[1629] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[1630] 4. Gather feedback and update the model

[1631] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[1632] The server receives the feedback and updates the AI ​​model.

[1633] This allows the AI-powered anger management system to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[1634] The processing flow will be explained below.

[1635] Step 1:

[1636] The device (smartwatch) measures the user's pulse at regular intervals. Specifically, it uses a sensor to record the user's heart rate in real time.

[1637] Step 2:

[1638] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[1639] Step 3:

[1640] The device periodically transmits the collected pulse and voice data to a server via wireless communication.

[1641] Step 4:

[1642] The server receives vital data (pulse data) and voice data sent from the device and stores this data for analysis.

[1643] Step 5:

[1644] The AI ​​model on the server analyzes the received vital data, detects sudden spikes in pulse rate and abnormal fluctuations, and evaluates the heart rate change pattern to estimate the user's emotional state.

[1645] Step 6:

[1646] The server analyzes the voice data and recognizes the tone and vocabulary of the user's voice. It then uses a voice analysis algorithm to evaluate the expression and intensity of anger.

[1647] Step 7:

[1648] The server combines vital and voice data to assess the user's emotional state, and an AI model uses this data to quantify the user's anger level.

[1649] Step 8:

[1650] The server generates an alert when the user's anger level exceeds a certain threshold, and determines the composition of the warning message and instructions for next steps.

[1651] Step 9:

[1652] The server sends the generated alert to the terminal, and the alert data is transferred to the terminal via wireless communication.

[1653] Step 10:

[1654] The device will then prompt the user with appropriate anger management techniques, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you take a break from the situation."

[1655] Step 11:

[1656] The user acts according to the system's instructions, taking specific actions such as taking a deep breath or leaving the area.

[1657] Step 12:

[1658] After the user follows the system's instructions, they enter their feedback into a terminal, where they are provided with input fields to report how their emotions changed and whether the technique was helpful.

[1659] Step 13:

[1660] The terminal sends the user's feedback to the server. The input feedback data is periodically transferred to the server.

[1661] Step 14:

[1662] The server receives the feedback data and updates the AI ​​model, training the model based on the feedback and using it for the next emotion assessment.

[1663] Through these steps, the system supports users in managing their emotions and provides real-time information to promote a healthy lifestyle.

[1664] Example 1

[1665] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1666] Conventional emotion management systems lack the functionality to integrate a user's biometric and voice data to assess their emotional state in real time and provide appropriate measures. Furthermore, they lack the functionality to improve the system based on user feedback, making it difficult to continuously provide users with appropriate advice.

[1667] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1668] In this invention, the server includes means for collecting biometric data of a user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotion management, means for collecting post-action feedback from the user, and means for analyzing the feedback and updating the evaluation model. This makes it possible to continuously monitor the user's emotional state in real time, provide an appropriate management method, and automatically improve the system based on the feedback.

[1669] "Biometric data" refers to information related to the user's bodily functions, and primarily includes vital signs such as pulse data.

[1670] "Voice data" refers to information related to the user's voice, including, among other things, tone and volume of speech.

[1671] "Analysis" refers to the process of detecting patterns and anomalies from collected data and extracting meaningful information.

[1672] The "emotional state" is an index that indicates the type and intensity of the emotion that the user is feeling.

[1673] "Notification" refers to a warning or suggestion message generated by the system to the user.

[1674] "Feedback" refers to the evaluation and opinion information that a user provides to a system after taking an action.

[1675] "Assessment Model" refers to a mathematical or statistical model, including artificial intelligence, used to analyze and assess a user's emotional state.

[1676] "Collection methods" refers to the devices, software, and methods used to obtain data.

[1677] "Analysis means" refers to devices, software, and methods for analyzing collected data and extracting information.

[1678] "Notification means" refers to a device or software for conveying the generated message to the user.

[1679] "Feedback collection means" refers to a device, software, or method for obtaining feedback from users.

[1680] "Evaluation model update means" refers to devices, software, and methods for improving the model based on collected feedback.

[1681] The system of the present invention comprises an apparatus and a method for collecting and analyzing a user's biometric and voice data to evaluate the user's emotional state and provide the user with appropriate anger management techniques as needed. Specifically, the system is implemented using the following hardware and software.

[1682] Hardware and software used

[1683] 1. Hardware: Smartwatches and wearable devices

[1684] 2. Software: AI models, servers, speech recognition software

[1685] Implementation details

[1686] 1. Data Collection

[1687] The device (smartwatch) collects the user's biometric data. Specifically, the device's optical sensor measures the user's pulse every 60 seconds and records heart rate fluctuations.

[1688] The device recognizes the user's surrounding sounds in real time, particularly capturing the tone and volume of the user's speech, by collecting voice data using the device's built-in microphone and temporarily storing it in local memory.

[1689] The collected pulse and voice data is periodically sent to a server via Wi-Fi or Bluetooth.

[1690] 2. Data Analysis

[1691] The server receives the biometric data and voice data sent from the terminal and stores them in a database.

[1692] The AI ​​model deployed on the server analyzes biometric data and detects sudden increases or abnormal fluctuations in pulse rate over a certain period of time. Specifically, if the heart rate rises by more than 30% from a normal resting level, an abnormality is detected.

[1693] The server uses voice recognition software to analyze the audio data, analyzing changes in tone and volume of the user's speech and identifying patterns that indicate anger, such as a sudden increase in volume or high-pitched sounds.

[1694] 3. Emotional assessment

[1695] The server combines the analysis results of biometric data and voice data and uses an AI model to assess the user's emotional state. If a sudden increase in pulse rate and an increase in high-pitched voice are observed simultaneously, it is determined to be angry.

[1696] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10).

[1697] 4. Alert Generation and Notification

[1698] The server generates an immediate alert if the emotion assessment reveals that the anger level exceeds a threshold.

[1699] The server generates a message suggesting appropriate anger management techniques to the user (e.g., "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend that you leave the situation for a while") and sends it to the device.

[1700] The device displays messages from the server on its display and prompts the user to take action, including text messages and audio alerts.

[1701] 5. Gather feedback and update the model

[1702] After following the system's suggestions, the user inputs feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they use the device's interface to input information such as "I felt calmer" or "The suggestions were helpful."

[1703] The device sends the collected feedback data to a server. Data transfer is secure and uses communication infrastructure.

[1704] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model, thereby improving the accuracy of the next emotion assessment.

[1705] Specific examples

[1706] Example 1: Frustration at work

[1707] 1. Data Collection

[1708] The device measures the user's pulse every 60 seconds and detects any sudden increases in heart rate.

[1709] The device collects the user's voice in real time and transmits it to the server.

[1710] 2. Data analysis and emotion assessment

[1711] The server detects a spike in pulse rate and an angry tone.

[1712] The server quantifies the anger level and assesses it as reaching an alert level.

[1713] 3. Display of notifications

[1714] The device displays the message "Take a deep breath for six seconds."

[1715] 4. Gather feedback and update the model

[1716] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[1717] The server receives the feedback and fine-tunes the AI ​​model.

[1718] Example 2: Stress during child-rearing

[1719] 1. Data Collection

[1720] The device measures the user's pulse every 30 seconds.

[1721] The device collects surrounding interactions and sends them to the server.

[1722] 2. Data analysis and emotion assessment

[1723] The server detects pulse spikes and vocal anger.

[1724] The server evaluates the anger level and determines the user's stress state.

[1725] 3. Display of notifications

[1726] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[1727] 4. Gather feedback and update the model

[1728] The user leaves the scene and inputs into the terminal as feedback that their emotions have calmed down, and this is sent to the server.

[1729] The server receives the feedback and updates the AI ​​model.

[1730] Prompt Sentence Examples

[1731] "Suggest appropriate actions when the user's pulse rate spikes and indicates anger."

[1732] The above is an embodiment of the present invention. This system makes it possible to monitor a user's emotions in real time and to effectively control their emotions by recommending management techniques.

[1733] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1734] Step 1: Data collection

[1735] The device (smartwatch) measures the user's pulse every 60 seconds. Specifically, the device's optical sensor uses light reflection to detect blood flow and calculate the pulse rate. The input is light reflection data from the skin on the user's wrist, and the output is the user's pulse rate.

[1736] The device recognizes the user's surrounding sounds in real time. It uses a built-in microphone to capture the tone and volume of the user's speech and generates audio data. The input is ambient sound, and the output is digital audio data.

[1737] The collected pulse and voice data are periodically sent to a server via Wi-Fi or Bluetooth. The input is the pulse and voice data collected on the device, and the output is the data sent to the server.

[1738] Step 2: Data analysis

[1739] The server receives the biometric data and voice data sent from the device and stores them in a database. The input is the pulse data and voice data sent from the device, and the output is the data stored in the database.

[1740] An AI model on the server analyzes biometric data to detect pulse rate spikes and abnormal fluctuations. For example, if the heart rate spikes by more than 30% from a normal resting level, an alert is issued. The input is pulse rate data retrieved from a database, and the output is a flag indicating the spike or abnormality.

[1741] The server uses speech recognition software to analyze the voice data, detect tones and patterns of anger in the user's voice, and assess their emotional state. The input is the voice data retrieved from the database, and the output is the analysis results showing patterns of anger.

[1742] Step 3: Emotional assessment

[1743] The server combines the results of biometric and voice data analysis to assess the user's emotional state. This process is performed using an AI model, which quantifies emotions based on the combined data. The input is the pulse data analysis results and voice data analysis results, and the output is a number that indicates the user's emotional state.

[1744] The server quantifies the user's anger level and determines whether the value exceeds a certain threshold (e.g., 7 or more out of 10). The input is a number indicating the emotional state, and the output is a judgment result indicating whether the threshold has been exceeded.

[1745] Step 4: Alerting and Notification

[1746] If the server determines that the anger level exceeds a threshold as a result of emotion evaluation, it immediately generates an alert. The input is the judgment result indicating that the threshold has been exceeded, and the output is the alert data.

[1747] The server generates a message suggesting appropriate anger management techniques to the user. Specifically, it creates a notification with the following content: "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. It is recommended that you leave the area for a while." The input is the alert data, and the output is a text message.

[1748] The device displays a message from the server on the display, prompting the user to take action. Display methods include text messages and audio alerts. The input is a text message from the server, and the output is a notification through the device's display or speaker.

[1749] Step 5: Gather feedback and update the model

[1750] After the user acts according to the system's suggestions, they input feedback into the device about changes in their emotions and the effectiveness of the system's suggestions. Specifically, they input information such as "I feel calmer" or "The suggestions were helpful." The input is information about changes in the user's emotions, and the output is the feedback data recorded on the device.

[1751] The feedback data collected by the device is sent to the server. The data is securely transferred using the communication infrastructure. The input is the feedback data, and the output is the data sent to the server.

[1752] The server analyzes the received feedback data and performs continuous learning to improve the AI ​​model. Specifically, it adjusts the model according to the individual characteristics of each user. The input is the feedback data, and the output is an updated AI model.

[1753] (Application example 1)

[1754] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1755] Conventional emotion management systems have difficulty detecting anger in real time during a user's daily life or work and taking effective measures. Furthermore, they only analyze vital and voice data when assessing emotional states, making it difficult to provide appropriate feedback based on the user's behavior and actions. This can leave users unsure of what actions to take, leading to inadequate stress management.

[1756] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1757] In this invention, the server includes means for collecting a user's vital data, means for collecting voice data, means for analyzing the vital data and voice data to evaluate the user's emotional state, means for generating a notification when the user's anger level exceeds a certain threshold, means for presenting the notification to the user to support the user's emotional management, means for an activity detection device to monitor the user's movements and suggest specific actions, means for collecting feedback from the user after the suggestion and updating the analytical model, and means for contactlessly collecting vital data. This allows the user to understand their own emotional state in real time and take appropriate action. Furthermore, the feedback improves the accuracy of the system, enabling more effective emotional management.

[1758] The "means for collecting user's vital data" refers to a device or system for collecting the user's biological information such as heart rate, blood pressure, and body temperature in real time.

[1759] A "means for collecting voice data" is a device or system for recording a user's voice and analyzing the tone and volume of the voice.

[1760] The "means for assessing the user's emotional state" is a device or system for analyzing the collected vital data and voice data to identify the user's emotions, such as stress or anger.

[1761] A "means for generating a notification" is a device or system that generates a warning or alert to notify a user when their emotional state exceeds a certain threshold.

[1762] A "means for supporting a user's emotional management" is a device or system for providing the user with advice and behavioral instructions necessary to control their emotional state.

[1763] An "activity detection device" is a device or system that monitors a user's activity or behavior and grasps the situation.

[1764] The "means for suggesting a specific action" is a device or system for presenting recommended actions or measures to the user based on the detected emotional state or behavior.

[1765] A "means for collecting feedback" is a device or system that allows a user to input and collect information about actions taken and the results of those actions.

[1766] The "means for updating the analytical model" is a device or system that continuously improves the AI ​​model to more accurately assess emotional states based on collected feedback data.

[1767] "Means for collecting vital data non-contact" refers to technology or devices for collecting vital data without requiring physical contact with the user.

[1768] This invention relates to a system that evaluates a user's emotional state in real time and provides appropriate management techniques. Specifically, it proposes a method for appropriately managing the user's anger and stress by collecting and analyzing the user's vital signs and voice data.

[1769] The system includes the following main components:

[1770] 1. Data Collection

[1771] The server collects the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data using the user's contactless pulse sensor and voice recognition microphone (e.g., TensorFlow voice recognition model). This allows the user's biological state and emotional tone to be understood in real time.

[1772] 2. Data Analysis

[1773] An analysis engine (e.g., SciKit-Learn + Keras) deployed on the server analyzes the collected vital signs and voice data. The data is stored and analyzed using a cloud-based platform (e.g., AWS Lambda + S3 + SageMaker). The system detects sudden increases in pulse rate and changes in voice tone to assess the user's emotional state. Based on this analysis, the system quantifies the user's anger and stress levels.

[1774] 3. Notices and Suggestions

[1775] If the user's emotional state exceeds a certain threshold, the server generates a warning or alert, which is displayed on the task-sensing device (such as a factory robot's display). The notification includes specific actions the user should take, such as "Take a deep breath for 6 seconds" or "Stress level: 7 / 10. We recommend taking a short break."

[1776] 4. Gather feedback and update the model

[1777] The user performs the suggested action and then inputs feedback to the server. The feedback includes the change in the user's emotions after the action and the effectiveness of the suggestion. The analysis model is updated based on this feedback, improving the accuracy of the next evaluation and suggestion.

[1778] As a specific example, if a worker in a factory experiences a sudden increase in pulse rate due to being in a loud environment, the work detection device may suggest, "Take a deep breath." If the worker follows this suggestion, they can input feedback that "stress has been alleviated," which will improve the analysis model.

[1779] Below are some example prompts to input to a generative AI model:

[1780] "Implement a system that monitors the pulse and voice of workers in factories to assess their stress levels in real time. If a rapid rise in pulse or angry tone is detected, provide appropriate anger management techniques."

[1781] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1782] Step 1:

[1783] The device uses the user's contactless pulse sensor and voice recognition microphone to obtain the user's vital data (heart rate, blood pressure, body temperature, etc.) and voice data in real time. Specifically, the pulse sensor measures the user's pulse every 30 seconds, and the microphone collects the user's speech. The input is vital data and voice data, and the output is a real-time record of these data.

[1784] Step 2:

[1785] The device periodically transmits the collected vital and voice data to a server, where the data is uploaded to a cloud-based platform. Specifically, the data is securely stored using a storage service such as AWS S3. The input is the recorded data from the device, and the output is the data transferred to the server.

[1786] Step 3:

[1787] The server analyzes the received vital signs and voice data. Specifically, it uses SciKit-Learn and Keras to detect sudden increases in pulse rate and changes in voice tone. The analysis model evaluates the user's emotional state based on the data and quantifies levels of anger and stress. The input is the collected data, and the output is the evaluation of the emotional state.

[1788] Step 4:

[1789] The server generates an alert when the user's emotional state exceeds a certain threshold. The generated alert includes a specific action suggestion, such as "Take a deep breath for six seconds." The input is the evaluation result of the emotional state, and the output is the alert and the action suggestion.

[1790] Step 5:

[1791] The terminal presents the alerts and action suggestions from the server to the user. Specifically, it displays messages on the display and helps the user act according to the suggestions. The input is the server's alert, and the output is the message presented to the user.

[1792] Step 6:

[1793] The user inputs feedback on the results of following the suggestions into the terminal. Specifically, a feedback form is used to record changes in emotions and the effectiveness of the suggestions. The input is feedback after the user's actions, and the output is feedback data.

[1794] Step 7:

[1795] The device sends user feedback to the server, which then updates the AI ​​analysis model based on the feedback. This improves the accuracy of future emotion assessments. The input is the user feedback, and the output is the updated analysis model.

[1796] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1797] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects a user's vital signs and voice data in real time, analyzes this data to evaluate the user's emotional state, and provides the user with appropriate anger management techniques when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[1798] Program processing

[1799] 1. Data Collection

[1800] The device (smartwatch) measures the user's pulse at regular intervals. Pulse data is important for recording the user's heart rate fluctuations in real time.

[1801] The device recognizes surrounding sounds and collects user voice data, capturing the user's speech and the surrounding acoustic environment through a microphone.

[1802] The terminal periodically transmits the collected pulse data and voice data to the server.

[1803] 2. Data Analysis

[1804] The server receives the vital data (pulse data) and voice data sent from the terminal.

[1805] An AI model placed on the server analyzes the received vital data and detects sudden increases in pulse rate or abnormal fluctuations.

[1806] The server analyzes the voice data and recognizes angry tones and patterns in the user's voice.

[1807] 3. Emotional assessment

[1808] The server integrates vital data and voice data and uses an emotion engine to assess the user's emotional state, using an AI model to provide a customized assessment of the user's emotional state.

[1809] The emotion engine recognizes the user's multiple emotions (anger, stress, anxiety, etc.) and quantifies their overall emotional state.

[1810] 4. Alert Generation and Notification

[1811] The server generates an alert if the emotion assessment indicates that the anger level exceeds a certain threshold.

[1812] The server suggests appropriate anger management techniques to the user, such as "Take a deep breath for six seconds" or "Anger level: 7 / 10. We recommend you leave the area for a while."

[1813] The terminal displays the message presented to the user and prompts the user to take appropriate action.

[1814] 5. Gather feedback and update the model

[1815] After the user follows the instructions of the system, they input feedback into the device, providing information such as how their emotions changed and whether the instructions were helpful.

[1816] The device sends the collected feedback to the server.

[1817] The server updates the AI ​​model and emotion engine based on the feedback, improving the accuracy of the next emotion assessment.

[1818] Specific examples

[1819] Example 1: Frustration at work

[1820] 1. Data Collection

[1821] The device measures the user's pulse every 60 seconds and detects any fluctuations.

[1822] The device collects the user's voice in real time and transmits it to the server.

[1823] 2. Data analysis and emotion assessment

[1824] The server detects a spike in pulse rate and an angry tone.

[1825] The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[1826] 3. Display of notifications

[1827] The device displays the message "Take a deep breath for 6 seconds" to the user.

[1828] 4. Gather feedback and update the model

[1829] After taking a deep breath, the user inputs "I feel calm" into the terminal and sends it to the server.

[1830] The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[1831] Example 2: Stress during child-rearing

[1832] 1. Data Collection

[1833] The device measures the user's pulse every 30 seconds.

[1834] The device collects conversations with family members and sends them to a server.

[1835] 2. Data analysis and emotion assessment

[1836] The server detects pulse spikes and angry tones in the voice.

[1837] The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[1838] 3. Display of notifications

[1839] The device displays the message "Anger level: 7 / 10. We recommend you step away from the area."

[1840] 4. Gather feedback and update the model

[1841] The user actually leaves the scene and inputs feedback that their emotions have calmed down.

[1842] The server receives the feedback and updates the AI ​​model and emotion engine.

[1843] This allows the anger management system, which utilizes AI and an emotion engine, to monitor users' emotions in real time and provide appropriate management techniques, enabling users to manage their emotions in a healthy way and respond appropriately even in stressful situations.

[1844] The processing flow will be explained below.

[1845] Step 1:

[1846] The device (smartwatch) measures the user's pulse at regular intervals, records the pulse data in real time using a sensor, and stores it in a data logger.

[1847] Step 2:

[1848] The device recognizes surrounding sounds and collects user voice data. It uses a microphone to capture the user's speech and environmental sounds, which are also stored in a data logger.

[1849] Step 3:

[1850] The terminal transmits the collected pulse data and voice data to the server at regular intervals. The data is transferred via wireless communication using the communication module.

[1851] Step 4:

[1852] The server receives vital data (pulse data) and voice data sent from the device, and stores this data in a database for analysis.

[1853] Step 5:

[1854] The AI ​​model on the server analyzes the received vital signs and detects sudden spikes or abnormal fluctuations in the pulse rate. The model evaluates the pulse rate change pattern and identifies abnormal values.

[1855] Step 6:

[1856] The server analyzes the voice data and recognizes angry tones and language in the user's voice. It uses a voice analysis algorithm to detect changes in voice intensity and tone.

[1857] Step 7:

[1858] The server integrates vital data and voice data and evaluates the user's emotional state using an emotion engine, which comprehensively analyzes the data and quantifies emotions such as anger, stress, and anxiety.

[1859] Step 8:

[1860] The server evaluates the user's emotional state and generates an alert if the anger level exceeds a certain threshold, with tailored countermeasures for the user.

[1861] Step 9:

[1862] The server generates an alert and sends it to the terminal. The alert message is wirelessly transferred to the terminal via the communication module.

[1863] Step 10:

[1864] The device will display notifications to the user, such as "Take a deep breath for 6 seconds" or "Anger level: 7 / 10. We recommend you step away from the situation."

[1865] Step 11:

[1866] The user acts according to the system's instructions, such as taking a deep breath or leaving the area.

[1867] Step 12:

[1868] After the user follows the system's instructions, they input their feedback into the terminal, describing how their emotions changed and the effectiveness of the system's instructions in an input form.

[1869] Step 13:

[1870] The device sends the user's feedback to the server, and the collected feedback data is sent to the server via wireless communication.

[1871] Step 14:

[1872] The server receives the feedback data and updates the emotion engine and AI model, using the feedback to retrain the model and improve the accuracy of emotion assessment.

[1873] Through these steps, the system evaluates the user's emotional state in real time and provides appropriate anger management techniques at the appropriate time, helping the user to better control their emotions and respond appropriately even in stressful situations.

[1874] Example 2

[1875] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1876] Conventional emotion management systems lack the means to analyze users' biometric and voice data in real time, making it difficult to properly and timely evaluate the user's emotional state and prompt appropriate responses. Furthermore, they lack a mechanism for improving the accuracy of the system based on feedback, making it impossible to provide emotion management methods customized for individual users. This has led to the issue of being unable to properly manage users' stress and anger.

[1877] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data of the user, means for collecting voice data, means for analyzing the biometric data and voice data to evaluate the user's emotional state, means for generating a notification when the user's emotional state exceeds a specific threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting feedback from the user and updating the analysis means, and means for integrating the biometric data and voice data to evaluate the user's emotional state in detail. This makes it possible to evaluate the user's emotional state in real time and provide an appropriate management method. Furthermore, by improving the accuracy of the system based on the feedback, it is possible to provide an emotion management method customized for each individual user.

[1878] "Biometric data" refers to data that indicates the physiological state of the user, and mainly includes heart rate and biorhythms.

[1879] "Audio data" is data that collects the sounds of the user and the surrounding area, and includes conversations and environmental sounds acquired through a microphone.

[1880] "Analysis means" refers to a means for analyzing collected biometric data and voice data to evaluate the user's emotional state, and is primarily a means that uses AI models and natural language processing technology.

[1881] An "emotional state" is a state that indicates the user's current psychological or emotional state, and includes multiple emotional elements such as anger, stress, and anxiety.

[1882] "Notifications" are warning and advice messages generated by the system for the user, and are messages that suggest appropriate emotion management techniques.

[1883] "Feedback" refers to data on reactions and evaluations provided by users, and indicates how appropriate the system's instructions were and how their emotions have changed.

[1884] "Update methods" are methods for improving and correcting analytical methods and models based on collected feedback, and are methods for improving the accuracy and effectiveness of the system.

[1885] "Integration" refers to the technique of combining different types of data (biometric data and audio data) and processing them as a single analysis target.

[1886] A "threshold" is a reference value set by the system when assessing an emotional state, which, if exceeded, triggers a particular action (such as generating a notification).

[1887] The "emotion engine" is a mechanism within the system that evaluates the user's emotional state in detail and analyzes it from multiple angles, and is an engine that is primarily built using AI technology.

[1888] The system of the present invention comprises an apparatus and method for effectively managing anger experienced by users in their daily lives and at work. Specifically, the system collects biometric and voice data from users in real time, analyzes this data to evaluate the user's emotional state, and provides appropriate anger management techniques to the user when necessary. Furthermore, by combining it with an emotion engine, it becomes possible to evaluate the user's emotions in more detail and respond individually.

[1889] Data collection

[1890] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[1891] The device collects the user's voice data in real time through a microphone, and is equipped with a noise-canceling function to improve sound quality.

[1892] The device collects pulse and voice data and periodically transmits it to a server via Wi-Fi or cellular networks.

[1893] Data analysis

[1894] The server stores the biometric data and voice data received from the terminal in a database.

[1895] The server is home to an AI model built using TensorFlow that analyzes biometric data and identifies sudden spikes and abnormal fluctuations in heart rate.

[1896] The server's natural language processing (NLP) model analyzes the audio data to detect angry tones and specific patterns in the user's voice, using pre-trained models such as BERT and WaveNet.

[1897] Emotional evaluation

[1898] The server integrates vital data and voice data and synchronizes them based on timestamps.

[1899] The emotion engine on the server analyzes this integrated data and evaluates the user's emotional state. The emotion engine generates multiple numerical values ​​(anger, stress, anxiety, etc.) to evaluate the overall emotional state.

[1900] Alerting and Notification

[1901] The server generates an alert if the emotional state exceeds a certain threshold as a result of the emotion assessment.

[1902] The server sends a message to the device suggesting appropriate anger management techniques to the user, such as "Your heart rate is increasing. Please take a deep breath."

[1903] Notifications received by the device are immediately displayed to the user, either as a vibration or a pop-up message.

[1904] Gathering feedback and updating the model

[1905] After the user follows the system's instructions, they input feedback from the device, such as how their emotions changed and whether the instructions were helpful.

[1906] The terminal transmits feedback data from the user to the server.

[1907] The server analyzes the collected feedback data and updates the AI ​​model and emotion engine, which improves the accuracy of the next emotion evaluation.

[1908] Specific examples

[1909] Example 1: Frustration at work

[1910] 1. The device measures the user's pulse every 60 seconds and detects any fluctuations.

[1911] 2. The device collects the user's voice in real time and sends it to the server.

[1912] 3. The server detects a spike in pulse rate and an angry tone.

[1913] 4. The server uses an emotion engine to quantify the anger level and determines that it has reached an alert level.

[1914] 5. The device displays the message "Take a deep breath for 6 seconds" to the user.

[1915] 6. After taking a deep breath, the user enters "I feel calm" into the device and sends it to the server.

[1916] 7. The server receives the feedback and fine-tunes the AI ​​model and emotion engine.

[1917] Example 2: Stress during child-rearing

[1918] 1. The device measures the user's pulse every 30 seconds.

[1919] 2. The device collects conversations with family members and sends them to the server.

[1920] 3. The server detects pulse spikes and angry tones in the voice.

[1921] 4. The server uses an emotion engine to assess not only anger but also levels of stress, anxiety, etc.

[1922] 5. The device displays the message "Anger level: 7 / 10. We recommend you take some time away."

[1923] 6. The user actually leaves the situation and inputs feedback that their emotions have calmed down.

[1924] 7. The server receives the feedback and updates the AI ​​model and emotion engine.

[1925] Prompt Sentence Examples

[1926] "Your heart rate is increasing. Take six deep breaths."

[1927] "Your anger levels are high. I suggest you step away from the area for a bit."

[1928] The system and method of the present invention allows for real-time monitoring of a user's emotional state and provides appropriate management techniques, thereby enabling users to manage their emotions in a healthy way and respond appropriately to stressful situations.

[1929] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1930] Step 1: Data collection

[1931] Input: Real-time biometric data (pulse) and voice data of the user.

[1932] Operation:

[1933] The device (smartwatch) measures the user's pulse every 60 seconds using an optical heart rate sensor.

[1934] The pulse data measured by the device is stored in the internal memory.

[1935] The device uses a built-in microphone to collect the user's voice data in real time, and the voice is captured clearly through noise cancellation.

[1936] Output: Collected biometric data (pulse data) and audio data.

[1937] Step 2: Send data

[1938] Input: Collected biometric data (pulse data) and audio data.

[1939] Operation:

[1940] The device compresses the biometric and audio data it collects and periodically transmits it to a server via Wi-Fi or cellular networks.

[1941] Output: Pulse data and audio data sent to the server.

[1942] Step 3: Analyzing vital data

[1943] Input: Pulse data sent to the server.

[1944] Operation:

[1945] The server stores the received pulse data in a database.

[1946] The received pulse data is analyzed using a server-based AI model (built using TensorFlow), specifically to identify sudden spikes and abnormal fluctuations in heart rate.

[1947] Output: Analyzed heart rate data (anomaly detection results).

[1948] Step 4: Analyzing the audio data

[1949] Input: The audio data sent to the server.

[1950] Operation:

[1951] The server stores the received audio data in cloud storage.

[1952] A server-based natural language processing (NLP) model (using BERT or WaveNet) analyzes the audio data and detects angry tones and patterns in the user's voice.

[1953] Output: Analyzed audio data (angry tone detection results).

[1954] Step 5: Data Integration

[1955] Input: Analyzed heart rate data and audio data (anomaly detection results and angry tone detection results).

[1956] Operation:

[1957] The server synchronizes and integrates the pulse data and voice data based on timestamps.

[1958] Output: A consolidated dataset (consolidated data based on a uniform timestamp).

[1959] Step 6: Emotional assessment

[1960] Input: The integrated dataset.

[1961] Operation:

[1962] The emotion engine on the server analyzes the integrated data and evaluates the user's emotional state, which is expressed as a numerical value such as anger, stress, or anxiety.

[1963] Output: User's emotional state assessment result (quantified emotional level).

[1964] Step 7: Alert Generation

[1965] Input: User's emotional state assessment results.

[1966] Operation:

[1967] The server generates an alert based on the emotion assessment results if the emotional state exceeds a certain threshold.

[1968] Output: Alert information (alert message when threshold is exceeded).

[1969] Step 8: Send and display notifications

[1970] Input: Alert information.

[1971] Operation:

[1972] The server sends an alert message to the terminal.

[1973] The device will receive notifications and display them to the user as a vibration or a pop-up message, such as "Your heart rate is increasing. Please take a deep breath."

[1974] Output: The notification message presented to the user.

[1975] Step 9: Gather feedback

[1976] Input: User feedback information (e.g., reaction to notifications, emotional changes, etc.).

[1977] Operation:

[1978] After the user acts according to the system's instructions, they input feedback into the device about changes in their emotions and the effectiveness of the notification.

[1979] The terminal transmits feedback data from the user to the server.

[1980] Output: Feedback data sent to the server.

[1981] Step 10: Model Update

[1982] Input: Feedback data.

[1983] Operation:

[1984] The server analyzes the collected feedback data.

[1985] The AI ​​model and emotion engine on the server are updated to improve the accuracy of the next emotion evaluation.

[1986] Output: Updated AI model and emotion engine.

[1987] (Application example 2)

[1988] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1989] In conventional food delivery work, delivery workers often experience various stressful situations, such as congested roads and complaints from customers, which can lead to problems such as reduced work efficiency and delivery quality. In particular, if delivery workers do not manage their emotions properly, the risk of accidents and trouble increases. Therefore, there is a need for a system that can immediately detect the stress and anger that delivery workers feel while working and provide appropriate management methods.

[1990] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1991] In this invention, the server includes means for collecting biometric data of a user, means for collecting acoustic data, means for analyzing the biometric data and acoustic data and evaluating the user's emotional state, means for generating a notification when the user's anger exceeds a certain threshold, means for presenting the notification to the user and supporting the user's emotional management, means for collecting user behavior based on the notification as feedback to support the delivery person's stress management and using the feedback to update the AI ​​model, and means for analyzing events that cause the delivery person to feel stressed and presenting countermeasures. This makes it possible to detect the stress and anger that delivery people feel during their work in real time and provide appropriate management methods.

[1992] "Biometric data" refers to data obtained from the user's body, including, for example, heart rate, pulse, body temperature, and the like.

[1993] "Acoustic data" refers to audio information of the user and the surroundings, including the user's voice and surrounding environmental sounds.

[1994] "Emotional state" indicates the user's psychological state, and is a numerical representation of emotions such as anger, stress, and anxiety.

[1995] A "notification" is information, including warnings and instructions, sent to a user, such as a message displayed on the user's device.

[1996] "Emotion management" refers to techniques that allow users to appropriately control their emotions and reduce stress and anger.

[1997] An "AI model" is an algorithm that uses artificial intelligence to analyze user data and assess their emotional state.

[1998] "Feedback" refers to information that a user inputs after following instructions from the system and taking action, and is used to improve the system.

[1999] "Delivery person" refers to an employee of a food delivery service, specifically someone who delivers meals.

[2000] "Stress management" means that the system provides a way to reduce the mental burden on delivery personnel and improve their work efficiency.

[2001] "Countermeasures" means specific measures or instructions to help delivery personnel deal with stress or problems they face in the course of their work.

[2002] This invention is a system that manages the stress and anger felt by food delivery workers in real time and provides appropriate measures. The configuration and operation for realizing this system are described in detail below.

[2003] Hardware Configuration

[2004] 1. Smartwatch

[2005] Collects biometric data such as heart rate and pulse in real time.

[2006] 2. Smartphone

[2007] It has a built-in microphone for collecting acoustic data (voice data and environmental sounds).

[2008] A dedicated app is installed to collect and transmit data.

[2009] 3. Server

[2010] An AI model is deployed to perform data analysis.

[2011] Save the information to the database and update the model.

[2012] Software Configuration

[2013] 1. Data Collection App

[2014] Pulse data is collected from the smartwatch and transferred to a smartphone.

[2015] Audio data is collected using the smartphone's microphone.

[2016] 2. Server AI model

[2017] It is built using machine learning libraries such as TensorFlow and PyTorch.

[2018] Biometric and acoustic data is analyzed to assess the user's emotional state.

[2019] 3. Cloud Messaging System

[2020] Send real-time notifications to users using something like Firebase Cloud Messaging.

[2021] Data flow and processing overview

[2022] 1. Data Collection

[2023] The smartwatch measures the delivery person's heart rate every 10 seconds and transfers the data to a smartphone.

[2024] The smartphone's microphone collects ambient sounds and the delivery person's voice, and transmits them along with biometric data to a server.

[2025] 2. Data Analysis

[2026] The server analyzes the data it receives using an AI model to detect sudden increases in heart rate and angry tones in the audio data.

[2027] The emotion engine uses this data to quantify and evaluate the delivery person's emotional state.

[2028] 3. Notification Generation and Delivery

[2029] If the emotion assessment reveals that the anger or stress level exceeds a certain threshold, the server generates a notification with appropriate measures (e.g., "Take a deep breath" or "Take a 5-minute break") and sends it to the smartphone.

[2030] A notification will appear on the smartphone, prompting the delivery person to take specific action.

[2031] 4. Gather feedback and update the model

[2032] After the delivery person follows the system's instructions, they enter feedback on the effectiveness of their actions.

[2033] This feedback is received by the server and used to improve the accuracy of the AI ​​model.

[2034] Specific examples

[2035] A delivery driver received a complaint call from a customer while driving on a busy road. At this time, the smartwatch detected a sudden increase in the driver's heart rate, and the smartphone recognized an angry tone from the surrounding audio. The server's emotion engine analyzed this data and assessed the driver's anger level as 8 / 10, so a notification "Take a 5-minute break" was displayed on the driver's smartphone. The delivery driver followed the instructions, took a break, and entered feedback that his emotions had calmed down. The feedback data was sent to the server and used to improve the accuracy of the next assessment.

[2036] Prompt Sentence Examples

[2037] Generate a description for the following application: A food delivery app that manages the emotions of delivery drivers. It collects vital data and voice data, analyzes them using an emotion engine, and suggests appropriate anger management techniques in real time.

[2038] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2039] Step 1:

[2040] Data collection

[2041] The devices (smartwatches and smartphones) collect the delivery person's heart rate and voice data.

[2042] Specifically, the smartwatch measures the driver's heart rate every 10 seconds and transmits the data to the smartphone, whose microphone captures surrounding sounds and the voice of the delivery person.

[2043] Input: Delivery person's heart rate data, voice data

[2044] Output: Heart rate data and audio data are sent to a smartphone

[2045] Step 2:

[2046] Data Transfer

[2047] The terminal transmits the collected biometric data and acoustic data to a server.

[2048] Specifically, the smartphone uses Wi-Fi or mobile data communication to upload the collected data to a server in real time.

[2049] Input: Heart rate data, audio data

[2050] Output: Data is saved to the server

[2051] Step 3:

[2052] Data analysis

[2053] The server parses the received data.

[2054] Specifically, the AI ​​model on the server analyzes fluctuations in heart rate data to detect sudden changes, and analyzes voice data to recognize angry tones and specific voice patterns.

[2055] Input: Heart rate data, audio data

[2056] Output: Analysis results (heart rate fluctuations, anger tone detection)

[2057] Step 4:

[2058] Emotional evaluation

[2059] Based on the analysis results, the server's emotion engine quantifies and evaluates the delivery person's emotional state.

[2060] Specifically, the AI ​​model integrates heart rate variability and the tone of the voice data to calculate a score for anger and stress levels.

[2061] Input: Analysis results (heart rate fluctuations, anger tone detection)

[2062] Output: Emotional state evaluation result (e.g., anger level 8 / 10)

[2063] Step 5:

[2064] Notification generation

[2065] Based on the emotion evaluation results, the server generates a notification with specific measures to be taken by the delivery person.

[2066] Specifically, if the server detects an emotion score above a certain threshold, it generates messages such as "Take a deep breath" or "Take a five-minute break."

[2067] Input: Emotional state assessment results

[2068] Output: Notification message

[2069] Step 6:

[2070] Send notifications

[2071] The server generates a notification message and sends it to the device (smartphone).

[2072] Specifically, it uses cloud messaging services such as Firebase Cloud Messaging to send notifications to devices in real time.

[2073] Input: Notification message

[2074] Output: A notification appears on your smartphone

[2075] Step 7:

[2076] Feedback collection

[2077] After the user (delivery person) takes the action instructed in accordance with the notification, they input feedback about the effect of the action.

[2078] Specifically, delivery workers enter into a smartphone app whether they feel calmer after a break.

[2079] Input: Delivery person feedback information

[2080] Output: Feedback information is sent to the server

[2081] Step 8:

[2082] Model Update

[2083] Based on the feedback information received by the server, the AI ​​model is updated to improve its accuracy.

[2084] Specifically, the collected feedback data is used to fine-tune emotion evaluation algorithms and thresholds.

[2085] Input: Feedback information

[2086] Output: Updated AI model

[2087] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2088] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2089] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2090] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2091] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2092] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2093] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2094] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2095] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2096] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2097] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2098] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2099] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2100] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2101] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2102] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2103] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2104] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2105] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2106] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2107] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2108] The following is further disclosed regarding the above embodiment.

[2109] (Claim 1)

[2110] means for collecting vital data of a user;

[2111] means for collecting audio data;

[2112] means for analyzing the vital data and voice data to assess the emotional state of the user;

[2113] means for generating a notification when the user's anger emotion exceeds a certain threshold;

[2114] The system includes means for presenting the notification to a user and assisting the user in managing their emotions.

[2115] (Claim 2)

[2116] 10. The system of claim 1, wherein the vital data includes pulse data.

[2117] (Claim 3)

[2118] 10. The system of claim 1, wherein the voice data is analyzed to detect angry tones.

[2119] (Claim 4)

[2120] 2. The system according to claim 1, further comprising means for presenting a message to the user encouraging deep breathing.

[2121] (Claim 5)

[2122] The system of claim 1, further comprising means for collecting feedback from the user and updating the AI ​​model.

[2123] "Example 1"

[2124] (Claim 1)

[2125] means for collecting biometric data of a user;

[2126] means for collecting audio data;

[2127] means for analyzing the biometric data and audio data to assess the emotional state of the user;

[2128] means for generating a notification when the user's anger emotion exceeds a certain threshold;

[2129] means for presenting the notification to a user to assist the user in managing their emotions;

[2130] a means for collecting post-action feedback from users;

[2131] The system includes means for analyzing said feedback and updating a rating model.

[2132] (Claim 2)

[2133] The system of claim 1 , wherein the biometric data includes pulse data.

[2134] (Claim 3)

[2135] The system of claim 1, wherein the voice data is analyzed to detect voice patterns of an angry subject.

[2136] "Application Example 1"

[2137] (Claim 1)

[2138] means for collecting vital data of a user;

[2139] means for collecting audio data;

[2140] means for analyzing the vital data and voice data to assess the emotional state of the user;

[2141] means for generating a notification when the user's anger emotion exceeds a certain threshold;

[2142] means for presenting the notification to a user to assist the user in managing their emotions;

[2143] a means for the task detection device to monitor the user's movements and suggest specific actions;

[2144] means for collecting feedback from users after the proposal and updating the analytical model;

[2145] A means for contactless collection of vital data;

[2146] A system including:

[2147] (Claim 2)

[2148] 10. The system of claim 1, wherein the vital data includes pulse data.

[2149] (Claim 3)

[2150] 10. The system of claim 1, wherein the voice data is analyzed to detect angry tones.

[2151] "Example 2: Combining Emotion Engines"

[2152] (Claim 1)

[2153] means for collecting biometric data of a user;

[2154] means for collecting audio data;

[2155] means for analyzing the biometric data and audio data to assess the emotional state of the user;

[2156] means for generating a notification when the user's emotional state exceeds a certain threshold;

[2157] means for presenting the notification to a user to assist the user in managing their emotions;

[2158] a means for collecting user feedback and updating the analytics means;

[2159] a means for integrating biometric and audio data to provide a detailed assessment of emotional state;

[2160] A system including:

[2161] (Claim 2)

[2162] 10. The system of claim 1, wherein the biometric data includes heart rate data.

[2163] (Claim 3)

[2164] The system of claim 1, wherein the audio data is analyzed to detect anger patterns.

[2165] "Application example 2 when combining emotion engines"

[2166] (Claim 1)

[2167] means for collecting biometric data of a user;

[2168] means for collecting acoustic data;

[2169] means for analyzing the biometric and acoustic data to assess the emotional state of the user;

[2170] means for generating a notification when the user's anger emotion exceeds a certain threshold;

[2171] means for presenting the notification to a user to assist the user in managing their emotions;

[2172] To support delivery workers' stress management, a method is provided to collect user behavior based on notifications as feedback and use it to update the AI ​​model.

[2173] A system that includes a means for analyzing events that cause stress to delivery personnel and suggesting countermeasures.

[2174] (Claim 2)

[2175] The system of claim 1 , wherein the biometric data includes heart rate data.

[2176] (Claim 3)

[2177] 10. The system of claim 1, wherein the acoustic data is analyzed to detect angry speech patterns. [Explanation of symbols]

[2178] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting vital data of a user; means for collecting audio data; means for analyzing the vital data and voice data to assess the emotional state of the user; means for generating a notification when the user's anger emotion exceeds a certain threshold; The system includes means for presenting the notification to a user and assisting the user in managing their emotions.

2. The system of claim 1 , wherein the vital data includes pulse data.

3. The system of claim 1 , wherein the voice data is analyzed to detect angry tones.

4. The system according to claim 1 , further comprising means for presenting a message to the user encouraging deep breathing.

5. The system of claim 1 , further comprising means for collecting feedback from the user and updating the AI ​​model.

Images