System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting chat and voice data entered by a user; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis result and generating an alert when the score exceeds a certain threshold; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; and means for notifying the user of the selected care plan and dialogue script.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern times, mental illnesses (such as depression and anxiety disorders) are serious problems that have a significant impact on individuals and society as a whole. In particular, if mental illnesses are left untreated in the workplace, they can not only reduce productivity but also have a significant impact on the health and quality of life of workers. Currently, many mental illnesses are not detected early, making it difficult for people to receive appropriate care. For this reason, it is necessary to detect users' mental states early and provide appropriate care. [Means for solving the problem]

[0005] The present invention provides a system that collects chat and voice data entered by users, analyzes it using natural language processing algorithms, performs sentiment analysis, and extracts keywords to detect signs of mental illness early and provide necessary care. The system calculates a sign score of mental illness based on the analysis results and generates an alert if a certain threshold is exceeded. The system then provides the user with an appropriate care plan and dialogue script based on the generated alert. When voice data is entered, it is converted into text data using voice recognition technology and then preprocessed using noise reduction technology. This allows the user's mental state to be quickly grasped and care to be provided early.

[0006] "User" refers to an individual who uses the system.

[0007] "Chat" refers to a method of communication that takes place in real time using text messages.

[0008] "Audio data" refers to data that records audio in digital format.

[0009] "Natural language processing algorithms" refers to a set of techniques and methods that allow computers to process human language.

[0010] "Sentiment analysis" refers to technology for inferring a speaker's emotions from text or audio data.

[0011] "Keyword extraction" refers to the technique of selecting important words and phrases from text data.

[0012] "Psychopathic symptom score" refers to an index that shows the result of quantifying the risk of psychopathic illness based on analyzed data.

[0013] An "alert" is a warning or notification that occurs when certain conditions are met.

[0014] A "care plan" refers to specific support and measures proposed based on the user's mental state.

[0015] A "dialogue script" refers to a dialogue script prepared in advance to support interaction with a user.

[0016] "Voice recognition technology" refers to technology for analyzing voice data and converting it into text data.

[0017] "Noise reduction technology" refers to technology for removing unnecessary noise from audio data.

[0018] "Text data" refers to data that represents character information in digital form. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention provides a system for detecting early signs of mental illness in a user and providing appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat or voice. Below, an embodiment of the present invention will be described with specific examples.

[0041] System Overview

[0042] The system collects data entered by users via chat or voice and analyzes it using natural language processing algorithms. The analysis involves sentiment analysis and keyword extraction, and calculates a mental illness symptom score. If the score exceeds a certain level, an alert is generated and the user is provided with an appropriate care plan and dialogue script.

[0043] Technologies used

[0044] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[0045] Speech recognition technology: Converts voice data into text data.

[0046] Noise reduction technology: Removes unwanted noise from audio data.

[0047] Program processing

[0048] Data collection

[0049] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[0050] Data analysis

[0051] Server: The received text data is passed to a natural language processing module for sentiment analysis and keyword extraction. For example, keywords such as "stress," "tension," and "can't sleep" are extracted from the text data to analyze the speaker's emotions.

[0052] Identifying symptoms

[0053] Server: Calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the calculated score exceeds a certain threshold, an alert is generated. For example, a specific numerical value such as "Depression risk score: 85" or "Anxiety disorder risk score: 90" is displayed.

[0054] Alert Generation

[0055] Server: The generated alert includes the user ID, type of mental illness, score, and specific keywords. This information can be saved in a database and notified to an administrator.

[0056] Providing care plans

[0057] Server: Based on the alert information, the server selects a care plan to provide to the user. For example, it provides specific and actionable plans such as "recommending regular exercise" and "how to schedule counseling appointments."

[0058] Conversation script recommendations

[0059] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. For example, for depression, it generates a message like "You seem tired lately, are you okay?", and for anxiety disorders, it generates a message like "Why don't you try taking a deep breath?"

[0060] User Notification

[0061] Terminal: The care plan and dialogue script are notified to the user. They are displayed in a format that is easy for the user to check, and are provided as pop-ups or notification messages as necessary.

[0062] Specific examples

[0063] Detecting and Treating Signs of Depression

[0064] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0065] Server: Sentiment analysis detects negative emotions and keywords such as "tired" and "not happy."

[0066] Server: Generate alert due to high depression risk score.

[0067] Server: Provides care plans such as "exercise recommendations" and "appointment with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[0068] Terminal: Notify User A of this.

[0069] Detecting and Treating Anxiety Disorder Signs

[0070] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0071] Terminal: Converts voice into text data and sends it to the server.

[0072] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[0073] Server: High score generates an alert.

[0074] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[0075] Terminal: Notify User B of this.

[0076] The present invention enables early detection of signs of mental illness and prompt provision of appropriate care, thereby maintaining the user's health and improving their quality of life, which is expected to help maintain productivity in the workplace and prevent the worsening of mental illness.

[0077] The processing flow will be explained below.

[0078] Step 1: The user inputs text or voice using a chat app or speech recognition app. For example, the user might input text such as "Work has been really stressful lately," or say something like "I'm so nervous about the meeting I'm having trouble sleeping."

[0079] Step 2: The device receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[0080] Step 3: The server passes the received text data to a natural language processing (NLP) module for text preprocessing, such as tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[0081] Step 4: The server passes the preprocessed text to a sentiment analysis algorithm to classify emotions such as positive, negative, or neutral, and extract key keywords, such as "stressed," "tense," and "unhappy."

[0082] Step 5: The server calculates a mental illness symptom score based on the results of the sentiment analysis and keyword extraction. The algorithm uses a machine learning model (e.g., SVM, deep learning, etc.) to quantify the severity of the symptoms.

[0083] Step 6: The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and specific keywords.

[0084] Step 7: The server selects a care plan for the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[0085] Step 8: The server generates an appropriate dialogue script depending on the type of mental illness detected, such as "You seem tired lately, are you okay?" or "Why don't you try some deep breathing to relax before the meeting?"

[0086] Step 9: The terminal notifies the user of the care plan and the dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[0087] Step 10: The user reviews the notification and takes action based on the provided care plan or dialogue script, for example, trying suggested relaxation techniques or booking a consultation.

[0088] In this way, the system can detect the user's mental state early and provide appropriate care promptly.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] While the importance of mental health management has increased in recent years, it often takes time for users to become aware of their own mental state and receive appropriate care. In particular, overlooking early signs of mental illness carries the risk of worsening symptoms. Conventional methods assume that users will consult with a specialist themselves, and lack a means to automatically and in real time monitor the user's mental state and provide appropriate care. The challenge is to address this shortcoming.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes means for collecting text data and voice data input by the user, means for preprocessing the collected data using noise reduction technology and converting the voice data into text data, and means for analyzing the converted text data using a natural language processing algorithm to perform sentiment analysis and keyword extraction, thereby enabling automatic and real-time monitoring of the user's mental state and instantly providing an appropriate care plan and dialogue script.

[0094] A "user" refers to an individual who uses the system and inputs data via text or voice.

[0095] "Text data" refers to information entered by a user in text form through a chat app.

[0096] "Voice data" refers to voice information spoken by a user, including recorded or real-time sound data.

[0097] "Noise reduction technology" refers to technology that removes unnecessary background sounds and noise from audio data.

[0098] "Preprocessing" refers to a series of steps to prepare data for easier analysis.

[0099] "Natural language processing algorithms" refer to technologies for analyzing text data, and are algorithms that perform sentiment analysis and keyword extraction.

[0100] "Sentiment analysis" refers to the technology of analyzing user emotions from text data.

[0101] "Keyword extraction" refers to the technique of extracting important words and phrases from text data.

[0102] The "symptom score" refers to a numerical assessment of a user's mental state, indicating their risk of mental illness.

[0103] An "alert" refers to warning information that is generated when a user's situation exceeds a certain threshold.

[0104] "Support plan" refers to a care plan provided to the user by the system, which includes specific measures and advice.

[0105] A "dialogue script" refers to a set phrase that is generated to facilitate communication with a user.

[0106] "Notification" refers to a means of conveying information to the user through the device, and is done in the form of a pop-up or push notification.

[0107] MODE FOR CARRYING OUT THE INVENTION

[0108] The present invention is a system that detects early signs of mental illness in users and provides appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat and voice.

[0109] Hardware and Software Used

[0110] 1. Terminal: A device such as a smartphone or computer used by a user.

[0111] 2. Server: A central server for data analysis. No specific hardware is required, but a computer with powerful processing power is recommended.

[0112] 3. Software:

[0113] Natural language processing (NLP) libraries: Python's "NLTK" and "spaCy".

[0114] Speech recognition technology: "Google (registered trademark) Cloud Speech-to-Text" API for converting voice data into text data.

[0115] Noise reduction techniques: "LibROSA" and "PyDub" for preprocessing audio data.

[0116] Database: MySQL (registered trademark) or MongoDB for storing alerts and care plans.

[0117] Generative AI models: "Scikit-learn" and "TENSORFLOW (registered trademark)" for machine learning.

[0118] Program processing explanation

[0119] Data collection

[0120] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[0121] Data Preprocessing

[0122] Server: Analyzes the received voice data using noise reduction technology to remove unwanted background sounds and noise, then converts the voice data into text using speech recognition technology.

[0123] Data analysis

[0124] Server: The converted text data is passed to a natural language processing algorithm for sentiment analysis and keyword extraction.

[0125] Identifying symptoms

[0126] Server: Calculates a psychosis symptom score based on the analysis results. If the score exceeds a certain threshold, an alert is generated.

[0127] Alert Generation

[0128] Server: The generated alert contains the user ID, type of mental illness, score, and specific keywords. This information is stored in a database and notified to the administrator.

[0129] Providing care plans

[0130] Server: Based on the alert information, selects a care plan to provide to the user. Recommendations for the care plan include "regular exercise" and "how to schedule counseling appointments."

[0131] Conversation script recommendations

[0132] Server: Generates an appropriate dialogue script depending on the type of mental illness detected.

[0133] User Notification

[0134] On the device: The care plan and dialogue script are notified to the user as a pop-up or push notification.

[0135] Specific examples

[0136] Detecting and Treating Signs of Depression

[0137] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0138] Server: Performs sentiment analysis on the received text data to detect negative emotions and keywords such as "tired" and "not having fun."

[0139] Server: Generate alert due to high depression risk score.

[0140] Server: Provides care plans such as "regular exercise" and "appointments with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[0141] Terminal: Notify User A of this.

[0142] Detecting and Treating Anxiety Disorder Signs

[0143] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0144] Terminal: Converts voice into text data and sends it to the server.

[0145] Server: The audio data is preprocessed using noise reduction technology and converted into text data using Google Cloud Speech-to-Text.

[0146] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[0147] Server: High score generates an alert.

[0148] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[0149] Terminal: Notify User B of this.

[0150] Prompt Sentence Examples

[0151] 1. "Analyze what users have been talking about in chat about their work recently and tell me if they are showing signs of stress."

[0152] 2. "If a user says, 'Work has been really stressful lately,' what would you do to address that?"

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Step 1: Data collection

[0155] Device: A user uses a chat app to type, "Work has been really stressful lately," or to say, "I'm so nervous before the meeting I can't sleep," as voice data. This data is collected on the device and sent to the server.

[0156] Input: User text or voice data.

[0157] Output: The data packet to be sent to the server.

[0158] Step 2: Data Preprocessing

[0159] Server: The received audio data is first preprocessed using noise reduction technology (e.g., LibROSA, PyDub) to remove unwanted background sounds and noise, and then converted into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[0160] Input: Audio data sent from the device.

[0161] Output: Text data.

[0162] Step 3: Data analysis

[0163] Server: The converted text data is passed to a natural language processing (NLP) algorithm (e.g., NLTK, spaCy) for sentiment analysis and keyword extraction. Based on the analysis results, keywords such as "stress," "tension," and "can't sleep" are extracted, and the emotional state is analyzed.

[0164] Input: Text data converted by speech recognition technology.

[0165] Output: Sentiment analysis results and extracted keywords.

[0166] Step 4: Identify symptoms

[0167] Server: Based on the results of sentiment analysis and keyword extraction, a generative AI model (e.g., Scikit-learn, TensorFlow) is used to calculate a mental illness symptom score. If the score exceeds a certain threshold, it is determined that an abnormality has been detected and an alert is generated.

[0168] Input: Sentiment analysis results and extracted keywords.

[0169] Output: Psychosis symptom score and whether an alert was generated.

[0170] Step 5: Alert Generation

[0171] Server: If the score exceeds the threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and specific keywords. The generated alert information is stored in a database (e.g., MySQL, MongoDB) and notified to the administrator.

[0172] Input: Psychosis symptom score.

[0173] Output: Alert information.

[0174] Step 6: Provide a care plan

[0175] Server: Based on the alert information, the server selects the care plan to be provided to the user. For example, it suggests "regular exercise recommendations" or "how to schedule counseling appointments." This information is also stored in the database.

[0176] Input: Alert information.

[0177] Output: Care plan.

[0178] Step 7: Recommend conversation scripts

[0179] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. Using a generative AI model, it generates messages such as "You seem tired lately, are you okay?" for depression, or "Why don't you try taking a deep breath?" for anxiety disorders.

[0180] Input: Care plan.

[0181] Output: Dialogue script.

[0182] Step 8: Notify users

[0183] Device: The care plan and dialogue script are notified in a format that is easy for the user to view. Specifically, this is displayed as a pop-up notification or push notification in the chat app.

[0184] Input: Care plan and dialogue script.

[0185] Output: User notification.

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Mental stress among employees and customers leads to reduced productivity and customer satisfaction, so it is necessary to detect mental anxiety early and provide appropriate care even in physical store environments. However, the technology to do this efficiently and in real time is currently immature, so there is a need for a system that can appropriately monitor the mental state of employees and customers and take prompt measures.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes: means for collecting chat and voice data entered by users; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for using a smart device to monitor the mental state of employees and customers; means for providing a care plan recommending the use of a relaxation room in a store environment; means for generating a dialogue script recommending relaxation techniques such as deep breathing; and means for calculating the stress score of employees using the collected data. This makes it possible to quickly detect changes in mental state even in a physical store environment and provide appropriate care.

[0191] "User" refers to an individual or end user who uses the System.

[0192] "Chat data" refers to conversation data entered by a user in text format.

[0193] "Voice data" refers to data collected by recording voices uttered by users.

[0194] "Natural language processing algorithm" refers to a technical system for analyzing human language using a computer.

[0195] "Sentiment analysis" is a technique for determining a user's emotional state from text or voice data.

[0196] "Keyword extraction" is a technique for identifying important words and phrases from text or audio data.

[0197] The "psychiatric symptom score" refers to a numerical value that quantifies a person's mental state based on the results of emotion analysis and keyword extraction.

[0198] "Alert" refers to warning information generated when the psychosis symptom score exceeds a certain threshold.

[0199] A "care plan" refers to specific support ideas and measures that should be provided to a user based on an alert.

[0200] A "dialogue script" refers to a dialogue template or recommended message that is generated according to a specific situation.

[0201] "Notification" refers to the process of conveying information from the system to the user.

[0202] "Smart device" refers to an electronic device with advanced technology that is used to collect data about a user.

[0203] A "relaxation room" refers to a dedicated space set up for users to relax.

[0204] "Relaxation techniques" refer to methods and techniques for relieving stress and tension in users.

[0205] A "stress score" refers to a numerical evaluation value that quantifies an employee's level of mental stress.

[0206] The present invention provides a system for monitoring the mental state of employees and customers in a brick-and-mortar store environment, detecting early signs of mental illness, and providing appropriate care. The following describes in detail the embodiments of the invention.

[0207] Program processing

[0208] Data collection

[0209] Users input voice and chat data using smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," the voice is collected and the voice data is sent from the device to the server.

[0210] Data analysis

[0211] The server first preprocesses the collected voice data using noise reduction technology and then converts it into text data. Software used includes, for example, Google Cloud Natural Language API and AWS (registered trademark) Transcribe. Next, natural language processing algorithms are used to analyze the sentiment and extract keywords from the text data. Keywords such as "stress" and "fatigue" are extracted, and the user's emotional state is analyzed.

[0212] Identifying symptoms

[0213] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the score exceeds a certain threshold, an alert is generated. For example, an alert is generated if the "stress score" is high.

[0214] Providing care plans

[0215] Based on the generated alert, the server selects a care plan to provide to the user. In a physical store environment, care plans might include "recommending use of the relaxation room" or "recommending relaxation techniques such as deep breathing." A dialogue script is also generated. For example, a recommended message might be, "You seem tired lately. Are you okay?"

[0216] User Notification

[0217] The selected care plan and dialogue script are notified to the user via the terminal, and the information is provided in a format that is easy for the user to view via a smartphone or smart glasses.

[0218] Technologies used

[0219] This system uses the following hardware and software:

[0220] Google Cloud Natural Language API: Used for sentiment analysis and keyword extraction.

[0221] AWS Transcribe: Converts audio data into text data.

[0222] TensorFlow: Used to build the algorithm that calculates the mental state score.

[0223] Node.js + Express: Used for server-side processing.

[0224] React Native: Used to develop smartphone apps.

[0225] Specific examples

[0226] For example, employee A says through his smart glasses, "Work has been tough lately." This voice data is converted into text data and analyzed on the server. As a result of the analysis, the keyword "stress" is extracted and the stress score is determined to be 80. An alert is generated, and a dialogue script is generated that includes a care plan such as "recommending use of the relaxation room" and "try taking deep breaths." These are then notified to employee A's smart glasses.

[0227] Prompt Sentence Examples

[0228] "Imagine a system that collects employees' daily voice and chat content, analyzes emotions and keywords through natural language processing, detects early signs of mental illness, and presents care plans and dialogue scripts."

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] Users input voice data and chat data through their smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," that voice data is collected. Voice data is input, and the collected voice data is stored on the device as output.

[0232] Step 2:

[0233] The terminal transmits the collected voice data to the server. The input is the collected voice data, and the output is the data transmitted to the server. The voice data is transmitted in an appropriate format using a data communication protocol.

[0234] Step 3:

[0235] The server preprocesses the received audio data using noise reduction technology. Specifically, it removes unnecessary background noise and improves the clarity of the audio. The input is the transmitted audio data, and the output is the audio data with noise removed.

[0236] Step 4:

[0237] The server converts the noise-removed audio data into text using speech recognition technology, such as software like "AWS Transcribe." The input is preprocessed audio data, and the output is converted text data.

[0238] Step 5:

[0239] The server uses a natural language processing algorithm to perform sentiment analysis and keyword extraction on the text data. Specifically, it uses the Google Cloud Natural Language API to analyze the emotional tone of the text and keywords such as "stress" and "fatigue." The input is the text data, and the output is the sentiment analysis results and a set of keywords.

[0240] Step 6:

[0241] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. It uses TensorFlow algorithms to calculate a "stress score" and other similar scores. The input is the sentiment analysis results and a set of keywords, and the output is the calculated mental illness symptom score.

[0242] Step 7:

[0243] The server generates an alert if the score exceeds a certain threshold. The input is the calculated psychosis symptom score, and the output is the generated alert, which includes the score and associated keywords.

[0244] Step 8:

[0245] Based on the generated alert, the server selects a care plan to be provided to the user and generates a specific dialogue script. For example, a script such as "We recommend using the relaxation room" or "Please try taking deep breaths" is generated. The input is the alert information, and the output is the care plan and dialogue script.

[0246] Step 9:

[0247] The terminal notifies the user of the care plan and dialogue script sent from the server. A pop-up notification or message is displayed on the smartphone or smart glasses. The input is the care plan and dialogue script, and the output is a notification to the user.

[0248] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0249] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[0250] System Configuration

[0251] The system consists of the following main components:

[0252] 1. Data collection device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[0253] 2. Server: A central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[0254] 3. Emotion engine: A device that analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[0255] Technologies used

[0256] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[0257] Emotion engine: Using specific algorithms, it analyzes emotions in real time based on user input data.

[0258] Speech recognition technology: Converts voice data into text data.

[0259] Noise reduction technology: Removes unwanted noise from audio data.

[0260] Program processing

[0261] Data collection

[0262] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device sends this data to the server.

[0263] Data analysis

[0264] Server: Passes the received text data to a natural language processing (NLP) module to preprocess the text, specifically tokenizing words, normalizing them, removing stop words, etc.

[0265] sentiment analysis

[0266] Server: The preprocessed text is passed to the sentiment engine for sentiment analysis, for example, to detect sentiment such as "positive," "negative," or "neutral" from the text and extract key keywords.

[0267] Identifying symptoms

[0268] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. Using a machine learning model, it quantifies the user's condition and determines the severity of the symptoms.

[0269] Alert Generation

[0270] Server: If the score exceeds the set threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and extracted keywords.

[0271] Providing care plans

[0272] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[0273] Generate conversation scripts

[0274] Server: Generates a specific dialogue script based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[0275] User Notification

[0276] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[0277] Specific examples

[0278] Detecting and Treating Signs of Depression

[0279] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0280] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[0281] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[0282] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[0283] Terminal: Notifies user A of the care plan and dialogue script.

[0284] Detecting and Treating the Signs of Anxiety Disorders

[0285] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0286] Terminal: Converts voice into text data and sends it to the server.

[0287] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[0288] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[0289] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[0290] Terminal: Notifies user B of the care plan and dialogue script.

[0291] This invention makes it possible to analyze a user's emotions in real time, detect signs of mental illness early, and provide appropriate care, which is expected to help maintain the user's health and improve their quality of life, maintain productivity at work, and prevent the mental illness from worsening.

[0292] The processing flow will be explained below.

[0293] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[0294] Step 1:

[0295] A user inputs text or voice using a chat app or speech recognition app. For example, the user might type, "Work has been really stressful lately," or say, "I'm so nervous about the meeting I'm having and can't sleep."

[0296] Step 2:

[0297] The terminal receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[0298] Step 3:

[0299] The server passes the received text data to a natural language processing (NLP) module, which preprocesses the text, specifically tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[0300] Step 4:

[0301] The server passes the preprocessed text to a sentiment analysis algorithm, which classifies emotions such as positive, negative, or neutral, and extracts key keywords, such as "stressed," "tense," and "unhappy."

[0302] Step 5:

[0303] The server passes the data to the emotion engine based on the results of emotion analysis and keyword extraction, and analyzes the user's emotional state in real time. For example, the emotional state may be determined as "very negative" or "very nervous."

[0304] Step 6:

[0305] The server calculates a mental illness symptom score based on the results of the emotion engine, and uses machine learning models (e.g., SVM, deep learning, etc.) to quantify the user's condition and determine the severity of the symptoms.

[0306] Step 7:

[0307] The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and the extracted keywords.

[0308] Step 8:

[0309] The server selects a care plan to provide to the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific and actionable care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[0310] Step 9:

[0311] The server generates a specific dialogue script based on the analysis results of the emotion engine, such as "You seem tired lately, are you okay?" or "Why don't you try taking some deep breaths to relax before the meeting?"

[0312] Step 10:

[0313] The terminal notifies the user of the care plan and dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[0314] Step 11:

[0315] The user reviews the notification and takes action based on the care plan and dialogue script provided, for example, trying suggested relaxation techniques or booking a consultation.

[0316] In this way, the system analyzes the user's emotional state in real time, detects early signs of mental illness, and provides appropriate care promptly, which is expected to help maintain the user's health and improve their quality of life, maintain productivity in the workplace, and prevent mental illness from worsening.

[0317] Example 2

[0318] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0319] Mental health problems are on the rise in modern society, and it is important to detect signs of mental illness early and provide appropriate care. However, it is difficult for users to accurately recognize their own emotions and mental state and take appropriate action. In addition, receiving professional support takes time and effort, making it difficult to respond quickly. To solve these issues, there is a need for a system that can monitor users' emotions and mental state in real time and provide appropriate care quickly.

[0320] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0321] In this invention, the server includes means for collecting text and voice data entered by a user, means for preprocessing and analyzing the collected data using a natural language processing algorithm to extract emotions and keywords, means for performing emotion analysis on the preprocessed data using an emotion engine, means for calculating a symptom score of mental illness based on the analysis result and generating an alert if a certain threshold is exceeded, means for selecting an appropriate care plan based on the generated alert and generating a specific dialogue script, and means for notifying the user of the selected care plan and dialogue script. This makes it possible to evaluate the user's emotions and mental state in real time, detect symptoms of mental illness early, and provide appropriate care promptly.

[0322] "User" refers to a person who uses the system to input text or voice data.

[0323] "Terminal" refers to a device or interface through which a user inputs text or voice data and transmits that data to a server.

[0324] "Server" refers to the central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[0325] "Data collection means" refers to a method or device for collecting user-entered text or voice data.

[0326] "Natural language processing algorithm" refers to an algorithm for preprocessing and analyzing text data to extract emotions and keywords.

[0327] "Emotion engine" refers to specific algorithms or software for analyzing user emotions from pre-processed data.

[0328] The "psychiatric symptom score" refers to a numerical value that quantifies the user's mental state based on analyzed data.

[0329] "Alert generation means" refers to a method or device for notifying a user of a warning when the psychosis symptom score exceeds a certain threshold.

[0330] The "care plan selection means" refers to a method or device for selecting an appropriate care method or action based on the generated alert information.

[0331] The "interaction script generating means" refers to a method or device for generating specific messages and interaction flows according to the user's emotional and mental state.

[0332] "Notification means" refers to a method or device for notifying the user of the selected care plan and dialogue script.

[0333] "Voice data" refers to voice information uttered by a user.

[0334] "Text data" refers to collected voice data converted into text information.

[0335] "Noise reduction technology" refers to technology for removing unnecessary noise from audio data.

[0336] This system collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. Furthermore, it calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script.

[0337] System Configuration

[0338] The system consists of the following main components:

[0339] 1. Device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[0340] 2. Central control device (server): Analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[0341] 3. Emotion engine: Analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[0342] Hardware and software used

[0343] Hardware:

[0344] 1. Terminal: A device, such as a smartphone or tablet, that allows a user to input text or voice data.

[0345] 2. Server: A central control unit for data analysis, scoring, alert generation, care plan selection, dialogue script generation, and notifications.

[0346] software:

[0347] 1. Natural Language Processing (NLP): Analyze sentiment and keywords from text data using Python's NLTK and SpaCy.

[0348] 2. Emotion Engine: Using TensorFlow and PyTorch, emotions are analyzed in real time based on user input data.

[0349] 3. Speech recognition technology: Uses the Google Speech-to-Text API to convert voice data into text data.

[0350] 4. Noise reduction technology: Use FFmpeg to remove unnecessary noise from audio data.

[0351] Program processing overview

[0352] Data collection

[0353] Device: The user provides data through text chat or voice input. For example, the user might use the chat function to type, "Work has been really stressful lately," or say, "I'm so nervous before the meeting I can't sleep." The device then sends this data to the server.

[0354] Data Preprocessing and Analysis

[0355] Server: The received data is passed to the natural language processing (NLP) module, which preprocesses the data. Specifically, it tokenizes words, normalizes them, and removes stop words. The preprocessed data is then passed to the emotion engine, where it performs sentiment analysis. For example, it detects emotions such as "positive," "negative," and "neutral" from the text and extracts key keywords.

[0356] Psychosis symptom scoring and alert generation

[0357] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. A machine learning model is used to quantify the user's condition and determine the severity of the symptoms. If the score exceeds a set threshold, an alert is generated. For example, the data includes the user ID, type of mental illness, score, and extracted keywords.

[0358] Care plan selection and dialogue script generation

[0359] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendation" or "rest recommendation" for mild depression, and "relaxation techniques" for anxiety disorders. It generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking some deep breaths?"

[0360] User Notification

[0361] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[0362] Specific examples

[0363] Detecting and Treating Signs of Depression

[0364] Device: The user types in chat, "I've been feeling really tired lately and nothing feels fun anymore."

[0365] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[0366] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[0367] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[0368] Terminal: Notifies the user of the care plan and dialogue script.

[0369] Detecting and Treating the Signs of Anxiety Disorders

[0370] Device: The user utters the following through voice input: "I'm so nervous before the meeting that I can't sleep."

[0371] Terminal: Converts voice into text data and sends it to the server.

[0372] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[0373] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[0374] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[0375] Terminal: Notifies the user of the care plan and dialogue script.

[0376] Example prompts for generative AI models

[0377] Below are some example prompts for using our generative AI model:

[0378] "Work has been really stressful lately. I get so nervous before certain meetings that I can't sleep. How can I relax?"

[0379] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0380] Step 1: Data collection

[0381] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device collects this text and voice data and sends it to the server.

[0382] Input: Text or voice data from the user.

[0383] Output: The text or audio data sent to the server.

[0384] Step 2: Convert the audio data

[0385] Server: When voice data is sent, the server uses voice recognition technology (e.g., Google Speech-to-Text API) to convert the voice data into text data.

[0386] Input: Audio data.

[0387] Output: The converted text data.

[0388] Step 3: Noise reduction

[0389] Server: Audio data often contains noise, so use noise reduction techniques such as FFmpeg to remove unwanted noise.

[0390] Input: Audio data.

[0391] Output: Clear audio data with noise removed.

[0392] Step 4: Preprocessing with Natural Language Processing (NLP)

[0393] Server: Preprocesses the received text data. Specifically, it uses Python's NLTK and SpaCy to perform the following steps:

[0394] Word tokenization: Splitting text into words.

[0395] Normalization: Standardizing the format. Example: Converting "daijoubu" to "daijoubu."

[0396] Stop word removal: Remove common words (e.g., "wa" and "ga") that are not necessary for analysis.

[0397] Input: Text data from the user.

[0398] Output: Preprocessed text data.

[0399] Step 5: Sentiment analysis

[0400] Server: Passes the preprocessed text data to an emotion engine (e.g., using TensorFlow or PyTorch) for sentiment analysis. Specifically, it performs the following processes:

[0401] Sentiment classification: Detecting emotions such as "positive," "negative," and "neutral" from text.

[0402] Keyword extraction: Extract key keywords.

[0403] Input: Preprocessed text data.

[0404] Output: Sentiment label (e.g. "negative") and key keywords.

[0405] Step 6: Scoring psychotic symptoms

[0406] Server: Calculates a numerical score of the symptoms of mental illness based on the emotion labels and keywords obtained from the emotion engine. It evaluates the user's condition using a scoring algorithm or machine learning model.

[0407] Input: emotion labels and keywords.

[0408] Output: Psychotic symptom score.

[0409] Step 7: Alert Generation

[0410] Server: Generate an alert if the psychosis symptom score exceeds a set threshold. The alert will include the following information:

[0411] User ID: The user for whom the alert is intended.

[0412] Type of mental illness: For example, "depression" or "anxiety."

[0413] Psychotic symptom score.

[0414] Extracted keywords.

[0415] Input: Psychotic symptom score and keywords.

[0416] Output: Alert information.

[0417] Step 8: Selecting a Care Plan

[0418] Server: Selects an appropriate care plan based on the generated alert information. Specifically, it proposes care methods according to the user's condition. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[0419] Input: Alert information.

[0420] Output: Care plan.

[0421] Step 9: Generate the interaction script

[0422] Server: Generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[0423] Input: Analysis results of the emotion engine.

[0424] Output: Dialogue script.

[0425] Step 10: Notify users

[0426] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[0427] Input: Care plan and dialogue script.

[0428] Output: A notification message to the user.

[0429] (Application example 2)

[0430] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0431] In modern society, it is important to monitor people's mental and emotional states in real time, detect abnormalities early, and take appropriate measures. In particular, rapid response is required when signs of mental illness are observed. There is also a need for security devices to quickly detect dangerous situations and abnormal mental states and provide appropriate countermeasures. However, current systems cannot fully meet these requirements, making effective monitoring and response difficult.

[0432] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting chat and voice data entered by the user; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for detecting danger or abnormal conditions in real time using voice recognition, natural language processing, and sentiment analysis technologies in the security device; and means for generating an appropriate response and dialogue script based on the detected abnormality and notifying the user or relevant parties from the security device. This enables early detection of changes in the user's mental state or security abnormalities and enables prompt and appropriate response.

[0433] "Chat data" refers to data that indicates the contents of conversations entered by users in text format.

[0434] "Voice data" refers to data that indicates the contents of a conversation input by a user in voice form.

[0435] A "natural language processing algorithm" is an algorithm that analyzes text data and extracts emotions and keywords from it.

[0436] "Sentiment analysis" is the process of analyzing a user's emotional state (e.g., positive, negative, neutral, etc.) from input text data or voice data.

[0437] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[0438] The "psychiatric symptom score" is a numerical value that quantifies the abnormality of the user's mental state based on the results of emotion analysis and keyword extraction.

[0439] An "alert" is a warning message that is generated when a psychosis symptom score exceeds a certain threshold.

[0440] A "care plan" is a plan that includes measures and suggestions to be provided depending on the user's mental state.

[0441] A "dialogue script" is a specific set of messages or questions that are generated to interact with a user.

[0442] A "security device" is a device equipped with voice recognition, natural language processing, and emotion analysis technologies that has the ability to detect danger or abnormal conditions in real time.

[0443] "Speech recognition" is a technology that converts voice data into text data.

[0444] "Real-time" is a temporal concept that refers to the instantaneous processing and updating of data.

[0445] "Notification means" refers to a method or system for communicating alerts, care plans, and dialogue scripts to users and related parties.

[0446] This system collects data entered by users via chat or voice, performs sentiment analysis and keyword extraction, detects signs of mental illness in real time, and provides appropriate care plans.Furthermore, by applying it to security devices, it can quickly detect danger or abnormal conditions and take countermeasures.

[0447] Components

[0448] 1. Data collection device (terminal)

[0449] This is an interface for users to chat and input voice. This device can be a PC, smartphone, tablet, etc. It collects data from users and sends it to a server.

[0450] 2. Server

[0451] The server is a central control unit that analyzes the received data, performs sentiment analysis, keyword extraction, calculation of mental illness symptom scores, generation of alerts, selection of care plans, generation of dialogue scripts, and notification. This server is built using a computer with high-performance processing capabilities or a cloud service (e.g., Amazon Web Services or Microsoft® Azure®).

[0452] Processing content

[0453] Voice Recognition

[0454] On the device, voice data input by the user is converted into text data using voice recognition technology. At this time, highly accurate voice recognition is achieved by using the speech_recognition library, etc.

[0455] Data Preprocessing

[0456] The server preprocesses the received text data using natural language processing algorithms (e.g., Hugging Face's transformers library), including tokenization, normalization, and stop-word removal.

[0457] Sentiment analysis and keyword extraction

[0458] The preprocessed data is passed to an emotion engine for sentiment analysis and keyword extraction. The emotion engine (e.g., Hugging Face sentiment-analysis model) detects positive, negative, and neutral emotions from the input data and extracts key keywords.

[0459] Calculation of psychotic symptom scores

[0460] The server calculates a psychosis symptom score based on the sentiment analysis results and keywords, using machine learning models (e.g., scaling algorithms and averaging) to quantify the user's state.

[0461] Generate alerts

[0462] If the score exceeds a certain threshold, the server generates an alert, which includes the user ID, the type of mental illness, the score, and the extracted keywords.

[0463] Care plan selection and dialogue script generation

[0464] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, it may suggest "exercise recommendations" and "rest recommendations" for mild depression, or "relaxation techniques" for anxiety disorders.

[0465] User Notification

[0466] The selected care plan and dialogue script are sent to the terminal and notified to the user. The notification is in the form of a pop-up or message so that the user can easily check it.

[0467] Security equipment applications

[0468] Real-time detection of dangers and abnormal conditions

[0469] The system, installed in the security device, analyzes surrounding sounds using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time, and if certain thresholds are exceeded, an alert and dialogue script is generated to notify the user immediately.

[0470] Specific examples

[0471] 1. Detecting and Treating Signs of Depression

[0472] User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0473] The server analyzes the keywords "very negative," "tired," and "not having fun."

[0474] Because the depression risk score is high, an alert is generated and care plans include "recommended exercise" and "appointment with a psychiatrist."

[0475] The server generates a dialogue script saying, "You seem tired lately. Do you need any help?" and sends it to the terminal.

[0476] The terminal notifies user A of the care plan and dialogue script.

[0477] 2. Detecting and Treating Symptoms of Anxiety Disorders

[0478] User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0479] The voice is converted into text data and sent to the server.

[0480] The server analyzes the keywords "very anxious," "tense," and "can't sleep."

[0481] Because the anxiety disorder risk score is high, an alert is generated and a care plan is suggested that includes relaxation techniques and a meeting with a specialist.

[0482] The server generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[0483] The device notifies user B of the care plan and dialogue script.

[0484] 3. Examples of danger detection by security devices

[0485] Detects nearby voices such as "Help, I'm scared!"

[0486] The voice is converted into text data and sent to the server.

[0487] The server analyzes the keywords "very negative," "help," and "scary."

[0488] Because the security risk score is high, an alert is generated and a care plan is suggested stating that "consultation with an expert is required."

[0489] The server generates a dialogue script "Do you need support?" and notifies the relevant person via the security device.

[0490] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0491] Step 1: User enters chat or voice input

[0492] The user uses a device (smartphone, PC, tablet) to input text or speak in chat. The input chat data and voice data are temporarily stored on the device. In this step, the user inputs any conversation content into the system. Input is done by text or voice.

[0493] Step 2: Convert the audio data to text

[0494] The device converts the voice input data into text data using the speech_recognition library. Specifically, the voice recognition algorithm analyzes the voice data and converts it into a corresponding string of characters. The input is voice data, and the output is the corresponding text data.

[0495] Step 3: Preprocess the data

[0496] The server receives the text data sent from the device and preprocesses it using a natural language processing algorithm (e.g., Hugging Face's transformers library). Specific preprocessing steps include tokenization, normalization, and stop word removal. The input is text data, and the output is the preprocessed text data.

[0497] Step 4: Sentiment analysis and keyword extraction

[0498] The server passes the preprocessed text data to an emotion engine (e.g., Hugging Face sentiment-analysis model) for sentiment analysis and keyword extraction. Specifically, the algorithm extracts emotional states (e.g., positive, negative, neutral) and key keywords from the text data. The input is the preprocessed text data, and the output is the sentiment analysis results and extracted keywords.

[0499] Step 5: Calculate the psychosis symptom score

[0500] The server calculates a mental illness symptom score based on the sentiment analysis results and extracted keywords. It uses a machine learning model to normalize the scores for each emotion and calculate an overall symptom score. The input is the sentiment analysis results and keywords, and the output is the mental illness symptom score.

[0501] Step 6: Generate an alert

[0502] The server generates an alert when the psychosis symptom score exceeds a certain threshold. The alert includes the user ID, the type of psychosis, the score, and the extracted keywords. The input is the psychosis symptom score, and the output is the generated alert.

[0503] Step 7: Select a care plan and generate a dialogue script

[0504] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, recommendations for exercise and rest are made for mild depression, and relaxation techniques are suggested for anxiety disorders. The input is the generated alert, and the output is the care plan and dialogue script.

[0505] Step 8: Inform the user of the care plan and dialogue script

[0506] The terminal receives the care plan and dialogue script sent from the server and notifies the user. The notification is in the form of a pop-up or message so that the user can easily check it. The input is the care plan and dialogue script, and the output is the notification to the user.

[0507] Step 9: Detect and respond to abnormalities using security devices

[0508] The server uses a system installed in the security device to analyze surrounding audio using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time. At this stage, an alert and dialogue script are generated if a certain threshold is exceeded. The input is surrounding audio data, and the output is the generated alert and dialogue script.

[0509] Step 10: Security Device Notification

[0510] The security device immediately notifies relevant parties and users based on the generated alert and dialogue script. This notification is made via the security device's display and audio output. The input is the generated alert and dialogue script, and the output is the notification to relevant parties.

[0511] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0512] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0513] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0514] [Second embodiment]

[0515] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0516] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0517] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0518] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0519] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0520] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0521] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0522] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0523] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0524] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0525] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0526] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0527] The present invention provides a system for detecting early signs of mental illness in a user and providing appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat or voice. Below, an embodiment of the present invention will be described with specific examples.

[0528] System Overview

[0529] The system collects data entered by users via chat or voice and analyzes it using natural language processing algorithms. The analysis involves sentiment analysis and keyword extraction, and calculates a mental illness symptom score. If the score exceeds a certain level, an alert is generated and the user is provided with an appropriate care plan and dialogue script.

[0530] Technologies used

[0531] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[0532] Speech recognition technology: Converts voice data into text data.

[0533] Noise reduction technology: Removes unwanted noise from audio data.

[0534] Program processing

[0535] Data collection

[0536] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[0537] Data analysis

[0538] Server: The received text data is passed to a natural language processing module for sentiment analysis and keyword extraction. For example, keywords such as "stress," "tension," and "can't sleep" are extracted from the text data to analyze the speaker's emotions.

[0539] Identifying symptoms

[0540] Server: Calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the calculated score exceeds a certain threshold, an alert is generated. For example, a specific numerical value such as "Depression risk score: 85" or "Anxiety disorder risk score: 90" is displayed.

[0541] Alert Generation

[0542] Server: The generated alert includes the user ID, type of mental illness, score, and specific keywords. This information can be saved in a database and notified to an administrator.

[0543] Providing care plans

[0544] Server: Based on the alert information, the server selects a care plan to provide to the user. For example, it provides specific and actionable plans such as "recommending regular exercise" and "how to schedule counseling appointments."

[0545] Conversation script recommendations

[0546] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. For example, for depression, it generates a message like "You seem tired lately, are you okay?", and for anxiety disorders, it generates a message like "Why don't you try taking a deep breath?"

[0547] User Notification

[0548] Terminal: The care plan and dialogue script are notified to the user. They are displayed in a format that is easy for the user to check, and are provided as pop-ups or notification messages as necessary.

[0549] Specific examples

[0550] Detecting and Treating Signs of Depression

[0551] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0552] Server: Sentiment analysis detects negative emotions and keywords such as "tired" and "not happy."

[0553] Server: Generate alert due to high depression risk score.

[0554] Server: Provides care plans such as "exercise recommendations" and "appointment with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[0555] Terminal: Notify User A of this.

[0556] Detecting and Treating Anxiety Disorder Signs

[0557] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0558] Terminal: Converts voice into text data and sends it to the server.

[0559] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[0560] Server: High score generates an alert.

[0561] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[0562] Terminal: Notify User B of this.

[0563] The present invention enables early detection of signs of mental illness and prompt provision of appropriate care, thereby maintaining the user's health and improving their quality of life, which is expected to help maintain productivity in the workplace and prevent the worsening of mental illness.

[0564] The processing flow will be explained below.

[0565] Step 1: The user inputs text or voice using a chat app or speech recognition app. For example, the user might input text such as "Work has been really stressful lately," or say something like "I'm so nervous about the meeting I'm having trouble sleeping."

[0566] Step 2: The device receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[0567] Step 3: The server passes the received text data to a natural language processing (NLP) module for text preprocessing, such as tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[0568] Step 4: The server passes the preprocessed text to a sentiment analysis algorithm to classify emotions such as positive, negative, or neutral, and extract key keywords, such as "stressed," "tense," and "unhappy."

[0569] Step 5: The server calculates a mental illness symptom score based on the results of the sentiment analysis and keyword extraction. The algorithm uses a machine learning model (e.g., SVM, deep learning, etc.) to quantify the severity of the symptoms.

[0570] Step 6: The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and specific keywords.

[0571] Step 7: The server selects a care plan for the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[0572] Step 8: The server generates an appropriate dialogue script depending on the type of mental illness detected, such as "You seem tired lately, are you okay?" or "Why don't you try some deep breathing to relax before the meeting?"

[0573] Step 9: The terminal notifies the user of the care plan and the dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[0574] Step 10: The user reviews the notification and takes action based on the provided care plan or dialogue script, for example, trying suggested relaxation techniques or booking a consultation.

[0575] In this way, the system can detect the user's mental state early and provide appropriate care promptly.

[0576] Example 1

[0577] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] While the importance of mental health management has increased in recent years, it often takes time for users to become aware of their own mental state and receive appropriate care. In particular, overlooking early signs of mental illness carries the risk of worsening symptoms. Conventional methods assume that users will consult with a specialist themselves, and lack a means to automatically and in real time monitor the user's mental state and provide appropriate care. The challenge is to address this shortcoming.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0580] In this invention, the server includes means for collecting text data and voice data input by the user, means for preprocessing the collected data using noise reduction technology and converting the voice data into text data, and means for analyzing the converted text data using a natural language processing algorithm to perform sentiment analysis and keyword extraction, thereby enabling automatic and real-time monitoring of the user's mental state and instantly providing an appropriate care plan and dialogue script.

[0581] A "user" refers to an individual who uses the system and inputs data via text or voice.

[0582] "Text data" refers to information entered by a user in text form through a chat app.

[0583] "Voice data" refers to voice information spoken by a user, including recorded or real-time sound data.

[0584] "Noise reduction technology" refers to technology that removes unnecessary background sounds and noise from audio data.

[0585] "Preprocessing" refers to a series of steps to prepare data for easier analysis.

[0586] "Natural language processing algorithms" refer to technologies for analyzing text data, and are algorithms that perform sentiment analysis and keyword extraction.

[0587] "Sentiment analysis" refers to the technology of analyzing user emotions from text data.

[0588] "Keyword extraction" refers to the technique of extracting important words and phrases from text data.

[0589] The "symptom score" refers to a numerical assessment of a user's mental state, indicating their risk of mental illness.

[0590] An "alert" refers to warning information that is generated when a user's situation exceeds a certain threshold.

[0591] "Support plan" refers to a care plan provided to the user by the system, which includes specific measures and advice.

[0592] A "dialogue script" refers to a set phrase that is generated to facilitate communication with a user.

[0593] "Notification" refers to a means of conveying information to the user through the device, and is done in the form of a pop-up or push notification.

[0594] MODE FOR CARRYING OUT THE INVENTION

[0595] The present invention is a system that detects early signs of mental illness in users and provides appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat and voice.

[0596] Hardware and Software Used

[0597] 1. Terminal: A device such as a smartphone or computer used by a user.

[0598] 2. Server: A central server for data analysis. No specific hardware is required, but a computer with powerful processing power is recommended.

[0599] 3. Software:

[0600] Natural language processing (NLP) libraries: Python's "NLTK" and "spaCy".

[0601] Speech recognition technology: "Google Cloud Speech-to-Text" API for converting voice data into text data.

[0602] Noise reduction techniques: "LibROSA" and "PyDub" for preprocessing audio data.

[0603] Database: MySQL or MongoDB for storing alerts and care plans.

[0604] Generative AI models: "Scikit-learn" and "TensorFlow" for machine learning.

[0605] Program processing explanation

[0606] Data collection

[0607] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[0608] Data Preprocessing

[0609] Server: Analyzes the received voice data using noise reduction technology to remove unwanted background sounds and noise, then converts the voice data into text using speech recognition technology.

[0610] Data analysis

[0611] Server: The converted text data is passed to a natural language processing algorithm for sentiment analysis and keyword extraction.

[0612] Identifying symptoms

[0613] Server: Calculates a psychosis symptom score based on the analysis results. If the score exceeds a certain threshold, an alert is generated.

[0614] Alert Generation

[0615] Server: The generated alert contains the user ID, type of mental illness, score, and specific keywords. This information is stored in a database and notified to the administrator.

[0616] Providing care plans

[0617] Server: Based on the alert information, selects a care plan to provide to the user. Recommendations for the care plan include "regular exercise" and "how to schedule counseling appointments."

[0618] Conversation script recommendations

[0619] Server: Generates an appropriate dialogue script depending on the type of mental illness detected.

[0620] User Notification

[0621] On the device: The care plan and dialogue script are notified to the user as a pop-up or push notification.

[0622] Specific examples

[0623] Detecting and Treating Signs of Depression

[0624] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0625] Server: Performs sentiment analysis on the received text data to detect negative emotions and keywords such as "tired" and "not having fun."

[0626] Server: Generate alert due to high depression risk score.

[0627] Server: Provides care plans such as "regular exercise" and "appointments with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[0628] Terminal: Notify User A of this.

[0629] Detecting and Treating Anxiety Disorder Signs

[0630] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0631] Terminal: Converts voice into text data and sends it to the server.

[0632] Server: The audio data is preprocessed using noise reduction technology and converted into text data using Google Cloud Speech-to-Text.

[0633] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[0634] Server: High score generates an alert.

[0635] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[0636] Terminal: Notify User B of this.

[0637] Prompt Sentence Examples

[0638] 1. "Analyze what users have been talking about in chat about their work recently and tell me if they are showing signs of stress."

[0639] 2. "If a user says, 'Work has been really stressful lately,' what would you do to address that?"

[0640] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0641] Step 1: Data collection

[0642] Device: A user uses a chat app to type, "Work has been really stressful lately," or to say, "I'm so nervous before the meeting I can't sleep," as voice data. This data is collected on the device and sent to the server.

[0643] Input: User text or voice data.

[0644] Output: The data packet to be sent to the server.

[0645] Step 2: Data Preprocessing

[0646] Server: The received audio data is first preprocessed using noise reduction technology (e.g., LibROSA, PyDub) to remove unwanted background sounds and noise, and then converted into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[0647] Input: Audio data sent from the device.

[0648] Output: Text data.

[0649] Step 3: Data analysis

[0650] Server: The converted text data is passed to a natural language processing (NLP) algorithm (e.g., NLTK, spaCy) for sentiment analysis and keyword extraction. Based on the analysis results, keywords such as "stress," "tension," and "can't sleep" are extracted, and the emotional state is analyzed.

[0651] Input: Text data converted by speech recognition technology.

[0652] Output: Sentiment analysis results and extracted keywords.

[0653] Step 4: Identify symptoms

[0654] Server: Based on the results of sentiment analysis and keyword extraction, a generative AI model (e.g., Scikit-learn, TensorFlow) is used to calculate a mental illness symptom score. If the score exceeds a certain threshold, it is determined that an abnormality has been detected and an alert is generated.

[0655] Input: Sentiment analysis results and extracted keywords.

[0656] Output: Psychosis symptom score and whether an alert was generated.

[0657] Step 5: Alert Generation

[0658] Server: If the score exceeds the threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and specific keywords. The generated alert information is stored in a database (e.g., MySQL, MongoDB) and notified to the administrator.

[0659] Input: Psychosis symptom score.

[0660] Output: Alert information.

[0661] Step 6: Provide a care plan

[0662] Server: Based on the alert information, the server selects the care plan to be provided to the user. For example, it suggests "regular exercise recommendations" or "how to schedule counseling appointments." This information is also stored in the database.

[0663] Input: Alert information.

[0664] Output: Care plan.

[0665] Step 7: Recommend conversation scripts

[0666] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. Using a generative AI model, it generates messages such as "You seem tired lately, are you okay?" for depression, or "Why don't you try taking a deep breath?" for anxiety disorders.

[0667] Input: Care plan.

[0668] Output: Dialogue script.

[0669] Step 8: Notify users

[0670] Device: The care plan and dialogue script are notified in a format that is easy for the user to view. Specifically, this is displayed as a pop-up notification or push notification in the chat app.

[0671] Input: Care plan and dialogue script.

[0672] Output: User notification.

[0673] (Application example 1)

[0674] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] Mental stress among employees and customers leads to reduced productivity and customer satisfaction, so it is necessary to detect mental anxiety early and provide appropriate care even in physical store environments. However, the technology to do this efficiently and in real time is currently immature, so there is a need for a system that can appropriately monitor the mental state of employees and customers and take prompt measures.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0677] In this invention, the server includes: means for collecting chat and voice data entered by users; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for using a smart device to monitor the mental state of employees and customers; means for providing a care plan recommending the use of a relaxation room in a store environment; means for generating a dialogue script recommending relaxation techniques such as deep breathing; and means for calculating the stress score of employees using the collected data. This makes it possible to quickly detect changes in mental state even in a physical store environment and provide appropriate care.

[0678] "User" refers to an individual or end user who uses the System.

[0679] "Chat data" refers to conversation data entered by a user in text format.

[0680] "Voice data" refers to data collected by recording voices uttered by users.

[0681] "Natural language processing algorithm" refers to a technical system for analyzing human language using a computer.

[0682] "Sentiment analysis" is a technique for determining a user's emotional state from text or voice data.

[0683] "Keyword extraction" is a technique for identifying important words and phrases from text or audio data.

[0684] The "psychiatric symptom score" refers to a numerical value that quantifies a person's mental state based on the results of emotion analysis and keyword extraction.

[0685] "Alert" refers to warning information generated when the psychosis symptom score exceeds a certain threshold.

[0686] A "care plan" refers to specific support ideas and measures that should be provided to a user based on an alert.

[0687] A "dialogue script" refers to a dialogue template or recommended message that is generated according to a specific situation.

[0688] "Notification" refers to the process of conveying information from the system to the user.

[0689] "Smart device" refers to an electronic device with advanced technology that is used to collect data about a user.

[0690] A "relaxation room" refers to a dedicated space set up for users to relax.

[0691] "Relaxation techniques" refer to methods and techniques for relieving stress and tension in users.

[0692] A "stress score" refers to a numerical evaluation value that quantifies an employee's level of mental stress.

[0693] The present invention provides a system for monitoring the mental state of employees and customers in a brick-and-mortar store environment, detecting early signs of mental illness, and providing appropriate care. The following describes in detail the embodiments of the invention.

[0694] Program processing

[0695] Data collection

[0696] Users input voice and chat data using smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," the voice is collected and the voice data is sent from the device to the server.

[0697] Data analysis

[0698] The server first preprocesses the collected voice data using noise reduction technology and then converts it into text data. Software such as Google Cloud Natural Language API and AWS Transcribe are used. Next, natural language processing algorithms are used to analyze the sentiment and extract keywords from the text data. Keywords such as "stress" and "fatigue" are extracted, and the user's emotional state is analyzed.

[0699] Identifying symptoms

[0700] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the score exceeds a certain threshold, an alert is generated. For example, an alert is generated if the "stress score" is high.

[0701] Providing care plans

[0702] Based on the generated alert, the server selects a care plan to provide to the user. In a physical store environment, care plans might include "recommending use of the relaxation room" or "recommending relaxation techniques such as deep breathing." A dialogue script is also generated. For example, a recommended message might be, "You seem tired lately. Are you okay?"

[0703] User Notification

[0704] The selected care plan and dialogue script are notified to the user via the terminal, and the information is provided in a format that is easy for the user to view via a smartphone or smart glasses.

[0705] Technologies used

[0706] This system uses the following hardware and software:

[0707] Google Cloud Natural Language API: Used for sentiment analysis and keyword extraction.

[0708] AWS Transcribe: Converts audio data into text data.

[0709] TensorFlow: Used to build the algorithm that calculates the mental state score.

[0710] Node.js + Express: Used for server-side processing.

[0711] React Native: Used to develop smartphone apps.

[0712] Specific examples

[0713] For example, employee A says through his smart glasses, "Work has been tough lately." This voice data is converted into text data and analyzed on the server. As a result of the analysis, the keyword "stress" is extracted and the stress score is determined to be 80. An alert is generated, and a dialogue script is generated that includes a care plan such as "recommending use of the relaxation room" and "try taking deep breaths." These are then notified to employee A's smart glasses.

[0714] Prompt Sentence Examples

[0715] "Imagine a system that collects employees' daily voice and chat content, analyzes emotions and keywords through natural language processing, detects early signs of mental illness, and presents care plans and dialogue scripts."

[0716] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0717] Step 1:

[0718] Users input voice data and chat data through their smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," that voice data is collected. Voice data is input, and the collected voice data is stored on the device as output.

[0719] Step 2:

[0720] The terminal transmits the collected voice data to the server. The input is the collected voice data, and the output is the data transmitted to the server. The voice data is transmitted in an appropriate format using a data communication protocol.

[0721] Step 3:

[0722] The server preprocesses the received audio data using noise reduction technology. Specifically, it removes unnecessary background noise and improves the clarity of the audio. The input is the transmitted audio data, and the output is the audio data with noise removed.

[0723] Step 4:

[0724] The server converts the noise-removed audio data into text using speech recognition technology, such as software like "AWS Transcribe." The input is preprocessed audio data, and the output is converted text data.

[0725] Step 5:

[0726] The server uses a natural language processing algorithm to perform sentiment analysis and keyword extraction on the text data. Specifically, it uses the Google Cloud Natural Language API to analyze the emotional tone of the text and keywords such as "stress" and "fatigue." The input is the text data, and the output is the sentiment analysis results and a set of keywords.

[0727] Step 6:

[0728] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. It uses TensorFlow algorithms to calculate a "stress score" and other similar scores. The input is the sentiment analysis results and a set of keywords, and the output is the calculated mental illness symptom score.

[0729] Step 7:

[0730] The server generates an alert if the score exceeds a certain threshold. The input is the calculated psychosis symptom score, and the output is the generated alert, which includes the score and associated keywords.

[0731] Step 8:

[0732] Based on the generated alert, the server selects a care plan to be provided to the user and generates a specific dialogue script. For example, a script such as "We recommend using the relaxation room" or "Please try taking deep breaths" is generated. The input is the alert information, and the output is the care plan and dialogue script.

[0733] Step 9:

[0734] The terminal notifies the user of the care plan and dialogue script sent from the server. A pop-up notification or message is displayed on the smartphone or smart glasses. The input is the care plan and dialogue script, and the output is a notification to the user.

[0735] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0736] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[0737] System Configuration

[0738] The system consists of the following main components:

[0739] 1. Data collection device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[0740] 2. Server: A central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[0741] 3. Emotion engine: A device that analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[0742] Technologies used

[0743] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[0744] Emotion engine: Using specific algorithms, it analyzes emotions in real time based on user input data.

[0745] Speech recognition technology: Converts voice data into text data.

[0746] Noise reduction technology: Removes unwanted noise from audio data.

[0747] Program processing

[0748] Data collection

[0749] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device sends this data to the server.

[0750] Data analysis

[0751] Server: Passes the received text data to a natural language processing (NLP) module to preprocess the text, specifically tokenizing words, normalizing them, removing stop words, etc.

[0752] sentiment analysis

[0753] Server: The preprocessed text is passed to the sentiment engine for sentiment analysis, for example, to detect sentiment such as "positive," "negative," or "neutral" from the text and extract key keywords.

[0754] Identifying symptoms

[0755] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. Using a machine learning model, it quantifies the user's condition and determines the severity of the symptoms.

[0756] Alert Generation

[0757] Server: If the score exceeds the set threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and extracted keywords.

[0758] Providing care plans

[0759] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[0760] Generate conversation scripts

[0761] Server: Generates a specific dialogue script based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[0762] User Notification

[0763] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[0764] Specific examples

[0765] Detecting and Treating Signs of Depression

[0766] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0767] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[0768] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[0769] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[0770] Terminal: Notifies user A of the care plan and dialogue script.

[0771] Detecting and Treating the Signs of Anxiety Disorders

[0772] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0773] Terminal: Converts voice into text data and sends it to the server.

[0774] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[0775] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[0776] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[0777] Terminal: Notifies user B of the care plan and dialogue script.

[0778] This invention makes it possible to analyze a user's emotions in real time, detect signs of mental illness early, and provide appropriate care, which is expected to help maintain the user's health and improve their quality of life, maintain productivity at work, and prevent the mental illness from worsening.

[0779] The processing flow will be explained below.

[0780] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[0781] Step 1:

[0782] A user inputs text or voice using a chat app or speech recognition app. For example, the user might type, "Work has been really stressful lately," or say, "I'm so nervous about the meeting I'm having and can't sleep."

[0783] Step 2:

[0784] The terminal receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[0785] Step 3:

[0786] The server passes the received text data to a natural language processing (NLP) module, which preprocesses the text, specifically tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[0787] Step 4:

[0788] The server passes the preprocessed text to a sentiment analysis algorithm, which classifies emotions such as positive, negative, or neutral, and extracts key keywords, such as "stressed," "tense," and "unhappy."

[0789] Step 5:

[0790] The server passes the data to the emotion engine based on the results of emotion analysis and keyword extraction, and analyzes the user's emotional state in real time. For example, the emotional state may be determined as "very negative" or "very nervous."

[0791] Step 6:

[0792] The server calculates a mental illness symptom score based on the results of the emotion engine, and uses machine learning models (e.g., SVM, deep learning, etc.) to quantify the user's condition and determine the severity of the symptoms.

[0793] Step 7:

[0794] The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and the extracted keywords.

[0795] Step 8:

[0796] The server selects a care plan to provide to the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific and actionable care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[0797] Step 9:

[0798] The server generates a specific dialogue script based on the analysis results of the emotion engine, such as "You seem tired lately, are you okay?" or "Why don't you try taking some deep breaths to relax before the meeting?"

[0799] Step 10:

[0800] The terminal notifies the user of the care plan and dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[0801] Step 11:

[0802] The user reviews the notification and takes action based on the care plan and dialogue script provided, for example, trying suggested relaxation techniques or booking a consultation.

[0803] In this way, the system analyzes the user's emotional state in real time, detects early signs of mental illness, and provides appropriate care promptly, which is expected to help maintain the user's health and improve their quality of life, maintain productivity in the workplace, and prevent mental illness from worsening.

[0804] Example 2

[0805] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0806] Mental health problems are on the rise in modern society, and it is important to detect signs of mental illness early and provide appropriate care. However, it is difficult for users to accurately recognize their own emotions and mental state and take appropriate action. In addition, receiving professional support takes time and effort, making it difficult to respond quickly. To solve these issues, there is a need for a system that can monitor users' emotions and mental state in real time and provide appropriate care quickly.

[0807] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0808] In this invention, the server includes means for collecting text and voice data entered by a user, means for preprocessing and analyzing the collected data using a natural language processing algorithm to extract emotions and keywords, means for performing emotion analysis on the preprocessed data using an emotion engine, means for calculating a symptom score of mental illness based on the analysis result and generating an alert if a certain threshold is exceeded, means for selecting an appropriate care plan based on the generated alert and generating a specific dialogue script, and means for notifying the user of the selected care plan and dialogue script. This makes it possible to evaluate the user's emotions and mental state in real time, detect symptoms of mental illness early, and provide appropriate care promptly.

[0809] "User" refers to a person who uses the system to input text or voice data.

[0810] "Terminal" refers to a device or interface through which a user inputs text or voice data and transmits that data to a server.

[0811] "Server" refers to the central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[0812] "Data collection means" refers to a method or device for collecting user-entered text or voice data.

[0813] "Natural language processing algorithm" refers to an algorithm for preprocessing and analyzing text data to extract emotions and keywords.

[0814] "Emotion engine" refers to specific algorithms or software for analyzing user emotions from pre-processed data.

[0815] The "psychiatric symptom score" refers to a numerical value that quantifies the user's mental state based on analyzed data.

[0816] "Alert generation means" refers to a method or device for notifying a user of a warning when the psychosis symptom score exceeds a certain threshold.

[0817] The "care plan selection means" refers to a method or device for selecting an appropriate care method or action based on the generated alert information.

[0818] The "interaction script generating means" refers to a method or device for generating specific messages and interaction flows according to the user's emotional and mental state.

[0819] "Notification means" refers to a method or device for notifying the user of the selected care plan and dialogue script.

[0820] "Voice data" refers to voice information uttered by a user.

[0821] "Text data" refers to collected voice data converted into text information.

[0822] "Noise reduction technology" refers to technology for removing unnecessary noise from audio data.

[0823] This system collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. Furthermore, it calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script.

[0824] System Configuration

[0825] The system consists of the following main components:

[0826] 1. Device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[0827] 2. Central control device (server): Analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[0828] 3. Emotion engine: Analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[0829] Hardware and software used

[0830] Hardware:

[0831] 1. Terminal: A device, such as a smartphone or tablet, that allows a user to input text or voice data.

[0832] 2. Server: A central control unit for data analysis, scoring, alert generation, care plan selection, dialogue script generation, and notifications.

[0833] software:

[0834] 1. Natural Language Processing (NLP): Analyze sentiment and keywords from text data using Python's NLTK and SpaCy.

[0835] 2. Emotion Engine: Using TensorFlow and PyTorch, emotions are analyzed in real time based on user input data.

[0836] 3. Speech recognition technology: Uses the Google Speech-to-Text API to convert voice data into text data.

[0837] 4. Noise reduction technology: Use FFmpeg to remove unnecessary noise from audio data.

[0838] Program processing overview

[0839] Data collection

[0840] Device: The user provides data through text chat or voice input. For example, the user might use the chat function to type, "Work has been really stressful lately," or say, "I'm so nervous before the meeting I can't sleep." The device then sends this data to the server.

[0841] Data Preprocessing and Analysis

[0842] Server: The received data is passed to the natural language processing (NLP) module, which preprocesses the data. Specifically, it tokenizes words, normalizes them, and removes stop words. The preprocessed data is then passed to the emotion engine, where it performs sentiment analysis. For example, it detects emotions such as "positive," "negative," and "neutral" from the text and extracts key keywords.

[0843] Psychosis symptom scoring and alert generation

[0844] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. A machine learning model is used to quantify the user's condition and determine the severity of the symptoms. If the score exceeds a set threshold, an alert is generated. For example, the data includes the user ID, type of mental illness, score, and extracted keywords.

[0845] Care plan selection and dialogue script generation

[0846] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendation" or "rest recommendation" for mild depression, and "relaxation techniques" for anxiety disorders. It generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking some deep breaths?"

[0847] User Notification

[0848] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[0849] Specific examples

[0850] Detecting and Treating Signs of Depression

[0851] Device: The user types in chat, "I've been feeling really tired lately and nothing feels fun anymore."

[0852] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[0853] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[0854] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[0855] Terminal: Notifies the user of the care plan and dialogue script.

[0856] Detecting and Treating the Signs of Anxiety Disorders

[0857] Device: The user utters the following through voice input: "I'm so nervous before the meeting that I can't sleep."

[0858] Terminal: Converts voice into text data and sends it to the server.

[0859] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[0860] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[0861] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[0862] Terminal: Notifies the user of the care plan and dialogue script.

[0863] Example prompts for generative AI models

[0864] Below are some example prompts for using our generative AI model:

[0865] "Work has been really stressful lately. I get so nervous before certain meetings that I can't sleep. How can I relax?"

[0866] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0867] Step 1: Data collection

[0868] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device collects this text and voice data and sends it to the server.

[0869] Input: Text or voice data from the user.

[0870] Output: The text or audio data sent to the server.

[0871] Step 2: Convert the audio data

[0872] Server: When voice data is sent, the server uses voice recognition technology (e.g., Google Speech-to-Text API) to convert the voice data into text data.

[0873] Input: Audio data.

[0874] Output: The converted text data.

[0875] Step 3: Noise reduction

[0876] Server: Audio data often contains noise, so use noise reduction techniques such as FFmpeg to remove unwanted noise.

[0877] Input: Audio data.

[0878] Output: Clear audio data with noise removed.

[0879] Step 4: Preprocessing with Natural Language Processing (NLP)

[0880] Server: Preprocesses the received text data. Specifically, it uses Python's NLTK and SpaCy to perform the following steps:

[0881] Word tokenization: Splitting text into words.

[0882] Normalization: Standardizing the format. Example: Converting "daijoubu" to "daijoubu."

[0883] Stop word removal: Remove common words (e.g., "wa" and "ga") that are not necessary for analysis.

[0884] Input: Text data from the user.

[0885] Output: Preprocessed text data.

[0886] Step 5: Sentiment analysis

[0887] Server: Passes the preprocessed text data to an emotion engine (e.g., using TensorFlow or PyTorch) for sentiment analysis. Specifically, it performs the following processes:

[0888] Sentiment classification: Detecting emotions such as "positive," "negative," and "neutral" from text.

[0889] Keyword extraction: Extract key keywords.

[0890] Input: Preprocessed text data.

[0891] Output: Sentiment label (e.g. "negative") and key keywords.

[0892] Step 6: Scoring psychotic symptoms

[0893] Server: Calculates a numerical score of the symptoms of mental illness based on the emotion labels and keywords obtained from the emotion engine. It evaluates the user's condition using a scoring algorithm or machine learning model.

[0894] Input: emotion labels and keywords.

[0895] Output: Psychotic symptom score.

[0896] Step 7: Alert Generation

[0897] Server: Generate an alert if the psychosis symptom score exceeds a set threshold. The alert will include the following information:

[0898] User ID: The user for whom the alert is intended.

[0899] Type of mental illness: For example, "depression" or "anxiety."

[0900] Psychotic symptom score.

[0901] Extracted keywords.

[0902] Input: Psychotic symptom score and keywords.

[0903] Output: Alert information.

[0904] Step 8: Selecting a Care Plan

[0905] Server: Selects an appropriate care plan based on the generated alert information. Specifically, it proposes care methods according to the user's condition. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[0906] Input: Alert information.

[0907] Output: Care plan.

[0908] Step 9: Generate the interaction script

[0909] Server: Generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[0910] Input: Analysis results of the emotion engine.

[0911] Output: Dialogue script.

[0912] Step 10: Notify users

[0913] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[0914] Input: Care plan and dialogue script.

[0915] Output: A notification message to the user.

[0916] (Application example 2)

[0917] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0918] In modern society, it is important to monitor people's mental and emotional states in real time, detect abnormalities early, and take appropriate measures. In particular, rapid response is required when signs of mental illness are observed. There is also a need for security devices to quickly detect dangerous situations and abnormal mental states and provide appropriate countermeasures. However, current systems cannot fully meet these requirements, making effective monitoring and response difficult.

[0919] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting chat and voice data entered by the user; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for detecting danger or abnormal conditions in real time using voice recognition, natural language processing, and sentiment analysis technologies in the security device; and means for generating an appropriate response and dialogue script based on the detected abnormality and notifying the user or relevant parties from the security device. This enables early detection of changes in the user's mental state or security abnormalities and enables prompt and appropriate response.

[0920] "Chat data" refers to data that indicates the contents of conversations entered by users in text format.

[0921] "Voice data" refers to data that indicates the contents of a conversation input by a user in voice form.

[0922] A "natural language processing algorithm" is an algorithm that analyzes text data and extracts emotions and keywords from it.

[0923] "Sentiment analysis" is the process of analyzing a user's emotional state (e.g., positive, negative, neutral, etc.) from input text data or voice data.

[0924] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[0925] The "psychiatric symptom score" is a numerical value that quantifies the abnormality of the user's mental state based on the results of emotion analysis and keyword extraction.

[0926] An "alert" is a warning message that is generated when a psychosis symptom score exceeds a certain threshold.

[0927] A "care plan" is a plan that includes measures and suggestions to be provided depending on the user's mental state.

[0928] A "dialogue script" is a specific set of messages or questions that are generated to interact with a user.

[0929] A "security device" is a device equipped with voice recognition, natural language processing, and emotion analysis technologies that has the ability to detect danger or abnormal conditions in real time.

[0930] "Speech recognition" is a technology that converts voice data into text data.

[0931] "Real-time" is a temporal concept that refers to the instantaneous processing and updating of data.

[0932] "Notification means" refers to a method or system for communicating alerts, care plans, and dialogue scripts to users and related parties.

[0933] This system collects data entered by users via chat or voice, performs sentiment analysis and keyword extraction, detects signs of mental illness in real time, and provides appropriate care plans.Furthermore, by applying it to security devices, it can quickly detect danger or abnormal conditions and take countermeasures.

[0934] Components

[0935] 1. Data collection device (terminal)

[0936] This is an interface for users to chat and input voice. This device can be a PC, smartphone, tablet, etc. It collects data from users and sends it to a server.

[0937] 2. Server

[0938] The server is a central control unit that analyzes the received data, performs sentiment analysis, keyword extraction, calculates the mental illness symptom score, generates alerts, selects care plans, generates dialogue scripts, and notifies users. This server is built using a computer with high-performance processing power or a cloud service (e.g., Amazon Web Services or Microsoft Azure).

[0939] Processing content

[0940] Voice Recognition

[0941] On the device, voice data input by the user is converted into text data using voice recognition technology. At this time, highly accurate voice recognition is achieved by using the speech_recognition library, etc.

[0942] Data Preprocessing

[0943] The server preprocesses the received text data using natural language processing algorithms (e.g., Hugging Face's transformers library), including tokenization, normalization, and stop-word removal.

[0944] Sentiment analysis and keyword extraction

[0945] The preprocessed data is passed to an emotion engine for sentiment analysis and keyword extraction. The emotion engine (e.g., Hugging Face sentiment-analysis model) detects positive, negative, and neutral emotions from the input data and extracts key keywords.

[0946] Calculation of psychotic symptom scores

[0947] The server calculates a psychosis symptom score based on the sentiment analysis results and keywords, using machine learning models (e.g., scaling algorithms and averaging) to quantify the user's state.

[0948] Generate alerts

[0949] If the score exceeds a certain threshold, the server generates an alert, which includes the user ID, the type of mental illness, the score, and the extracted keywords.

[0950] Care plan selection and dialogue script generation

[0951] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, it may suggest "exercise recommendations" and "rest recommendations" for mild depression, or "relaxation techniques" for anxiety disorders.

[0952] User Notification

[0953] The selected care plan and dialogue script are sent to the terminal and notified to the user. The notification is in the form of a pop-up or message so that the user can easily check it.

[0954] Security equipment applications

[0955] Real-time detection of dangers and abnormal conditions

[0956] The system, installed in the security device, analyzes surrounding sounds using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time, and if certain thresholds are exceeded, an alert and dialogue script is generated to notify the user immediately.

[0957] Specific examples

[0958] 1. Detecting and Treating Signs of Depression

[0959] User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[0960] The server analyzes the keywords "very negative," "tired," and "not having fun."

[0961] Because the depression risk score is high, an alert is generated and care plans include "recommended exercise" and "appointment with a psychiatrist."

[0962] The server generates a dialogue script saying, "You seem tired lately. Do you need any help?" and sends it to the terminal.

[0963] The terminal notifies user A of the care plan and dialogue script.

[0964] 2. Detecting and Treating Symptoms of Anxiety Disorders

[0965] User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[0966] The voice is converted into text data and sent to the server.

[0967] The server analyzes the keywords "very anxious," "tense," and "can't sleep."

[0968] Because the anxiety disorder risk score is high, an alert is generated and a care plan is suggested that includes relaxation techniques and a meeting with a specialist.

[0969] The server generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[0970] The device notifies user B of the care plan and dialogue script.

[0971] 3. Examples of danger detection by security devices

[0972] Detects nearby voices such as "Help, I'm scared!"

[0973] The voice is converted into text data and sent to the server.

[0974] The server analyzes the keywords "very negative," "help," and "scary."

[0975] Because the security risk score is high, an alert is generated and a care plan is suggested stating that "consultation with an expert is required."

[0976] The server generates a dialogue script "Do you need support?" and notifies the relevant person via the security device.

[0977] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0978] Step 1: User enters chat or voice input

[0979] The user uses a device (smartphone, PC, tablet) to input text or speak in chat. The input chat data and voice data are temporarily stored on the device. In this step, the user inputs any conversation content into the system. Input is done by text or voice.

[0980] Step 2: Convert the audio data to text

[0981] The device converts the voice input data into text data using the speech_recognition library. Specifically, the voice recognition algorithm analyzes the voice data and converts it into a corresponding string of characters. The input is voice data, and the output is the corresponding text data.

[0982] Step 3: Preprocess the data

[0983] The server receives the text data sent from the device and preprocesses it using a natural language processing algorithm (e.g., Hugging Face's transformers library). Specific preprocessing steps include tokenization, normalization, and stop word removal. The input is text data, and the output is the preprocessed text data.

[0984] Step 4: Sentiment analysis and keyword extraction

[0985] The server passes the preprocessed text data to an emotion engine (e.g., Hugging Face sentiment-analysis model) for sentiment analysis and keyword extraction. Specifically, the algorithm extracts emotional states (e.g., positive, negative, neutral) and key keywords from the text data. The input is the preprocessed text data, and the output is the sentiment analysis results and extracted keywords.

[0986] Step 5: Calculate the psychosis symptom score

[0987] The server calculates a mental illness symptom score based on the sentiment analysis results and extracted keywords. It uses a machine learning model to normalize the scores for each emotion and calculate an overall symptom score. The input is the sentiment analysis results and keywords, and the output is the mental illness symptom score.

[0988] Step 6: Generate an alert

[0989] The server generates an alert when the psychosis symptom score exceeds a certain threshold. The alert includes the user ID, the type of psychosis, the score, and the extracted keywords. The input is the psychosis symptom score, and the output is the generated alert.

[0990] Step 7: Select a care plan and generate a dialogue script

[0991] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, recommendations for exercise and rest are made for mild depression, and relaxation techniques are suggested for anxiety disorders. The input is the generated alert, and the output is the care plan and dialogue script.

[0992] Step 8: Inform the user of the care plan and dialogue script

[0993] The terminal receives the care plan and dialogue script sent from the server and notifies the user. The notification is in the form of a pop-up or message so that the user can easily check it. The input is the care plan and dialogue script, and the output is the notification to the user.

[0994] Step 9: Detect and respond to abnormalities using security devices

[0995] The server uses a system installed in the security device to analyze surrounding audio using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time. At this stage, an alert and dialogue script are generated if a certain threshold is exceeded. The input is surrounding audio data, and the output is the generated alert and dialogue script.

[0996] Step 10: Security Device Notification

[0997] The security device immediately notifies relevant parties and users based on the generated alert and dialogue script. This notification is made via the security device's display and audio output. The input is the generated alert and dialogue script, and the output is the notification to relevant parties.

[0998] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0999] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1000] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1001] [Third embodiment]

[1002] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1003] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1004] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1005] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1006] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1007] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1008] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1009] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1010] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1011] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1012] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1013] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1014] The present invention provides a system for detecting early signs of mental illness in a user and providing appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat or voice. Below, an embodiment of the present invention will be described with specific examples.

[1015] System Overview

[1016] The system collects data entered by users via chat or voice and analyzes it using natural language processing algorithms. The analysis involves sentiment analysis and keyword extraction, and calculates a mental illness symptom score. If the score exceeds a certain level, an alert is generated and the user is provided with an appropriate care plan and dialogue script.

[1017] Technologies used

[1018] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[1019] Speech recognition technology: Converts voice data into text data.

[1020] Noise reduction technology: Removes unwanted noise from audio data.

[1021] Program processing

[1022] Data collection

[1023] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[1024] Data analysis

[1025] Server: The received text data is passed to a natural language processing module for sentiment analysis and keyword extraction. For example, keywords such as "stress," "tension," and "can't sleep" are extracted from the text data to analyze the speaker's emotions.

[1026] Identifying symptoms

[1027] Server: Calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the calculated score exceeds a certain threshold, an alert is generated. For example, a specific numerical value such as "Depression risk score: 85" or "Anxiety disorder risk score: 90" is displayed.

[1028] Alert Generation

[1029] Server: The generated alert includes the user ID, type of mental illness, score, and specific keywords. This information can be saved in a database and notified to an administrator.

[1030] Providing care plans

[1031] Server: Based on the alert information, the server selects a care plan to provide to the user. For example, it provides specific and actionable plans such as "recommending regular exercise" and "how to schedule counseling appointments."

[1032] Conversation script recommendations

[1033] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. For example, for depression, it generates a message like "You seem tired lately, are you okay?", and for anxiety disorders, it generates a message like "Why don't you try taking a deep breath?"

[1034] User Notification

[1035] Terminal: The care plan and dialogue script are notified to the user. They are displayed in a format that is easy for the user to check, and are provided as pop-ups or notification messages as necessary.

[1036] Specific examples

[1037] Detecting and Treating Signs of Depression

[1038] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1039] Server: Sentiment analysis detects negative emotions and keywords such as "tired" and "not happy."

[1040] Server: Generate alert due to high depression risk score.

[1041] Server: Provides care plans such as "exercise recommendations" and "appointment with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[1042] Terminal: Notify User A of this.

[1043] Detecting and Treating Anxiety Disorder Signs

[1044] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1045] Terminal: Converts voice into text data and sends it to the server.

[1046] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[1047] Server: High score generates an alert.

[1048] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[1049] Terminal: Notify User B of this.

[1050] The present invention enables early detection of signs of mental illness and prompt provision of appropriate care, thereby maintaining the user's health and improving their quality of life, which is expected to help maintain productivity in the workplace and prevent the worsening of mental illness.

[1051] The processing flow will be explained below.

[1052] Step 1: The user inputs text or voice using a chat app or speech recognition app. For example, the user might input text such as "Work has been really stressful lately," or say something like "I'm so nervous about the meeting I'm having trouble sleeping."

[1053] Step 2: The device receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[1054] Step 3: The server passes the received text data to a natural language processing (NLP) module for text preprocessing, such as tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[1055] Step 4: The server passes the preprocessed text to a sentiment analysis algorithm to classify emotions such as positive, negative, or neutral, and extract key keywords, such as "stressed," "tense," and "unhappy."

[1056] Step 5: The server calculates a mental illness symptom score based on the results of the sentiment analysis and keyword extraction. The algorithm uses a machine learning model (e.g., SVM, deep learning, etc.) to quantify the severity of the symptoms.

[1057] Step 6: The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and specific keywords.

[1058] Step 7: The server selects a care plan for the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[1059] Step 8: The server generates an appropriate dialogue script depending on the type of mental illness detected, such as "You seem tired lately, are you okay?" or "Why don't you try some deep breathing to relax before the meeting?"

[1060] Step 9: The terminal notifies the user of the care plan and the dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[1061] Step 10: The user reviews the notification and takes action based on the provided care plan or dialogue script, for example, trying suggested relaxation techniques or booking a consultation.

[1062] In this way, the system can detect the user's mental state early and provide appropriate care promptly.

[1063] Example 1

[1064] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1065] While the importance of mental health management has increased in recent years, it often takes time for users to become aware of their own mental state and receive appropriate care. In particular, overlooking early signs of mental illness carries the risk of worsening symptoms. Conventional methods assume that users will consult with a specialist themselves, and lack a means to automatically and in real time monitor the user's mental state and provide appropriate care. The challenge is to address this shortcoming.

[1066] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1067] In this invention, the server includes means for collecting text data and voice data input by the user, means for preprocessing the collected data using noise reduction technology and converting the voice data into text data, and means for analyzing the converted text data using a natural language processing algorithm to perform sentiment analysis and keyword extraction, thereby enabling automatic and real-time monitoring of the user's mental state and instantly providing an appropriate care plan and dialogue script.

[1068] A "user" refers to an individual who uses the system and inputs data via text or voice.

[1069] "Text data" refers to information entered by a user in text form through a chat app.

[1070] "Voice data" refers to voice information spoken by a user, including recorded or real-time sound data.

[1071] "Noise reduction technology" refers to technology that removes unnecessary background sounds and noise from audio data.

[1072] "Preprocessing" refers to a series of steps to prepare data for easier analysis.

[1073] "Natural language processing algorithms" refer to technologies for analyzing text data, and are algorithms that perform sentiment analysis and keyword extraction.

[1074] "Sentiment analysis" refers to the technology of analyzing user emotions from text data.

[1075] "Keyword extraction" refers to the technique of extracting important words and phrases from text data.

[1076] The "symptom score" refers to a numerical assessment of a user's mental state, indicating their risk of mental illness.

[1077] An "alert" refers to warning information that is generated when a user's situation exceeds a certain threshold.

[1078] "Support plan" refers to a care plan provided to the user by the system, which includes specific measures and advice.

[1079] A "dialogue script" refers to a set phrase that is generated to facilitate communication with a user.

[1080] "Notification" refers to a means of conveying information to the user through the device, and is done in the form of a pop-up or push notification.

[1081] MODE FOR CARRYING OUT THE INVENTION

[1082] The present invention is a system that detects early signs of mental illness in users and provides appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat and voice.

[1083] Hardware and Software Used

[1084] 1. Terminal: A device such as a smartphone or computer used by a user.

[1085] 2. Server: A central server for data analysis. No specific hardware is required, but a computer with powerful processing power is recommended.

[1086] 3. Software:

[1087] Natural language processing (NLP) libraries: Python's "NLTK" and "spaCy".

[1088] Speech recognition technology: "Google Cloud Speech-to-Text" API for converting voice data into text data.

[1089] Noise reduction techniques: "LibROSA" and "PyDub" for preprocessing audio data.

[1090] Database: MySQL or MongoDB for storing alerts and care plans.

[1091] Generative AI models: "Scikit-learn" and "TensorFlow" for machine learning.

[1092] Program processing explanation

[1093] Data collection

[1094] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[1095] Data Preprocessing

[1096] Server: Analyzes the received voice data using noise reduction technology to remove unwanted background sounds and noise, then converts the voice data into text using speech recognition technology.

[1097] Data analysis

[1098] Server: The converted text data is passed to a natural language processing algorithm for sentiment analysis and keyword extraction.

[1099] Identifying symptoms

[1100] Server: Calculates a psychosis symptom score based on the analysis results. If the score exceeds a certain threshold, an alert is generated.

[1101] Alert Generation

[1102] Server: The generated alert contains the user ID, type of mental illness, score, and specific keywords. This information is stored in a database and notified to the administrator.

[1103] Providing care plans

[1104] Server: Based on the alert information, selects a care plan to provide to the user. Recommendations for the care plan include "regular exercise" and "how to schedule counseling appointments."

[1105] Conversation script recommendations

[1106] Server: Generates an appropriate dialogue script depending on the type of mental illness detected.

[1107] User Notification

[1108] On the device: The care plan and dialogue script are notified to the user as a pop-up or push notification.

[1109] Specific examples

[1110] Detecting and Treating Signs of Depression

[1111] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1112] Server: Performs sentiment analysis on the received text data to detect negative emotions and keywords such as "tired" and "not having fun."

[1113] Server: Generate alert due to high depression risk score.

[1114] Server: Provides care plans such as "regular exercise" and "appointments with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[1115] Terminal: Notify User A of this.

[1116] Detecting and Treating Anxiety Disorder Signs

[1117] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1118] Terminal: Converts voice into text data and sends it to the server.

[1119] Server: The audio data is preprocessed using noise reduction technology and converted into text data using Google Cloud Speech-to-Text.

[1120] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[1121] Server: High score generates an alert.

[1122] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[1123] Terminal: Notify User B of this.

[1124] Prompt Sentence Examples

[1125] 1. "Analyze what users have been talking about in chat about their work recently and tell me if they are showing signs of stress."

[1126] 2. "If a user says, 'Work has been really stressful lately,' what would you do to address that?"

[1127] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1128] Step 1: Data collection

[1129] Device: A user uses a chat app to type, "Work has been really stressful lately," or to say, "I'm so nervous before the meeting I can't sleep," as voice data. This data is collected on the device and sent to the server.

[1130] Input: User text or voice data.

[1131] Output: The data packet to be sent to the server.

[1132] Step 2: Data Preprocessing

[1133] Server: The received audio data is first preprocessed using noise reduction technology (e.g., LibROSA, PyDub) to remove unwanted background sounds and noise, and then converted into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[1134] Input: Audio data sent from the device.

[1135] Output: Text data.

[1136] Step 3: Data analysis

[1137] Server: The converted text data is passed to a natural language processing (NLP) algorithm (e.g., NLTK, spaCy) for sentiment analysis and keyword extraction. Based on the analysis results, keywords such as "stress," "tension," and "can't sleep" are extracted, and the emotional state is analyzed.

[1138] Input: Text data converted by speech recognition technology.

[1139] Output: Sentiment analysis results and extracted keywords.

[1140] Step 4: Identify symptoms

[1141] Server: Based on the results of sentiment analysis and keyword extraction, a generative AI model (e.g., Scikit-learn, TensorFlow) is used to calculate a mental illness symptom score. If the score exceeds a certain threshold, it is determined that an abnormality has been detected and an alert is generated.

[1142] Input: Sentiment analysis results and extracted keywords.

[1143] Output: Psychosis symptom score and whether an alert was generated.

[1144] Step 5: Alert Generation

[1145] Server: If the score exceeds the threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and specific keywords. The generated alert information is stored in a database (e.g., MySQL, MongoDB) and notified to the administrator.

[1146] Input: Psychosis symptom score.

[1147] Output: Alert information.

[1148] Step 6: Provide a care plan

[1149] Server: Based on the alert information, the server selects the care plan to be provided to the user. For example, it suggests "regular exercise recommendations" or "how to schedule counseling appointments." This information is also stored in the database.

[1150] Input: Alert information.

[1151] Output: Care plan.

[1152] Step 7: Recommend conversation scripts

[1153] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. Using a generative AI model, it generates messages such as "You seem tired lately, are you okay?" for depression, or "Why don't you try taking a deep breath?" for anxiety disorders.

[1154] Input: Care plan.

[1155] Output: Dialogue script.

[1156] Step 8: Notify users

[1157] Device: The care plan and dialogue script are notified in a format that is easy for the user to view. Specifically, this is displayed as a pop-up notification or push notification in the chat app.

[1158] Input: Care plan and dialogue script.

[1159] Output: User notification.

[1160] (Application example 1)

[1161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1162] Mental stress among employees and customers leads to reduced productivity and customer satisfaction, so it is necessary to detect mental anxiety early and provide appropriate care even in physical store environments. However, the technology to do this efficiently and in real time is currently immature, so there is a need for a system that can appropriately monitor the mental state of employees and customers and take prompt measures.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1164] In this invention, the server includes: means for collecting chat and voice data entered by users; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for using a smart device to monitor the mental state of employees and customers; means for providing a care plan recommending the use of a relaxation room in a store environment; means for generating a dialogue script recommending relaxation techniques such as deep breathing; and means for calculating the stress score of employees using the collected data. This makes it possible to quickly detect changes in mental state even in a physical store environment and provide appropriate care.

[1165] "User" refers to an individual or end user who uses the System.

[1166] "Chat data" refers to conversation data entered by a user in text format.

[1167] "Voice data" refers to data collected by recording voices uttered by users.

[1168] "Natural language processing algorithm" refers to a technical system for analyzing human language using a computer.

[1169] "Sentiment analysis" is a technique for determining a user's emotional state from text or voice data.

[1170] "Keyword extraction" is a technique for identifying important words and phrases from text or audio data.

[1171] The "psychiatric symptom score" refers to a numerical value that quantifies a person's mental state based on the results of emotion analysis and keyword extraction.

[1172] "Alert" refers to warning information generated when the psychosis symptom score exceeds a certain threshold.

[1173] A "care plan" refers to specific support ideas and measures that should be provided to a user based on an alert.

[1174] A "dialogue script" refers to a dialogue template or recommended message that is generated according to a specific situation.

[1175] "Notification" refers to the process of conveying information from the system to the user.

[1176] "Smart device" refers to an electronic device with advanced technology that is used to collect data about a user.

[1177] A "relaxation room" refers to a dedicated space set up for users to relax.

[1178] "Relaxation techniques" refer to methods and techniques for relieving stress and tension in users.

[1179] A "stress score" refers to a numerical evaluation value that quantifies an employee's level of mental stress.

[1180] The present invention provides a system for monitoring the mental state of employees and customers in a brick-and-mortar store environment, detecting early signs of mental illness, and providing appropriate care. The following describes in detail the embodiments of the invention.

[1181] Program processing

[1182] Data collection

[1183] Users input voice and chat data using smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," the voice is collected and the voice data is sent from the device to the server.

[1184] Data analysis

[1185] The server first preprocesses the collected voice data using noise reduction technology and then converts it into text data. Software such as Google Cloud Natural Language API and AWS Transcribe are used. Next, natural language processing algorithms are used to analyze the sentiment and extract keywords from the text data. Keywords such as "stress" and "fatigue" are extracted, and the user's emotional state is analyzed.

[1186] Identifying symptoms

[1187] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the score exceeds a certain threshold, an alert is generated. For example, an alert is generated if the "stress score" is high.

[1188] Providing care plans

[1189] Based on the generated alert, the server selects a care plan to provide to the user. In a physical store environment, care plans might include "recommending use of the relaxation room" or "recommending relaxation techniques such as deep breathing." A dialogue script is also generated. For example, a recommended message might be, "You seem tired lately. Are you okay?"

[1190] User Notification

[1191] The selected care plan and dialogue script are notified to the user via the terminal, and the information is provided in a format that is easy for the user to view via a smartphone or smart glasses.

[1192] Technologies used

[1193] This system uses the following hardware and software:

[1194] Google Cloud Natural Language API: Used for sentiment analysis and keyword extraction.

[1195] AWS Transcribe: Converts audio data into text data.

[1196] TensorFlow: Used to build the algorithm that calculates the mental state score.

[1197] Node.js + Express: Used for server-side processing.

[1198] React Native: Used to develop smartphone apps.

[1199] Specific examples

[1200] For example, employee A says through his smart glasses, "Work has been tough lately." This voice data is converted into text data and analyzed on the server. As a result of the analysis, the keyword "stress" is extracted and the stress score is determined to be 80. An alert is generated, and a dialogue script is generated that includes a care plan such as "recommending use of the relaxation room" and "try taking deep breaths." These are then notified to employee A's smart glasses.

[1201] Prompt Sentence Examples

[1202] "Imagine a system that collects employees' daily voice and chat content, analyzes emotions and keywords through natural language processing, detects early signs of mental illness, and presents care plans and dialogue scripts."

[1203] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1204] Step 1:

[1205] Users input voice data and chat data through their smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," that voice data is collected. Voice data is input, and the collected voice data is stored on the device as output.

[1206] Step 2:

[1207] The terminal transmits the collected voice data to the server. The input is the collected voice data, and the output is the data transmitted to the server. The voice data is transmitted in an appropriate format using a data communication protocol.

[1208] Step 3:

[1209] The server preprocesses the received audio data using noise reduction technology. Specifically, it removes unnecessary background noise and improves the clarity of the audio. The input is the transmitted audio data, and the output is the audio data with noise removed.

[1210] Step 4:

[1211] The server converts the noise-removed audio data into text using speech recognition technology, such as software like "AWS Transcribe." The input is preprocessed audio data, and the output is converted text data.

[1212] Step 5:

[1213] The server uses a natural language processing algorithm to perform sentiment analysis and keyword extraction on the text data. Specifically, it uses the Google Cloud Natural Language API to analyze the emotional tone of the text and keywords such as "stress" and "fatigue." The input is the text data, and the output is the sentiment analysis results and a set of keywords.

[1214] Step 6:

[1215] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. It uses TensorFlow algorithms to calculate a "stress score" and other similar scores. The input is the sentiment analysis results and a set of keywords, and the output is the calculated mental illness symptom score.

[1216] Step 7:

[1217] The server generates an alert if the score exceeds a certain threshold. The input is the calculated psychosis symptom score, and the output is the generated alert, which includes the score and associated keywords.

[1218] Step 8:

[1219] Based on the generated alert, the server selects a care plan to be provided to the user and generates a specific dialogue script. For example, a script such as "We recommend using the relaxation room" or "Please try taking deep breaths" is generated. The input is the alert information, and the output is the care plan and dialogue script.

[1220] Step 9:

[1221] The terminal notifies the user of the care plan and dialogue script sent from the server. A pop-up notification or message is displayed on the smartphone or smart glasses. The input is the care plan and dialogue script, and the output is a notification to the user.

[1222] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1223] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[1224] System Configuration

[1225] The system consists of the following main components:

[1226] 1. Data collection device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[1227] 2. Server: A central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[1228] 3. Emotion engine: A device that analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[1229] Technologies used

[1230] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[1231] Emotion engine: Using specific algorithms, it analyzes emotions in real time based on user input data.

[1232] Speech recognition technology: Converts voice data into text data.

[1233] Noise reduction technology: Removes unwanted noise from audio data.

[1234] Program processing

[1235] Data collection

[1236] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device sends this data to the server.

[1237] Data analysis

[1238] Server: Passes the received text data to a natural language processing (NLP) module to preprocess the text, specifically tokenizing words, normalizing them, removing stop words, etc.

[1239] sentiment analysis

[1240] Server: The preprocessed text is passed to the sentiment engine for sentiment analysis, for example, to detect sentiment such as "positive," "negative," or "neutral" from the text and extract key keywords.

[1241] Identifying symptoms

[1242] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. Using a machine learning model, it quantifies the user's condition and determines the severity of the symptoms.

[1243] Alert Generation

[1244] Server: If the score exceeds the set threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and extracted keywords.

[1245] Providing care plans

[1246] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[1247] Generate conversation scripts

[1248] Server: Generates a specific dialogue script based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[1249] User Notification

[1250] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[1251] Specific examples

[1252] Detecting and Treating Signs of Depression

[1253] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1254] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[1255] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[1256] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[1257] Terminal: Notifies user A of the care plan and dialogue script.

[1258] Detecting and Treating the Signs of Anxiety Disorders

[1259] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1260] Terminal: Converts voice into text data and sends it to the server.

[1261] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[1262] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[1263] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[1264] Terminal: Notifies user B of the care plan and dialogue script.

[1265] This invention makes it possible to analyze a user's emotions in real time, detect signs of mental illness early, and provide appropriate care, which is expected to help maintain the user's health and improve their quality of life, maintain productivity at work, and prevent the mental illness from worsening.

[1266] The processing flow will be explained below.

[1267] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[1268] Step 1:

[1269] A user inputs text or voice using a chat app or speech recognition app. For example, the user might type, "Work has been really stressful lately," or say, "I'm so nervous about the meeting I'm having and can't sleep."

[1270] Step 2:

[1271] The terminal receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[1272] Step 3:

[1273] The server passes the received text data to a natural language processing (NLP) module, which preprocesses the text, specifically tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[1274] Step 4:

[1275] The server passes the preprocessed text to a sentiment analysis algorithm, which classifies emotions such as positive, negative, or neutral, and extracts key keywords, such as "stressed," "tense," and "unhappy."

[1276] Step 5:

[1277] The server passes the data to the emotion engine based on the results of emotion analysis and keyword extraction, and analyzes the user's emotional state in real time. For example, the emotional state may be determined as "very negative" or "very nervous."

[1278] Step 6:

[1279] The server calculates a mental illness symptom score based on the results of the emotion engine, and uses machine learning models (e.g., SVM, deep learning, etc.) to quantify the user's condition and determine the severity of the symptoms.

[1280] Step 7:

[1281] The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and the extracted keywords.

[1282] Step 8:

[1283] The server selects a care plan to provide to the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific and actionable care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[1284] Step 9:

[1285] The server generates a specific dialogue script based on the analysis results of the emotion engine, such as "You seem tired lately, are you okay?" or "Why don't you try taking some deep breaths to relax before the meeting?"

[1286] Step 10:

[1287] The terminal notifies the user of the care plan and dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[1288] Step 11:

[1289] The user reviews the notification and takes action based on the care plan and dialogue script provided, for example, trying suggested relaxation techniques or booking a consultation.

[1290] In this way, the system analyzes the user's emotional state in real time, detects early signs of mental illness, and provides appropriate care promptly, which is expected to help maintain the user's health and improve their quality of life, maintain productivity in the workplace, and prevent mental illness from worsening.

[1291] Example 2

[1292] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1293] Mental health problems are on the rise in modern society, and it is important to detect signs of mental illness early and provide appropriate care. However, it is difficult for users to accurately recognize their own emotions and mental state and take appropriate action. In addition, receiving professional support takes time and effort, making it difficult to respond quickly. To solve these issues, there is a need for a system that can monitor users' emotions and mental state in real time and provide appropriate care quickly.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1295] In this invention, the server includes means for collecting text and voice data entered by a user, means for preprocessing and analyzing the collected data using a natural language processing algorithm to extract emotions and keywords, means for performing emotion analysis on the preprocessed data using an emotion engine, means for calculating a symptom score of mental illness based on the analysis result and generating an alert if a certain threshold is exceeded, means for selecting an appropriate care plan based on the generated alert and generating a specific dialogue script, and means for notifying the user of the selected care plan and dialogue script. This makes it possible to evaluate the user's emotions and mental state in real time, detect symptoms of mental illness early, and provide appropriate care promptly.

[1296] "User" refers to a person who uses the system to input text or voice data.

[1297] "Terminal" refers to a device or interface through which a user inputs text or voice data and transmits that data to a server.

[1298] "Server" refers to the central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[1299] "Data collection means" refers to a method or device for collecting user-entered text or voice data.

[1300] "Natural language processing algorithm" refers to an algorithm for preprocessing and analyzing text data to extract emotions and keywords.

[1301] "Emotion engine" refers to specific algorithms or software for analyzing user emotions from pre-processed data.

[1302] The "psychiatric symptom score" refers to a numerical value that quantifies the user's mental state based on analyzed data.

[1303] "Alert generation means" refers to a method or device for notifying a user of a warning when the psychosis symptom score exceeds a certain threshold.

[1304] The "care plan selection means" refers to a method or device for selecting an appropriate care method or action based on the generated alert information.

[1305] The "interaction script generating means" refers to a method or device for generating specific messages and interaction flows according to the user's emotional and mental state.

[1306] "Notification means" refers to a method or device for notifying the user of the selected care plan and dialogue script.

[1307] "Voice data" refers to voice information uttered by a user.

[1308] "Text data" refers to collected voice data converted into text information.

[1309] "Noise reduction technology" refers to technology for removing unnecessary noise from audio data.

[1310] This system collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. Furthermore, it calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script.

[1311] System Configuration

[1312] The system consists of the following main components:

[1313] 1. Device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[1314] 2. Central control device (server): Analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[1315] 3. Emotion engine: Analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[1316] Hardware and software used

[1317] Hardware:

[1318] 1. Terminal: A device, such as a smartphone or tablet, that allows a user to input text or voice data.

[1319] 2. Server: A central control unit for data analysis, scoring, alert generation, care plan selection, dialogue script generation, and notifications.

[1320] software:

[1321] 1. Natural Language Processing (NLP): Analyze sentiment and keywords from text data using Python's NLTK and SpaCy.

[1322] 2. Emotion Engine: Using TensorFlow and PyTorch, emotions are analyzed in real time based on user input data.

[1323] 3. Speech recognition technology: Uses the Google Speech-to-Text API to convert voice data into text data.

[1324] 4. Noise reduction technology: Use FFmpeg to remove unnecessary noise from audio data.

[1325] Program processing overview

[1326] Data collection

[1327] Device: The user provides data through text chat or voice input. For example, the user might use the chat function to type, "Work has been really stressful lately," or say, "I'm so nervous before the meeting I can't sleep." The device then sends this data to the server.

[1328] Data Preprocessing and Analysis

[1329] Server: The received data is passed to the natural language processing (NLP) module, which preprocesses the data. Specifically, it tokenizes words, normalizes them, and removes stop words. The preprocessed data is then passed to the emotion engine, where it performs sentiment analysis. For example, it detects emotions such as "positive," "negative," and "neutral" from the text and extracts key keywords.

[1330] Psychosis symptom scoring and alert generation

[1331] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. A machine learning model is used to quantify the user's condition and determine the severity of the symptoms. If the score exceeds a set threshold, an alert is generated. For example, the data includes the user ID, type of mental illness, score, and extracted keywords.

[1332] Care plan selection and dialogue script generation

[1333] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendation" or "rest recommendation" for mild depression, and "relaxation techniques" for anxiety disorders. It generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking some deep breaths?"

[1334] User Notification

[1335] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[1336] Specific examples

[1337] Detecting and Treating Signs of Depression

[1338] Device: The user types in chat, "I've been feeling really tired lately and nothing feels fun anymore."

[1339] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[1340] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[1341] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[1342] Terminal: Notifies the user of the care plan and dialogue script.

[1343] Detecting and Treating the Signs of Anxiety Disorders

[1344] Device: The user utters the following through voice input: "I'm so nervous before the meeting that I can't sleep."

[1345] Terminal: Converts voice into text data and sends it to the server.

[1346] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[1347] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[1348] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[1349] Terminal: Notifies the user of the care plan and dialogue script.

[1350] Example prompts for generative AI models

[1351] Below are some example prompts for using our generative AI model:

[1352] "Work has been really stressful lately. I get so nervous before certain meetings that I can't sleep. How can I relax?"

[1353] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1354] Step 1: Data collection

[1355] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device collects this text and voice data and sends it to the server.

[1356] Input: Text or voice data from the user.

[1357] Output: The text or audio data sent to the server.

[1358] Step 2: Convert the audio data

[1359] Server: When voice data is sent, the server uses voice recognition technology (e.g., Google Speech-to-Text API) to convert the voice data into text data.

[1360] Input: Audio data.

[1361] Output: The converted text data.

[1362] Step 3: Noise reduction

[1363] Server: Audio data often contains noise, so use noise reduction techniques such as FFmpeg to remove unwanted noise.

[1364] Input: Audio data.

[1365] Output: Clear audio data with noise removed.

[1366] Step 4: Preprocessing with Natural Language Processing (NLP)

[1367] Server: Preprocesses the received text data. Specifically, it uses Python's NLTK and SpaCy to perform the following steps:

[1368] Word tokenization: Splitting text into words.

[1369] Normalization: Standardizing the format. Example: Converting "daijoubu" to "daijoubu."

[1370] Stop word removal: Remove common words (e.g., "wa" and "ga") that are not necessary for analysis.

[1371] Input: Text data from the user.

[1372] Output: Preprocessed text data.

[1373] Step 5: Sentiment analysis

[1374] Server: Passes the preprocessed text data to an emotion engine (e.g., using TensorFlow or PyTorch) for sentiment analysis. Specifically, it performs the following processes:

[1375] Sentiment classification: Detecting emotions such as "positive," "negative," and "neutral" from text.

[1376] Keyword extraction: Extract key keywords.

[1377] Input: Preprocessed text data.

[1378] Output: Sentiment label (e.g. "negative") and key keywords.

[1379] Step 6: Scoring psychotic symptoms

[1380] Server: Calculates a numerical score of the symptoms of mental illness based on the emotion labels and keywords obtained from the emotion engine. It evaluates the user's condition using a scoring algorithm or machine learning model.

[1381] Input: emotion labels and keywords.

[1382] Output: Psychotic symptom score.

[1383] Step 7: Alert Generation

[1384] Server: Generate an alert if the psychosis symptom score exceeds a set threshold. The alert will include the following information:

[1385] User ID: The user for whom the alert is intended.

[1386] Type of mental illness: For example, "depression" or "anxiety."

[1387] Psychotic symptom score.

[1388] Extracted keywords.

[1389] Input: Psychotic symptom score and keywords.

[1390] Output: Alert information.

[1391] Step 8: Selecting a Care Plan

[1392] Server: Selects an appropriate care plan based on the generated alert information. Specifically, it proposes care methods according to the user's condition. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[1393] Input: Alert information.

[1394] Output: Care plan.

[1395] Step 9: Generate the interaction script

[1396] Server: Generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[1397] Input: Analysis results of the emotion engine.

[1398] Output: Dialogue script.

[1399] Step 10: Notify users

[1400] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[1401] Input: Care plan and dialogue script.

[1402] Output: A notification message to the user.

[1403] (Application example 2)

[1404] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1405] In modern society, it is important to monitor people's mental and emotional states in real time, detect abnormalities early, and take appropriate measures. In particular, rapid response is required when signs of mental illness are observed. There is also a need for security devices to quickly detect dangerous situations and abnormal mental states and provide appropriate countermeasures. However, current systems cannot fully meet these requirements, making effective monitoring and response difficult.

[1406] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting chat and voice data entered by the user; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for detecting danger or abnormal conditions in real time using voice recognition, natural language processing, and sentiment analysis technologies in the security device; and means for generating an appropriate response and dialogue script based on the detected abnormality and notifying the user or relevant parties from the security device. This enables early detection of changes in the user's mental state or security abnormalities and enables prompt and appropriate response.

[1407] "Chat data" refers to data that indicates the contents of conversations entered by users in text format.

[1408] "Voice data" refers to data that indicates the contents of a conversation input by a user in voice form.

[1409] A "natural language processing algorithm" is an algorithm that analyzes text data and extracts emotions and keywords from it.

[1410] "Sentiment analysis" is the process of analyzing a user's emotional state (e.g., positive, negative, neutral, etc.) from input text data or voice data.

[1411] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[1412] The "psychiatric symptom score" is a numerical value that quantifies the abnormality of the user's mental state based on the results of emotion analysis and keyword extraction.

[1413] An "alert" is a warning message that is generated when a psychosis symptom score exceeds a certain threshold.

[1414] A "care plan" is a plan that includes measures and suggestions to be provided depending on the user's mental state.

[1415] A "dialogue script" is a specific set of messages or questions that are generated to interact with a user.

[1416] A "security device" is a device equipped with voice recognition, natural language processing, and emotion analysis technologies that has the ability to detect danger or abnormal conditions in real time.

[1417] "Speech recognition" is a technology that converts voice data into text data.

[1418] "Real-time" is a temporal concept that refers to the instantaneous processing and updating of data.

[1419] "Notification means" refers to a method or system for communicating alerts, care plans, and dialogue scripts to users and related parties.

[1420] This system collects data entered by users via chat or voice, performs sentiment analysis and keyword extraction, detects signs of mental illness in real time, and provides appropriate care plans.Furthermore, by applying it to security devices, it can quickly detect danger or abnormal conditions and take countermeasures.

[1421] Components

[1422] 1. Data collection device (terminal)

[1423] This is an interface for users to chat and input voice. This device can be a PC, smartphone, tablet, etc. It collects data from users and sends it to a server.

[1424] 2. Server

[1425] The server is a central control unit that analyzes the received data, performs sentiment analysis, keyword extraction, calculates the mental illness symptom score, generates alerts, selects care plans, generates dialogue scripts, and notifies users. This server is built using a computer with high-performance processing power or a cloud service (e.g., Amazon Web Services or Microsoft Azure).

[1426] Processing content

[1427] Voice Recognition

[1428] On the device, voice data input by the user is converted into text data using voice recognition technology. At this time, highly accurate voice recognition is achieved by using the speech_recognition library, etc.

[1429] Data Preprocessing

[1430] The server preprocesses the received text data using natural language processing algorithms (e.g., Hugging Face's transformers library), including tokenization, normalization, and stop-word removal.

[1431] Sentiment analysis and keyword extraction

[1432] The preprocessed data is passed to an emotion engine for sentiment analysis and keyword extraction. The emotion engine (e.g., Hugging Face sentiment-analysis model) detects positive, negative, and neutral emotions from the input data and extracts key keywords.

[1433] Calculation of psychotic symptom scores

[1434] The server calculates a psychosis symptom score based on the sentiment analysis results and keywords, using machine learning models (e.g., scaling algorithms and averaging) to quantify the user's state.

[1435] Generate alerts

[1436] If the score exceeds a certain threshold, the server generates an alert, which includes the user ID, the type of mental illness, the score, and the extracted keywords.

[1437] Care plan selection and dialogue script generation

[1438] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, it may suggest "exercise recommendations" and "rest recommendations" for mild depression, or "relaxation techniques" for anxiety disorders.

[1439] User Notification

[1440] The selected care plan and dialogue script are sent to the terminal and notified to the user. The notification is in the form of a pop-up or message so that the user can easily check it.

[1441] Security equipment applications

[1442] Real-time detection of dangers and abnormal conditions

[1443] The system, installed in the security device, analyzes surrounding sounds using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time, and if certain thresholds are exceeded, an alert and dialogue script is generated to notify the user immediately.

[1444] Specific examples

[1445] 1. Detecting and Treating Signs of Depression

[1446] User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1447] The server analyzes the keywords "very negative," "tired," and "not having fun."

[1448] Because the depression risk score is high, an alert is generated and care plans include "recommended exercise" and "appointment with a psychiatrist."

[1449] The server generates a dialogue script saying, "You seem tired lately. Do you need any help?" and sends it to the terminal.

[1450] The terminal notifies user A of the care plan and dialogue script.

[1451] 2. Detecting and Treating Symptoms of Anxiety Disorders

[1452] User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1453] The voice is converted into text data and sent to the server.

[1454] The server analyzes the keywords "very anxious," "tense," and "can't sleep."

[1455] Because the anxiety disorder risk score is high, an alert is generated and a care plan is suggested that includes relaxation techniques and a meeting with a specialist.

[1456] The server generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[1457] The device notifies user B of the care plan and dialogue script.

[1458] 3. Examples of danger detection by security devices

[1459] Detects nearby voices such as "Help, I'm scared!"

[1460] The voice is converted into text data and sent to the server.

[1461] The server analyzes the keywords "very negative," "help," and "scary."

[1462] Because the security risk score is high, an alert is generated and a care plan is suggested stating that "consultation with an expert is required."

[1463] The server generates a dialogue script "Do you need support?" and notifies the relevant person via the security device.

[1464] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1465] Step 1: User enters chat or voice input

[1466] The user uses a device (smartphone, PC, tablet) to input text or speak in chat. The input chat data and voice data are temporarily stored on the device. In this step, the user inputs any conversation content into the system. Input is done by text or voice.

[1467] Step 2: Convert the audio data to text

[1468] The device converts the voice input data into text data using the speech_recognition library. Specifically, the voice recognition algorithm analyzes the voice data and converts it into a corresponding string of characters. The input is voice data, and the output is the corresponding text data.

[1469] Step 3: Preprocess the data

[1470] The server receives the text data sent from the device and preprocesses it using a natural language processing algorithm (e.g., Hugging Face's transformers library). Specific preprocessing steps include tokenization, normalization, and stop word removal. The input is text data, and the output is the preprocessed text data.

[1471] Step 4: Sentiment analysis and keyword extraction

[1472] The server passes the preprocessed text data to an emotion engine (e.g., Hugging Face sentiment-analysis model) for sentiment analysis and keyword extraction. Specifically, the algorithm extracts emotional states (e.g., positive, negative, neutral) and key keywords from the text data. The input is the preprocessed text data, and the output is the sentiment analysis results and extracted keywords.

[1473] Step 5: Calculate the psychosis symptom score

[1474] The server calculates a mental illness symptom score based on the sentiment analysis results and extracted keywords. It uses a machine learning model to normalize the scores for each emotion and calculate an overall symptom score. The input is the sentiment analysis results and keywords, and the output is the mental illness symptom score.

[1475] Step 6: Generate an alert

[1476] The server generates an alert when the psychosis symptom score exceeds a certain threshold. The alert includes the user ID, the type of psychosis, the score, and the extracted keywords. The input is the psychosis symptom score, and the output is the generated alert.

[1477] Step 7: Select a care plan and generate a dialogue script

[1478] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, recommendations for exercise and rest are made for mild depression, and relaxation techniques are suggested for anxiety disorders. The input is the generated alert, and the output is the care plan and dialogue script.

[1479] Step 8: Inform the user of the care plan and dialogue script

[1480] The terminal receives the care plan and dialogue script sent from the server and notifies the user. The notification is in the form of a pop-up or message so that the user can easily check it. The input is the care plan and dialogue script, and the output is the notification to the user.

[1481] Step 9: Detect and respond to abnormalities using security devices

[1482] The server uses a system installed in the security device to analyze surrounding audio using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time. At this stage, an alert and dialogue script are generated if a certain threshold is exceeded. The input is surrounding audio data, and the output is the generated alert and dialogue script.

[1483] Step 10: Security Device Notification

[1484] The security device immediately notifies relevant parties and users based on the generated alert and dialogue script. This notification is made via the security device's display and audio output. The input is the generated alert and dialogue script, and the output is the notification to relevant parties.

[1485] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1486] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1487] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1488] [Fourth embodiment]

[1489] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1490] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1491] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1492] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1493] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1494] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1495] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1496] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1497] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1498] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1499] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1500] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1501] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1502] The present invention provides a system for detecting early signs of mental illness in a user and providing appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat or voice. Below, an embodiment of the present invention will be described with specific examples.

[1503] System Overview

[1504] The system collects data entered by users via chat or voice and analyzes it using natural language processing algorithms. The analysis involves sentiment analysis and keyword extraction, and calculates a mental illness symptom score. If the score exceeds a certain level, an alert is generated and the user is provided with an appropriate care plan and dialogue script.

[1505] Technologies used

[1506] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[1507] Speech recognition technology: Converts voice data into text data.

[1508] Noise reduction technology: Removes unwanted noise from audio data.

[1509] Program processing

[1510] Data collection

[1511] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[1512] Data analysis

[1513] Server: The received text data is passed to a natural language processing module for sentiment analysis and keyword extraction. For example, keywords such as "stress," "tension," and "can't sleep" are extracted from the text data to analyze the speaker's emotions.

[1514] Identifying symptoms

[1515] Server: Calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the calculated score exceeds a certain threshold, an alert is generated. For example, a specific numerical value such as "Depression risk score: 85" or "Anxiety disorder risk score: 90" is displayed.

[1516] Alert Generation

[1517] Server: The generated alert includes the user ID, type of mental illness, score, and specific keywords. This information can be saved in a database and notified to an administrator.

[1518] Providing care plans

[1519] Server: Based on the alert information, the server selects a care plan to provide to the user. For example, it provides specific and actionable plans such as "recommending regular exercise" and "how to schedule counseling appointments."

[1520] Conversation script recommendations

[1521] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. For example, for depression, it generates a message like "You seem tired lately, are you okay?", and for anxiety disorders, it generates a message like "Why don't you try taking a deep breath?"

[1522] User Notification

[1523] Terminal: The care plan and dialogue script are notified to the user. They are displayed in a format that is easy for the user to check, and are provided as pop-ups or notification messages as necessary.

[1524] Specific examples

[1525] Detecting and Treating Signs of Depression

[1526] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1527] Server: Sentiment analysis detects negative emotions and keywords such as "tired" and "not happy."

[1528] Server: Generate alert due to high depression risk score.

[1529] Server: Provides care plans such as "exercise recommendations" and "appointment with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[1530] Terminal: Notify User A of this.

[1531] Detecting and Treating Anxiety Disorder Signs

[1532] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1533] Terminal: Converts voice into text data and sends it to the server.

[1534] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[1535] Server: High score generates an alert.

[1536] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[1537] Terminal: Notify User B of this.

[1538] The present invention enables early detection of signs of mental illness and prompt provision of appropriate care, thereby maintaining the user's health and improving their quality of life, which is expected to help maintain productivity in the workplace and prevent the worsening of mental illness.

[1539] The processing flow will be explained below.

[1540] Step 1: The user inputs text or voice using a chat app or speech recognition app. For example, the user might input text such as "Work has been really stressful lately," or say something like "I'm so nervous about the meeting I'm having trouble sleeping."

[1541] Step 2: The device receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[1542] Step 3: The server passes the received text data to a natural language processing (NLP) module for text preprocessing, such as tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[1543] Step 4: The server passes the preprocessed text to a sentiment analysis algorithm to classify emotions such as positive, negative, or neutral, and extract key keywords, such as "stressed," "tense," and "unhappy."

[1544] Step 5: The server calculates a mental illness symptom score based on the results of the sentiment analysis and keyword extraction. The algorithm uses a machine learning model (e.g., SVM, deep learning, etc.) to quantify the severity of the symptoms.

[1545] Step 6: The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and specific keywords.

[1546] Step 7: The server selects a care plan for the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[1547] Step 8: The server generates an appropriate dialogue script depending on the type of mental illness detected, such as "You seem tired lately, are you okay?" or "Why don't you try some deep breathing to relax before the meeting?"

[1548] Step 9: The terminal notifies the user of the care plan and the dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[1549] Step 10: The user reviews the notification and takes action based on the provided care plan or dialogue script, for example, trying suggested relaxation techniques or booking a consultation.

[1550] In this way, the system can detect the user's mental state early and provide appropriate care promptly.

[1551] Example 1

[1552] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1553] While the importance of mental health management has increased in recent years, it often takes time for users to become aware of their own mental state and receive appropriate care. In particular, overlooking early signs of mental illness carries the risk of worsening symptoms. Conventional methods assume that users will consult with a specialist themselves, and lack a means to automatically and in real time monitor the user's mental state and provide appropriate care. The challenge is to address this shortcoming.

[1554] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1555] In this invention, the server includes means for collecting text data and voice data input by the user, means for preprocessing the collected data using noise reduction technology and converting the voice data into text data, and means for analyzing the converted text data using a natural language processing algorithm to perform sentiment analysis and keyword extraction, thereby enabling automatic and real-time monitoring of the user's mental state and instantly providing an appropriate care plan and dialogue script.

[1556] A "user" refers to an individual who uses the system and inputs data via text or voice.

[1557] "Text data" refers to information entered by a user in text form through a chat app.

[1558] "Voice data" refers to voice information spoken by a user, including recorded or real-time sound data.

[1559] "Noise reduction technology" refers to technology that removes unnecessary background sounds and noise from audio data.

[1560] "Preprocessing" refers to a series of steps to prepare data for easier analysis.

[1561] "Natural language processing algorithms" refer to technologies for analyzing text data, and are algorithms that perform sentiment analysis and keyword extraction.

[1562] "Sentiment analysis" refers to the technology of analyzing user emotions from text data.

[1563] "Keyword extraction" refers to the technique of extracting important words and phrases from text data.

[1564] The "symptom score" refers to a numerical assessment of a user's mental state, indicating their risk of mental illness.

[1565] An "alert" refers to warning information that is generated when a user's situation exceeds a certain threshold.

[1566] "Support plan" refers to a care plan provided to the user by the system, which includes specific measures and advice.

[1567] A "dialogue script" refers to a set phrase that is generated to facilitate communication with a user.

[1568] "Notification" refers to a means of conveying information to the user through the device, and is done in the form of a pop-up or push notification.

[1569] MODE FOR CARRYING OUT THE INVENTION

[1570] The present invention is a system that detects early signs of mental illness in users and provides appropriate care. This system understands the user's mental state by collecting and analyzing data entered by the user through chat and voice.

[1571] Hardware and Software Used

[1572] 1. Terminal: A device such as a smartphone or computer used by a user.

[1573] 2. Server: A central server for data analysis. No specific hardware is required, but a computer with powerful processing power is recommended.

[1574] 3. Software:

[1575] Natural language processing (NLP) libraries: Python's "NLTK" and "spaCy".

[1576] Speech recognition technology: "Google Cloud Speech-to-Text" API for converting voice data into text data.

[1577] Noise reduction techniques: "LibROSA" and "PyDub" for preprocessing audio data.

[1578] Database: MySQL or MongoDB for storing alerts and care plans.

[1579] Generative AI models: "Scikit-learn" and "TensorFlow" for machine learning.

[1580] Program processing explanation

[1581] Data collection

[1582] Device: The user uses a chat app to type, "Work has been really stressful lately." Or, they say, "I'm so nervous before the meeting I can't sleep," as voice data. The device then sends this data to the server.

[1583] Data Preprocessing

[1584] Server: Analyzes the received voice data using noise reduction technology to remove unwanted background sounds and noise, then converts the voice data into text using speech recognition technology.

[1585] Data analysis

[1586] Server: The converted text data is passed to a natural language processing algorithm for sentiment analysis and keyword extraction.

[1587] Identifying symptoms

[1588] Server: Calculates a psychosis symptom score based on the analysis results. If the score exceeds a certain threshold, an alert is generated.

[1589] Alert Generation

[1590] Server: The generated alert contains the user ID, type of mental illness, score, and specific keywords. This information is stored in a database and notified to the administrator.

[1591] Providing care plans

[1592] Server: Based on the alert information, selects a care plan to provide to the user. Recommendations for the care plan include "regular exercise" and "how to schedule counseling appointments."

[1593] Conversation script recommendations

[1594] Server: Generates an appropriate dialogue script depending on the type of mental illness detected.

[1595] User Notification

[1596] On the device: The care plan and dialogue script are notified to the user as a pop-up or push notification.

[1597] Specific examples

[1598] Detecting and Treating Signs of Depression

[1599] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1600] Server: Performs sentiment analysis on the received text data to detect negative emotions and keywords such as "tired" and "not having fun."

[1601] Server: Generate alert due to high depression risk score.

[1602] Server: Provides care plans such as "regular exercise" and "appointments with a psychiatrist," and generates a dialogue script such as "You seem tired lately. Do you need any support?"

[1603] Terminal: Notify User A of this.

[1604] Detecting and Treating Anxiety Disorder Signs

[1605] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1606] Terminal: Converts voice into text data and sends it to the server.

[1607] Server: The audio data is preprocessed using noise reduction technology and converted into text data using Google Cloud Speech-to-Text.

[1608] Server: Detects the keywords "tension" and "inability to sleep" and calculates an anxiety disorder risk score.

[1609] Server: High score generates an alert.

[1610] Server: Suggests "relaxation techniques" and "meeting with a specialist" as care plans, and generates a dialogue script: "Why don't you try taking some deep breaths to relax before the meeting?"

[1611] Terminal: Notify User B of this.

[1612] Prompt Sentence Examples

[1613] 1. "Analyze what users have been talking about in chat about their work recently and tell me if they are showing signs of stress."

[1614] 2. "If a user says, 'Work has been really stressful lately,' what would you do to address that?"

[1615] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1616] Step 1: Data collection

[1617] Device: A user uses a chat app to type, "Work has been really stressful lately," or to say, "I'm so nervous before the meeting I can't sleep," as voice data. This data is collected on the device and sent to the server.

[1618] Input: User text or voice data.

[1619] Output: The data packet to be sent to the server.

[1620] Step 2: Data Preprocessing

[1621] Server: The received audio data is first preprocessed using noise reduction technology (e.g., LibROSA, PyDub) to remove unwanted background sounds and noise, and then converted into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[1622] Input: Audio data sent from the device.

[1623] Output: Text data.

[1624] Step 3: Data analysis

[1625] Server: The converted text data is passed to a natural language processing (NLP) algorithm (e.g., NLTK, spaCy) for sentiment analysis and keyword extraction. Based on the analysis results, keywords such as "stress," "tension," and "can't sleep" are extracted, and the emotional state is analyzed.

[1626] Input: Text data converted by speech recognition technology.

[1627] Output: Sentiment analysis results and extracted keywords.

[1628] Step 4: Identify symptoms

[1629] Server: Based on the results of sentiment analysis and keyword extraction, a generative AI model (e.g., Scikit-learn, TensorFlow) is used to calculate a mental illness symptom score. If the score exceeds a certain threshold, it is determined that an abnormality has been detected and an alert is generated.

[1630] Input: Sentiment analysis results and extracted keywords.

[1631] Output: Psychosis symptom score and whether an alert was generated.

[1632] Step 5: Alert Generation

[1633] Server: If the score exceeds the threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and specific keywords. The generated alert information is stored in a database (e.g., MySQL, MongoDB) and notified to the administrator.

[1634] Input: Psychosis symptom score.

[1635] Output: Alert information.

[1636] Step 6: Provide a care plan

[1637] Server: Based on the alert information, the server selects the care plan to be provided to the user. For example, it suggests "regular exercise recommendations" or "how to schedule counseling appointments." This information is also stored in the database.

[1638] Input: Alert information.

[1639] Output: Care plan.

[1640] Step 7: Recommend conversation scripts

[1641] Server: Generates an appropriate dialogue script depending on the type of mental illness detected. Using a generative AI model, it generates messages such as "You seem tired lately, are you okay?" for depression, or "Why don't you try taking a deep breath?" for anxiety disorders.

[1642] Input: Care plan.

[1643] Output: Dialogue script.

[1644] Step 8: Notify users

[1645] Device: The care plan and dialogue script are notified in a format that is easy for the user to view. Specifically, this is displayed as a pop-up notification or push notification in the chat app.

[1646] Input: Care plan and dialogue script.

[1647] Output: User notification.

[1648] (Application example 1)

[1649] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1650] Mental stress among employees and customers leads to reduced productivity and customer satisfaction, so it is necessary to detect mental anxiety early and provide appropriate care even in physical store environments. However, the technology to do this efficiently and in real time is currently immature, so there is a need for a system that can appropriately monitor the mental state of employees and customers and take prompt measures.

[1651] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1652] In this invention, the server includes: means for collecting chat and voice data entered by users; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for using a smart device to monitor the mental state of employees and customers; means for providing a care plan recommending the use of a relaxation room in a store environment; means for generating a dialogue script recommending relaxation techniques such as deep breathing; and means for calculating the stress score of employees using the collected data. This makes it possible to quickly detect changes in mental state even in a physical store environment and provide appropriate care.

[1653] "User" refers to an individual or end user who uses the System.

[1654] "Chat data" refers to conversation data entered by a user in text format.

[1655] "Voice data" refers to data collected by recording voices uttered by users.

[1656] "Natural language processing algorithm" refers to a technical system for analyzing human language using a computer.

[1657] "Sentiment analysis" is a technique for determining a user's emotional state from text or voice data.

[1658] "Keyword extraction" is a technique for identifying important words and phrases from text or audio data.

[1659] The "psychiatric symptom score" refers to a numerical value that quantifies a person's mental state based on the results of emotion analysis and keyword extraction.

[1660] "Alert" refers to warning information generated when the psychosis symptom score exceeds a certain threshold.

[1661] A "care plan" refers to specific support ideas and measures that should be provided to a user based on an alert.

[1662] A "dialogue script" refers to a dialogue template or recommended message that is generated according to a specific situation.

[1663] "Notification" refers to the process of conveying information from the system to the user.

[1664] "Smart device" refers to an electronic device with advanced technology that is used to collect data about a user.

[1665] A "relaxation room" refers to a dedicated space set up for users to relax.

[1666] "Relaxation techniques" refer to methods and techniques for relieving stress and tension in users.

[1667] A "stress score" refers to a numerical evaluation value that quantifies an employee's level of mental stress.

[1668] The present invention provides a system for monitoring the mental state of employees and customers in a brick-and-mortar store environment, detecting early signs of mental illness, and providing appropriate care. The following describes in detail the embodiments of the invention.

[1669] Program processing

[1670] Data collection

[1671] Users input voice and chat data using smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," the voice is collected and the voice data is sent from the device to the server.

[1672] Data analysis

[1673] The server first preprocesses the collected voice data using noise reduction technology and then converts it into text data. Software such as Google Cloud Natural Language API and AWS Transcribe are used. Next, natural language processing algorithms are used to analyze the sentiment and extract keywords from the text data. Keywords such as "stress" and "fatigue" are extracted, and the user's emotional state is analyzed.

[1674] Identifying symptoms

[1675] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. If the score exceeds a certain threshold, an alert is generated. For example, an alert is generated if the "stress score" is high.

[1676] Providing care plans

[1677] Based on the generated alert, the server selects a care plan to provide to the user. In a physical store environment, care plans might include "recommending use of the relaxation room" or "recommending relaxation techniques such as deep breathing." A dialogue script is also generated. For example, a recommended message might be, "You seem tired lately. Are you okay?"

[1678] User Notification

[1679] The selected care plan and dialogue script are notified to the user via the terminal, and the information is provided in a format that is easy for the user to view via a smartphone or smart glasses.

[1680] Technologies used

[1681] This system uses the following hardware and software:

[1682] Google Cloud Natural Language API: Used for sentiment analysis and keyword extraction.

[1683] AWS Transcribe: Converts audio data into text data.

[1684] TensorFlow: Used to build the algorithm that calculates the mental state score.

[1685] Node.js + Express: Used for server-side processing.

[1686] React Native: Used to develop smartphone apps.

[1687] Specific examples

[1688] For example, employee A says through his smart glasses, "Work has been tough lately." This voice data is converted into text data and analyzed on the server. As a result of the analysis, the keyword "stress" is extracted and the stress score is determined to be 80. An alert is generated, and a dialogue script is generated that includes a care plan such as "recommending use of the relaxation room" and "try taking deep breaths." These are then notified to employee A's smart glasses.

[1689] Prompt Sentence Examples

[1690] "Imagine a system that collects employees' daily voice and chat content, analyzes emotions and keywords through natural language processing, detects early signs of mental illness, and presents care plans and dialogue scripts."

[1691] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1692] Step 1:

[1693] Users input voice data and chat data through their smartphones or smart glasses. For example, if an employee says, "Work has been tough lately," that voice data is collected. Voice data is input, and the collected voice data is stored on the device as output.

[1694] Step 2:

[1695] The terminal transmits the collected voice data to the server. The input is the collected voice data, and the output is the data transmitted to the server. The voice data is transmitted in an appropriate format using a data communication protocol.

[1696] Step 3:

[1697] The server preprocesses the received audio data using noise reduction technology. Specifically, it removes unnecessary background noise and improves the clarity of the audio. The input is the transmitted audio data, and the output is the audio data with noise removed.

[1698] Step 4:

[1699] The server converts the noise-removed audio data into text using speech recognition technology, such as software like "AWS Transcribe." The input is preprocessed audio data, and the output is converted text data.

[1700] Step 5:

[1701] The server uses a natural language processing algorithm to perform sentiment analysis and keyword extraction on the text data. Specifically, it uses the Google Cloud Natural Language API to analyze the emotional tone of the text and keywords such as "stress" and "fatigue." The input is the text data, and the output is the sentiment analysis results and a set of keywords.

[1702] Step 6:

[1703] The server calculates a mental illness symptom score based on the results of sentiment analysis and keyword extraction. It uses TensorFlow algorithms to calculate a "stress score" and other similar scores. The input is the sentiment analysis results and a set of keywords, and the output is the calculated mental illness symptom score.

[1704] Step 7:

[1705] The server generates an alert if the score exceeds a certain threshold. The input is the calculated psychosis symptom score, and the output is the generated alert, which includes the score and associated keywords.

[1706] Step 8:

[1707] Based on the generated alert, the server selects a care plan to be provided to the user and generates a specific dialogue script. For example, a script such as "We recommend using the relaxation room" or "Please try taking deep breaths" is generated. The input is the alert information, and the output is the care plan and dialogue script.

[1708] Step 9:

[1709] The terminal notifies the user of the care plan and dialogue script sent from the server. A pop-up notification or message is displayed on the smartphone or smart glasses. The input is the care plan and dialogue script, and the output is a notification to the user.

[1710] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1711] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[1712] System Configuration

[1713] The system consists of the following main components:

[1714] 1. Data collection device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[1715] 2. Server: A central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[1716] 3. Emotion engine: A device that analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[1717] Technologies used

[1718] Natural Language Processing (NLP): Analyzing sentiment and keywords from text data.

[1719] Emotion engine: Using specific algorithms, it analyzes emotions in real time based on user input data.

[1720] Speech recognition technology: Converts voice data into text data.

[1721] Noise reduction technology: Removes unwanted noise from audio data.

[1722] Program processing

[1723] Data collection

[1724] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device sends this data to the server.

[1725] Data analysis

[1726] Server: Passes the received text data to a natural language processing (NLP) module to preprocess the text, specifically tokenizing words, normalizing them, removing stop words, etc.

[1727] sentiment analysis

[1728] Server: The preprocessed text is passed to the sentiment engine for sentiment analysis, for example, to detect sentiment such as "positive," "negative," or "neutral" from the text and extract key keywords.

[1729] Identifying symptoms

[1730] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. Using a machine learning model, it quantifies the user's condition and determines the severity of the symptoms.

[1731] Alert Generation

[1732] Server: If the score exceeds the set threshold, an alert is generated. The alert includes the user ID, type of mental illness, score, and extracted keywords.

[1733] Providing care plans

[1734] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[1735] Generate conversation scripts

[1736] Server: Generates a specific dialogue script based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[1737] User Notification

[1738] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[1739] Specific examples

[1740] Detecting and Treating Signs of Depression

[1741] Device: User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1742] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[1743] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[1744] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[1745] Terminal: Notifies user A of the care plan and dialogue script.

[1746] Detecting and Treating the Signs of Anxiety Disorders

[1747] Device: User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1748] Terminal: Converts voice into text data and sends it to the server.

[1749] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[1750] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[1751] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[1752] Terminal: Notifies user B of the care plan and dialogue script.

[1753] This invention makes it possible to analyze a user's emotions in real time, detect signs of mental illness early, and provide appropriate care, which is expected to help maintain the user's health and improve their quality of life, maintain productivity at work, and prevent the mental illness from worsening.

[1754] The processing flow will be explained below.

[1755] This invention is a system that collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. This system calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Furthermore, based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script. In addition, by combining it with an emotion engine, it is possible to recognize the user's emotions in real time and provide more accurate feedback and care.

[1756] Step 1:

[1757] A user inputs text or voice using a chat app or speech recognition app. For example, the user might type, "Work has been really stressful lately," or say, "I'm so nervous about the meeting I'm having and can't sleep."

[1758] Step 2:

[1759] The terminal receives the input data. In the case of voice input, the voice data is converted into text data using a voice recognition system. The converted text data or the original text data is sent to the server.

[1760] Step 3:

[1761] The server passes the received text data to a natural language processing (NLP) module, which preprocesses the text, specifically tokenizing words, normalizing them, and removing unnecessary fillers and stop words.

[1762] Step 4:

[1763] The server passes the preprocessed text to a sentiment analysis algorithm, which classifies emotions such as positive, negative, or neutral, and extracts key keywords, such as "stressed," "tense," and "unhappy."

[1764] Step 5:

[1765] The server passes the data to the emotion engine based on the results of emotion analysis and keyword extraction, and analyzes the user's emotional state in real time. For example, the emotional state may be determined as "very negative" or "very nervous."

[1766] Step 6:

[1767] The server calculates a mental illness symptom score based on the results of the emotion engine, and uses machine learning models (e.g., SVM, deep learning, etc.) to quantify the user's condition and determine the severity of the symptoms.

[1768] Step 7:

[1769] The server compares the calculated psychosis symptom score with a threshold and generates an alert if the threshold is exceeded. The alert includes the user ID, the type of psychosis (e.g., depression, anxiety disorder), the score, and the extracted keywords.

[1770] Step 8:

[1771] The server selects a care plan to provide to the user based on the generated alert information. For example, if the user is at high risk of depression, the server will propose specific and actionable care plans such as "regular exercise," "hobby recommendations," and "appointments with a psychiatrist."

[1772] Step 9:

[1773] The server generates a specific dialogue script based on the analysis results of the emotion engine, such as "You seem tired lately, are you okay?" or "Why don't you try taking some deep breaths to relax before the meeting?"

[1774] Step 10:

[1775] The terminal notifies the user of the care plan and dialogue script received from the server in a manner that is easy to understand, such as a pop-up, a message display, or other format.

[1776] Step 11:

[1777] The user reviews the notification and takes action based on the care plan and dialogue script provided, for example, trying suggested relaxation techniques or booking a consultation.

[1778] In this way, the system analyzes the user's emotional state in real time, detects early signs of mental illness, and provides appropriate care promptly, which is expected to help maintain the user's health and improve their quality of life, maintain productivity in the workplace, and prevent mental illness from worsening.

[1779] Example 2

[1780] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1781] Mental health problems are on the rise in modern society, and it is important to detect signs of mental illness early and provide appropriate care. However, it is difficult for users to accurately recognize their own emotions and mental state and take appropriate action. In addition, receiving professional support takes time and effort, making it difficult to respond quickly. To solve these issues, there is a need for a system that can monitor users' emotions and mental state in real time and provide appropriate care quickly.

[1782] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1783] In this invention, the server includes means for collecting text and voice data entered by a user, means for preprocessing and analyzing the collected data using a natural language processing algorithm to extract emotions and keywords, means for performing emotion analysis on the preprocessed data using an emotion engine, means for calculating a symptom score of mental illness based on the analysis result and generating an alert if a certain threshold is exceeded, means for selecting an appropriate care plan based on the generated alert and generating a specific dialogue script, and means for notifying the user of the selected care plan and dialogue script. This makes it possible to evaluate the user's emotions and mental state in real time, detect symptoms of mental illness early, and provide appropriate care promptly.

[1784] "User" refers to a person who uses the system to input text or voice data.

[1785] "Terminal" refers to a device or interface through which a user inputs text or voice data and transmits that data to a server.

[1786] "Server" refers to the central control device that analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[1787] "Data collection means" refers to a method or device for collecting user-entered text or voice data.

[1788] "Natural language processing algorithm" refers to an algorithm for preprocessing and analyzing text data to extract emotions and keywords.

[1789] "Emotion engine" refers to specific algorithms or software for analyzing user emotions from pre-processed data.

[1790] The "psychiatric symptom score" refers to a numerical value that quantifies the user's mental state based on analyzed data.

[1791] "Alert generation means" refers to a method or device for notifying a user of a warning when the psychosis symptom score exceeds a certain threshold.

[1792] The "care plan selection means" refers to a method or device for selecting an appropriate care method or action based on the generated alert information.

[1793] The "interaction script generating means" refers to a method or device for generating specific messages and interaction flows according to the user's emotional and mental state.

[1794] "Notification means" refers to a method or device for notifying the user of the selected care plan and dialogue script.

[1795] "Voice data" refers to voice information uttered by a user.

[1796] "Text data" refers to collected voice data converted into text information.

[1797] "Noise reduction technology" refers to technology for removing unnecessary noise from audio data.

[1798] This system collects chat and voice data entered by users, analyzes the data using natural language processing algorithms, and performs sentiment analysis and keyword extraction. Furthermore, it calculates a mental illness symptom score based on the analysis results and generates an alert if a certain threshold is exceeded. Based on the generated alert, the system selects a care plan to be provided to the user and generates and notifies a specific dialogue script.

[1799] System Configuration

[1800] The system consists of the following main components:

[1801] 1. Device (terminal): An interface through which users input chat and voice, and sends this data to the server.

[1802] 2. Central control device (server): Analyzes data, performs scoring, generates alerts, selects care plans, generates dialogue scripts, and provides notifications.

[1803] 3. Emotion engine: Analyzes the user's emotions in real time and generates feedback and dialogue scripts according to the user's emotional state.

[1804] Hardware and software used

[1805] Hardware:

[1806] 1. Terminal: A device, such as a smartphone or tablet, that allows a user to input text or voice data.

[1807] 2. Server: A central control unit for data analysis, scoring, alert generation, care plan selection, dialogue script generation, and notifications.

[1808] software:

[1809] 1. Natural Language Processing (NLP): Analyze sentiment and keywords from text data using Python's NLTK and SpaCy.

[1810] 2. Emotion Engine: Using TensorFlow and PyTorch, emotions are analyzed in real time based on user input data.

[1811] 3. Speech recognition technology: Uses the Google Speech-to-Text API to convert voice data into text data.

[1812] 4. Noise reduction technology: Use FFmpeg to remove unnecessary noise from audio data.

[1813] Program processing overview

[1814] Data collection

[1815] Device: The user provides data through text chat or voice input. For example, the user might use the chat function to type, "Work has been really stressful lately," or say, "I'm so nervous before the meeting I can't sleep." The device then sends this data to the server.

[1816] Data Preprocessing and Analysis

[1817] Server: The received data is passed to the natural language processing (NLP) module, which preprocesses the data. Specifically, it tokenizes words, normalizes them, and removes stop words. The preprocessed data is then passed to the emotion engine, where it performs sentiment analysis. For example, it detects emotions such as "positive," "negative," and "neutral" from the text and extracts key keywords.

[1818] Psychosis symptom scoring and alert generation

[1819] Server: Calculates a mental illness symptom score based on the analysis results of the emotion engine. A machine learning model is used to quantify the user's condition and determine the severity of the symptoms. If the score exceeds a set threshold, an alert is generated. For example, the data includes the user ID, type of mental illness, score, and extracted keywords.

[1820] Care plan selection and dialogue script generation

[1821] Server: Selects an appropriate care plan based on the generated alert information. For example, it suggests "exercise recommendation" or "rest recommendation" for mild depression, and "relaxation techniques" for anxiety disorders. It generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking some deep breaths?"

[1822] User Notification

[1823] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[1824] Specific examples

[1825] Detecting and Treating Signs of Depression

[1826] Device: The user types in chat, "I've been feeling really tired lately and nothing feels fun anymore."

[1827] Server: The emotion engine analyzes the keywords "very negative," "tired," and "not having fun."

[1828] Server: Because the depression risk score is high, an alert is generated and the care plan includes "recommended exercise" and "appointment with a psychiatrist."

[1829] Server: Generates a dialogue script saying "You seem tired lately. Do you need any help?" and sends it to the device.

[1830] Terminal: Notifies the user of the care plan and dialogue script.

[1831] Detecting and Treating the Signs of Anxiety Disorders

[1832] Device: The user utters the following through voice input: "I'm so nervous before the meeting that I can't sleep."

[1833] Terminal: Converts voice into text data and sends it to the server.

[1834] Server: The emotion engine analyzes the keywords "very anxious," "tense," and "can't sleep."

[1835] Server: Because the anxiety disorder risk score is high, an alert is generated and a care plan is proposed that includes relaxation techniques and a consultation with a specialist.

[1836] Server: Generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[1837] Terminal: Notifies the user of the care plan and dialogue script.

[1838] Example prompts for generative AI models

[1839] Below are some example prompts for using our generative AI model:

[1840] "Work has been really stressful lately. I get so nervous before certain meetings that I can't sleep. How can I relax?"

[1841] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1842] Step 1: Data collection

[1843] Device: The user uses the chat function to type, "Work has been really stressful lately," or speaks, "I'm so nervous before the meeting I can't sleep." The device collects this text and voice data and sends it to the server.

[1844] Input: Text or voice data from the user.

[1845] Output: The text or audio data sent to the server.

[1846] Step 2: Convert the audio data

[1847] Server: When voice data is sent, the server uses voice recognition technology (e.g., Google Speech-to-Text API) to convert the voice data into text data.

[1848] Input: Audio data.

[1849] Output: The converted text data.

[1850] Step 3: Noise reduction

[1851] Server: Audio data often contains noise, so use noise reduction techniques such as FFmpeg to remove unwanted noise.

[1852] Input: Audio data.

[1853] Output: Clear audio data with noise removed.

[1854] Step 4: Preprocessing with Natural Language Processing (NLP)

[1855] Server: Preprocesses the received text data. Specifically, it uses Python's NLTK and SpaCy to perform the following steps:

[1856] Word tokenization: Splitting text into words.

[1857] Normalization: Standardizing the format. Example: Converting "daijoubu" to "daijoubu."

[1858] Stop word removal: Remove common words (e.g., "wa" and "ga") that are not necessary for analysis.

[1859] Input: Text data from the user.

[1860] Output: Preprocessed text data.

[1861] Step 5: Sentiment analysis

[1862] Server: Passes the preprocessed text data to an emotion engine (e.g., using TensorFlow or PyTorch) for sentiment analysis. Specifically, it performs the following processes:

[1863] Sentiment classification: Detecting emotions such as "positive," "negative," and "neutral" from text.

[1864] Keyword extraction: Extract key keywords.

[1865] Input: Preprocessed text data.

[1866] Output: Sentiment label (e.g. "negative") and key keywords.

[1867] Step 6: Scoring psychotic symptoms

[1868] Server: Calculates a numerical score of the symptoms of mental illness based on the emotion labels and keywords obtained from the emotion engine. It evaluates the user's condition using a scoring algorithm or machine learning model.

[1869] Input: emotion labels and keywords.

[1870] Output: Psychotic symptom score.

[1871] Step 7: Alert Generation

[1872] Server: Generate an alert if the psychosis symptom score exceeds a set threshold. The alert will include the following information:

[1873] User ID: The user for whom the alert is intended.

[1874] Type of mental illness: For example, "depression" or "anxiety."

[1875] Psychotic symptom score.

[1876] Extracted keywords.

[1877] Input: Psychotic symptom score and keywords.

[1878] Output: Alert information.

[1879] Step 8: Selecting a Care Plan

[1880] Server: Selects an appropriate care plan based on the generated alert information. Specifically, it proposes care methods according to the user's condition. For example, it suggests "exercise recommendations" and "rest recommendations" for mild depression, and "relaxation techniques" for anxiety disorders.

[1881] Input: Alert information.

[1882] Output: Care plan.

[1883] Step 9: Generate the interaction script

[1884] Server: Generates specific dialogue scripts based on the analysis results of the emotion engine. For example, it generates messages such as "You seem tired lately. Are you okay?" or "Why don't you try taking a deep breath?"

[1885] Input: Analysis results of the emotion engine.

[1886] Output: Dialogue script.

[1887] Step 10: Notify users

[1888] Terminal: Notifies the user of the care plan and dialogue script sent from the server. Notifications are made via pop-ups or message displays, and are provided in a format that the user can easily check.

[1889] Input: Care plan and dialogue script.

[1890] Output: A notification message to the user.

[1891] (Application example 2)

[1892] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1893] In modern society, it is important to monitor people's mental and emotional states in real time, detect abnormalities early, and take appropriate measures. In particular, rapid response is required when signs of mental illness are observed. There is also a need for security devices to quickly detect dangerous situations and abnormal mental states and provide appropriate countermeasures. However, current systems cannot fully meet these requirements, making effective monitoring and response difficult.

[1894] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting chat and voice data entered by the user; means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; means for calculating a mental illness symptom score based on the analysis results and generating an alert if a certain threshold is exceeded; means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; means for notifying the user of the selected care plan and dialogue script; means for detecting danger or abnormal conditions in real time using voice recognition, natural language processing, and sentiment analysis technologies in the security device; and means for generating an appropriate response and dialogue script based on the detected abnormality and notifying the user or relevant parties from the security device. This enables early detection of changes in the user's mental state or security abnormalities and enables prompt and appropriate response.

[1895] "Chat data" refers to data that indicates the contents of conversations entered by users in text format.

[1896] "Voice data" refers to data that indicates the contents of a conversation input by a user in voice form.

[1897] A "natural language processing algorithm" is an algorithm that analyzes text data and extracts emotions and keywords from it.

[1898] "Sentiment analysis" is the process of analyzing a user's emotional state (e.g., positive, negative, neutral, etc.) from input text data or voice data.

[1899] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[1900] The "psychiatric symptom score" is a numerical value that quantifies the abnormality of the user's mental state based on the results of emotion analysis and keyword extraction.

[1901] An "alert" is a warning message that is generated when a psychosis symptom score exceeds a certain threshold.

[1902] A "care plan" is a plan that includes measures and suggestions to be provided depending on the user's mental state.

[1903] A "dialogue script" is a specific set of messages or questions that are generated to interact with a user.

[1904] A "security device" is a device equipped with voice recognition, natural language processing, and emotion analysis technologies that has the ability to detect danger or abnormal conditions in real time.

[1905] "Speech recognition" is a technology that converts voice data into text data.

[1906] "Real-time" is a temporal concept that refers to the instantaneous processing and updating of data.

[1907] "Notification means" refers to a method or system for communicating alerts, care plans, and dialogue scripts to users and related parties.

[1908] This system collects data entered by users via chat or voice, performs sentiment analysis and keyword extraction, detects signs of mental illness in real time, and provides appropriate care plans.Furthermore, by applying it to security devices, it can quickly detect danger or abnormal conditions and take countermeasures.

[1909] Components

[1910] 1. Data collection device (terminal)

[1911] This is an interface for users to chat and input voice. This device can be a PC, smartphone, tablet, etc. It collects data from users and sends it to a server.

[1912] 2. Server

[1913] The server is a central control unit that analyzes the received data, performs sentiment analysis, keyword extraction, calculates the mental illness symptom score, generates alerts, selects care plans, generates dialogue scripts, and notifies users. This server is built using a computer with high-performance processing power or a cloud service (e.g., Amazon Web Services or Microsoft Azure).

[1914] Processing content

[1915] Voice Recognition

[1916] On the device, voice data input by the user is converted into text data using voice recognition technology. At this time, highly accurate voice recognition is achieved by using the speech_recognition library, etc.

[1917] Data Preprocessing

[1918] The server preprocesses the received text data using natural language processing algorithms (e.g., Hugging Face's transformers library), including tokenization, normalization, and stop-word removal.

[1919] Sentiment analysis and keyword extraction

[1920] The preprocessed data is passed to an emotion engine for sentiment analysis and keyword extraction. The emotion engine (e.g., Hugging Face sentiment-analysis model) detects positive, negative, and neutral emotions from the input data and extracts key keywords.

[1921] Calculation of psychotic symptom scores

[1922] The server calculates a psychosis symptom score based on the sentiment analysis results and keywords, using machine learning models (e.g., scaling algorithms and averaging) to quantify the user's state.

[1923] Generate alerts

[1924] If the score exceeds a certain threshold, the server generates an alert, which includes the user ID, the type of mental illness, the score, and the extracted keywords.

[1925] Care plan selection and dialogue script generation

[1926] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, it may suggest "exercise recommendations" and "rest recommendations" for mild depression, or "relaxation techniques" for anxiety disorders.

[1927] User Notification

[1928] The selected care plan and dialogue script are sent to the terminal and notified to the user. The notification is in the form of a pop-up or message so that the user can easily check it.

[1929] Security equipment applications

[1930] Real-time detection of dangers and abnormal conditions

[1931] The system, installed in the security device, analyzes surrounding sounds using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time, and if certain thresholds are exceeded, an alert and dialogue script is generated to notify the user immediately.

[1932] Specific examples

[1933] 1. Detecting and Treating Signs of Depression

[1934] User A types in chat, "I've been feeling very tired lately and nothing feels fun anymore."

[1935] The server analyzes the keywords "very negative," "tired," and "not having fun."

[1936] Because the depression risk score is high, an alert is generated and care plans include "recommended exercise" and "appointment with a psychiatrist."

[1937] The server generates a dialogue script saying, "You seem tired lately. Do you need any help?" and sends it to the terminal.

[1938] The terminal notifies user A of the care plan and dialogue script.

[1939] 2. Detecting and Treating Symptoms of Anxiety Disorders

[1940] User B says via voice input, "I'm so nervous before the meeting I can't sleep."

[1941] The voice is converted into text data and sent to the server.

[1942] The server analyzes the keywords "very anxious," "tense," and "can't sleep."

[1943] Because the anxiety disorder risk score is high, an alert is generated and a care plan is suggested that includes relaxation techniques and a meeting with a specialist.

[1944] The server generates a dialogue script, "Why don't you try taking a deep breath to relax before the meeting?" and sends it to the terminal.

[1945] The device notifies user B of the care plan and dialogue script.

[1946] 3. Examples of danger detection by security devices

[1947] Detects nearby voices such as "Help, I'm scared!"

[1948] The voice is converted into text data and sent to the server.

[1949] The server analyzes the keywords "very negative," "help," and "scary."

[1950] Because the security risk score is high, an alert is generated and a care plan is suggested stating that "consultation with an expert is required."

[1951] The server generates a dialogue script "Do you need support?" and notifies the relevant person via the security device.

[1952] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1953] Step 1: User enters chat or voice input

[1954] The user uses a device (smartphone, PC, tablet) to input text or speak in chat. The input chat data and voice data are temporarily stored on the device. In this step, the user inputs any conversation content into the system. Input is done by text or voice.

[1955] Step 2: Convert the audio data to text

[1956] The device converts the voice input data into text data using the speech_recognition library. Specifically, the voice recognition algorithm analyzes the voice data and converts it into a corresponding string of characters. The input is voice data, and the output is the corresponding text data.

[1957] Step 3: Preprocess the data

[1958] The server receives the text data sent from the device and preprocesses it using a natural language processing algorithm (e.g., Hugging Face's transformers library). Specific preprocessing steps include tokenization, normalization, and stop word removal. The input is text data, and the output is the preprocessed text data.

[1959] Step 4: Sentiment analysis and keyword extraction

[1960] The server passes the preprocessed text data to an emotion engine (e.g., Hugging Face sentiment-analysis model) for sentiment analysis and keyword extraction. Specifically, the algorithm extracts emotional states (e.g., positive, negative, neutral) and key keywords from the text data. The input is the preprocessed text data, and the output is the sentiment analysis results and extracted keywords.

[1961] Step 5: Calculate the psychosis symptom score

[1962] The server calculates a mental illness symptom score based on the sentiment analysis results and extracted keywords. It uses a machine learning model to normalize the scores for each emotion and calculate an overall symptom score. The input is the sentiment analysis results and keywords, and the output is the mental illness symptom score.

[1963] Step 6: Generate an alert

[1964] The server generates an alert when the psychosis symptom score exceeds a certain threshold. The alert includes the user ID, the type of psychosis, the score, and the extracted keywords. The input is the psychosis symptom score, and the output is the generated alert.

[1965] Step 7: Select a care plan and generate a dialogue script

[1966] The server selects an appropriate care plan based on the generated alert information and generates a dialogue script. For example, recommendations for exercise and rest are made for mild depression, and relaxation techniques are suggested for anxiety disorders. The input is the generated alert, and the output is the care plan and dialogue script.

[1967] Step 8: Inform the user of the care plan and dialogue script

[1968] The terminal receives the care plan and dialogue script sent from the server and notifies the user. The notification is in the form of a pop-up or message so that the user can easily check it. The input is the care plan and dialogue script, and the output is the notification to the user.

[1969] Step 9: Detect and respond to abnormalities using security devices

[1970] The server uses a system installed in the security device to analyze surrounding audio using voice recognition, natural language processing, and sentiment analysis technologies to detect danger or abnormal conditions in real time. At this stage, an alert and dialogue script are generated if a certain threshold is exceeded. The input is surrounding audio data, and the output is the generated alert and dialogue script.

[1971] Step 10: Security Device Notification

[1972] The security device immediately notifies relevant parties and users based on the generated alert and dialogue script. This notification is made via the security device's display and audio output. The input is the generated alert and dialogue script, and the output is the notification to relevant parties.

[1973] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1974] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1975] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1976] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1977] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1978] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1979] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1980] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1981] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1982] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1983] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the for...

Claims

1. a means for collecting chat and voice data entered by users; A means for analyzing the collected data using a natural language processing algorithm to perform sentiment analysis and keyword extraction; A means for calculating a psychosis symptom score based on the analysis results and generating an alert when a certain threshold is exceeded; A means for selecting a care plan to be provided to the user based on the generated alert and generating a specific dialogue script; The system includes a means for notifying the user of the selected care plan and interaction script.

2. 2. The system according to claim 1, further comprising means for converting voice data input by a user into text data.

3. 3. The system of claim 2, further comprising means for preprocessing the audio data using noise reduction techniques and converting the audio data to text data.

Images