System

Abstract

A system is provided.SOLUTION: A system comprising: means for monitoring a user's search history and remarks in a social networking service to detect a specific keyword; means for evaluating a user's emotional state based on the keyword detected by said means; and means for transmitting a counseling message generated based on an evaluation result to a user's terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem of suicide among young people and students is becoming more serious every year. Behind this is the inability to find someone to talk to appropriately about their worries and suffering. In such situations, the problem worsens, and in many cases, it ultimately results in suicide. Conventional methods make it difficult to provide appropriate support early on, and there are particularly limited means of understanding a user's emotional state from online comments and search history. Therefore, to solve these issues, a new system is needed that can continuously monitor a user's emotional state in real time and provide support at the appropriate time. [Means for solving the problem]

[0005] The present invention is a system that includes a means for monitoring a user's search history and comments on social networking services to detect specific keywords, a means for evaluating the user's emotional state based on the keywords detected by the above means, and a means for sending a counseling message to the user's device based on the evaluation results. Furthermore, by adding a means for sending a notification to a live counselor when the evaluation results meet certain conditions, the system can address even more serious issues. Furthermore, by introducing a means for users to create and customize virtual characters, the system allows users to use applications with a sense of familiarity. By adding a means for periodically collecting input data from users and storing it in a database to improve the accuracy of the evaluation results, and a means for periodically displaying status check prompts on the user's device, continuous support and real-time responses are possible.

[0006] "Search history" refers to a record of past search queries a user has made on the Internet.

[0007] A "social networking service" is an online platform that allows people to communicate over the Internet.

[0008] "Specific keywords" are predefined important words or phrases that relate to the user's mental state or emotion.

[0009] "Mood" refers to a user's mental and emotional state, including stress, sadness, joy, etc.

[0010] "Counseling messages" are text messages automatically created by generative AI that contain support and advice for users.

[0011] "User terminal" refers to an electronic device used by a user, such as a smartphone, tablet, or PC.

[0012] A "staffed counselor" is a trained professional who intervenes in emergencies or serious problems.

[0013] A "virtual character" is a digital avatar or character that users can create and customize.

[0014] A "periodic status check prompt" is a question or message that is displayed to the user periodically to check their state of mind.

[0015] A "database" is a repository of information that stores and manages collected data and makes it accessible as needed.

[0016] "Real-time response" refers to the ability to react with very short latency, meaning continuous and immediate support. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system then sends a notification to a live counselor. The system also allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0039] Server Processing

[0040] The server receives and analyzes the user's search history and comments on social networking services. If specific keywords are detected, a machine learning algorithm is used to evaluate the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[0041] Examples:

[0042] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[0043] Terminal handling

[0044] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[0045] Examples:

[0046] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0047] User operations

[0048] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0049] Examples:

[0050] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0051] Backup function

[0052] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then begins the process of providing direct support to the user.

[0053] Examples:

[0054] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[0055] Through these processes, the present invention is able to monitor the user's emotional state in real time, provide support at the appropriate time, and deal with serious situations.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[0059] Step 2:

[0060] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[0061] Step 3:

[0062] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[0063] Step 4:

[0064] The server analyzes the received data, detects specific keywords, and uses machine learning algorithms to evaluate the user's emotional state.

[0065] Step 5:

[0066] The server uses a generative AI model to generate a counseling message based on the evaluation results, and the generated message is immediately sent to the device.

[0067] Step 6:

[0068] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[0069] Step 7:

[0070] The server again analyzes the user's reply data and generates and sends additional counseling messages as needed.

[0071] Step 8:

[0072] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[0073] Step 9:

[0074] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[0075] Step 10:

[0076] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then provides emergency dialogue and assistance.

[0077] Example 1

[0078] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0079] In modern society, as users make daily comments and searches over the Internet, there is a need for systems that can accurately grasp their emotional state and provide counseling messages at the appropriate time. However, existing systems have difficulty analyzing users' emotions in real time and taking appropriate action. Furthermore, they often lack emergency notification functions for dealing with serious situations and user-friendly interfaces. Therefore, there is an urgent need to develop a system that can maintain users' mental health and provide prompt and individualized support.

[0080] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0081] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a generated counseling message to the user's terminal based on the evaluation result; means for periodically collecting the user's operation log and data; means for creating and customizing a virtual character; and means for using a generative AI model to generate the counseling message. This allows the system to grasp the user's emotional state in real time, provide an appropriate counseling message, and notify a live counselor as needed. Furthermore, by customizing the virtual character, the user can increase familiarity and promote interaction with the system.

[0082] A "user's search history" is a history of terms and phrases that a user searches for using an Internet search engine.

[0083] A "social networking service" is a platform that enables users to interact with other users online, including features such as posting, commenting, and messaging.

[0084] A "specific keyword" is a specific word or phrase that provides important clues when assessing a user's emotional state.

[0085] "Mood" refers to the user's psychological state and emotions.

[0086] "Means for evaluation" refers to methods or techniques for analyzing a user's emotional state based on collected data and obtaining a specific score or classification result.

[0087] A "counseling message" is a message that is generated according to the user's emotional state and provides psychological support and advice.

[0088] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0089] An "operation log" is recorded data related to a user's device operations, including app usage status and input content.

[0090] A "generative AI model" is a model that uses machine learning and artificial intelligence techniques to generate output for specific purposes.

[0091] A "virtual character" is a digital character image that can be used as an interface by a user and can be customized.

[0092] A "manned counselor" is a person who is actually a human counselor and provides direct support to the user.

[0093] A "means for sending notifications" is a method or technology for sending a message or alert to a specific person when a specific condition is met by the system.

[0094] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system notifies a live counselor. Furthermore, the system allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0095] Server Processing

[0096] The server uses Apache Kafka to receive the user's search history and social networking service comment data. The received data is temporarily stored and then used to evaluate the user's emotional state using a TensorFlow model. The evaluation is performed by detecting specific keywords (e.g., "I want to die" or "It's painful"). An analysis script preprocesses the data and inputs it into the model, which calculates an emotion score from the inference results. Based on the evaluation results, a counseling message is generated using a generative AI model (e.g., a GPT model) and sent to the user's device via the Firebase Cloud Messaging (FCM) API.

[0097] Specific examples

[0098] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[0099] Prompt Sentence Examples

[0100] "Create a message that will be generated when a user posts 'I want to die'."

[0101] Terminal handling

[0102] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. A background service runs and periodically saves the operation logs and search history in a local database and sends them to the server. It also displays counseling messages sent from the server to the user in real time.

[0103] Specific examples

[0104] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0105] Prompt Sentence Examples

[0106] "Generate a message to display if the user responds 'I'm a little tired.'"

[0107] User operations

[0108] Users install the app and complete the initial setup. Once setup is complete, a background task will start, automatically monitoring the user's search history and comments on social networking services. Users can also create and customize their own virtual characters, input their emotional state within the app, and interact with a generated AI counselor. On the character creation screen, users can select their avatar's appearance and clothing, preview their customizations in real time, and save them.

[0109] Specific examples

[0110] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0111] Prompt Sentence Examples

[0112] "Generate a response when a user types, 'School has been really stressful lately.'"

[0113] Backup function

[0114] If the evaluation result meets certain conditions (e.g., a high emotional score), the server uses the Twilio API to send an SOS notification to a live counselor. The live counselor receives the notification and begins the process of providing direct assistance to the user.

[0115] Specific examples

[0116] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[0117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0118] Step 1:

[0119] The user installs the app and completes the initial setup.

[0120] Specific behavior: The user downloads and installs the app. When the app is launched for the first time, a setup wizard appears and the user grants the necessary permissions (e.g., Internet access, storage access, etc.). The user clicks the Finish setting button and the settings are saved.

[0121] Input: User interactions (touches, clicks), preference information.

[0122] Output: Setting completion flag, user initial setting data.

[0123] Step 2:

[0124] The device collects the user's search history and comments on social media in real time.

[0125] How it works: The device runs a background service that monitors the user's browser history and in-app social media posts to collect data, which is then stored in a local database.

[0126] Input: User search history, social media posts.

[0127] Output: Collected data stored in a local database.

[0128] Step 3:

[0129] The terminal periodically transmits the collected data to the server.

[0130] Specific operation: The device executes a scheduled task and uploads the collected data from the local database to the server. The upload is performed at the optimal time taking into account the network conditions.

[0131] Input: Collected data in local database.

[0132] Output: The data sent to the server.

[0133] Step 4:

[0134] The server temporarily stores the received data and prepares it for analysis.

[0135] Specific operation: The server receives data using Apache Kafka and stores it in temporary storage. The received data is preprocessed by the analysis script to prepare a dataset for analysis.

[0136] Input: Data sent from the terminal.

[0137] Output: Dataset for analysis.

[0138] Step 5:

[0139] The server analyzes the data and detects specific keywords.

[0140] How it works: The server inputs the preprocessed data into a TensorFlow model to detect specific keywords (e.g., "I want to die" or "It's painful"). It then scans the text data using regular expressions to extract matching keywords.

[0141] Input: Dataset for analysis.

[0142] Output: Detected results for specific keywords.

[0143] Step 6:

[0144] The server evaluates the user's emotional state.

[0145] Specific operation: Based on the detected keywords, the server uses a machine learning algorithm to evaluate the user's emotional state, and outputs the evaluation results as an emotion score or classification result.

[0146] Input: Detected results for a specific keyword.

[0147] Output: Evaluation result of the user's emotional state.

[0148] Step 7:

[0149] The server generates a counseling message based on the evaluation result.

[0150] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate an appropriate counseling message based on the evaluation results. A prompt sentence is input, and the model generates a message.

[0151] Input: User's emotional state assessment result.

[0152] Output: The generated counseling message.

[0153] Step 8:

[0154] The server sends a counseling message to the user's terminal.

[0155] Specific operation: The server uses the Firebase Cloud Messaging (FCM) API to send the generated counseling message to the user's device.

[0156] Input: The generated counseling message.

[0157] Output: The message sent to the user's terminal.

[0158] Step 9:

[0159] The terminal receives the counseling message and displays it to the user.

[0160] Specific behavior: The device receives the FCM notification and triggers a local notification. A message is displayed on a specific screen in the app according to the notification.

[0161] Input: The counseling message sent by the server.

[0162] Output: The message displayed to the user.

[0163] Step 10:

[0164] If the evaluation results meet certain conditions, the server sends an SOS notification to a manned counselor.

[0165] What it does: When the server detects a certain condition, such as a high emotion score, it uses the Twilio API to send an emergency notification to a live counselor.

[0166] Input: Evaluation results, such as high sentiment scores.

[0167] Output: SOS notification sent to manned counselors.

[0168] Step 11:

[0169] A live counselor receives the notification and initiates the process of providing direct assistance to the user.

[0170] Specific actions: When a counselor receives a notification, they will provide emergency assistance, such as calling the user directly, and take appropriate action if necessary.

[0171] Input: SOS notification from the server.

[0172] Output: Emergency assistance to the user.

[0173] (Application example 1)

[0174] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0175] Maintaining mental health is an important issue in modern society. In particular, with the spread of the Internet and social networking services, the information users send out on a daily basis often reveals mental distress and stress. However, collecting and analyzing this information in real time and providing appropriate counseling and support is currently difficult. Conventional systems lack the immediacy and accuracy required to provide effective mental support.

[0176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0177] In this invention, the server includes means for monitoring a user's search history and comments on a social networking service and detecting specific keywords, means for evaluating the user's emotional state based on the keywords detected by the above means, means for generating a counseling message based on the evaluation result and sending the message to the user's terminal, means for periodically collecting the user's posts via a social media API and evaluating the emotional state in real time, means for generating an AI counseling message based on the evaluation of the emotional state and sending the message to the user, and means for sending a notification to a human counselor according to the severity of the evaluated emotional state. This makes it possible to quickly and accurately evaluate the user's mental state and provide counseling messages and support at an appropriate time.

[0178] "Search history" is a history of keywords and phrases that a user has searched for on the Internet using a search engine.

[0179] A "social networking service" is a platform that allows users to communicate and share information with each other over the Internet.

[0180] A "specific keyword" is an important word or phrase that indicates a predetermined psychological state from a user's comments or search history.

[0181] "Evaluation means" refers to algorithms or programs that analyze collected data and determine the user's emotional state.

[0182] A "generated counseling message" is a message of advice or comfort that is generated for transmission to a user based on the emotional state determined by the evaluation means.

[0183] A "terminal" is a computer device used by a user, such as a smartphone, tablet, or PC.

[0184] A "social media API" is an application program interface for automatically obtaining data from social networking services.

[0185] "Mood assessment" is the process of determining a user's psychological health from their statements and behavior.

[0186] "AI counseling messages" are messages generated using artificial intelligence technology with the aim of providing psychological support to users.

[0187] A "manned counselor" is a human counselor who provides direct support and intervention when the user's emotional state is determined to be serious.

[0188] This system monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends a generated counseling message to the user's device. Furthermore, if the evaluation results meet certain conditions, the system also includes a function to notify a live counselor.

[0189] Server Processing

[0190] The server first collects the user's search history and comments on social networking services. This is done through social media APIs, and data is periodically retrieved. This data includes search keywords and the content of social media posts. The server then analyzes this data to determine whether specific keywords are included.

[0191] Sentiment analysis is performed on the analyzed data to evaluate the user's emotional state. Based on the evaluation results, an AI-generated counseling message is generated. This AI-generated counseling message is generated in real time using a generative AI model, and the content is determined by using appropriate prompt sentences.

[0192] The generated counseling message is sent to the user's terminal and notified to the user. Depending on the severity of the evaluation result, a notification is also sent to a live counselor.

[0193] Terminal handling

[0194] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to the server. Information entered by the user and operation logs are also collected periodically, and this data is used for analysis on the server. Counseling messages sent from the server are notified to the device and displayed to the user.

[0195] User operations

[0196] Users install the app and complete the initial setup. Once setup is complete, search history and comments on social networking services are automatically monitored. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0197] Specific examples

[0198] For example, if a user posts on social media, "I can't take it anymore," the content of the post is sent to the server via a social media API. The server performs sentiment analysis, assesses the severity, and determines it to be "high." Based on this result, an AI-generated counseling message is generated, and a message saying "Let me tell you something" is sent to the user's device. At the same time, because the condition is serious, a notification is also sent to a live counselor.

[0199] Prompt Sentence Examples

[0200] An example prompt might have the following format:

[0201] User ID: 12345

[0202] Post content: I can't take it anymore

[0203] In this way, the present invention is able to monitor the user's emotional state in real time, provide timely support and deal with critical situations.

[0204] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0205] Step 1:

[0206] The server receives user search history and comment data from social networking services. Specifically, it periodically retrieves user posts and comments using social media APIs. The input is user posted data, and the output is raw data to be analyzed.

[0207] Step 2:

[0208] The server detects specific keywords from the received data using a text analysis engine. The input is raw data, and the output is data containing the detected keywords. Specifically, keyword extraction is performed using natural language processing tools.

[0209] Step 3:

[0210] The server evaluates the user's emotional state based on the detected keywords. It uses a sentiment analysis algorithm to analyze the user's posts and evaluate the emotional state with a score. The input is data containing keywords, and the output is a rating score.

[0211] Step 4:

[0212] The server generates an AI-generated counseling message based on the evaluation score. It uses a generative AI model to create an appropriate message according to the situation. The input is the evaluation score and prompt, and the output is the counseling message. Specifically, based on the example prompt, data is input into the AI-generated model and the generated message is obtained.

[0213] Step 5:

[0214] The server sends the generated counseling message to the user's device using a message sending API. The input is the generated counseling message, and the output is a notification to the user's device.

[0215] Step 6:

[0216] If the evaluation score meets certain conditions, the server sends a notification to a human counselor. The counselor is contacted using a notification system. The input is the evaluation score and the user's posted data, and the output is an alert to the counselor.

[0217] Step 7:

[0218] The user receives the counseling message sent to their terminal. The user terminal displays the message through the notification function. The input is the counseling message sent from the server, and the output is the display to the user. Specifically, the user checks the message and replies if necessary.

[0219] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0220] This invention combines a system that monitors a user's search history and comments on social networking services, detects specific keywords, and evaluates the user's emotional state with an emotion engine. The generated counseling message is sent to the user's device, and if necessary, a live counselor is notified. The system also allows users to create and customize virtual characters to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0221] Server Processing

[0222] The server receives the user's search history and comments on social networking services and analyzes this data using an emotion engine. The emotion engine has an algorithm that recognizes emotions from the user's comments and behavioral patterns, and when specific keywords are detected, it evaluates the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[0223] Examples:

[0224] The server detects users' social media posts containing keywords such as "I want to die" or "It's painful," and the emotion engine evaluates the emotions of "sadness" and "despair." Based on this, a counseling message is generated, such as "I've been very worried about you lately. Please tell me what you think," and sent to the user's device.

[0225] Terminal handling

[0226] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[0227] Examples:

[0228] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0229] User operations

[0230] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0231] Examples:

[0232] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0233] Additional Server Processing

[0234] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor. After receiving the notification, the on-site counselor initiates the procedure to provide direct assistance to the user. The emotion engine uses accumulated user data to track changes in the user's emotions and adjusts the counseling method accordingly.

[0235] Examples:

[0236] The server detects multiple serious posts from the user, such as "I can't take it anymore," and the emotion engine evaluates this as "despair" and determines that there is a serious problem. This causes an emergency notification to be sent to a manned counselor, who then provides emergency support, such as calling the user directly.

[0237] This allows the present invention to monitor the user's emotional state in real time, use the emotion engine to make a highly accurate assessment, and provide timely support to deal with serious situations.

[0238] The processing flow will be explained below.

[0239] Step 1:

[0240] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[0241] Step 2:

[0242] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[0243] Step 3:

[0244] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[0245] Step 4:

[0246] The server analyzes the received data and uses an emotion engine to detect specific keywords and recognize the user's emotions. For example, keywords such as "I want to die" and "It's painful" can be used to identify emotions such as "sadness" and "despair."

[0247] Step 5:

[0248] The server uses a generative AI model to generate a counseling message based on the recognized emotions and evaluation results, and the generated message is immediately sent to the device.

[0249] Step 6:

[0250] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[0251] Step 7:

[0252] The server analyzes the user's reply data again and generates and sends additional counseling messages as needed. For example, if the user replies, "I'm very tired today," the server generates a new message such as, "Do you have time to take a short break?"

[0253] Step 8:

[0254] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[0255] Step 9:

[0256] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[0257] Step 10:

[0258] The server accumulates the received periodic input data and tracks changes in the user's emotions through an emotion engine, which allows long-term emotional trends to be evaluated.

[0259] Step 11:

[0260] If the evaluation results meet certain criteria, the server sends an SOS notification to a live counselor, who then provides emergency dialogue and support. For example, if the server detects multiple serious statements such as "I can't take it anymore," it will send an emergency notification and have a counselor contact the user directly.

[0261] These steps enable the present invention to monitor the user's emotional state in real time, provide timely support, and use the emotion engine to make accurate assessments and handle critical situations.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] In modern society, fluctuations in users' emotional states have become a major problem, and early detection and appropriate response are particularly required for users experiencing stress or depression. However, current technology lacks a system that can accurately monitor these emotional fluctuations in real time and provide appropriate support. Furthermore, other methods can be distrustful of users and impose a heavy burden, so further improvement is needed.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0266] In this invention, the server includes means for monitoring a user's information usage history and comments on the networking service and detecting specific words and phrases, means for evaluating the user's emotional state based on the words and phrases detected by said means, means for transmitting a corresponding message generated based on the evaluation result to the user's information processing device, and means for periodically collecting user input data and improving the accuracy of the evaluation result. This makes it possible to monitor the user's emotional state in real time, perform highly accurate evaluations, and provide appropriate messages at appropriate times.

[0267] "User" means an individual who uses the System to input information and receive services.

[0268] "Information usage history" refers to data including the user's search and access history on the Internet.

[0269] "Networking service" refers to an online platform, such as a social networking site or messaging app, that enables users to exchange information with other users.

[0270] A "phrase" refers to a word or phrase that has a specific meaning in a user's utterance or input data.

[0271] "Emotional state" refers to the user's mental and psychological state, and includes emotions such as "joy," "sadness," and "anger."

[0272] The "evaluation result" refers to the result of the determination of the emotional state obtained by the emotion engine through analysis of the user's words and actions.

[0273] "Response message" refers to a message appropriate to the user's emotional state, created by the generative AI model based on the evaluation results.

[0274] "Information processing device" refers to devices used by users, such as smartphones, tablets, and personal computers.

[0275] "Input data" refers to all information that a user inputs into a system, including text, audio, images, etc.

[0276] An "emotion engine" refers to a system that integrates algorithms and technologies to analyze emotions from users' statements and actions.

[0277] A "generative AI model" refers to an artificial intelligence model that generates appropriate messages in natural language based on input data.

[0278] "Virtual presence" refers to an in-app character or avatar that users can customize.

[0279] "Manned" refers to human experts who monitor the system and intervene in emergencies.

[0280] This invention combines an emotion engine with a system that monitors a user's search history and comments on social networking services (SNS), detects specific words, and evaluates the user's emotional state. The generated counseling message is sent to the user's information processing device, and if necessary, notifies a human expert. The system also allows users to create and customize virtual beings to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0281] The server receives information through an API that collects the user's information usage history and SNS comment data. The collected data is passed to an emotion engine, which uses natural language processing technology to identify the user's emotional state from the comments. Specifically, context analysis and keyword extraction are performed. Once the evaluation results are obtained, a prompt is sent to the generative AI model based on the results, which generates a counseling message. The generated message is then sent to the user's information processing device.

[0282] Examples:

[0283] The server detects social media posts from users that contain phrases such as "I want to die" or "It's painful." The emotion engine evaluates these as "sadness" or "despair." Based on the evaluation results, it sends a prompt to the generative AI model saying, "If the user says 'I want to die,' please generate an appropriate counseling message." The generative AI model then generates a message such as, "I've been very worried about you lately. Please tell me your story," and sends it to the user's information processing device.

[0284] The device monitors the user's search history and social media postings in real time, periodically sending the collected data to the server, and also receives counseling messages sent from the server in real time and notifies the user.

[0285] Users install the app and perform the initial setup. This setup prepares the app to automatically monitor the user's search history and social media posts. Users can create a virtual presence within the app and customize it to their liking. They can also input their emotional state within the app and have conversations with a generated AI counselor.

[0286] Examples:

[0287] Within the app, users can choose and customize the appearance of their virtual presence, then type in something like, "I've been feeling stressed about school lately," to which the generated AI counselor responds, "What's causing you stress?"

[0288] Furthermore, if the evaluation results meet certain conditions, the server can send a notification to a human expert. If the emotion engine detects a serious emotional state, the human expert will respond directly to the user who is deemed to need urgent assistance.

[0289] Examples:

[0290] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[0291] As a result, the present invention makes it possible to monitor a user's emotional state in real time, use an emotion engine to make a highly accurate assessment, and provide support at the appropriate time to deal with serious situations.

[0292] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0293] Step 1: Data collection

[0294] The server receives information through an API to collect user information usage history and comment data from networking services. This process inputs the user's social media posts and browser history. The input data is received in JSON format and stored in a database.

[0295] Specific behavior:

[0296] The server uses the Twitter API to retrieve the user's recent tweets, including the tweet text, timestamp, and user ID.

[0297] Step 2: Data analysis

[0298] The server passes the collected data to the emotion engine for analysis. The emotion engine uses natural language processing technology to identify emotions from the utterances. The input to this analysis process is the text data of the user utterances collected in step 1, and the output indicates an emotion tag (e.g., "sadness," "joy," etc.) and its degree.

[0299] Specific behavior:

[0300] The emotion engine detects the keyword "I want to die" and evaluates it as "sadness" or "despair." The analysis algorithm calculates an emotion score for each utterance.

[0301] Step 3: Message generation using a generative AI model

[0302] The server sends prompts to the generative AI model based on the evaluation results of the emotion engine, and generates a counseling message. The input is the evaluation result data from the emotion engine, and the output is the counseling message.

[0303] Specific behavior:

[0304] The server sends the prompt "If the user says 'I want to die,' please generate an appropriate counseling message" to the generative AI model. Based on the input, the generative AI model generates the message "I've been very worried about you lately. Please tell me your story."

[0305] Step 4: Send the message

[0306] The server then sends the generated counseling message to the user's information processing device. The input of this process is the message from the generative AI model, and the output is a notification that arrives on the user's device.

[0307] Specific behavior:

[0308] After generating the message, it sends it to the user's smartphone via an SMS API or notification service, saying, "I've been really worried about you lately. Let me tell you something."

[0309] Step 5: Receiving and viewing counseling messages

[0310] The terminal receives counseling messages sent from the server in real time and notifies the user. The input is the message sent from the server, and the output is the message notification displayed on the terminal.

[0311] Specific behavior:

[0312] The device uses push notifications to display the message, "I've been really worried about you lately. Tell me your story."

[0313] Step 6: User interaction and data entry

[0314] Users install the app and perform the initial setup. This setup allows the app to automatically monitor the user's search history and social media posts. Users can also input their emotional state and interact with the generated AI counselor.

[0315] Specific behavior:

[0316] The user enters in the app, "I've been feeling stressed about school lately," and the generated AI counselor responds, "What is causing you stress?" The input data is then sent back to the server for further analysis.

[0317] Step 7: Emergency response determination and notification

[0318] The server sends notifications to live experts if the evaluation results meet certain criteria. If the emotion engine detects a serious emotional state, it provides a means to intervene for users who are deemed to need urgent assistance.

[0319] Specific behavior:

[0320] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[0321] (Application example 2)

[0322] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] In today's digital society, it is extremely important to monitor a user's emotional state in real time and provide appropriate support and intervention. However, conventional systems have difficulty accurately assessing a user's emotional state and responding immediately. Furthermore, they lack elements that enhance the sense of familiarity between the user and the system, and their functionality for quickly responding to changes in the user's stress and emotions is limited. The present invention aims to solve these problems and provide more effective and accurate emotional state monitoring and counseling.

[0324] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0325] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a counseling message generated based on the evaluation result to the user's terminal; means for displaying the counseling message in real time on the user's terminal; means for using the evaluation result to make an emergency notification to a live counselor as needed; and means for enhancing a sense of intimacy with the user through a selectable and customizable virtual character. This makes it possible to accurately monitor the user's emotional state in real time and provide support at the appropriate time. Furthermore, the introduction of virtual characters enhances a sense of intimacy with the user and provides an environment in which the user feels natural interacting with the system.

[0326] "User search history" is a record of searches a user has conducted on the Internet.

[0327] "Comments on social networking services" refer to the text and comments posted by users on social networking services.

[0328] "Specific keywords" are specific words or phrases that the system places importance on.

[0329] "Mood" refers to the user's emotional or mental state.

[0330] The "evaluation result" is the result of analysis by the emotion engine and classification of the user's emotional state.

[0331] A "counseling message" is a message of advice or encouragement generated according to the user's emotional state.

[0332] "User's device" refers to a device used by a user, such as a mobile phone, tablet, or computer.

[0333] "Means for displaying in real time" is a function that allows the counseling message to be displayed to the user immediately.

[0334] "Emergency notification to manned counselors" is a function that makes emergency contact with experts when a critical emotional state is detected.

[0335] A "virtual character" is a digital avatar that can converse and interact with users.

[0336] "Means for enhancing familiarity" are functions and methods that make the user feel natural and familiar with the system.

[0337] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and generates appropriate counseling messages. Specific embodiments of this invention will be described below.

[0338] The server monitors users' search history and comments on social networking services, and analyzes the text data using a sentiment analysis library such as TextBlob. If the sentiment engine detects text containing specific keywords and evaluates it as a negative emotional state, it uses a generative AI model to generate an appropriate counseling message.

[0339] This message is sent to the user's device and displayed in real time. The user's device also has the function of sending collected data to the server, which periodically updates the user's emotional state. If the evaluation results meet certain conditions, the server will send an emergency notification to a manned counselor.

[0340] Users can begin using the system by installing the application and completing the initial setup. They can create and customize a virtual character and receive counseling messages through that character. This increases the sense of intimacy with the user, enabling accurate assessment of their emotional state and appropriate support.

[0341] As an example of specific processing, the server detects a user's social media posts containing keywords such as "I want to die" or "painful," and the emotion engine evaluates the emotions of "sadness" and "despair." As a result, a counseling message is generated saying, "I've been very worried about you lately. Please tell me what you think," and sent to the user's device. Furthermore, in the case of an emergency, a counselor is notified.

[0342] For example, you can input the following prompts into a generative AI model:

[0343] A user posted "I want to die" on social media. Please provide a counseling message to generate.

[0344] This makes it possible to monitor the user's emotional state in real time and provide support at the appropriate time. In addition, if the user feels natural and familiar with the system, accurate assessment of emotions and appropriate responses can be achieved.

[0345] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0346] Step 1:

[0347] The user installs the application and completes the initial setup, which includes downloading, installing, and configuring the application on the device, so the system is ready to monitor the user's search history and social media posts.

[0348] Input: User settings information

[0349] Output: Initialization completion status

[0350] Step 2:

[0351] The device monitors users' search history and comments on social networking services in real time. The collected text data is sent to a server. This data includes keywords searched by users and comments posted by users.

[0352] Input: User search history and social media posts

[0353] Output: Collected text data

[0354] Step 3:

[0355] The server analyzes the received text data using a sentiment analysis library such as TextBlob. If certain keywords are detected, the data is further analyzed by the sentiment engine to evaluate the user's emotional state.

[0356] Input: Collected text data

[0357] Output: Evaluation result of emotional state

[0358] Step 4:

[0359] The server uses a generative AI model to generate appropriate counseling messages based on the evaluation results. In this generation process, counseling messages that correspond to the user's emotional state are automatically created.

[0360] Input: Emotional state evaluation result

[0361] Output: The generated counseling message

[0362] Step 5:

[0363] The server sends the generated counseling message to the user's terminal, which displays the message in real time, allowing the user to receive the message immediately and respond as needed.

[0364] Input: Generated counseling message

[0365] Output: Messages displayed in real time

[0366] Step 6:

[0367] Users can read the counseling messages displayed on their devices, interact with the system as needed, or contact a live counselor. Users can also customize their virtual characters to enhance their sense of intimacy.

[0368] Input: Messages displayed in real time and customization information for virtual characters

[0369] Output: User responses and interaction data

[0370] Step 7:

[0371] The server collects the user's responses and dialogue data, and if the evaluation results meet certain conditions, it sends an emergency notification to a live counselor, allowing the live counselor to provide direct support to the user.

[0372] Input: User responses and interaction data

[0373] Output: Urgent notification to manned counselors

[0374] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0375] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0376] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0377] [Second embodiment]

[0378] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0379] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0380] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0381] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0382] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0383] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0384] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0385] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0386] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0387] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0388] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0389] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0390] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system then sends a notification to a live counselor. The system also allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0391] Server Processing

[0392] The server receives and analyzes the user's search history and comments on social networking services. If specific keywords are detected, a machine learning algorithm is used to evaluate the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[0393] Examples:

[0394] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[0395] Terminal handling

[0396] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[0397] Examples:

[0398] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0399] User operations

[0400] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0401] Examples:

[0402] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0403] Backup function

[0404] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then begins the process of providing direct support to the user.

[0405] Examples:

[0406] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[0407] Through these processes, the present invention is able to monitor the user's emotional state in real time, provide support at the appropriate time, and deal with serious situations.

[0408] The processing flow will be explained below.

[0409] Step 1:

[0410] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[0411] Step 2:

[0412] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[0413] Step 3:

[0414] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[0415] Step 4:

[0416] The server analyzes the received data, detects specific keywords, and uses machine learning algorithms to evaluate the user's emotional state.

[0417] Step 5:

[0418] The server uses a generative AI model to generate a counseling message based on the evaluation results, and the generated message is immediately sent to the device.

[0419] Step 6:

[0420] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[0421] Step 7:

[0422] The server again analyzes the user's reply data and generates and sends additional counseling messages as needed.

[0423] Step 8:

[0424] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[0425] Step 9:

[0426] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[0427] Step 10:

[0428] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then provides emergency dialogue and assistance.

[0429] Example 1

[0430] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0431] In modern society, as users make daily comments and searches over the Internet, there is a need for systems that can accurately grasp their emotional state and provide counseling messages at the appropriate time. However, existing systems have difficulty analyzing users' emotions in real time and taking appropriate action. Furthermore, they often lack emergency notification functions for dealing with serious situations and user-friendly interfaces. Therefore, there is an urgent need to develop a system that can maintain users' mental health and provide prompt and individualized support.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0433] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a generated counseling message to the user's terminal based on the evaluation result; means for periodically collecting the user's operation log and data; means for creating and customizing a virtual character; and means for using a generative AI model to generate the counseling message. This allows the system to grasp the user's emotional state in real time, provide an appropriate counseling message, and notify a live counselor as needed. Furthermore, by customizing the virtual character, the user can increase familiarity and promote interaction with the system.

[0434] A "user's search history" is a history of terms and phrases that a user searches for using an Internet search engine.

[0435] A "social networking service" is a platform that enables users to interact with other users online, including features such as posting, commenting, and messaging.

[0436] A "specific keyword" is a specific word or phrase that provides important clues when assessing a user's emotional state.

[0437] "Mood" refers to the user's psychological state and emotions.

[0438] "Means for evaluation" refers to methods or techniques for analyzing a user's emotional state based on collected data and obtaining a specific score or classification result.

[0439] A "counseling message" is a message that is generated according to the user's emotional state and provides psychological support and advice.

[0440] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0441] An "operation log" is recorded data related to a user's device operations, including app usage status and input content.

[0442] A "generative AI model" is a model that uses machine learning and artificial intelligence techniques to generate output for specific purposes.

[0443] A "virtual character" is a digital character image that can be used as an interface by a user and can be customized.

[0444] A "manned counselor" is a person who is actually a human counselor and provides direct support to the user.

[0445] A "means for sending notifications" is a method or technology for sending a message or alert to a specific person when a specific condition is met by the system.

[0446] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system notifies a live counselor. Furthermore, the system allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0447] Server Processing

[0448] The server uses Apache Kafka to receive the user's search history and social networking service comment data. The received data is temporarily stored and then used to evaluate the user's emotional state using a TensorFlow model. The evaluation is performed by detecting specific keywords (e.g., "I want to die" or "It's painful"). An analysis script preprocesses the data and inputs it into the model, which calculates an emotion score from the inference results. Based on the evaluation results, a counseling message is generated using a generative AI model (e.g., a GPT model) and sent to the user's device via the Firebase Cloud Messaging (FCM) API.

[0449] Specific examples

[0450] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[0451] Prompt Sentence Examples

[0452] "Create a message that will be generated when a user posts 'I want to die'."

[0453] Terminal handling

[0454] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. A background service runs and periodically saves the operation logs and search history in a local database and sends them to the server. It also displays counseling messages sent from the server to the user in real time.

[0455] Specific examples

[0456] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0457] Prompt Sentence Examples

[0458] "Generate a message to display if the user responds 'I'm a little tired.'"

[0459] User operations

[0460] Users install the app and complete the initial setup. Once setup is complete, a background task will start, automatically monitoring the user's search history and comments on social networking services. Users can also create and customize their own virtual characters, input their emotional state within the app, and interact with a generated AI counselor. On the character creation screen, users can select their avatar's appearance and clothing, preview their customizations in real time, and save them.

[0461] Specific examples

[0462] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0463] Prompt Sentence Examples

[0464] "Generate a response when a user types, 'School has been really stressful lately.'"

[0465] Backup function

[0466] If the evaluation result meets certain conditions (e.g., a high emotional score), the server uses the Twilio API to send an SOS notification to a live counselor. The live counselor receives the notification and begins the process of providing direct assistance to the user.

[0467] Specific examples

[0468] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[0469] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0470] Step 1:

[0471] The user installs the app and completes the initial setup.

[0472] Specific behavior: The user downloads and installs the app. When the app is launched for the first time, a setup wizard appears and the user grants the necessary permissions (e.g., Internet access, storage access, etc.). The user clicks the Finish setting button and the settings are saved.

[0473] Input: User interactions (touches, clicks), preference information.

[0474] Output: Setting completion flag, user initial setting data.

[0475] Step 2:

[0476] The device collects the user's search history and comments on social media in real time.

[0477] How it works: The device runs a background service that monitors the user's browser history and in-app social media posts to collect data, which is then stored in a local database.

[0478] Input: User search history, social media posts.

[0479] Output: Collected data stored in a local database.

[0480] Step 3:

[0481] The terminal periodically transmits the collected data to the server.

[0482] Specific operation: The device executes a scheduled task and uploads the collected data from the local database to the server. The upload is performed at the optimal time taking into account the network conditions.

[0483] Input: Collected data in local database.

[0484] Output: The data sent to the server.

[0485] Step 4:

[0486] The server temporarily stores the received data and prepares it for analysis.

[0487] Specific operation: The server receives data using Apache Kafka and stores it in temporary storage. The received data is preprocessed by the analysis script to prepare a dataset for analysis.

[0488] Input: Data sent from the terminal.

[0489] Output: Dataset for analysis.

[0490] Step 5:

[0491] The server analyzes the data and detects specific keywords.

[0492] How it works: The server inputs the preprocessed data into a TensorFlow model to detect specific keywords (e.g., "I want to die" or "It's painful"). It then scans the text data using regular expressions to extract matching keywords.

[0493] Input: Dataset for analysis.

[0494] Output: Detected results for specific keywords.

[0495] Step 6:

[0496] The server evaluates the user's emotional state.

[0497] Specific operation: Based on the detected keywords, the server uses a machine learning algorithm to evaluate the user's emotional state, and outputs the evaluation results as an emotion score or classification result.

[0498] Input: Detected results for a specific keyword.

[0499] Output: Evaluation result of the user's emotional state.

[0500] Step 7:

[0501] The server generates a counseling message based on the evaluation result.

[0502] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate an appropriate counseling message based on the evaluation results. A prompt sentence is input, and the model generates a message.

[0503] Input: User's emotional state assessment result.

[0504] Output: The generated counseling message.

[0505] Step 8:

[0506] The server sends a counseling message to the user's terminal.

[0507] Specific operation: The server uses the Firebase Cloud Messaging (FCM) API to send the generated counseling message to the user's device.

[0508] Input: The generated counseling message.

[0509] Output: The message sent to the user's terminal.

[0510] Step 9:

[0511] The terminal receives the counseling message and displays it to the user.

[0512] Specific behavior: The device receives the FCM notification and triggers a local notification. A message is displayed on a specific screen in the app according to the notification.

[0513] Input: The counseling message sent by the server.

[0514] Output: The message displayed to the user.

[0515] Step 10:

[0516] If the evaluation results meet certain conditions, the server sends an SOS notification to a manned counselor.

[0517] What it does: When the server detects a certain condition, such as a high emotion score, it uses the Twilio API to send an emergency notification to a live counselor.

[0518] Input: Evaluation results, such as high sentiment scores.

[0519] Output: SOS notification sent to manned counselors.

[0520] Step 11:

[0521] A live counselor receives the notification and initiates the process of providing direct assistance to the user.

[0522] Specific actions: When a counselor receives a notification, they will provide emergency assistance, such as calling the user directly, and take appropriate action if necessary.

[0523] Input: SOS notification from the server.

[0524] Output: Emergency assistance to the user.

[0525] (Application example 1)

[0526] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0527] Maintaining mental health is an important issue in modern society. In particular, with the spread of the Internet and social networking services, the information users send out on a daily basis often reveals mental distress and stress. However, collecting and analyzing this information in real time and providing appropriate counseling and support is currently difficult. Conventional systems lack the immediacy and accuracy required to provide effective mental support.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0529] In this invention, the server includes means for monitoring a user's search history and comments on a social networking service and detecting specific keywords, means for evaluating the user's emotional state based on the keywords detected by the above means, means for generating a counseling message based on the evaluation result and sending the message to the user's terminal, means for periodically collecting the user's posts via a social media API and evaluating the emotional state in real time, means for generating an AI counseling message based on the evaluation of the emotional state and sending the message to the user, and means for sending a notification to a human counselor according to the severity of the evaluated emotional state. This makes it possible to quickly and accurately evaluate the user's mental state and provide counseling messages and support at an appropriate time.

[0530] "Search history" is a history of keywords and phrases that a user has searched for on the Internet using a search engine.

[0531] A "social networking service" is a platform that allows users to communicate and share information with each other over the Internet.

[0532] A "specific keyword" is an important word or phrase that indicates a predetermined psychological state from a user's comments or search history.

[0533] "Evaluation means" refers to algorithms or programs that analyze collected data and determine the user's emotional state.

[0534] A "generated counseling message" is a message of advice or comfort that is generated for transmission to a user based on the emotional state determined by the evaluation means.

[0535] A "terminal" is a computer device used by a user, such as a smartphone, tablet, or PC.

[0536] A "social media API" is an application program interface for automatically obtaining data from social networking services.

[0537] "Mood assessment" is the process of determining a user's psychological health from their statements and behavior.

[0538] "AI counseling messages" are messages generated using artificial intelligence technology with the aim of providing psychological support to users.

[0539] A "manned counselor" is a human counselor who provides direct support and intervention when the user's emotional state is determined to be serious.

[0540] This system monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends a generated counseling message to the user's device. Furthermore, if the evaluation results meet certain conditions, the system also includes a function to notify a live counselor.

[0541] Server Processing

[0542] The server first collects the user's search history and comments on social networking services. This is done through social media APIs, and data is periodically retrieved. This data includes search keywords and the content of social media posts. The server then analyzes this data to determine whether specific keywords are included.

[0543] Sentiment analysis is performed on the analyzed data to evaluate the user's emotional state. Based on the evaluation results, an AI-generated counseling message is generated. This AI-generated counseling message is generated in real time using a generative AI model, and the content is determined by using appropriate prompt sentences.

[0544] The generated counseling message is sent to the user's terminal and notified to the user. Depending on the severity of the evaluation result, a notification is also sent to a live counselor.

[0545] Terminal handling

[0546] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to the server. Information entered by the user and operation logs are also collected periodically, and this data is used for analysis on the server. Counseling messages sent from the server are notified to the device and displayed to the user.

[0547] User operations

[0548] Users install the app and complete the initial setup. Once setup is complete, search history and comments on social networking services are automatically monitored. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0549] Specific examples

[0550] For example, if a user posts on social media, "I can't take it anymore," the content of the post is sent to the server via a social media API. The server performs sentiment analysis, assesses the severity, and determines it to be "high." Based on this result, an AI-generated counseling message is generated, and a message saying "Let me tell you something" is sent to the user's device. At the same time, because the condition is serious, a notification is also sent to a live counselor.

[0551] Prompt Sentence Examples

[0552] An example prompt might have the following format:

[0553] User ID: 12345

[0554] Post content: I can't take it anymore

[0555] In this way, the present invention is able to monitor the user's emotional state in real time, provide timely support and deal with critical situations.

[0556] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0557] Step 1:

[0558] The server receives user search history and comment data from social networking services. Specifically, it periodically retrieves user posts and comments using social media APIs. The input is user posted data, and the output is raw data to be analyzed.

[0559] Step 2:

[0560] The server detects specific keywords from the received data using a text analysis engine. The input is raw data, and the output is data containing the detected keywords. Specifically, keyword extraction is performed using natural language processing tools.

[0561] Step 3:

[0562] The server evaluates the user's emotional state based on the detected keywords. It uses a sentiment analysis algorithm to analyze the user's posts and evaluate the emotional state with a score. The input is data containing keywords, and the output is a rating score.

[0563] Step 4:

[0564] The server generates an AI-generated counseling message based on the evaluation score. It uses a generative AI model to create an appropriate message according to the situation. The input is the evaluation score and prompt, and the output is the counseling message. Specifically, based on the example prompt, data is input into the AI-generated model and the generated message is obtained.

[0565] Step 5:

[0566] The server sends the generated counseling message to the user's device using a message sending API. The input is the generated counseling message, and the output is a notification to the user's device.

[0567] Step 6:

[0568] If the evaluation score meets certain conditions, the server sends a notification to a human counselor. The counselor is contacted using a notification system. The input is the evaluation score and the user's posted data, and the output is an alert to the counselor.

[0569] Step 7:

[0570] The user receives the counseling message sent to their terminal. The user terminal displays the message through the notification function. The input is the counseling message sent from the server, and the output is the display to the user. Specifically, the user checks the message and replies if necessary.

[0571] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0572] This invention combines a system that monitors a user's search history and comments on social networking services, detects specific keywords, and evaluates the user's emotional state with an emotion engine. The generated counseling message is sent to the user's device, and if necessary, a live counselor is notified. The system also allows users to create and customize virtual characters to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0573] Server Processing

[0574] The server receives the user's search history and comments on social networking services and analyzes this data using an emotion engine. The emotion engine has an algorithm that recognizes emotions from the user's comments and behavioral patterns, and when specific keywords are detected, it evaluates the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[0575] Examples:

[0576] The server detects users' social media posts containing keywords such as "I want to die" or "It's painful," and the emotion engine evaluates the emotions of "sadness" and "despair." Based on this, a counseling message is generated, such as "I've been very worried about you lately. Please tell me what you think," and sent to the user's device.

[0577] Terminal handling

[0578] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[0579] Examples:

[0580] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0581] User operations

[0582] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0583] Examples:

[0584] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0585] Additional Server Processing

[0586] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor. After receiving the notification, the on-site counselor initiates the procedure to provide direct assistance to the user. The emotion engine uses accumulated user data to track changes in the user's emotions and adjusts the counseling method accordingly.

[0587] Examples:

[0588] The server detects multiple serious posts from the user, such as "I can't take it anymore," and the emotion engine evaluates this as "despair" and determines that there is a serious problem. This causes an emergency notification to be sent to a manned counselor, who then provides emergency support, such as calling the user directly.

[0589] This allows the present invention to monitor the user's emotional state in real time, use the emotion engine to make a highly accurate assessment, and provide timely support to deal with serious situations.

[0590] The processing flow will be explained below.

[0591] Step 1:

[0592] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[0593] Step 2:

[0594] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[0595] Step 3:

[0596] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[0597] Step 4:

[0598] The server analyzes the received data and uses an emotion engine to detect specific keywords and recognize the user's emotions. For example, keywords such as "I want to die" and "It's painful" can be used to identify emotions such as "sadness" and "despair."

[0599] Step 5:

[0600] The server uses a generative AI model to generate a counseling message based on the recognized emotions and evaluation results, and the generated message is immediately sent to the device.

[0601] Step 6:

[0602] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[0603] Step 7:

[0604] The server analyzes the user's reply data again and generates and sends additional counseling messages as needed. For example, if the user replies, "I'm very tired today," the server generates a new message such as, "Do you have time to take a short break?"

[0605] Step 8:

[0606] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[0607] Step 9:

[0608] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[0609] Step 10:

[0610] The server accumulates the received periodic input data and tracks changes in the user's emotions through an emotion engine, which allows long-term emotional trends to be evaluated.

[0611] Step 11:

[0612] If the evaluation results meet certain criteria, the server sends an SOS notification to a live counselor, who then provides emergency dialogue and support. For example, if the server detects multiple serious statements such as "I can't take it anymore," it will send an emergency notification and have a counselor contact the user directly.

[0613] These steps enable the present invention to monitor the user's emotional state in real time, provide timely support, and use the emotion engine to make accurate assessments and handle critical situations.

[0614] Example 2

[0615] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0616] In modern society, fluctuations in users' emotional states have become a major problem, and early detection and appropriate response are particularly required for users experiencing stress or depression. However, current technology lacks a system that can accurately monitor these emotional fluctuations in real time and provide appropriate support. Furthermore, other methods can be distrustful of users and impose a heavy burden, so further improvement is needed.

[0617] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0618] In this invention, the server includes means for monitoring a user's information usage history and comments on the networking service and detecting specific words and phrases, means for evaluating the user's emotional state based on the words and phrases detected by said means, means for transmitting a corresponding message generated based on the evaluation result to the user's information processing device, and means for periodically collecting user input data and improving the accuracy of the evaluation result. This makes it possible to monitor the user's emotional state in real time, perform highly accurate evaluations, and provide appropriate messages at appropriate times.

[0619] "User" means an individual who uses the System to input information and receive services.

[0620] "Information usage history" refers to data including the user's search and access history on the Internet.

[0621] "Networking service" refers to an online platform, such as a social networking site or messaging app, that enables users to exchange information with other users.

[0622] A "phrase" refers to a word or phrase that has a specific meaning in a user's utterance or input data.

[0623] "Emotional state" refers to the user's mental and psychological state, and includes emotions such as "joy," "sadness," and "anger."

[0624] The "evaluation result" refers to the result of the determination of the emotional state obtained by the emotion engine through analysis of the user's words and actions.

[0625] "Response message" refers to a message appropriate to the user's emotional state, created by the generative AI model based on the evaluation results.

[0626] "Information processing device" refers to devices used by users, such as smartphones, tablets, and personal computers.

[0627] "Input data" refers to all information that a user inputs into a system, including text, audio, images, etc.

[0628] An "emotion engine" refers to a system that integrates algorithms and technologies to analyze emotions from users' statements and actions.

[0629] A "generative AI model" refers to an artificial intelligence model that generates appropriate messages in natural language based on input data.

[0630] "Virtual presence" refers to an in-app character or avatar that users can customize.

[0631] "Manned" refers to human experts who monitor the system and intervene in emergencies.

[0632] This invention combines an emotion engine with a system that monitors a user's search history and comments on social networking services (SNS), detects specific words, and evaluates the user's emotional state. The generated counseling message is sent to the user's information processing device, and if necessary, notifies a human expert. The system also allows users to create and customize virtual beings to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0633] The server receives information through an API that collects the user's information usage history and SNS comment data. The collected data is passed to an emotion engine, which uses natural language processing technology to identify the user's emotional state from the comments. Specifically, context analysis and keyword extraction are performed. Once the evaluation results are obtained, a prompt is sent to the generative AI model based on the results, which generates a counseling message. The generated message is then sent to the user's information processing device.

[0634] Examples:

[0635] The server detects social media posts from users that contain phrases such as "I want to die" or "It's painful." The emotion engine evaluates these as "sadness" or "despair." Based on the evaluation results, it sends a prompt to the generative AI model saying, "If the user says 'I want to die,' please generate an appropriate counseling message." The generative AI model then generates a message such as, "I've been very worried about you lately. Please tell me your story," and sends it to the user's information processing device.

[0636] The device monitors the user's search history and social media postings in real time, periodically sending the collected data to the server, and also receives counseling messages sent from the server in real time and notifies the user.

[0637] Users install the app and perform the initial setup. This setup prepares the app to automatically monitor the user's search history and social media posts. Users can create a virtual presence within the app and customize it to their liking. They can also input their emotional state within the app and have conversations with a generated AI counselor.

[0638] Examples:

[0639] Within the app, users can choose and customize the appearance of their virtual presence, then type in something like, "I've been feeling stressed about school lately," to which the generated AI counselor responds, "What's causing you stress?"

[0640] Furthermore, if the evaluation results meet certain conditions, the server can send a notification to a human expert. If the emotion engine detects a serious emotional state, the human expert will respond directly to the user who is deemed to need urgent assistance.

[0641] Examples:

[0642] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[0643] As a result, the present invention makes it possible to monitor a user's emotional state in real time, use an emotion engine to make a highly accurate assessment, and provide support at the appropriate time to deal with serious situations.

[0644] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0645] Step 1: Data collection

[0646] The server receives information through an API to collect user information usage history and comment data from networking services. This process inputs the user's social media posts and browser history. The input data is received in JSON format and stored in a database.

[0647] Specific behavior:

[0648] The server uses the Twitter API to retrieve the user's recent tweets, including the tweet text, timestamp, and user ID.

[0649] Step 2: Data analysis

[0650] The server passes the collected data to the emotion engine for analysis. The emotion engine uses natural language processing technology to identify emotions from the utterances. The input to this analysis process is the text data of the user utterances collected in step 1, and the output indicates an emotion tag (e.g., "sadness," "joy," etc.) and its degree.

[0651] Specific behavior:

[0652] The emotion engine detects the keyword "I want to die" and evaluates it as "sadness" or "despair." The analysis algorithm calculates an emotion score for each utterance.

[0653] Step 3: Message generation using a generative AI model

[0654] The server sends prompts to the generative AI model based on the evaluation results of the emotion engine, and generates a counseling message. The input is the evaluation result data from the emotion engine, and the output is the counseling message.

[0655] Specific behavior:

[0656] The server sends the prompt "If the user says 'I want to die,' please generate an appropriate counseling message" to the generative AI model. Based on the input, the generative AI model generates the message "I've been very worried about you lately. Please tell me your story."

[0657] Step 4: Send the message

[0658] The server then sends the generated counseling message to the user's information processing device. The input of this process is the message from the generative AI model, and the output is a notification that arrives on the user's device.

[0659] Specific behavior:

[0660] After generating the message, it sends it to the user's smartphone via an SMS API or notification service, saying, "I've been really worried about you lately. Let me tell you something."

[0661] Step 5: Receiving and viewing counseling messages

[0662] The terminal receives counseling messages sent from the server in real time and notifies the user. The input is the message sent from the server, and the output is the message notification displayed on the terminal.

[0663] Specific behavior:

[0664] The device uses push notifications to display the message, "I've been really worried about you lately. Tell me your story."

[0665] Step 6: User interaction and data entry

[0666] Users install the app and perform the initial setup. This setup allows the app to automatically monitor the user's search history and social media posts. Users can also input their emotional state and interact with the generated AI counselor.

[0667] Specific behavior:

[0668] The user enters in the app, "I've been feeling stressed about school lately," and the generated AI counselor responds, "What is causing you stress?" The input data is then sent back to the server for further analysis.

[0669] Step 7: Emergency response determination and notification

[0670] The server sends notifications to live experts if the evaluation results meet certain criteria. If the emotion engine detects a serious emotional state, it provides a means to intervene for users who are deemed to need urgent assistance.

[0671] Specific behavior:

[0672] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[0673] (Application example 2)

[0674] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] In today's digital society, it is extremely important to monitor a user's emotional state in real time and provide appropriate support and intervention. However, conventional systems have difficulty accurately assessing a user's emotional state and responding immediately. Furthermore, they lack elements that enhance the sense of familiarity between the user and the system, and their functionality for quickly responding to changes in the user's stress and emotions is limited. The present invention aims to solve these problems and provide more effective and accurate emotional state monitoring and counseling.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0677] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a counseling message generated based on the evaluation result to the user's terminal; means for displaying the counseling message in real time on the user's terminal; means for using the evaluation result to make an emergency notification to a live counselor as needed; and means for enhancing a sense of intimacy with the user through a selectable and customizable virtual character. This makes it possible to accurately monitor the user's emotional state in real time and provide support at the appropriate time. Furthermore, the introduction of virtual characters enhances a sense of intimacy with the user and provides an environment in which the user feels natural interacting with the system.

[0678] "User search history" is a record of searches a user has conducted on the Internet.

[0679] "Comments on social networking services" refer to the text and comments posted by users on social networking services.

[0680] "Specific keywords" are specific words or phrases that the system places importance on.

[0681] "Mood" refers to the user's emotional or mental state.

[0682] The "evaluation result" is the result of analysis by the emotion engine and classification of the user's emotional state.

[0683] A "counseling message" is a message of advice or encouragement generated according to the user's emotional state.

[0684] "User's device" refers to a device used by a user, such as a mobile phone, tablet, or computer.

[0685] "Means for displaying in real time" is a function that allows the counseling message to be displayed to the user immediately.

[0686] "Emergency notification to manned counselors" is a function that makes emergency contact with experts when a critical emotional state is detected.

[0687] A "virtual character" is a digital avatar that can converse and interact with users.

[0688] "Means for enhancing familiarity" are functions and methods that make the user feel natural and familiar with the system.

[0689] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and generates appropriate counseling messages. Specific embodiments of this invention will be described below.

[0690] The server monitors users' search history and comments on social networking services, and analyzes the text data using a sentiment analysis library such as TextBlob. If the sentiment engine detects text containing specific keywords and evaluates it as a negative emotional state, it uses a generative AI model to generate an appropriate counseling message.

[0691] This message is sent to the user's device and displayed in real time. The user's device also has the function of sending collected data to the server, which periodically updates the user's emotional state. If the evaluation results meet certain conditions, the server will send an emergency notification to a manned counselor.

[0692] Users can begin using the system by installing the application and completing the initial setup. They can create and customize a virtual character and receive counseling messages through that character. This increases the sense of intimacy with the user, enabling accurate assessment of their emotional state and appropriate support.

[0693] As an example of specific processing, the server detects a user's social media posts containing keywords such as "I want to die" or "painful," and the emotion engine evaluates the emotions of "sadness" and "despair." As a result, a counseling message is generated saying, "I've been very worried about you lately. Please tell me what you think," and sent to the user's device. Furthermore, in the case of an emergency, a counselor is notified.

[0694] For example, you can input the following prompts into a generative AI model:

[0695] A user posted "I want to die" on social media. Please provide a counseling message to generate.

[0696] This makes it possible to monitor the user's emotional state in real time and provide support at the appropriate time. In addition, if the user feels natural and familiar with the system, accurate assessment of emotions and appropriate responses can be achieved.

[0697] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0698] Step 1:

[0699] The user installs the application and completes the initial setup, which includes downloading, installing, and configuring the application on the device, so the system is ready to monitor the user's search history and social media posts.

[0700] Input: User settings information

[0701] Output: Initialization completion status

[0702] Step 2:

[0703] The device monitors users' search history and comments on social networking services in real time. The collected text data is sent to a server. This data includes keywords searched by users and comments posted by users.

[0704] Input: User search history and social media posts

[0705] Output: Collected text data

[0706] Step 3:

[0707] The server analyzes the received text data using a sentiment analysis library such as TextBlob. If certain keywords are detected, the data is further analyzed by the sentiment engine to evaluate the user's emotional state.

[0708] Input: Collected text data

[0709] Output: Evaluation result of emotional state

[0710] Step 4:

[0711] The server uses a generative AI model to generate appropriate counseling messages based on the evaluation results. In this generation process, counseling messages that correspond to the user's emotional state are automatically created.

[0712] Input: Emotional state evaluation result

[0713] Output: The generated counseling message

[0714] Step 5:

[0715] The server sends the generated counseling message to the user's terminal, which displays the message in real time, allowing the user to receive the message immediately and respond as needed.

[0716] Input: Generated counseling message

[0717] Output: Messages displayed in real time

[0718] Step 6:

[0719] Users can read the counseling messages displayed on their devices, interact with the system as needed, or contact a live counselor. Users can also customize their virtual characters to enhance their sense of intimacy.

[0720] Input: Messages displayed in real time and customization information for virtual characters

[0721] Output: User responses and interaction data

[0722] Step 7:

[0723] The server collects the user's responses and dialogue data, and if the evaluation results meet certain conditions, it sends an emergency notification to a live counselor, allowing the live counselor to provide direct support to the user.

[0724] Input: User responses and interaction data

[0725] Output: Urgent notification to manned counselors

[0726] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0727] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0728] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0729] [Third embodiment]

[0730] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0731] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0732] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0733] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0734] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0735] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0736] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0737] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0738] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0739] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0740] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0741] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0742] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system then sends a notification to a live counselor. The system also allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0743] Server Processing

[0744] The server receives and analyzes the user's search history and comments on social networking services. If specific keywords are detected, a machine learning algorithm is used to evaluate the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[0745] Examples:

[0746] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[0747] Terminal handling

[0748] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[0749] Examples:

[0750] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0751] User operations

[0752] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0753] Examples:

[0754] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0755] Backup function

[0756] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then begins the process of providing direct support to the user.

[0757] Examples:

[0758] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[0759] Through these processes, the present invention is able to monitor the user's emotional state in real time, provide support at the appropriate time, and deal with serious situations.

[0760] The processing flow will be explained below.

[0761] Step 1:

[0762] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[0763] Step 2:

[0764] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[0765] Step 3:

[0766] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[0767] Step 4:

[0768] The server analyzes the received data, detects specific keywords, and uses machine learning algorithms to evaluate the user's emotional state.

[0769] Step 5:

[0770] The server uses a generative AI model to generate a counseling message based on the evaluation results, and the generated message is immediately sent to the device.

[0771] Step 6:

[0772] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[0773] Step 7:

[0774] The server again analyzes the user's reply data and generates and sends additional counseling messages as needed.

[0775] Step 8:

[0776] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[0777] Step 9:

[0778] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[0779] Step 10:

[0780] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then provides emergency dialogue and assistance.

[0781] Example 1

[0782] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0783] In modern society, as users make daily comments and searches over the Internet, there is a need for systems that can accurately grasp their emotional state and provide counseling messages at the appropriate time. However, existing systems have difficulty analyzing users' emotions in real time and taking appropriate action. Furthermore, they often lack emergency notification functions for dealing with serious situations and user-friendly interfaces. Therefore, there is an urgent need to develop a system that can maintain users' mental health and provide prompt and individualized support.

[0784] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0785] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a generated counseling message to the user's terminal based on the evaluation result; means for periodically collecting the user's operation log and data; means for creating and customizing a virtual character; and means for using a generative AI model to generate the counseling message. This allows the system to grasp the user's emotional state in real time, provide an appropriate counseling message, and notify a live counselor as needed. Furthermore, by customizing the virtual character, the user can increase familiarity and promote interaction with the system.

[0786] A "user's search history" is a history of terms and phrases that a user searches for using an Internet search engine.

[0787] A "social networking service" is a platform that enables users to interact with other users online, including features such as posting, commenting, and messaging.

[0788] A "specific keyword" is a specific word or phrase that provides important clues when assessing a user's emotional state.

[0789] "Mood" refers to the user's psychological state and emotions.

[0790] "Means for evaluation" refers to methods or techniques for analyzing a user's emotional state based on collected data and obtaining a specific score or classification result.

[0791] A "counseling message" is a message that is generated according to the user's emotional state and provides psychological support and advice.

[0792] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0793] An "operation log" is recorded data related to a user's device operations, including app usage status and input content.

[0794] A "generative AI model" is a model that uses machine learning and artificial intelligence techniques to generate output for specific purposes.

[0795] A "virtual character" is a digital character image that can be used as an interface by a user and can be customized.

[0796] A "manned counselor" is a person who is actually a human counselor and provides direct support to the user.

[0797] A "means for sending notifications" is a method or technology for sending a message or alert to a specific person when a specific condition is met by the system.

[0798] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system notifies a live counselor. Furthermore, the system allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0799] Server Processing

[0800] The server uses Apache Kafka to receive the user's search history and social networking service comment data. The received data is temporarily stored and then used to evaluate the user's emotional state using a TensorFlow model. The evaluation is performed by detecting specific keywords (e.g., "I want to die" or "It's painful"). An analysis script preprocesses the data and inputs it into the model, which calculates an emotion score from the inference results. Based on the evaluation results, a counseling message is generated using a generative AI model (e.g., a GPT model) and sent to the user's device via the Firebase Cloud Messaging (FCM) API.

[0801] Specific examples

[0802] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[0803] Prompt Sentence Examples

[0804] "Create a message that will be generated when a user posts 'I want to die'."

[0805] Terminal handling

[0806] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. A background service runs and periodically saves the operation logs and search history in a local database and sends them to the server. It also displays counseling messages sent from the server to the user in real time.

[0807] Specific examples

[0808] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0809] Prompt Sentence Examples

[0810] "Generate a message to display if the user responds 'I'm a little tired.'"

[0811] User operations

[0812] Users install the app and complete the initial setup. Once setup is complete, a background task will start, automatically monitoring the user's search history and comments on social networking services. Users can also create and customize their own virtual characters, input their emotional state within the app, and interact with a generated AI counselor. On the character creation screen, users can select their avatar's appearance and clothing, preview their customizations in real time, and save them.

[0813] Specific examples

[0814] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0815] Prompt Sentence Examples

[0816] "Generate a response when a user types, 'School has been really stressful lately.'"

[0817] Backup function

[0818] If the evaluation result meets certain conditions (e.g., a high emotional score), the server uses the Twilio API to send an SOS notification to a live counselor. The live counselor receives the notification and begins the process of providing direct assistance to the user.

[0819] Specific examples

[0820] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[0821] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0822] Step 1:

[0823] The user installs the app and completes the initial setup.

[0824] Specific behavior: The user downloads and installs the app. When the app is launched for the first time, a setup wizard appears and the user grants the necessary permissions (e.g., Internet access, storage access, etc.). The user clicks the Finish setting button and the settings are saved.

[0825] Input: User interactions (touches, clicks), preference information.

[0826] Output: Setting completion flag, user initial setting data.

[0827] Step 2:

[0828] The device collects the user's search history and comments on social media in real time.

[0829] How it works: The device runs a background service that monitors the user's browser history and in-app social media posts to collect data, which is then stored in a local database.

[0830] Input: User search history, social media posts.

[0831] Output: Collected data stored in a local database.

[0832] Step 3:

[0833] The terminal periodically transmits the collected data to the server.

[0834] Specific operation: The device executes a scheduled task and uploads the collected data from the local database to the server. The upload is performed at the optimal time taking into account the network conditions.

[0835] Input: Collected data in local database.

[0836] Output: The data sent to the server.

[0837] Step 4:

[0838] The server temporarily stores the received data and prepares it for analysis.

[0839] Specific operation: The server receives data using Apache Kafka and stores it in temporary storage. The received data is preprocessed by the analysis script to prepare a dataset for analysis.

[0840] Input: Data sent from the terminal.

[0841] Output: Dataset for analysis.

[0842] Step 5:

[0843] The server analyzes the data and detects specific keywords.

[0844] How it works: The server inputs the preprocessed data into a TensorFlow model to detect specific keywords (e.g., "I want to die" or "It's painful"). It then scans the text data using regular expressions to extract matching keywords.

[0845] Input: Dataset for analysis.

[0846] Output: Detected results for specific keywords.

[0847] Step 6:

[0848] The server evaluates the user's emotional state.

[0849] Specific operation: Based on the detected keywords, the server uses a machine learning algorithm to evaluate the user's emotional state, and outputs the evaluation results as an emotion score or classification result.

[0850] Input: Detected results for a specific keyword.

[0851] Output: Evaluation result of the user's emotional state.

[0852] Step 7:

[0853] The server generates a counseling message based on the evaluation result.

[0854] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate an appropriate counseling message based on the evaluation results. A prompt sentence is input, and the model generates a message.

[0855] Input: User's emotional state assessment result.

[0856] Output: The generated counseling message.

[0857] Step 8:

[0858] The server sends a counseling message to the user's terminal.

[0859] Specific operation: The server uses the Firebase Cloud Messaging (FCM) API to send the generated counseling message to the user's device.

[0860] Input: The generated counseling message.

[0861] Output: The message sent to the user's terminal.

[0862] Step 9:

[0863] The terminal receives the counseling message and displays it to the user.

[0864] Specific behavior: The device receives the FCM notification and triggers a local notification. A message is displayed on a specific screen in the app according to the notification.

[0865] Input: The counseling message sent by the server.

[0866] Output: The message displayed to the user.

[0867] Step 10:

[0868] If the evaluation results meet certain conditions, the server sends an SOS notification to a manned counselor.

[0869] What it does: When the server detects a certain condition, such as a high emotion score, it uses the Twilio API to send an emergency notification to a live counselor.

[0870] Input: Evaluation results, such as high sentiment scores.

[0871] Output: SOS notification sent to manned counselors.

[0872] Step 11:

[0873] A live counselor receives the notification and initiates the process of providing direct assistance to the user.

[0874] Specific actions: When a counselor receives a notification, they will provide emergency assistance, such as calling the user directly, and take appropriate action if necessary.

[0875] Input: SOS notification from the server.

[0876] Output: Emergency assistance to the user.

[0877] (Application example 1)

[0878] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0879] Maintaining mental health is an important issue in modern society. In particular, with the spread of the Internet and social networking services, the information users send out on a daily basis often reveals mental distress and stress. However, collecting and analyzing this information in real time and providing appropriate counseling and support is currently difficult. Conventional systems lack the immediacy and accuracy required to provide effective mental support.

[0880] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0881] In this invention, the server includes means for monitoring a user's search history and comments on a social networking service and detecting specific keywords, means for evaluating the user's emotional state based on the keywords detected by the above means, means for generating a counseling message based on the evaluation result and sending the message to the user's terminal, means for periodically collecting the user's posts via a social media API and evaluating the emotional state in real time, means for generating an AI counseling message based on the evaluation of the emotional state and sending the message to the user, and means for sending a notification to a human counselor according to the severity of the evaluated emotional state. This makes it possible to quickly and accurately evaluate the user's mental state and provide counseling messages and support at an appropriate time.

[0882] "Search history" is a history of keywords and phrases that a user has searched for on the Internet using a search engine.

[0883] A "social networking service" is a platform that allows users to communicate and share information with each other over the Internet.

[0884] A "specific keyword" is an important word or phrase that indicates a predetermined psychological state from a user's comments or search history.

[0885] "Evaluation means" refers to algorithms or programs that analyze collected data and determine the user's emotional state.

[0886] A "generated counseling message" is a message of advice or comfort that is generated for transmission to a user based on the emotional state determined by the evaluation means.

[0887] A "terminal" is a computer device used by a user, such as a smartphone, tablet, or PC.

[0888] A "social media API" is an application program interface for automatically obtaining data from social networking services.

[0889] "Mood assessment" is the process of determining a user's psychological health from their statements and behavior.

[0890] "AI counseling messages" are messages generated using artificial intelligence technology with the aim of providing psychological support to users.

[0891] A "manned counselor" is a human counselor who provides direct support and intervention when the user's emotional state is determined to be serious.

[0892] This system monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends a generated counseling message to the user's device. Furthermore, if the evaluation results meet certain conditions, the system also includes a function to notify a live counselor.

[0893] Server Processing

[0894] The server first collects the user's search history and comments on social networking services. This is done through social media APIs, and data is periodically retrieved. This data includes search keywords and the content of social media posts. The server then analyzes this data to determine whether specific keywords are included.

[0895] Sentiment analysis is performed on the analyzed data to evaluate the user's emotional state. Based on the evaluation results, an AI-generated counseling message is generated. This AI-generated counseling message is generated in real time using a generative AI model, and the content is determined by using appropriate prompt sentences.

[0896] The generated counseling message is sent to the user's terminal and notified to the user. Depending on the severity of the evaluation result, a notification is also sent to a live counselor.

[0897] Terminal handling

[0898] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to the server. Information entered by the user and operation logs are also collected periodically, and this data is used for analysis on the server. Counseling messages sent from the server are notified to the device and displayed to the user.

[0899] User operations

[0900] Users install the app and complete the initial setup. Once setup is complete, search history and comments on social networking services are automatically monitored. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0901] Specific examples

[0902] For example, if a user posts on social media, "I can't take it anymore," the content of the post is sent to the server via a social media API. The server performs sentiment analysis, assesses the severity, and determines it to be "high." Based on this result, an AI-generated counseling message is generated, and a message saying "Let me tell you something" is sent to the user's device. At the same time, because the condition is serious, a notification is also sent to a live counselor.

[0903] Prompt Sentence Examples

[0904] An example prompt might have the following format:

[0905] User ID: 12345

[0906] Post content: I can't take it anymore

[0907] In this way, the present invention is able to monitor the user's emotional state in real time, provide timely support and deal with critical situations.

[0908] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0909] Step 1:

[0910] The server receives user search history and comment data from social networking services. Specifically, it periodically retrieves user posts and comments using social media APIs. The input is user posted data, and the output is raw data to be analyzed.

[0911] Step 2:

[0912] The server detects specific keywords from the received data using a text analysis engine. The input is raw data, and the output is data containing the detected keywords. Specifically, keyword extraction is performed using natural language processing tools.

[0913] Step 3:

[0914] The server evaluates the user's emotional state based on the detected keywords. It uses a sentiment analysis algorithm to analyze the user's posts and evaluate the emotional state with a score. The input is data containing keywords, and the output is a rating score.

[0915] Step 4:

[0916] The server generates an AI-generated counseling message based on the evaluation score. It uses a generative AI model to create an appropriate message according to the situation. The input is the evaluation score and prompt, and the output is the counseling message. Specifically, based on the example prompt, data is input into the AI-generated model and the generated message is obtained.

[0917] Step 5:

[0918] The server sends the generated counseling message to the user's device using a message sending API. The input is the generated counseling message, and the output is a notification to the user's device.

[0919] Step 6:

[0920] If the evaluation score meets certain conditions, the server sends a notification to a human counselor. The counselor is contacted using a notification system. The input is the evaluation score and the user's posted data, and the output is an alert to the counselor.

[0921] Step 7:

[0922] The user receives the counseling message sent to their terminal. The user terminal displays the message through the notification function. The input is the counseling message sent from the server, and the output is the display to the user. Specifically, the user checks the message and replies if necessary.

[0923] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0924] This invention combines a system that monitors a user's search history and comments on social networking services, detects specific keywords, and evaluates the user's emotional state with an emotion engine. The generated counseling message is sent to the user's device, and if necessary, a live counselor is notified. The system also allows users to create and customize virtual characters to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0925] Server Processing

[0926] The server receives the user's search history and comments on social networking services and analyzes this data using an emotion engine. The emotion engine has an algorithm that recognizes emotions from the user's comments and behavioral patterns, and when specific keywords are detected, it evaluates the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[0927] Examples:

[0928] The server detects users' social media posts containing keywords such as "I want to die" or "It's painful," and the emotion engine evaluates the emotions of "sadness" and "despair." Based on this, a counseling message is generated, such as "I've been very worried about you lately. Please tell me what you think," and sent to the user's device.

[0929] Terminal handling

[0930] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[0931] Examples:

[0932] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[0933] User operations

[0934] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[0935] Examples:

[0936] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[0937] Additional Server Processing

[0938] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor. After receiving the notification, the on-site counselor initiates the procedure to provide direct assistance to the user. The emotion engine uses accumulated user data to track changes in the user's emotions and adjusts the counseling method accordingly.

[0939] Examples:

[0940] The server detects multiple serious posts from the user, such as "I can't take it anymore," and the emotion engine evaluates this as "despair" and determines that there is a serious problem. This causes an emergency notification to be sent to a manned counselor, who then provides emergency support, such as calling the user directly.

[0941] This allows the present invention to monitor the user's emotional state in real time, use the emotion engine to make a highly accurate assessment, and provide timely support to deal with serious situations.

[0942] The processing flow will be explained below.

[0943] Step 1:

[0944] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[0945] Step 2:

[0946] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[0947] Step 3:

[0948] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[0949] Step 4:

[0950] The server analyzes the received data and uses an emotion engine to detect specific keywords and recognize the user's emotions. For example, keywords such as "I want to die" and "It's painful" can be used to identify emotions such as "sadness" and "despair."

[0951] Step 5:

[0952] The server uses a generative AI model to generate a counseling message based on the recognized emotions and evaluation results, and the generated message is immediately sent to the device.

[0953] Step 6:

[0954] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[0955] Step 7:

[0956] The server analyzes the user's reply data again and generates and sends additional counseling messages as needed. For example, if the user replies, "I'm very tired today," the server generates a new message such as, "Do you have time to take a short break?"

[0957] Step 8:

[0958] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[0959] Step 9:

[0960] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[0961] Step 10:

[0962] The server accumulates the received periodic input data and tracks changes in the user's emotions through an emotion engine, which allows long-term emotional trends to be evaluated.

[0963] Step 11:

[0964] If the evaluation results meet certain criteria, the server sends an SOS notification to a live counselor, who then provides emergency dialogue and support. For example, if the server detects multiple serious statements such as "I can't take it anymore," it will send an emergency notification and have a counselor contact the user directly.

[0965] These steps enable the present invention to monitor the user's emotional state in real time, provide timely support, and use the emotion engine to make accurate assessments and handle critical situations.

[0966] Example 2

[0967] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] In modern society, fluctuations in users' emotional states have become a major problem, and early detection and appropriate response are particularly required for users experiencing stress or depression. However, current technology lacks a system that can accurately monitor these emotional fluctuations in real time and provide appropriate support. Furthermore, other methods can be distrustful of users and impose a heavy burden, so further improvement is needed.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0970] In this invention, the server includes means for monitoring a user's information usage history and comments on the networking service and detecting specific words and phrases, means for evaluating the user's emotional state based on the words and phrases detected by said means, means for transmitting a corresponding message generated based on the evaluation result to the user's information processing device, and means for periodically collecting user input data and improving the accuracy of the evaluation result. This makes it possible to monitor the user's emotional state in real time, perform highly accurate evaluations, and provide appropriate messages at appropriate times.

[0971] "User" means an individual who uses the System to input information and receive services.

[0972] "Information usage history" refers to data including the user's search and access history on the Internet.

[0973] "Networking service" refers to an online platform, such as a social networking site or messaging app, that enables users to exchange information with other users.

[0974] A "phrase" refers to a word or phrase that has a specific meaning in a user's utterance or input data.

[0975] "Emotional state" refers to the user's mental and psychological state, and includes emotions such as "joy," "sadness," and "anger."

[0976] The "evaluation result" refers to the result of the determination of the emotional state obtained by the emotion engine through analysis of the user's words and actions.

[0977] "Response message" refers to a message appropriate to the user's emotional state, created by the generative AI model based on the evaluation results.

[0978] "Information processing device" refers to devices used by users, such as smartphones, tablets, and personal computers.

[0979] "Input data" refers to all information that a user inputs into a system, including text, audio, images, etc.

[0980] An "emotion engine" refers to a system that integrates algorithms and technologies to analyze emotions from users' statements and actions.

[0981] A "generative AI model" refers to an artificial intelligence model that generates appropriate messages in natural language based on input data.

[0982] "Virtual presence" refers to an in-app character or avatar that users can customize.

[0983] "Manned" refers to human experts who monitor the system and intervene in emergencies.

[0984] This invention combines an emotion engine with a system that monitors a user's search history and comments on social networking services (SNS), detects specific words, and evaluates the user's emotional state. The generated counseling message is sent to the user's information processing device, and if necessary, notifies a human expert. The system also allows users to create and customize virtual beings to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[0985] The server receives information through an API that collects the user's information usage history and SNS comment data. The collected data is passed to an emotion engine, which uses natural language processing technology to identify the user's emotional state from the comments. Specifically, context analysis and keyword extraction are performed. Once the evaluation results are obtained, a prompt is sent to the generative AI model based on the results, which generates a counseling message. The generated message is then sent to the user's information processing device.

[0986] Examples:

[0987] The server detects social media posts from users that contain phrases such as "I want to die" or "It's painful." The emotion engine evaluates these as "sadness" or "despair." Based on the evaluation results, it sends a prompt to the generative AI model saying, "If the user says 'I want to die,' please generate an appropriate counseling message." The generative AI model then generates a message such as, "I've been very worried about you lately. Please tell me your story," and sends it to the user's information processing device.

[0988] The device monitors the user's search history and social media postings in real time, periodically sending the collected data to the server, and also receives counseling messages sent from the server in real time and notifies the user.

[0989] Users install the app and perform the initial setup. This setup prepares the app to automatically monitor the user's search history and social media posts. Users can create a virtual presence within the app and customize it to their liking. They can also input their emotional state within the app and have conversations with a generated AI counselor.

[0990] Examples:

[0991] Within the app, users can choose and customize the appearance of their virtual presence, then type in something like, "I've been feeling stressed about school lately," to which the generated AI counselor responds, "What's causing you stress?"

[0992] Furthermore, if the evaluation results meet certain conditions, the server can send a notification to a human expert. If the emotion engine detects a serious emotional state, the human expert will respond directly to the user who is deemed to need urgent assistance.

[0993] Examples:

[0994] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[0995] As a result, the present invention makes it possible to monitor a user's emotional state in real time, use an emotion engine to make a highly accurate assessment, and provide support at the appropriate time to deal with serious situations.

[0996] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0997] Step 1: Data collection

[0998] The server receives information through an API to collect user information usage history and comment data from networking services. This process inputs the user's social media posts and browser history. The input data is received in JSON format and stored in a database.

[0999] Specific behavior:

[1000] The server uses the Twitter API to retrieve the user's recent tweets, including the tweet text, timestamp, and user ID.

[1001] Step 2: Data analysis

[1002] The server passes the collected data to the emotion engine for analysis. The emotion engine uses natural language processing technology to identify emotions from the utterances. The input to this analysis process is the text data of the user utterances collected in step 1, and the output indicates an emotion tag (e.g., "sadness," "joy," etc.) and its degree.

[1003] Specific behavior:

[1004] The emotion engine detects the keyword "I want to die" and evaluates it as "sadness" or "despair." The analysis algorithm calculates an emotion score for each utterance.

[1005] Step 3: Message generation using a generative AI model

[1006] The server sends prompts to the generative AI model based on the evaluation results of the emotion engine, and generates a counseling message. The input is the evaluation result data from the emotion engine, and the output is the counseling message.

[1007] Specific behavior:

[1008] The server sends the prompt "If the user says 'I want to die,' please generate an appropriate counseling message" to the generative AI model. Based on the input, the generative AI model generates the message "I've been very worried about you lately. Please tell me your story."

[1009] Step 4: Send the message

[1010] The server then sends the generated counseling message to the user's information processing device. The input of this process is the message from the generative AI model, and the output is a notification that arrives on the user's device.

[1011] Specific behavior:

[1012] After generating the message, it sends it to the user's smartphone via an SMS API or notification service, saying, "I've been really worried about you lately. Let me tell you something."

[1013] Step 5: Receiving and viewing counseling messages

[1014] The terminal receives counseling messages sent from the server in real time and notifies the user. The input is the message sent from the server, and the output is the message notification displayed on the terminal.

[1015] Specific behavior:

[1016] The device uses push notifications to display the message, "I've been really worried about you lately. Tell me your story."

[1017] Step 6: User interaction and data entry

[1018] Users install the app and perform the initial setup. This setup allows the app to automatically monitor the user's search history and social media posts. Users can also input their emotional state and interact with the generated AI counselor.

[1019] Specific behavior:

[1020] The user enters in the app, "I've been feeling stressed about school lately," and the generated AI counselor responds, "What is causing you stress?" The input data is then sent back to the server for further analysis.

[1021] Step 7: Emergency response determination and notification

[1022] The server sends notifications to live experts if the evaluation results meet certain criteria. If the emotion engine detects a serious emotional state, it provides a means to intervene for users who are deemed to need urgent assistance.

[1023] Specific behavior:

[1024] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[1025] (Application example 2)

[1026] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1027] In today's digital society, it is extremely important to monitor a user's emotional state in real time and provide appropriate support and intervention. However, conventional systems have difficulty accurately assessing a user's emotional state and responding immediately. Furthermore, they lack elements that enhance the sense of familiarity between the user and the system, and their functionality for quickly responding to changes in the user's stress and emotions is limited. The present invention aims to solve these problems and provide more effective and accurate emotional state monitoring and counseling.

[1028] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1029] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a counseling message generated based on the evaluation result to the user's terminal; means for displaying the counseling message in real time on the user's terminal; means for using the evaluation result to make an emergency notification to a live counselor as needed; and means for enhancing a sense of intimacy with the user through a selectable and customizable virtual character. This makes it possible to accurately monitor the user's emotional state in real time and provide support at the appropriate time. Furthermore, the introduction of virtual characters enhances a sense of intimacy with the user and provides an environment in which the user feels natural interacting with the system.

[1030] "User search history" is a record of searches a user has conducted on the Internet.

[1031] "Comments on social networking services" refer to the text and comments posted by users on social networking services.

[1032] "Specific keywords" are specific words or phrases that the system places importance on.

[1033] "Mood" refers to the user's emotional or mental state.

[1034] The "evaluation result" is the result of analysis by the emotion engine and classification of the user's emotional state.

[1035] A "counseling message" is a message of advice or encouragement generated according to the user's emotional state.

[1036] "User's device" refers to a device used by a user, such as a mobile phone, tablet, or computer.

[1037] "Means for displaying in real time" is a function that allows the counseling message to be displayed to the user immediately.

[1038] "Emergency notification to manned counselors" is a function that makes emergency contact with experts when a critical emotional state is detected.

[1039] A "virtual character" is a digital avatar that can converse and interact with users.

[1040] "Means for enhancing familiarity" are functions and methods that make the user feel natural and familiar with the system.

[1041] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and generates appropriate counseling messages. Specific embodiments of this invention will be described below.

[1042] The server monitors users' search history and comments on social networking services, and analyzes the text data using a sentiment analysis library such as TextBlob. If the sentiment engine detects text containing specific keywords and evaluates it as a negative emotional state, it uses a generative AI model to generate an appropriate counseling message.

[1043] This message is sent to the user's device and displayed in real time. The user's device also has the function of sending collected data to the server, which periodically updates the user's emotional state. If the evaluation results meet certain conditions, the server will send an emergency notification to a manned counselor.

[1044] Users can begin using the system by installing the application and completing the initial setup. They can create and customize a virtual character and receive counseling messages through that character. This increases the sense of intimacy with the user, enabling accurate assessment of their emotional state and appropriate support.

[1045] As an example of specific processing, the server detects a user's social media posts containing keywords such as "I want to die" or "painful," and the emotion engine evaluates the emotions of "sadness" and "despair." As a result, a counseling message is generated saying, "I've been very worried about you lately. Please tell me what you think," and sent to the user's device. Furthermore, in the case of an emergency, a counselor is notified.

[1046] For example, you can input the following prompts into a generative AI model:

[1047] A user posted "I want to die" on social media. Please provide a counseling message to generate.

[1048] This makes it possible to monitor the user's emotional state in real time and provide support at the appropriate time. In addition, if the user feels natural and familiar with the system, accurate assessment of emotions and appropriate responses can be achieved.

[1049] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1050] Step 1:

[1051] The user installs the application and completes the initial setup, which includes downloading, installing, and configuring the application on the device, so the system is ready to monitor the user's search history and social media posts.

[1052] Input: User settings information

[1053] Output: Initialization completion status

[1054] Step 2:

[1055] The device monitors users' search history and comments on social networking services in real time. The collected text data is sent to a server. This data includes keywords searched by users and comments posted by users.

[1056] Input: User search history and social media posts

[1057] Output: Collected text data

[1058] Step 3:

[1059] The server analyzes the received text data using a sentiment analysis library such as TextBlob. If certain keywords are detected, the data is further analyzed by the sentiment engine to evaluate the user's emotional state.

[1060] Input: Collected text data

[1061] Output: Evaluation result of emotional state

[1062] Step 4:

[1063] The server uses a generative AI model to generate appropriate counseling messages based on the evaluation results. In this generation process, counseling messages that correspond to the user's emotional state are automatically created.

[1064] Input: Emotional state evaluation result

[1065] Output: The generated counseling message

[1066] Step 5:

[1067] The server sends the generated counseling message to the user's terminal, which displays the message in real time, allowing the user to receive the message immediately and respond as needed.

[1068] Input: Generated counseling message

[1069] Output: Messages displayed in real time

[1070] Step 6:

[1071] Users can read the counseling messages displayed on their devices, interact with the system as needed, or contact a live counselor. Users can also customize their virtual characters to enhance their sense of intimacy.

[1072] Input: Messages displayed in real time and customization information for virtual characters

[1073] Output: User responses and interaction data

[1074] Step 7:

[1075] The server collects the user's responses and dialogue data, and if the evaluation results meet certain conditions, it sends an emergency notification to a live counselor, allowing the live counselor to provide direct support to the user.

[1076] Input: User responses and interaction data

[1077] Output: Urgent notification to manned counselors

[1078] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1079] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1080] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1081] [Fourth embodiment]

[1082] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1083] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1084] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1085] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1086] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1087] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1088] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1089] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1090] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1091] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1092] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1093] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1094] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1095] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system then sends a notification to a live counselor. The system also allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[1096] Server Processing

[1097] The server receives and analyzes the user's search history and comments on social networking services. If specific keywords are detected, a machine learning algorithm is used to evaluate the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[1098] Examples:

[1099] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[1100] Terminal handling

[1101] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[1102] Examples:

[1103] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[1104] User operations

[1105] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[1106] Examples:

[1107] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[1108] Backup function

[1109] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then begins the process of providing direct support to the user.

[1110] Examples:

[1111] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[1112] Through these processes, the present invention is able to monitor the user's emotional state in real time, provide support at the appropriate time, and deal with serious situations.

[1113] The processing flow will be explained below.

[1114] Step 1:

[1115] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[1116] Step 2:

[1117] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[1118] Step 3:

[1119] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[1120] Step 4:

[1121] The server analyzes the received data, detects specific keywords, and uses machine learning algorithms to evaluate the user's emotional state.

[1122] Step 5:

[1123] The server uses a generative AI model to generate a counseling message based on the evaluation results, and the generated message is immediately sent to the device.

[1124] Step 6:

[1125] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[1126] Step 7:

[1127] The server again analyzes the user's reply data and generates and sends additional counseling messages as needed.

[1128] Step 8:

[1129] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[1130] Step 9:

[1131] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[1132] Step 10:

[1133] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor, who then provides emergency dialogue and assistance.

[1134] Example 1

[1135] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1136] In modern society, as users make daily comments and searches over the Internet, there is a need for systems that can accurately grasp their emotional state and provide counseling messages at the appropriate time. However, existing systems have difficulty analyzing users' emotions in real time and taking appropriate action. Furthermore, they often lack emergency notification functions for dealing with serious situations and user-friendly interfaces. Therefore, there is an urgent need to develop a system that can maintain users' mental health and provide prompt and individualized support.

[1137] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1138] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a generated counseling message to the user's terminal based on the evaluation result; means for periodically collecting the user's operation log and data; means for creating and customizing a virtual character; and means for using a generative AI model to generate the counseling message. This allows the system to grasp the user's emotional state in real time, provide an appropriate counseling message, and notify a live counselor as needed. Furthermore, by customizing the virtual character, the user can increase familiarity and promote interaction with the system.

[1139] A "user's search history" is a history of terms and phrases that a user searches for using an Internet search engine.

[1140] A "social networking service" is a platform that enables users to interact with other users online, including features such as posting, commenting, and messaging.

[1141] A "specific keyword" is a specific word or phrase that provides important clues when assessing a user's emotional state.

[1142] "Mood" refers to the user's psychological state and emotions.

[1143] "Means for evaluation" refers to methods or techniques for analyzing a user's emotional state based on collected data and obtaining a specific score or classification result.

[1144] A "counseling message" is a message that is generated according to the user's emotional state and provides psychological support and advice.

[1145] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[1146] An "operation log" is recorded data related to a user's device operations, including app usage status and input content.

[1147] A "generative AI model" is a model that uses machine learning and artificial intelligence techniques to generate output for specific purposes.

[1148] A "virtual character" is a digital character image that can be used as an interface by a user and can be customized.

[1149] A "manned counselor" is a person who is actually a human counselor and provides direct support to the user.

[1150] A "means for sending notifications" is a method or technology for sending a message or alert to a specific person when a specific condition is met by the system.

[1151] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends the generated counseling message to the user's device. If the evaluation results meet certain conditions, the system notifies a live counselor. Furthermore, the system allows users to create and customize virtual characters to enhance intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[1152] Server Processing

[1153] The server uses Apache Kafka to receive the user's search history and social networking service comment data. The received data is temporarily stored and then used to evaluate the user's emotional state using a TensorFlow model. The evaluation is performed by detecting specific keywords (e.g., "I want to die" or "It's painful"). An analysis script preprocesses the data and inputs it into the model, which calculates an emotion score from the inference results. Based on the evaluation results, a counseling message is generated using a generative AI model (e.g., a GPT model) and sent to the user's device via the Firebase Cloud Messaging (FCM) API.

[1154] Specific examples

[1155] The server detects user social media posts containing keywords such as "I want to die" or "It's painful," and uses a machine learning algorithm to assess the severity as "high." Based on this, it generates a counseling message saying, "I've been really worried about you lately. Please tell me what you think," and sends it to the user's device.

[1156] Prompt Sentence Examples

[1157] "Create a message that will be generated when a user posts 'I want to die'."

[1158] Terminal handling

[1159] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. A background service runs and periodically saves the operation logs and search history in a local database and sends them to the server. It also displays counseling messages sent from the server to the user in real time.

[1160] Specific examples

[1161] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[1162] Prompt Sentence Examples

[1163] "Generate a message to display if the user responds 'I'm a little tired.'"

[1164] User operations

[1165] Users install the app and complete the initial setup. Once setup is complete, a background task will start, automatically monitoring the user's search history and comments on social networking services. Users can also create and customize their own virtual characters, input their emotional state within the app, and interact with a generated AI counselor. On the character creation screen, users can select their avatar's appearance and clothing, preview their customizations in real time, and save them.

[1166] Specific examples

[1167] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[1168] Prompt Sentence Examples

[1169] "Generate a response when a user types, 'School has been really stressful lately.'"

[1170] Backup function

[1171] If the evaluation result meets certain conditions (e.g., a high emotional score), the server uses the Twilio API to send an SOS notification to a live counselor. The live counselor receives the notification and begins the process of providing direct assistance to the user.

[1172] Specific examples

[1173] If the server detects multiple posts from a user saying "I can't take it anymore," it determines that there is a serious problem. This triggers a notification to a live counselor, who then provides emergency support, such as calling the user directly.

[1174] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1175] Step 1:

[1176] The user installs the app and completes the initial setup.

[1177] Specific behavior: The user downloads and installs the app. When the app is launched for the first time, a setup wizard appears and the user grants the necessary permissions (e.g., Internet access, storage access, etc.). The user clicks the Finish setting button and the settings are saved.

[1178] Input: User interactions (touches, clicks), preference information.

[1179] Output: Setting completion flag, user initial setting data.

[1180] Step 2:

[1181] The device collects the user's search history and comments on social media in real time.

[1182] How it works: The device runs a background service that monitors the user's browser history and in-app social media posts to collect data, which is then stored in a local database.

[1183] Input: User search history, social media posts.

[1184] Output: Collected data stored in a local database.

[1185] Step 3:

[1186] The terminal periodically transmits the collected data to the server.

[1187] Specific operation: The device executes a scheduled task and uploads the collected data from the local database to the server. The upload is performed at the optimal time taking into account the network conditions.

[1188] Input: Collected data in local database.

[1189] Output: The data sent to the server.

[1190] Step 4:

[1191] The server temporarily stores the received data and prepares it for analysis.

[1192] Specific operation: The server receives data using Apache Kafka and stores it in temporary storage. The received data is preprocessed by the analysis script to prepare a dataset for analysis.

[1193] Input: Data sent from the terminal.

[1194] Output: Dataset for analysis.

[1195] Step 5:

[1196] The server analyzes the data and detects specific keywords.

[1197] How it works: The server inputs the preprocessed data into a TensorFlow model to detect specific keywords (e.g., "I want to die" or "It's painful"). It then scans the text data using regular expressions to extract matching keywords.

[1198] Input: Dataset for analysis.

[1199] Output: Detected results for specific keywords.

[1200] Step 6:

[1201] The server evaluates the user's emotional state.

[1202] Specific operation: Based on the detected keywords, the server uses a machine learning algorithm to evaluate the user's emotional state, and outputs the evaluation results as an emotion score or classification result.

[1203] Input: Detected results for a specific keyword.

[1204] Output: Evaluation result of the user's emotional state.

[1205] Step 7:

[1206] The server generates a counseling message based on the evaluation result.

[1207] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate an appropriate counseling message based on the evaluation results. A prompt sentence is input, and the model generates a message.

[1208] Input: User's emotional state assessment result.

[1209] Output: The generated counseling message.

[1210] Step 8:

[1211] The server sends a counseling message to the user's terminal.

[1212] Specific operation: The server uses the Firebase Cloud Messaging (FCM) API to send the generated counseling message to the user's device.

[1213] Input: The generated counseling message.

[1214] Output: The message sent to the user's terminal.

[1215] Step 9:

[1216] The terminal receives the counseling message and displays it to the user.

[1217] Specific behavior: The device receives the FCM notification and triggers a local notification. A message is displayed on a specific screen in the app according to the notification.

[1218] Input: The counseling message sent by the server.

[1219] Output: The message displayed to the user.

[1220] Step 10:

[1221] If the evaluation results meet certain conditions, the server sends an SOS notification to a manned counselor.

[1222] What it does: When the server detects a certain condition, such as a high emotion score, it uses the Twilio API to send an emergency notification to a live counselor.

[1223] Input: Evaluation results, such as high sentiment scores.

[1224] Output: SOS notification sent to manned counselors.

[1225] Step 11:

[1226] A live counselor receives the notification and initiates the process of providing direct assistance to the user.

[1227] Specific actions: When a counselor receives a notification, they will provide emergency assistance, such as calling the user directly, and take appropriate action if necessary.

[1228] Input: SOS notification from the server.

[1229] Output: Emergency assistance to the user.

[1230] (Application example 1)

[1231] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1232] Maintaining mental health is an important issue in modern society. In particular, with the spread of the Internet and social networking services, the information users send out on a daily basis often reveals mental distress and stress. However, collecting and analyzing this information in real time and providing appropriate counseling and support is currently difficult. Conventional systems lack the immediacy and accuracy required to provide effective mental support.

[1233] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1234] In this invention, the server includes means for monitoring a user's search history and comments on a social networking service and detecting specific keywords, means for evaluating the user's emotional state based on the keywords detected by the above means, means for generating a counseling message based on the evaluation result and sending the message to the user's terminal, means for periodically collecting the user's posts via a social media API and evaluating the emotional state in real time, means for generating an AI counseling message based on the evaluation of the emotional state and sending the message to the user, and means for sending a notification to a human counselor according to the severity of the evaluated emotional state. This makes it possible to quickly and accurately evaluate the user's mental state and provide counseling messages and support at an appropriate time.

[1235] "Search history" is a history of keywords and phrases that a user has searched for on the Internet using a search engine.

[1236] A "social networking service" is a platform that allows users to communicate and share information with each other over the Internet.

[1237] A "specific keyword" is an important word or phrase that indicates a predetermined psychological state from a user's comments or search history.

[1238] "Evaluation means" refers to algorithms or programs that analyze collected data and determine the user's emotional state.

[1239] A "generated counseling message" is a message of advice or comfort that is generated for transmission to a user based on the emotional state determined by the evaluation means.

[1240] A "terminal" is a computer device used by a user, such as a smartphone, tablet, or PC.

[1241] A "social media API" is an application program interface for automatically obtaining data from social networking services.

[1242] "Mood assessment" is the process of determining a user's psychological health from their statements and behavior.

[1243] "AI counseling messages" are messages generated using artificial intelligence technology with the aim of providing psychological support to users.

[1244] A "manned counselor" is a human counselor who provides direct support and intervention when the user's emotional state is determined to be serious.

[1245] This system monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and sends a generated counseling message to the user's device. Furthermore, if the evaluation results meet certain conditions, the system also includes a function to notify a live counselor.

[1246] Server Processing

[1247] The server first collects the user's search history and comments on social networking services. This is done through social media APIs, and data is periodically retrieved. This data includes search keywords and the content of social media posts. The server then analyzes this data to determine whether specific keywords are included.

[1248] Sentiment analysis is performed on the analyzed data to evaluate the user's emotional state. Based on the evaluation results, an AI-generated counseling message is generated. This AI-generated counseling message is generated in real time using a generative AI model, and the content is determined by using appropriate prompt sentences.

[1249] The generated counseling message is sent to the user's terminal and notified to the user. Depending on the severity of the evaluation result, a notification is also sent to a live counselor.

[1250] Terminal handling

[1251] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to the server. Information entered by the user and operation logs are also collected periodically, and this data is used for analysis on the server. Counseling messages sent from the server are notified to the device and displayed to the user.

[1252] User operations

[1253] Users install the app and complete the initial setup. Once setup is complete, search history and comments on social networking services are automatically monitored. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[1254] Specific examples

[1255] For example, if a user posts on social media, "I can't take it anymore," the content of the post is sent to the server via a social media API. The server performs sentiment analysis, assesses the severity, and determines it to be "high." Based on this result, an AI-generated counseling message is generated, and a message saying "Let me tell you something" is sent to the user's device. At the same time, because the condition is serious, a notification is also sent to a live counselor.

[1256] Prompt Sentence Examples

[1257] An example prompt might have the following format:

[1258] User ID: 12345

[1259] Post content: I can't take it anymore

[1260] In this way, the present invention is able to monitor the user's emotional state in real time, provide timely support and deal with critical situations.

[1261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1262] Step 1:

[1263] The server receives user search history and comment data from social networking services. Specifically, it periodically retrieves user posts and comments using social media APIs. The input is user posted data, and the output is raw data to be analyzed.

[1264] Step 2:

[1265] The server detects specific keywords from the received data using a text analysis engine. The input is raw data, and the output is data containing the detected keywords. Specifically, keyword extraction is performed using natural language processing tools.

[1266] Step 3:

[1267] The server evaluates the user's emotional state based on the detected keywords. It uses a sentiment analysis algorithm to analyze the user's posts and evaluate the emotional state with a score. The input is data containing keywords, and the output is a rating score.

[1268] Step 4:

[1269] The server generates an AI-generated counseling message based on the evaluation score. It uses a generative AI model to create an appropriate message according to the situation. The input is the evaluation score and prompt, and the output is the counseling message. Specifically, based on the example prompt, data is input into the AI-generated model and the generated message is obtained.

[1270] Step 5:

[1271] The server sends the generated counseling message to the user's device using a message sending API. The input is the generated counseling message, and the output is a notification to the user's device.

[1272] Step 6:

[1273] If the evaluation score meets certain conditions, the server sends a notification to a human counselor. The counselor is contacted using a notification system. The input is the evaluation score and the user's posted data, and the output is an alert to the counselor.

[1274] Step 7:

[1275] The user receives the counseling message sent to their terminal. The user terminal displays the message through the notification function. The input is the counseling message sent from the server, and the output is the display to the user. Specifically, the user checks the message and replies if necessary.

[1276] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1277] This invention combines a system that monitors a user's search history and comments on social networking services, detects specific keywords, and evaluates the user's emotional state with an emotion engine. The generated counseling message is sent to the user's device, and if necessary, a live counselor is notified. The system also allows users to create and customize virtual characters to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[1278] Server Processing

[1279] The server receives the user's search history and comments on social networking services and analyzes this data using an emotion engine. The emotion engine has an algorithm that recognizes emotions from the user's comments and behavioral patterns, and when specific keywords are detected, it evaluates the user's emotional state. Based on the evaluation results, a generative AI model generates a counseling message and sends it to the user's device.

[1280] Examples:

[1281] The server detects users' social media posts containing keywords such as "I want to die" or "It's painful," and the emotion engine evaluates the emotions of "sadness" and "despair." Based on this, a counseling message is generated, such as "I've been very worried about you lately. Please tell me what you think," and sent to the user's device.

[1282] Terminal handling

[1283] The device monitors the user's search history and comments on social networking services in real time and sends the collected data to a server. It also periodically collects information entered by the user and operation logs, and this data is used for analysis on the server. It also displays counseling messages sent from the server to the user in real time.

[1284] Examples:

[1285] The device asks the user, "How are you feeling these days?", and the user replies, "I'm a little tired." This information is sent to the server, which then generates an appropriate counseling message and sends it to the device. The device then displays the message, "Would you like to talk?" to the user.

[1286] User operations

[1287] Users install the app and complete the initial setup. Once setup is complete, the app automatically monitors the user's search history and comments on social networking services. Users can create and customize their own virtual characters. They can also input their emotional state within the app and interact with a generated AI counselor.

[1288] Examples:

[1289] Users can choose the appearance of their virtual character and customize it to their liking. They can then type something into the app like, "I've been really stressed out about school lately," and the generated AI counselor will respond with, "What exactly is causing you stress?"

[1290] Additional Server Processing

[1291] If the evaluation results meet certain conditions, the server sends an SOS notification to the on-site counselor. After receiving the notification, the on-site counselor initiates the procedure to provide direct assistance to the user. The emotion engine uses accumulated user data to track changes in the user's emotions and adjusts the counseling method accordingly.

[1292] Examples:

[1293] The server detects multiple serious posts from the user, such as "I can't take it anymore," and the emotion engine evaluates this as "despair" and determines that there is a serious problem. This causes an emergency notification to be sent to a manned counselor, who then provides emergency support, such as calling the user directly.

[1294] This allows the present invention to monitor the user's emotional state in real time, use the emotion engine to make a highly accurate assessment, and provide timely support to deal with serious situations.

[1295] The processing flow will be explained below.

[1296] Step 1:

[1297] The user installs the smartphone app and performs the initial setup. Here, they enter basic information such as their name, age, school name, etc. The device then sends this information to the server.

[1298] Step 2:

[1299] The user agrees to the privacy policy regarding data collection and analysis. Once the user presses the consent button, data collection will begin.

[1300] Step 3:

[1301] The device monitors the user's search history and comments on social networking services in real time, and if certain keywords are included, sends the data to a server.

[1302] Step 4:

[1303] The server analyzes the received data and uses an emotion engine to detect specific keywords and recognize the user's emotions. For example, keywords such as "I want to die" and "It's painful" can be used to identify emotions such as "sadness" and "despair."

[1304] Step 5:

[1305] The server uses a generative AI model to generate a counseling message based on the recognized emotions and evaluation results, and the generated message is immediately sent to the device.

[1306] Step 6:

[1307] The terminal notifies the user of the received counseling message and displays it. If the user replies to the message, the information is also sent to the server.

[1308] Step 7:

[1309] The server analyzes the user's reply data again and generates and sends additional counseling messages as needed. For example, if the user replies, "I'm very tired today," the server generates a new message such as, "Do you have time to take a short break?"

[1310] Step 8:

[1311] Users create and customize their virtual characters, and the device sends the customization information to the server, which reflects the changes in future interactions.

[1312] Step 9:

[1313] The terminal periodically displays a prompt to check the user's emotional state and asks for input, and the user's response is sent to the server.

[1314] Step 10:

[1315] The server accumulates the received periodic input data and tracks changes in the user's emotions through an emotion engine, which allows long-term emotional trends to be evaluated.

[1316] Step 11:

[1317] If the evaluation results meet certain criteria, the server sends an SOS notification to a live counselor, who then provides emergency dialogue and support. For example, if the server detects multiple serious statements such as "I can't take it anymore," it will send an emergency notification and have a counselor contact the user directly.

[1318] These steps enable the present invention to monitor the user's emotional state in real time, provide timely support, and use the emotion engine to make accurate assessments and handle critical situations.

[1319] Example 2

[1320] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1321] In modern society, fluctuations in users' emotional states have become a major problem, and early detection and appropriate response are particularly required for users experiencing stress or depression. However, current technology lacks a system that can accurately monitor these emotional fluctuations in real time and provide appropriate support. Furthermore, other methods can be distrustful of users and impose a heavy burden, so further improvement is needed.

[1322] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1323] In this invention, the server includes means for monitoring a user's information usage history and comments on the networking service and detecting specific words and phrases, means for evaluating the user's emotional state based on the words and phrases detected by said means, means for transmitting a corresponding message generated based on the evaluation result to the user's information processing device, and means for periodically collecting user input data and improving the accuracy of the evaluation result. This makes it possible to monitor the user's emotional state in real time, perform highly accurate evaluations, and provide appropriate messages at appropriate times.

[1324] "User" means an individual who uses the System to input information and receive services.

[1325] "Information usage history" refers to data including the user's search and access history on the Internet.

[1326] "Networking service" refers to an online platform, such as a social networking site or messaging app, that enables users to exchange information with other users.

[1327] A "phrase" refers to a word or phrase that has a specific meaning in a user's utterance or input data.

[1328] "Emotional state" refers to the user's mental and psychological state, and includes emotions such as "joy," "sadness," and "anger."

[1329] The "evaluation result" refers to the result of the determination of the emotional state obtained by the emotion engine through analysis of the user's words and actions.

[1330] "Response message" refers to a message appropriate to the user's emotional state, created by the generative AI model based on the evaluation results.

[1331] "Information processing device" refers to devices used by users, such as smartphones, tablets, and personal computers.

[1332] "Input data" refers to all information that a user inputs into a system, including text, audio, images, etc.

[1333] An "emotion engine" refers to a system that integrates algorithms and technologies to analyze emotions from users' statements and actions.

[1334] A "generative AI model" refers to an artificial intelligence model that generates appropriate messages in natural language based on input data.

[1335] "Virtual presence" refers to an in-app character or avatar that users can customize.

[1336] "Manned" refers to human experts who monitor the system and intervene in emergencies.

[1337] This invention combines an emotion engine with a system that monitors a user's search history and comments on social networking services (SNS), detects specific words, and evaluates the user's emotional state. The generated counseling message is sent to the user's information processing device, and if necessary, notifies a human expert. The system also allows users to create and customize virtual beings to increase intimacy, and periodically collects user input data to improve the accuracy of the evaluation results.

[1338] The server receives information through an API that collects the user's information usage history and SNS comment data. The collected data is passed to an emotion engine, which uses natural language processing technology to identify the user's emotional state from the comments. Specifically, context analysis and keyword extraction are performed. Once the evaluation results are obtained, a prompt is sent to the generative AI model based on the results, which generates a counseling message. The generated message is then sent to the user's information processing device.

[1339] Examples:

[1340] The server detects social media posts from users that contain phrases such as "I want to die" or "It's painful." The emotion engine evaluates these as "sadness" or "despair." Based on the evaluation results, it sends a prompt to the generative AI model saying, "If the user says 'I want to die,' please generate an appropriate counseling message." The generative AI model then generates a message such as, "I've been very worried about you lately. Please tell me your story," and sends it to the user's information processing device.

[1341] The device monitors the user's search history and social media postings in real time, periodically sending the collected data to the server, and also receives counseling messages sent from the server in real time and notifies the user.

[1342] Users install the app and perform the initial setup. This setup prepares the app to automatically monitor the user's search history and social media posts. Users can create a virtual presence within the app and customize it to their liking. They can also input their emotional state within the app and have conversations with a generated AI counselor.

[1343] Examples:

[1344] Within the app, users can choose and customize the appearance of their virtual presence, then type in something like, "I've been feeling stressed about school lately," to which the generated AI counselor responds, "What's causing you stress?"

[1345] Furthermore, if the evaluation results meet certain conditions, the server can send a notification to a human expert. If the emotion engine detects a serious emotional state, the human expert will respond directly to the user who is deemed to need urgent assistance.

[1346] Examples:

[1347] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[1348] As a result, the present invention makes it possible to monitor a user's emotional state in real time, use an emotion engine to make a highly accurate assessment, and provide support at the appropriate time to deal with serious situations.

[1349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1350] Step 1: Data collection

[1351] The server receives information through an API to collect user information usage history and comment data from networking services. This process inputs the user's social media posts and browser history. The input data is received in JSON format and stored in a database.

[1352] Specific behavior:

[1353] The server uses the Twitter API to retrieve the user's recent tweets, including the tweet text, timestamp, and user ID.

[1354] Step 2: Data analysis

[1355] The server passes the collected data to the emotion engine for analysis. The emotion engine uses natural language processing technology to identify emotions from the utterances. The input to this analysis process is the text data of the user utterances collected in step 1, and the output indicates an emotion tag (e.g., "sadness," "joy," etc.) and its degree.

[1356] Specific behavior:

[1357] The emotion engine detects the keyword "I want to die" and evaluates it as "sadness" or "despair." The analysis algorithm calculates an emotion score for each utterance.

[1358] Step 3: Message generation using a generative AI model

[1359] The server sends prompts to the generative AI model based on the evaluation results of the emotion engine, and generates a counseling message. The input is the evaluation result data from the emotion engine, and the output is the counseling message.

[1360] Specific behavior:

[1361] The server sends the prompt "If the user says 'I want to die,' please generate an appropriate counseling message" to the generative AI model. Based on the input, the generative AI model generates the message "I've been very worried about you lately. Please tell me your story."

[1362] Step 4: Send the message

[1363] The server then sends the generated counseling message to the user's information processing device. The input of this process is the message from the generative AI model, and the output is a notification that arrives on the user's device.

[1364] Specific behavior:

[1365] After generating the message, it sends it to the user's smartphone via an SMS API or notification service, saying, "I've been really worried about you lately. Let me tell you something."

[1366] Step 5: Receiving and viewing counseling messages

[1367] The terminal receives counseling messages sent from the server in real time and notifies the user. The input is the message sent from the server, and the output is the message notification displayed on the terminal.

[1368] Specific behavior:

[1369] The device uses push notifications to display the message, "I've been really worried about you lately. Tell me your story."

[1370] Step 6: User interaction and data entry

[1371] Users install the app and perform the initial setup. This setup allows the app to automatically monitor the user's search history and social media posts. Users can also input their emotional state and interact with the generated AI counselor.

[1372] Specific behavior:

[1373] The user enters in the app, "I've been feeling stressed about school lately," and the generated AI counselor responds, "What is causing you stress?" The input data is then sent back to the server for further analysis.

[1374] Step 7: Emergency response determination and notification

[1375] The server sends notifications to live experts if the evaluation results meet certain criteria. If the emotion engine detects a serious emotional state, it provides a means to intervene for users who are deemed to need urgent assistance.

[1376] Specific behavior:

[1377] If the server detects multiple posts saying "I can't take it anymore," the emotion engine evaluates this as "despair" and determines that there is a serious problem. As a result, an emergency notification is sent to a human expert, who then provides emergency support, such as calling the user directly.

[1378] (Application example 2)

[1379] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1380] In today's digital society, it is extremely important to monitor a user's emotional state in real time and provide appropriate support and intervention. However, conventional systems have difficulty accurately assessing a user's emotional state and responding immediately. Furthermore, they lack elements that enhance the sense of familiarity between the user and the system, and their functionality for quickly responding to changes in the user's stress and emotions is limited. The present invention aims to solve these problems and provide more effective and accurate emotional state monitoring and counseling.

[1381] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1382] In this invention, the server includes: means for monitoring a user's search history and comments on the social networking service to detect specific keywords; means for evaluating the user's emotional state based on the keywords detected by the above means; means for sending a counseling message generated based on the evaluation result to the user's terminal; means for displaying the counseling message in real time on the user's terminal; means for using the evaluation result to make an emergency notification to a live counselor as needed; and means for enhancing a sense of intimacy with the user through a selectable and customizable virtual character. This makes it possible to accurately monitor the user's emotional state in real time and provide support at the appropriate time. Furthermore, the introduction of virtual characters enhances a sense of intimacy with the user and provides an environment in which the user feels natural interacting with the system.

[1383] "User search history" is a record of searches a user has conducted on the Internet.

[1384] "Comments on social networking services" refer to the text and comments posted by users on social networking services.

[1385] "Specific keywords" are specific words or phrases that the system places importance on.

[1386] "Mood" refers to the user's emotional or mental state.

[1387] The "evaluation result" is the result of analysis by the emotion engine and classification of the user's emotional state.

[1388] A "counseling message" is a message of advice or encouragement generated according to the user's emotional state.

[1389] "User's device" refers to a device used by a user, such as a mobile phone, tablet, or computer.

[1390] "Means for displaying in real time" is a function that allows the counseling message to be displayed to the user immediately.

[1391] "Emergency notification to manned counselors" is a function that makes emergency contact with experts when a critical emotional state is detected.

[1392] A "virtual character" is a digital avatar that can converse and interact with users.

[1393] "Means for enhancing familiarity" are functions and methods that make the user feel natural and familiar with the system.

[1394] This invention is a system that monitors a user's search history and comments on social networking services, detects specific keywords, evaluates the user's emotional state, and generates appropriate counseling messages. Specific embodiments of this invention will be described below.

[1395] The server monitors users' search history and comments on social networking services, and analyzes the text data using a sentiment analysis library such as TextBlob. If the sentiment engine detects text containing specific keywords and evaluates it as a negative emotional state, it uses a generative AI model to generate an appropriate counseling message.

[1396] This message is sent to the user's device and displayed in real time. The user's device also has the function of sending collected data to the server, which periodically updates the user's emotional state. If the evaluation results meet certain conditions, the server will send an emergency notification to a manned counselor.

[1397] Users can begin using the system by installing the application and completing the initial setup. They can create and customize a virtual character and receive counseling messages through that character. This increases the sense of intimacy with the user, enabling accurate assessment of their emotional state and appropriate support.

[1398] As an example of specific processing, the server detects a user's social media posts containing keywords such as "I want to die" or "painful," and the emotion engine evaluates the emotions of "sadness" and "despair." As a result, a counseling message is generated saying, "I've been very worried about you lately. Please tell me what you think," and sent to the user's device. Furthermore, in the case of an emergency, a counselor is notified.

[1399] For example, you can input the following prompts into a generative AI model:

[1400] A user posted "I want to die" on social media. Please provide a counseling message to generate.

[1401] This makes it possible to monitor the user's emotional state in real time and provide support at the appropriate time. In addition, if the user feels natural and familiar with the system, accurate assessment of emotions and appropriate responses can be achieved.

[1402] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1403] Step 1:

[1404] The user installs the application and completes the initial setup, which includes downloading, installing, and configuring the application on the device, so the system is ready to monitor the user's search history and social media posts.

[1405] Input: User settings information

[1406] Output: Initialization completion status

[1407] Step 2:

[1408] The device monitors users' search history and comments on social networking services in real time. The collected text data is sent to a server. This data includes keywords searched by users and comments posted by users.

[1409] Input: User search history and social media posts

[1410] Output: Collected text data

[1411] Step 3:

[1412] The server analyzes the received text data using a sentiment analysis library such as TextBlob. If certain keywords are detected, the data is further analyzed by the sentiment engine to evaluate the user's emotional state.

[1413] Input: Collected text data

[1414] Output: Evaluation result of emotional state

[1415] Step 4:

[1416] The server uses a generative AI model to generate appropriate counseling messages based on the evaluation results. In this generation process, counseling messages that correspond to the user's emotional state are automatically created.

[1417] Input: Emotional state evaluation result

[1418] Output: The generated counseling message

[1419] Step 5:

[1420] The server sends the generated counseling message to the user's terminal, which displays the message in real time, allowing the user to receive the message immediately and respond as needed.

[1421] Input: Generated counseling message

[1422] Output: Messages displayed in real time

[1423] Step 6:

[1424] Users can read the counseling messages displayed on their devices, interact with the system as needed, or contact a live counselor. Users can also customize their virtual characters to enhance their sense of intimacy.

[1425] Input: Messages displayed in real time and customization information for virtual characters

[1426] Output: User responses and interaction data

[1427] Step 7:

[1428] The server collects the user's responses and dialogue data, and if the evaluation results meet certain conditions, it sends an emergency notification to a live counselor, allowing the live counselor to provide direct support to the user.

[1429] Input: User responses and interaction data

[1430] Output: Urgent notification to manned counselors

[1431] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1432] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1433] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1434] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1435] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1436] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1437] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1438] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1439] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1440] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1441] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1442] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1443] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1444] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1445] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1446] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1447] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1448] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1449] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1450] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1451] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1452] The following is further disclosed regarding the above embodiment.

[1453] (Claim 1)

[1454] A means for monitoring a user's search history and comments on social networking services to detect specific keywords;

[1455] means for evaluating a user's emotional state based on the keywords detected by said means;

[1456] means for transmitting a generated counseling message to the user's terminal based on the evaluation result;

[1457] A system including:

[1458] (Claim 2)

[1459] The method further includes a means for sending a notification to a manned counselor when the evaluation result satisfies a certain condition.

[1460] 10. The system of claim 1.

[1461] (Claim 3)

[1462] further including means for a user to create and customize a virtual character;

[1463] 10. The system of claim 1.

[1464] (Claim 4)

[1465] The method further includes means for periodically collecting input data from users and storing the data in a database to improve the accuracy of the evaluation results.

[1466] 10. The system of claim 1.

[1467] (Claim 5)

[1468] and means for displaying a periodic status check prompt to the user at the user's terminal.

[1469] 10. The system of claim 1.

[1470] "Example 1"

[1471] (Claim 1)

[1472] A means for monitoring a user's search history and comments on social networking services to detect specific keywords;

[1473] means for evaluating a user's emotional state based on the keywords detected by said means;

[1474] means for transmitting a generated counseling message to the user's terminal based on the evaluation result;

[1475] A means for periodically collecting user operation logs and data;

[1476] a means for creating and customizing a virtual character;

[1477] means for using a generative AI model to generate the counseling message;

[1478] ...

[1479] A system including:

[1480] (Claim 2)

[1481] The method further includes a means for sending a notification to a manned counselor when the evaluation result satisfies a certain condition.

[1482] 10. The system of claim 1.

[1483] (Claim 3)

[1484] further including means for a user to create and customize a virtual character;

[1485] 10. The system of claim 1.

[1486] "Application Example 1"

[1487] (Claim 1)

[1488] A means for monitoring a user's search history and comments on social networking services to detect specific keywords;

[1489] means for evaluating a user's emotional state based on the keywords detected by said means;

[1490] means for transmitting a generated counseling message to the user's terminal based on the evaluation result;

[1491] A means of periodically collecting user posts via social media APIs and assessing their sentiment in real time;

[1492] means for generating an AI counseling message based on the evaluation of the emotional state and sending the message to the user;

[1493] means for sending a notification to a staff counselor in response to the severity of the assessed emotional state;

[1494] A system including:

[1495] (Claim 2)

[1496] The method further includes a means for sending a notification to a manned counselor when the evaluation result satisfies a certain condition.

[1497] 10. The system of claim 1.

[1498] (Claim 3)

[1499] further including means for a user to create and customize a virtual character;

[1500] 10. The system of claim 1.

[1501] "Example 2: Combining Emotion Engines"

[1502] (Claim 1)

[1503] A means for monitoring a user's information usage history and comments on the networking service to detect specific words and phrases;

[1504] means for assessing the emotional state of the user based on the phrases detected by said means;

[1505] means for transmitting a response message generated based on the evaluation result to the user's information processing device;

[1506] a means for periodically collecting user input data to improve the accuracy of the evaluation results;

[1507] A system including:

[1508] (Claim 2)

[1509] and means for sending a notification to a human expert when the evaluation result satisfies a certain condition.

[1510] 10. The system of claim 1.

[1511] (Claim 3)

[1512] further comprising means for a user to create and customize a virtual presence;

[1513] 10. The system of claim 1.

[1514] "Application example 2 when combining emotion engines"

[1515] (Claim 1)

[1516] A means for monitoring a user's search history and comments on social networking services to detect specific keywords;

[1517] means for evaluating a user's emotional state based on the keywords detected by said means;

[1518] means for transmitting a generated counseling message to the user's terminal based on the evaluation result;

[1519] a means for displaying a counseling message on a user's device in real time;

[1520] a means for making an emergency notification to a manned counselor as necessary using the evaluation result;

[1521] A means to increase user intimacy through selectable and customizable virtual characters;

[1522] A system including:

[1523] (Claim 2)

[1524] The system of claim 1, further comprising means for sending an emergency notification to a staffed counselor when the evaluation result satisfies a certain condition.

[1525] (Claim 3)

[1526] a means for a user to create and customize a virtual character;

[1527] a means for displaying a counseling message generated based on the evaluation result through a virtual character;

[1528] 10. The system of claim 1, further comprising means for periodically updating and assessing the user's emotional state using said means. [Explanation of symbols]

[1529] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for monitoring a user's search history and comments on social networking services to detect specific keywords; means for evaluating a user's emotional state based on the keywords detected by said means; means for transmitting a generated counseling message to the user's terminal based on the evaluation result; A system including:

2. The method further includes a means for sending a notification to a manned counselor when the evaluation result satisfies a certain condition. The system of claim 1 .

3. further including means for a user to create and customize a virtual character; The system of claim 1 .

4. The method further includes means for periodically collecting input data from users and storing the data in a database to improve the accuracy of the evaluation results. The system of claim 1 .

5. and means for displaying a periodic status check prompt to the user at the user's terminal. The system of claim 1 .

Images