System

Abstract

A system is provided.SOLUTION: A system comprising: means for initializing a virtual character to be generated based on basic information set by a user; means for having a conversation with the generated virtual character on a daily basis; means for analyzing a content of the conversation to analyze a mental state of the user; means for providing an appropriate measure based on a result of the analysis; and means for monitoring the content of the conversation with another generation model to generate feedback when there is an inappropriate statement.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Mental illness is on the rise in modern society, and many people feel particularly isolated. Mild cases often have no noticeable symptoms, making early detection difficult. Another issue is the lack of someone to talk to due to geographical or time constraints. This invention aims to reduce feelings of loneliness and maintain mental health by detecting signs of mental illness in users' everyday conversations and providing appropriate treatment. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate countermeasures based on the analysis results, and means for monitoring the content of the conversation using a separate generation model and generating feedback if inappropriate remarks are made.

[0006] A "virtual character" is a human-like character that is generated based on information set by the user and is intended to converse with the user.

[0007] The "means for initializing" is a function for generating a virtual character based on basic information (appearance, personality, etc.) entered by the user and initializing the character.

[0008] "Means for conducting conversation" refers to the interface and technology that allows the user and the virtual character to exchange messages in a two-way manner.

[0009] The "means of analysis" refers to artificial intelligence models and algorithms that analyze the collected conversation content and evaluate and diagnose the user's mental state.

[0010] "Means for providing appropriate measures" refers to functions and technologies that provide measures such as suggestions for changing mood, encouraging medical attention by a specialist, and notifying contact information in case of an emergency, depending on the user's mental state.

[0011] The "monitoring method" is a technology that evaluates the conversation content in real time using a separate generative model to check whether it contains any inappropriate remarks.

[0012] "Means for generating feedback" is a function that, when an inappropriate comment is detected, suggests to the user the problem with the comment and ways to improve it. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[0035] Below, we will create a program for this system and explain the specific processing in natural language.

[0036] Program Overview

[0037] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on this information, the server initializes the character using a generative AI model and begins a daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state. Based on the evaluation, appropriate measures are offered and feedback is generated as necessary. The system also monitors the content of the conversation using a separate generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user.

[0038] Initial Setup

[0039] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0040] 2. The device sends the information entered by the user to the server.

[0041] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[0042] 4. The server sends the initialized virtual character to the device.

[0043] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[0044] Daily conversation

[0045] 1. The user uses the device to have everyday conversations with a virtual character.

[0046] 2. The device sends the message entered by the user to the server.

[0047] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[0048] 4. The server generates a response and sends it to the terminal.

[0049] 5. The terminal displays the generated response to the user.

[0050] Analysis of mental status

[0051] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[0052] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[0053] 3. The server generates appropriate measures according to the user's mental state.

[0054] Providing appropriate measures

[0055] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[0056] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[0057] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[0058] 4. The device displays the countermeasures from the server to the user.

[0059] Speech monitoring and feedback

[0060] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[0061] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[0062] 3. The device displays feedback to the user.

[0063] Specific examples

[0064] Example 1: Initial Setup

[0065] 1. The user enters their preference for a "cheerful and encouraging friend."

[0066] 2. The device sends the information to the server.

[0067] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[0068] 4. The device displays the virtual character to the user and begins a conversation.

[0069] Example 2: Analysis of everyday conversations and mental states

[0070] 1. The user types, "I just don't feel like doing anything these days."

[0071] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[0072] 3. The terminal displays the response to the user.

[0073] 4. The server analyzes the conversation and determines that the injury is minor.

[0074] Example 3: Severe cases

[0075] 1. The user types, "I'm sick of everything."

[0076] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[0077] 3. The device will display the message "Please talk to someone now."

[0078] This concludes the explanation of the specific form for carrying out the invention. With this system, users can receive mental care at any time by talking to a virtual character. In addition, if the condition is severe, a notification is sent to the appropriate contact person, so users can receive prompt and appropriate treatment without having to worry alone.

[0079] The processing flow will be explained below.

[0080] Initial Setup

[0081] Step 1:

[0082] The user accesses the terminal interface and inputs basic information (such as appearance and personality) about the virtual character they desire.

[0083] Step 2:

[0084] The device sends the basic information entered by the user to the server.

[0085] Step 3:

[0086] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[0087] Step 4:

[0088] The server transmits information about the initialized virtual character to the terminal.

[0089] Step 5:

[0090] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[0091] Daily conversation

[0092] Step 1:

[0093] The user uses the terminal to initiate a conversation with the virtual character.

[0094] Step 2:

[0095] The terminal transmits the conversation message input by the user to the server.

[0096] Step 3:

[0097] The server analyzes the received message using a generative AI model and generates an appropriate response.

[0098] Step 4:

[0099] The server sends the generated response message to the terminal.

[0100] Step 5:

[0101] The terminal displays the response message received from the server to the user.

[0102] Analysis of mental status

[0103] Step 1:

[0104] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[0105] Step 2:

[0106] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[0107] Step 3:

[0108] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[0109] Providing appropriate measures

[0110] Step 1:

[0111] The server generates appropriate measures depending on the user's mental state.

[0112] Step 2:

[0113] If the symptoms are mild, the server generates suggestions to encourage mood change (e.g., relaxation methods or activities).

[0114] Step 3:

[0115] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[0116] Step 4:

[0117] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[0118] Step 5:

[0119] The device displays the countermeasure information received from the server to the user.

[0120] Speech monitoring and feedback

[0121] Step 1:

[0122] The server monitors the conversation with the user in real time using a separate generative model.

[0123] Step 2:

[0124] The server analyzes and determines whether the content contains inappropriate comments.

[0125] Step 3:

[0126] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[0127] Step 4:

[0128] The server sends the generated feedback message to the terminal.

[0129] Step 5:

[0130] The device displays a feedback message to the user.

[0131] The above is a detailed description of the specific operations performed in each processing step. This system allows users to receive constant mental care and take appropriate measures.

[0132] Example 1

[0133] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0134] Mental health care has become an important issue in modern society. However, there are limited systems that allow individual users to easily receive mental health care at any time. There is also a lack of mechanisms that allow users in serious mental health conditions to quickly receive appropriate support. Furthermore, there is a need for a system that can immediately point out inappropriate comments made during conversations with users and provide appropriate feedback.

[0135] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0136] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to evaluate the user's mental state, means for providing appropriate measures based on the evaluation results, and means for monitoring the content of the conversation using a separate generative model and generating feedback if inappropriate comments are made. This allows the user to receive mental care through conversations with the virtual character, and in severe cases, appropriate measures are taken promptly, allowing the user to receive support without feeling isolated. In addition, immediate feedback is provided for inappropriate comments made during conversations, encouraging the user to improve.

[0137] "Basic information" is information about the appearance and temperament of the virtual character set by the user.

[0138] A "virtual character" is an interactable character that is initialized based on basic information about the user using a generative AI model.

[0139] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input and prompts.

[0140] "Means for daily conversation" refers to a system function that allows users and virtual characters to have ongoing dialogue.

[0141] "Means for analyzing conversation content" refers to technology for analyzing the content of conversation between a virtual character and a user and assessing the user's mental state.

[0142] "Mental state" refers to the user's psychological and emotional state, and is classified as mild, moderate, severe, etc.

[0143] "Means for providing appropriate measures" refers to a system function that provides measures based on the user's mental state, such as diversion, consultation with a specialist, or in some cases notification to emergency contacts.

[0144] A "separate generative model" is a separate artificial intelligence model designed to monitor conversation content and detect inappropriate remarks.

[0145] "Means for generating feedback" refers to a system function that provides users with information about the problem and possible solutions when inappropriate comments are made.

[0146] This invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made. Specific embodiments of this system are described below.

[0147] Hardware and software configuration used

[0148] The server is a high-performance computing environment for running generative AI models (e.g., GPT-3) and mental care AI models. The server sends and receives data to and from the user's device via the HTTP protocol.

[0149] The terminal is the device (e.g., smartphone, tablet, or PC) through which the user operates the interface. The terminal is responsible for collecting user input and sending it to the server. It also displays virtual characters and response messages received from the server to the user.

[0150] Program processing

[0151] Initial Setup

[0152] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0153] 2. The device sends the information entered by the user to the server.

[0154] 3. Based on the received user information, the server customizes the generative AI model and generates a virtual character.

[0155] 4. The server sends the generated virtual character to the device.

[0156] 5. The device displays the virtual character to the user, allowing them to start a daily conversation.

[0157] Daily conversation

[0158] 1. The user uses the device to converse with a virtual character.

[0159] 2. The device sends the message entered by the user to the server.

[0160] 3. The server analyzes the message sent using a generative AI model and generates an appropriate response.

[0161] 4. The server generates a response and sends it to the terminal.

[0162] 5. The terminal displays the response to the user.

[0163] Analysis of mental status

[0164] 1. The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[0165] 2. Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe.

[0166] 3. The server generates appropriate countermeasures based on the classification results.

[0167] Providing appropriate measures

[0168] 1. The server generates suggestions for users who are judged to have mild symptoms, such as "Why don't you take a walk to change your mood?"

[0169] 2. The server generates a message for users who are judged to be in moderate condition, such as "Please consult a specialist."

[0170] 3. The server automatically sends notifications to pre-defined contacts for users who are deemed to be in a serious condition.

[0171] 4. The device displays the countermeasures sent from the server to the user.

[0172] Speech monitoring and feedback

[0173] 1. The server uses a separate generative model to monitor the conversation in real time.

[0174] 2. When the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[0175] 3. The device displays this feedback to the user.

[0176] Examples of concrete examples and prompts

[0177] Example 1: Initial Setup

[0178] 1. The user enters their preference for a "cheerful and encouraging friend."

[0179] 2. The device sends the information to the server.

[0180] 3. The server trains a generative AI model to generate a "cheerful and encouraging" character.

[0181] 4. The device displays the virtual character to the user and begins a conversation.

[0182] Example prompt sentence:

[0183] "Create a character that is cheerful and encouraging."

[0184] Example 2: Analysis of everyday conversations and mental states

[0185] 1. The user types, "I just don't feel like doing anything these days."

[0186] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[0187] 3. The terminal displays the response to the user.

[0188] 4. The server analyzes the conversation and determines that the injury is minor.

[0189] Example 3: Severe cases

[0190] 1. The user types, "I'm sick of everything."

[0191] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[0192] 3. The device will display the message "Please talk to someone now."

[0193] This system allows users to receive mental health care through conversations with virtual characters at any time, and in severe cases, prompt and appropriate support is provided. It also provides immediate feedback on inappropriate remarks made during conversations, encouraging users to work on self-improvement.

[0194] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0195] Step 1:

[0196] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0197] Input: Basic information about the virtual character (appearance, personality) entered by the user.

[0198] Output: Basic information data entered into the terminal.

[0199] Step 2:

[0200] The terminal sends the information entered by the user to the server.

[0201] Input: Basic information data entered by the user.

[0202] Data processing: Convert the input data into JSON format.

[0203] Output: JSON data sent as an HTTP POST request to the server.

[0204] Step 3:

[0205] Based on the received user information, the server customizes the generative AI model to generate a virtual character.

[0206] Input: Basic user information data received by the server.

[0207] Data computation: Using prompts from a generative AI model (e.g., GPT-3) to generate a virtual character based on basic information.

[0208] Output: Data of the generated virtual character.

[0209] Step 4:

[0210] The server transmits the generated virtual character to the terminal.

[0211] Input: Data of the generated virtual character.

[0212] Data processing: Encode the virtual character data.

[0213] Output: The encoded data that is sent as an HTTP response to the device.

[0214] Step 5:

[0215] The device displays a virtual character to the user, allowing them to start a daily conversation.

[0216] Input: Encoded virtual character data received from the server.

[0217] Data processing: Decoded virtual character data.

[0218] Output: A virtual character displayed in the user interface.

[0219] Step 6:

[0220] The user uses the terminal to converse with the virtual character.

[0221] Input: The conversation message that the user types.

[0222] Output: A conversation message with information entered on the device.

[0223] Step 7:

[0224] The terminal sends the message entered by the user to the server.

[0225] Input: The message entered by the user.

[0226] Data processing: Convert messages to JSON format.

[0227] Output: JSON data sent as an HTTP POST request to the server.

[0228] Step 8:

[0229] The server analyzes the message sent using a generative AI model and generates an appropriate response.

[0230] Input: The user's message data as received by the server.

[0231] Data Computation: Using generative AI models to analyze messages and generate appropriate responses.

[0232] Output: The generated response data.

[0233] Step 9:

[0234] The server sends the generated response to the terminal.

[0235] Input: The generated response data.

[0236] Data processing: Encode the response data.

[0237] Output: The encoded data that is sent as an HTTP response to the device.

[0238] Step 10:

[0239] The terminal displays the generated response to the user.

[0240] Input: The encoded response data received from the server.

[0241] Data processing: Decoded response data.

[0242] Output: The response message that is displayed in the user interface.

[0243] Step 11:

[0244] The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[0245] Input: Data from everyday conversations.

[0246] Data calculation: A mental state assessment process using a mental care AI model.

[0247] Output: Assessment result (mild, moderate, severe).

[0248] Step 12:

[0249] Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe and generates appropriate countermeasures.

[0250] Input: Mental status assessment results.

[0251] Data calculation: Generation of countermeasures according to mental state.

[0252] Output: Generated countermeasure data.

[0253] Step 13:

[0254] The server generates appropriate messages and notifications for mild, moderate, and severe cases and sends them to the device or to configured contacts.

[0255] Input: Mental state classification results and generated measures data.

[0256] Data processing: Encode notification data as needed.

[0257] Output: Notification data to user device or contacts.

[0258] Step 14:

[0259] The device displays the countermeasures sent from the server to the user.

[0260] Input: Countermeasure data received from the server.

[0261] Data processing: Decoded countermeasure data.

[0262] Output: The action message that is displayed in the user interface.

[0263] Step 15:

[0264] The server uses another generative model to monitor conversations in real time and detect inappropriate remarks.

[0265] Input: Real-time conversation data.

[0266] Data computation: Detecting profanity using alternative generative models.

[0267] Output: Detection result (whether or not there is any inappropriate speech).

[0268] Step 16:

[0269] If the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[0270] Input: Profanity detection result data.

[0271] Data computation: Speech content analysis and feedback generation.

[0272] Output: Feedback data including issues and improvements.

[0273] Step 17:

[0274] The device displays feedback to the user.

[0275] Input: Feedback data received from the server.

[0276] Data processing: Decoded feedback data.

[0277] Output: Feedback message displayed in the user interface.

[0278] (Application example 1)

[0279] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] In modern society, stress and mental health problems are on the rise, and companies are placing importance on supporting the mental health of their employees. However, it is difficult to provide appropriate measures in a timely manner using conventional care methods, and there is a need for a system that can respond quickly, especially in emergencies.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0282] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate measures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, and means for automatically sending a notification to a set emergency contact if the mental state is determined to be serious, thereby enabling real-time care for employee mental health and rapid response to emergencies.

[0283] "User" refers to a person who uses the system.

[0284] "Basic information" refers to the initial setting information such as the appearance and personality of the virtual character entered by the user.

[0285] A "virtual character" is a character generated by a generative AI model based on the user's basic information.

[0286] "Initialization" refers to the process of generating a virtual character based on basic information set by the user.

[0287] "Means for daily conversation" refers to a system that allows users and virtual characters to have daily conversations.

[0288] "Conversation content" refers to the text and audio data of the dialogue between the user and the virtual character.

[0289] "Mental state" refers to the user's psychological health and mood.

[0290] "Analysis" refers to the process of evaluating the content of a conversation and determining the user's mental state.

[0291] "Appropriate measures" refer to countermeasures such as advice and suggested actions provided to users based on the analysis results.

[0292] A "generative model" refers to an AI algorithm that has been trained to accomplish a specific task.

[0293] "Inappropriate comments" refer to comments that may have a negative impact on the user's mental health.

[0294] "Feedback" refers to advice provided in response to inappropriate comments for correction or improvement.

[0295] "Emergency Contacts" refers to configured contacts to whom automatic notifications are sent if a user's mental condition is deemed severe.

[0296] "Automatic Notification" refers to a message that the system automatically sends when certain conditions are met.

[0297] This system generates a virtual character based on basic information provided by the user, analyzes the user's mental state through daily conversations, and provides appropriate measures. The system also includes a function to monitor the conversation content using a separate generative model and generate feedback if inappropriate remarks are made. It also has a function to automatically send a notification to emergency contacts if the user's mental state is determined to be severe.

[0298] System configuration

[0299] The system for implementing the present invention is mainly composed of the following hardware and software.

[0300] Hardware: Smartphone (Android or iOS), Server (Cloud service such as AWS, Google Cloud, Microsoft Azure, etc.)

[0301] Software: Smartphone applications (e.g., React Native, Swift, Kotlin), generative AI models (e.g., OpenAI's GPT-4), and mental health AI models (e.g., BERT-based filtering technology).

[0302] Program Overview

[0303] Initial Setup

[0304] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0305] 2. The device sends the information entered by the user to the server.

[0306] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[0307] 4. The server sends the initialized virtual character to the device.

[0308] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[0309] Daily conversation

[0310] 1. The user uses the device to have everyday conversations with a virtual character.

[0311] 2. The device sends the message entered by the user to the server.

[0312] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[0313] 4. The server generates a response and sends it to the terminal.

[0314] 5. The terminal displays the generated response to the user.

[0315] Analysis of mental status

[0316] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[0317] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[0318] 3. The server generates appropriate measures according to the user's mental state.

[0319] Providing appropriate measures

[0320] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[0321] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[0322] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[0323] 4. The device displays the countermeasures from the server to the user.

[0324] Speech monitoring and feedback

[0325] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[0326] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[0327] 3. The device displays feedback to the user.

[0328] Description and examples

[0329] Hardware and software used

[0330] The system begins by sending basic information from the user via a smartphone application to a server, which then uses a generative AI model (such as OpenAI's GPT-4) to generate a virtual character and analyzes the conversation using a mental health AI model (such as BERT-based filtering technology).

[0331] Examples of concrete examples and prompts

[0332] 1. The user inputs their preference for a "character they can talk to casually, like a friend."

[0333] 2. The app will display the initialized character and begin the conversation.

[0334] 3. The user types, "I've been feeling stressed lately."

[0335] 4. The server generates a response saying, "Take a break and relax."

[0336] 5. The server analyzes the conversation and determines that the injury is minor.

[0337] 6. The app will display suggestions for relaxation to the user.

[0338] Example prompts

[0339] "Generate encouraging responses based on user-entered messages."

[0340] "Evaluate the user's mental state based on this text and classify it as mild, moderate, or severe."

[0341] "Depending on the user's mental state, generate appropriate measures, such as suggestions for relaxation, recommendations for medical attention, or notification to emergency contacts."

[0342] This will enable real-time care for employee mental health and rapid response to emergencies.

[0343] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0344] Step 1:

[0345] A user launches a smartphone application and inputs basic information (appearance and personality) about the desired virtual character. The input information is necessary to generate the virtual character, providing specific data (appearance, personality traits, etc.) according to the user's preferences.

[0346] Step 2:

[0347] The terminal sends the basic information entered by the user to the server. The input here is the basic information provided by the user, and by sending this to the server, data initialization is performed. The terminal performs the process of sending the user information.

[0348] Step 3:

[0349] The server customizes the generative AI model based on the received basic information and initializes the virtual character. At this time, the server inputs the user's basic information into the virtual character generation algorithm, and obtains the virtual character's profile as the output. Specifically, the data is input into the generative AI model to generate the character's appearance and personality.

[0350] Step 4:

[0351] The server transmits the initialized virtual character profile to the terminal. The input here is the generated virtual character profile, and executes a process to transmit the profile to the terminal.

[0352] Step 5:

[0353] The terminal displays the virtual character to the user, enabling the user to start daily conversation with the virtual character. The user begins to interact with the virtual character displayed on the terminal screen.

[0354] Step 6:

[0355] The user can have everyday conversations with the virtual character and input messages, which are then saved as conversation content and sent to the virtual character.

[0356] Step 7:

[0357] The terminal sends the message entered by the user to the server. The input here is the user's dialogue message, and sending this to the server starts processing the conversation content.

[0358] Step 8:

[0359] The server analyzes the input message and generates an appropriate response using a generative AI model. The server inputs the user's message into the analysis algorithm and generates an appropriate response as the output. Specific operations include the processes of text analysis and response generation.

[0360] Step 9:

[0361] The server sends the generated response to the terminal. The input here is the generated response message, and processing to send it to the terminal is executed.

[0362] Step 10:

[0363] The terminal displays the generated response to the user, where the user can confirm the response message from the virtual character.

[0364] Step 11:

[0365] The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model. The analyzed conversation content is used as input, and the mental state evaluation result is obtained as output. Specific operations include analyzing the conversation content and classifying the mental state.

[0366] Step 12:

[0367] Based on the evaluation results, the server classifies the user's mental condition as mild, moderate, or severe, which serves as the basis for providing appropriate measures.

[0368] Step 13:

[0369] The server generates appropriate measures according to the user's mental state and sends them to the device. In mild cases, it suggests a change of mood, in moderate cases it generates a message urging a specialist to examine the patient, and in severe cases it automatically sends a notification to an emergency contact. The content of the generated measures is input, and data to be sent to the user or emergency contact is obtained as output.

[0370] Step 14:

[0371] The device will display the countermeasures from the server to the user, and if necessary, will notify emergency contacts.

[0372] Step 15:

[0373] The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made. The input conversation data is fed into the monitoring algorithm, which performs an error check for inappropriate comments.

[0374] Step 16:

[0375] If the server detects an inappropriate comment, it generates feedback that presents the problem with the comment and how to improve it to the user. The generated feedback is used as input, and data to be sent to the user is obtained as output.

[0376] Step 17:

[0377] The device will display the feedback to the user, who can then review the feedback and take appropriate action.

[0378] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0379] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. In addition, it combines an emotion engine to recognize the user's emotions and respond individually. This system also includes a function that monitors the content of conversations with a separate generative model and generates feedback if inappropriate remarks are made.

[0380] Below, we will create a program for this system and explain the specific processing in natural language.

[0381] Program Overview

[0382] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on the obtained information, the server initializes the character using a generative AI model and begins daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state and emotions. Based on the evaluation, appropriate countermeasures are provided and feedback is generated as necessary. The conversation content is also monitored using another generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user. The emotion engine recognizes the user's emotions from the content of the conversation, facial expressions, voice, etc., and adjusts the response content and countermeasures.

[0383] Initial Setup

[0384] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0385] 2. The device sends the information entered by the user to the server.

[0386] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[0387] 4. The server sends information about the initialized virtual character to the terminal.

[0388] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[0389] Daily conversation

[0390] 1. The user uses the device to start a conversation with a virtual character.

[0391] 2. The terminal sends the conversation message entered by the user to the server.

[0392] 3. The server analyzes the received message using a generative AI model and generates an appropriate response.

[0393] 4. The server sends the generated response message to the terminal.

[0394] 5. The terminal displays the generated response to the user.

[0395] Emotion recognition

[0396] 1. The server uses an emotion engine to recognize emotions from the content of the conversation, the user's facial expressions, and their voice.

[0397] 2. The server adjusts the response content of the generative AI model based on the recognized emotion data.

[0398] 3. The device displays the adjusted response to the user.

[0399] Analysis of mental status

[0400] 1. The server collects and stores the conversation content with the user and inputs it into the mental care AI model.

[0401] 2. The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[0402] 3. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[0403] Providing appropriate measures

[0404] 1. The server generates appropriate measures according to the user's mental state.

[0405] 2. If the symptoms are mild, the server generates suggestions to encourage relaxation (e.g., relaxation methods or activities).

[0406] 3. If the condition is moderate, the server generates a message urging the patient to seek medical advice from a specialist.

[0407] 4. In case of serious illness, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[0408] 5. The device displays the countermeasure information received from the server to the user.

[0409] Speech monitoring and feedback

[0410] 1. The server monitors the conversation with the user in real time using a separate generative model.

[0411] 2. The server analyzes and determines whether the content contains inappropriate comments.

[0412] 3. If the server detects an inappropriate comment, it generates a feedback message containing the problem with the comment and suggestions for improvement.

[0413] 4. The server sends the generated feedback message to the terminal.

[0414] 5. The device displays a feedback message to the user.

[0415] Specific examples

[0416] Example 1: Initial Setup

[0417] 1. The user enters their preference for a "cheerful and encouraging friend."

[0418] 2. The device sends the information to the server.

[0419] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[0420] 4. The device displays the virtual character to the user and begins a conversation.

[0421] Example 2: Everyday conversation and emotion recognition

[0422] 1. The user types, "I just don't feel like doing anything these days."

[0423] 2. The server analyzes the content of the conversation and the user's tone, and uses an emotion engine to recognize "melancholy."

[0424] 3. The server generates a response and adjusts the tone: "Why don't you take a break and refresh yourself?"

[0425] 4. The terminal displays the response to the user.

[0426] Example 3: Severe cases

[0427] 1. The user types, "I'm sick of everything."

[0428] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[0429] 3. The device will display the message "Please talk to someone now."

[0430] This concludes the description of a specific embodiment for carrying out the invention. This system allows users to receive constant mental health care and take appropriate measures. The addition of an emotion engine provides more personalized and effective responses and support.

[0431] The processing flow will be explained below.

[0432] Initial Setup

[0433] Step 1:

[0434] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the virtual character they desire.

[0435] Step 2:

[0436] The device sends the basic information entered by the user to the server.

[0437] Step 3:

[0438] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[0439] Step 4:

[0440] The server transmits information about the initialized virtual character to the terminal.

[0441] Step 5:

[0442] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[0443] Daily conversation

[0444] Step 1:

[0445] The user uses the terminal to initiate a conversation with the virtual character.

[0446] Step 2:

[0447] The terminal transmits the conversation message input by the user to the server.

[0448] Step 3:

[0449] The server analyzes the received message using a generative AI model and generates an appropriate response.

[0450] Step 4:

[0451] The server sends the generated response message to the terminal.

[0452] Step 5:

[0453] The terminal displays the generated response to the user.

[0454] Emotion recognition

[0455] Step 1:

[0456] During a conversation with a user, the server uses an emotion engine to recognize the user's emotions from input text, voice, and facial expression data.

[0457] Step 2:

[0458] The server adjusts the response content of the generative AI model based on the recognized emotion.

[0459] Step 3:

[0460] The server sends the adjusted response to the terminal.

[0461] Step 4:

[0462] The terminal displays the adjusted response to the user.

[0463] Analysis of mental status

[0464] Step 1:

[0465] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[0466] Step 2:

[0467] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[0468] Step 3:

[0469] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[0470] Providing appropriate measures

[0471] Step 1:

[0472] The server generates appropriate measures depending on the user's mental state.

[0473] Step 2:

[0474] If the symptoms are mild, the server generates suggestions to encourage relaxation (e.g., relaxation methods or activities).

[0475] Step 3:

[0476] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[0477] Step 4:

[0478] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[0479] Step 5:

[0480] The device displays the countermeasure information received from the server to the user.

[0481] Speech monitoring and feedback

[0482] Step 1:

[0483] The server monitors the conversation with the user in real time using a separate generative model.

[0484] Step 2:

[0485] The server analyzes and determines whether the content contains inappropriate comments.

[0486] Step 3:

[0487] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[0488] Step 4:

[0489] The server sends the generated feedback message to the terminal.

[0490] Step 5:

[0491] The device displays a feedback message to the user.

[0492] Specific examples

[0493] Initial Setup

[0494] Step 1:

[0495] The user inputs desired information about a "cheerful and encouraging friend" into the terminal interface.

[0496] Step 2:

[0497] The terminal sends this information to the server.

[0498] Step 3:

[0499] The server trains a generative AI model and initializes a virtual character based on the user's preferences.

[0500] Step 4:

[0501] The terminal displays the initialized virtual character to the user.

[0502] Everyday conversation and emotion recognition

[0503] Step 1:

[0504] A user types, "I feel very depressed today."

[0505] Step 2:

[0506] The terminal sends the input message to the server.

[0507] Step 3:

[0508] The server analyzes the message and uses an emotion engine to recognize the user's "depressed" emotion.

[0509] Step 4:

[0510] The server generates a response based on the recognized emotion: "How about a walk to change your mood?"

[0511] Step 5:

[0512] The terminal displays the generated response to the user.

[0513] Severe cases

[0514] Step 1:

[0515] The user types, "I'm sick of everything."

[0516] Step 2:

[0517] The device sends a message to the server.

[0518] Step 3:

[0519] The server determines this to be a sign of serious illness and automatically sends a notification to pre-set emergency contacts.

[0520] Step 4:

[0521] The device displays the message, "Let's talk to someone now."

[0522] The above are the detailed processing steps of a specific embodiment for carrying out the invention. With this system, users can receive mental care by conversing with a virtual character at any time. In addition, the emotion engine enables personalized responses based on the user's emotions, providing more effective support.

[0523] Example 2

[0524] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0525] In modern society, mental health management is becoming increasingly important due to busy daily lives and increased stress. In particular, the number of individuals suffering from loneliness and mental health problems is increasing, while there is a lack of mental health care systems that can provide appropriate measures. In this situation, there is a need for a system that is easily accessible to users, analyzes each individual's mental state, and can take appropriate measures.

[0526] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0527] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate countermeasures based on the analysis results, means for monitoring the content of the conversation using a separate generative model and generating feedback when inappropriate remarks are made, and means for recognizing the user's emotions using an emotion engine and adjusting the response content. This allows the user to easily monitor their own mental state and receive appropriate feedback and countermeasures.

[0528] A "virtual character" is a pseudo-being that is generated based on basic information set by the user and provides emotional support to the user through dialogue.

[0529] "Basic information" refers to personal setting data such as appearance and personality that a user provides when generating a virtual character.

[0530] A "generative AI model" is an artificial intelligence technology used to generate virtual characters based on basic information provided by users, and to provide conversation and feedback.

[0531] "Means of everyday conversation" refers to the processes and functions that allow users and virtual characters to communicate on an ongoing basis.

[0532] "Means for analyzing mental state" refers to methods or systems for analyzing the content of conversations with users and assessing their psychological and emotional state.

[0533] "Means for providing appropriate measures" refers to functions and mechanisms that provide necessary actions and suggestions to users based on the results of an analysis of their mental state.

[0534] "Means of monitoring using generative models" refers to a mechanism for analyzing conversation content in real time and detecting and monitoring inappropriate remarks.

[0535] "Means for generating feedback" refers to a function that reports inappropriate comments to users when they are detected and suggests ways to improve them.

[0536] An "emotion engine" is a technology that analyzes a user's facial expressions, voice, and conversation content to recognize their emotions.

[0537] The "means for adjusting response content" is a function for appropriately changing the response of the virtual character based on the recognized emotions of the user.

[0538] "Mild" refers to a state in which the user's mental state shows relatively mild stress or anxiety.

[0539] "Moderate" refers to a state in which the user's mental state indicates moderate stress or anxiety and may require specialized care.

[0540] "Severe" refers to a user's mental health condition that is severe enough to require immediate professional intervention.

[0541] "Emergency Contacts" refers to pre-defined contacts such as family and friends who will be automatically notified in the event of a serious illness.

[0542] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through everyday conversations with the virtual character, and provides appropriate countermeasures. It also includes a function that recognizes the user's emotions using an emotion engine and adjusts the response content. It also has a function that monitors the conversation content with a separate generative model and generates feedback if inappropriate remarks are made.

[0543] Initial Setup

[0544] 1. The user inputs basic information about the desired virtual character through the terminal interface. This basic information includes the virtual character's appearance and personality. For example, the user may input their desire for a "cheerful and encouraging friend."

[0545] 2. The device sends the information entered by the user to the server in the form of an HTTP request.

[0546] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character. A general artificial intelligence model can be used as a specific example of the generative AI model used here. The server generates the virtual character based on the prompt provided by the user.

[0547] 4. The server sends information about the virtual character that has been initialized to the device. This data is often sent in JSON format.

[0548] 5. The device displays the virtual character to the user, ready for daily conversation. Specifically, the virtual character's avatar and initial message are displayed on the screen.

[0549] Everyday conversation and emotion recognition

[0550] 1. The user starts a conversation with a virtual character using the terminal. For example, the user may type, "I haven't felt motivated to do anything recently."

[0551] 2. The device sends the conversation message entered by the user to the server. This data is also sent via an HTTP request.

[0552] 3. The server analyzes the received message using a generative AI model and generates an appropriate response. The generative model uses prompts to generate natural-sounding dialogue.

[0553] 4. The server sends the generated response message to the terminal. The generated data is sent again in JSON format.

[0554] 5. The terminal displays the generated response message to the user, adjusting the displayed message to fit the flow of the conversation.

[0555] Recognizing emotions and regulating responses

[0556] 1. The server uses an emotion engine to recognize emotions from the conversation content, the user's facial expressions, and their voice. For example, it can use Google Cloud's Speech-to-Text API or Emotion Detection API.

[0557] 2. The server adjusts the response of the generative AI model based on the recognized emotion data. For example, if the user is recognized as "depressed," the server generates a response such as "Why don't you take a break and refresh yourself?"

[0558] 3. The device displays the tailored response to the user. The displayed message is appropriately tailored to match the user's emotions.

[0559] Analysis of mental health conditions and provision of appropriate measures

[0560] 1. The server collects and stores the content of conversations with users and inputs it into the mental care AI model, including conversation logs and emotional data.

[0561] 2. The server analyzes the user's mental state from the conversation content using a mental care AI model, such as IBM Watson or Azure Cognitive Services.

[0562] 3. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[0563] 4. The server generates appropriate measures based on the user's mental state. For example, if the condition is mild, it suggests relaxation methods or activities, if the condition is moderate, it generates a message urging the user to seek medical advice, and if the condition is severe, it sends an automatic notification to pre-defined emergency contacts.

[0564] 5. The device displays the countermeasure information received from the server to the user.

[0565] Speech monitoring and feedback

[0566] 1. The server monitors the conversation with the user in real time using a separate generative model, which checks for inappropriate comments throughout the conversation.

[0567] 2. The server analyzes and determines whether the message contains inappropriate content. This analysis uses natural language processing technology.

[0568] 3. If the server detects inappropriate comments, it generates a feedback message containing the problem with the comment and suggestions for improvement.

[0569] 4. The server sends the generated feedback message to the terminal.

[0570] 5. The terminal displays a feedback message to the user.

[0571] Through the above process, the system allows users to easily monitor their own mental state and receive appropriate measures and feedback, allowing users to receive daily mental care, and the addition of an emotion engine provides more personalized and effective support.

[0572] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0573] Step 1:

[0574] The user inputs basic information about the desired virtual character through the device interface. The input information includes the virtual character's concept, appearance, personality, etc. As for specific actions, the user sets the character as a "cheerful and encouraging friend." The input data is then sent to the device.

[0575] Input: Basic information about the virtual character (name, appearance, personality)

[0576] Output: Basic information sent to the terminal

[0577] Step 2:

[0578] The device sends the basic information entered by the user to the server. The device generates an HTTP request and includes the data entered by the user. The information is sent to the server.

[0579] Input: Basic information from the user

[0580] Output: HTTP request sent to the server

[0581] Step 3:

[0582] The server customizes the generated AI model and initializes the virtual character based on the received basic information. The server uses prompts to give instructions to the generated AI model and configure the character. Data for the generated virtual character is obtained.

[0583] Input: Basic information (prompt statement)

[0584] Output: Data of the generated virtual character

[0585] Step 4:

[0586] The server sends information about the initialized virtual character in JSON format to the device. The server generates an HTTP response and sends it to the device, including the initialized character data.

[0587] Input: Data of the generated virtual character

[0588] Output: JSON formatted character data sent to the terminal

[0589] Step 5:

[0590] The terminal displays the virtual character to the user and prepares the user for daily conversation. The terminal analyzes the received character data and displays the character's avatar and initial message on the screen.

[0591] Input: Character data from the server

[0592] Output: The virtual character and initial message displayed to the user.

[0593] Step 6:

[0594] The user starts a conversation with the virtual character using the terminal. For example, the user inputs a message such as "I haven't been feeling motivated to do anything lately." This message is input into the terminal.

[0595] Input: User's message

[0596] Output: Message typed into the terminal

[0597] Step 7:

[0598] The terminal sends the message entered by the user to the server. The terminal generates an HTTP request and sends it to the server including the message data.

[0599] Input: User's message

[0600] Output: HTTP request sent to the server

[0601] Step 8:

[0602] The server analyzes the received message using a generative AI model and generates an appropriate response. The generative AI model creates a prompt based on the received message and generates an appropriate response. Response data is generated.

[0603] Input: User's message

[0604] Output: The generated response data

[0605] Step 9:

[0606] The server sends the generated response message to the terminal. The server converts the response data into JSON format, generates an HTTP response, and sends it to the terminal.

[0607] Input: Generated response data

[0608] Output: JSON-formatted response data sent to the terminal

[0609] Step 10:

[0610] The terminal displays the generated response to the user, analyzes the received response data, and displays it on the screen.

[0611] Input: Response data from the server

[0612] Output: The response message displayed to the user

[0613] Step 11:

[0614] The server uses an emotion engine to recognize emotions from the content of the conversation, the user's facial expressions, and their voice. For example, it runs an emotion recognition algorithm based on data input by the user through a camera or microphone, and evaluates the user's emotions.

[0615] Input: User's facial expressions and voice data

[0616] Output: Recognized emotion data

[0617] Step 12:

[0618] The server adjusts the response content of the generative AI model based on the recognized emotion data, and uses the emotion data to modify the response content and generate a message that is appropriate for the user's emotion.

[0619] Input: Recognized emotion data

[0620] Output: Adjusted response data

[0621] Step 13:

[0622] The terminal displays the adjusted response to the user. The adjusted message is parsed and displayed on the screen.

[0623] Input: Reconciled response data from the server

[0624] Output: The adjusted response message displayed to the user

[0625] Step 14:

[0626] The server collects and stores the content of conversations with users and inputs it into the mental care AI model. The server also stores the conversation log in a database and provides the content of the conversation to the mental care model.

[0627] Input: conversation log

[0628] Output: Conversation data input into the mental care model

[0629] Step 15:

[0630] The server uses a mental care AI model to analyze the user's mental state from the conversation content, executes the analysis algorithm, and evaluates the user's mental state.

[0631] Input: Conversation data

[0632] Output: Analyzed mental state data

[0633] Step 16:

[0634] Based on the analysis results, the server classifies the user's mental state into mild, moderate, or severe. Based on the analyzed data, the server classifies the user into the appropriate category.

[0635] Input: Parsed mental state

[0636] Output: Classified mental state data

[0637] Step 17:

[0638] The server generates appropriate measures based on the user's mental state, such as a suggestion to encourage relaxation in mild cases, a message to seek medical advice in moderate cases, and a notification to emergency contacts in severe cases.

[0639] Input: Classified mental state data

[0640] Output: Generated countermeasure data

[0641] Step 18:

[0642] The terminal displays the countermeasure information received from the server to the user, analyzes the generated countermeasure message, and displays it on the screen.

[0643] Input: Countermeasure data from the server

[0644] Output: The action message displayed to the user

[0645] Step 19:

[0646] The server monitors the conversation with the user in real time using a separate generative model, checking for inappropriate comments throughout the conversation.

[0647] Input: User conversation

[0648] Output: Inappropriate remark detection results

[0649] Step 20:

[0650] The server analyzes and determines whether the remarks contain inappropriate content, using natural language processing technology to evaluate the content of the remarks.

[0651] Input: User conversation

[0652] Output: Inappropriate remarks judgement result

[0653] Step 21:

[0654] When an inappropriate comment is detected, the server generates a feedback message including the problem with the comment and a remedy for the comment. Feedback is generated for the user based on the content of the inappropriate comment.

[0655] Input: Inappropriate comment judgment result

[0656] Output: The generated feedback message

[0657] Step 22:

[0658] The server sends the generated feedback message to the device. The feedback data is converted into JSON format and sent to the device.

[0659] Input: The generated feedback message

[0660] Output: Feedback data sent to the device

[0661] Step 23:

[0662] The terminal displays the feedback message to the user, parses the feedback message, and displays it on the screen.

[0663] Input: Feedback data from the server

[0664] Output: The feedback message displayed to the user

[0665] (Application example 2)

[0666] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0667] In modern society, there is a demand for real-time recognition of changes in a user's mental state and emotions and for prompt provision of appropriate measures. However, conventional systems are unable to perform a sufficient detailed analysis of a user's mental state and emotions, which often delays appropriate measures. In particular, there is a lack of means for providing prompt and effective feedback when inappropriate comments are made. The problem that this invention aims to solve is to comprehensively solve these problems and provide more personalized mental care and feedback to users.

[0668] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0669] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the user's mental state, means for providing appropriate countermeasures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, means for classifying the condition as mild, moderate, or severe based on the analysis results, means for recognizing the user's emotions from the content of the conversation, facial expressions, and voice, and means for generating a feedback message containing problems with the remarks and measures for improving them if inappropriate remarks are detected. This makes it possible to analyze the user's mental state and emotions in detail in real time and quickly provide appropriate countermeasures and feedback.

[0670] Below we create a definition sentence for each important word:

[0671] "User" refers to a user who interacts with a virtual character.

[0672] "Basic information" refers to information about appearance and personality provided by the user to generate a virtual character.

[0673] A "virtual character" is a character that is generated based on basic information set by the user and that engages in conversation.

[0674] "Initializing" means using a generative AI model to set the appearance and personality of a virtual character and generate that character.

[0675] "Conversation" refers to communication between a user and a virtual character through text or voice.

[0676] "Analyzing" means processing data such as conversation content, facial expressions, and voice, and extracting specific information.

[0677] "Mental state" refers to a user's psychological health and emotional state.

[0678] An "emotion engine" refers to a software component that recognizes a user's emotions from conversation content, voice, facial expressions, etc.

[0679] "Mild, moderate, severe" refers to different levels of psychological health classified based on the analysis of the user's mental state.

[0680] "Appropriate measures" refer to mental care and feedback provided according to the user's mental state.

[0681] A "generative model" refers to an algorithm or system for generating natural language using AI.

[0682] "Inappropriate comments" refers to comments made in a user's conversation that may be offensive or dangerous.

[0683] "Feedback" refers to responses or advice provided to users by virtual characters or systems.

[0684] A "feedback message" is a message that includes solutions or suggestions for improvement to inappropriate comments.

[0685] This system initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system includes a function that recognizes the user's emotions by combining it with an emotion engine and responds individually. It also has a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[0686] Program Overview

[0687] 1. Initial Setup:

[0688] The user accesses the device's interface and inputs basic information (e.g., appearance, personality) about the desired virtual character. The device then sends the information the user entered to the server. The server customizes the generative AI model based on the received user data and initializes the virtual character. The server then sends the initialized virtual character information to the device, and the device displays the virtual character to the user and begins daily conversation.

[0689] 2. Daily conversation:

[0690] The user initiates a conversation with a virtual character using the device. The device sends the conversation message entered by the user to the server. The server analyzes the received message using a generative AI model and generates an appropriate response. The server then sends the generated response message to the device, which then displays the generated response to the user.

[0691] 3. Emotion Recognition:

[0692] The server uses an emotion engine to recognize emotions from the conversation content, the user's facial expressions, and their voice. The server then adjusts the response content of the generative AI model based on the recognized emotion data. The device then displays the adjusted response to the user.

[0693] 4. Mental State Analysis:

[0694] The server inputs the conversation content with the user into the mental care AI model based on the data. The server then analyzes the user's mental state from the conversation content using the mental care AI model. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[0695] 5. Providing appropriate measures:

[0696] The server generates appropriate measures according to the user's mental state. In mild cases, it generates suggestions to encourage mood changes (e.g., relaxation methods or activities). In moderate cases, it generates a message urging the user to see a specialist. In severe cases, it sends an automatic notification to pre-set emergency contacts (family or friends). The device displays the measures information received from the server to the user.

[0697] 6. Speech monitoring and feedback:

[0698] The server monitors the conversation with the user in real time using a separate generative model. It analyzes and determines whether any inappropriate remarks are included. If an inappropriate remark is detected, it generates a feedback message containing the problem with the remark and a suggestion for improvement. The server sends the generated feedback message to the device, which then displays it to the user.

[0699] Hardware and software used

[0700] The system of the present invention uses the following hardware and software:

[0701] Smartphone: A device that provides a user interface

[0702] Server: Server for data processing and running the generative model (e.g., Amazon Web Services, Microsoft Azure)

[0703] Generative AI models: models for natural language processing (e.g., OpenAI GPT-3, ChatGPT)

[0704] Emotion engine: Software for emotion recognition (e.g., IBM Watson, Microsoft Azure Emotion API)

[0705] Profanity detection engine: Software for monitoring and analyzing profanity (e.g., Perspective API)

[0706] Specific examples

[0707] The following are specific examples of how this system can be used:

[0708] Example 1: Initial Setup

[0709] The user inputs their preference for a "cheerful and encouraging friend." The device sends this information to the server. The server trains a generative AI model and initializes a "cheerful and encouraging" character. The device then displays the virtual character to the user and begins a conversation.

[0710] Example 2: Everyday conversation and emotion recognition

[0711] The user types, "I've been feeling unmotivated lately." The server analyzes the conversation and the user's tone, and recognizes "depression" using an emotion engine. The server generates a response, such as, "Why don't you take a break and refresh yourself?" and adjusts the tone accordingly. The device displays the response to the user.

[0712] Example 3: Severe cases

[0713] The user types, "I've had enough of everything." The server determines that the condition is serious and automatically sends a notification to pre-defined family members. The device then displays the message, "Try to talk to someone right now."

[0714] Example prompt sentence:

[0715] "I'm looking for a friend who is cheerful and good at encouraging others."

[0716] "I just can't seem to get motivated to do anything these days. What should I do?"

[0717] In this way, the present invention allows users to receive constant mental health care and take appropriate measures. The addition of an emotion engine provides more personal and effective responses and support.

[0718] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0719] Program processing steps

[0720] Step 1:

[0721] The user enters basic information about the virtual character.

[0722] Input: Basic information about the virtual character the user desires (appearance, personality, etc.)

[0723] How it works: Through a smartphone app, users input information about the appearance and personality of their desired virtual character.

[0724] Output: The basic information of the virtual character entered is saved on the device.

[0725] Step 2:

[0726] The device sends the user's basic information to the server

[0727] Input: Basic information of the saved virtual character

[0728] How it works: The device sends the information entered by the user to the server using an HTTP request or similar.

[0729] Output: Basic information about the virtual character is sent to the server.

[0730] Step 3:

[0731] The server initializes the virtual character.

[0732] Input: User basic information

[0733] How it works: The server initializes a virtual character using a generative AI model (e.g. ChatGPT) and the basic information mentioned above. The generative AI model customizes the character's appearance and personality, and generates the virtual character.

[0734] Output: Initialized virtual character data

[0735] Step 4:

[0736] The server sends the initialized virtual character to the device.

[0737] Input: Initialized virtual character data

[0738] Operation: The server sends the virtual character data to the device.

[0739] Output: The virtual character data is transferred to the device.

[0740] Step 5:

[0741] The device displays the virtual character to the user.

[0742] Input: Virtual character data

[0743] How it works: The device displays an initialized virtual character on the user's screen and allows the user to initiate a conversation with the virtual character.

[0744] Output: The user sees a virtual character on the screen.

[0745] Step 6:

[0746] The user initiates a conversation with a virtual character

[0747] Input: User's conversation message

[0748] How it works: Users use a smartphone app to type messages to a virtual character.

[0749] Output: Conversation messages entered by the user are saved on the device.

[0750] Step 7:

[0751] The device sends the user's conversation messages to the server

[0752] Input: Saved conversation messages

[0753] Operation: The device sends the user's conversation message to the server.

[0754] Output: Conversation messages forwarded to the server

[0755] Step 8:

[0756] The server analyzes conversation messages using a generative AI model

[0757] Input: User's conversation message

[0758] How it works: The server uses a generative AI model (e.g. ChatGPT) to analyze conversational messages and generate appropriate responses.

[0759] Output: The generated response message

[0760] Step 9:

[0761] Sends the server-generated response message to the terminal

[0762] Input: Response message

[0763] Operation: The server generates a response message and sends it to the terminal.

[0764] Output: The response message is delivered to the terminal.

[0765] Step 10:

[0766] The terminal displays the generated response to the user

[0767] Input: Response message

[0768] Action: The terminal generates a response message and displays it on the user's screen.

[0769] Output: A response message that the user can see on their screen.

[0770] Step 11:

[0771] The server uses an emotion engine to analyze the content of the conversation, the user's facial expressions, and the voice.

[0772] Input: Conversation messages, user facial expression data, voice data

[0773] How it works: The server uses an emotion engine (e.g. IBM Watson) to analyze the conversation, facial expressions, and voice to recognize the user's emotions.

[0774] Output: User emotion data (e.g., sadness, joy)

[0775] Step 12:

[0776] The server adjusts the response based on the emotional data.

[0777] Input: Emotion data, response message

[0778] How it works: The server adjusts the response of the generative AI model based on the emotion data.

[0779] Output: Reconciled response message

[0780] Step 13:

[0781] The server inputs the conversation content into a mental care AI model to analyze the mental state.

[0782] Input: Conversation data

[0783] How it works: The server inputs the conversation content into a mental care AI model and analyzes the user's mental state.

[0784] Output: Mental status data (mild, moderate, severe)

[0785] Step 14:

[0786] The server generates countermeasures according to the mental state.

[0787] Input: Mental state data

[0788] How it works: The server generates corresponding measures based on the mental state (suggestions for relaxation, facilitating professional consultation, notifying emergency contacts).

[0789] Output: Solution message

[0790] Step 15:

[0791] The server monitors inappropriate comments using a separate generative model.

[0792] Input: Conversation data

[0793] How it works: The server uses another generative model (e.g., Perspective API) to monitor conversations in real time and detect inappropriate comments.

[0794] Output: Inappropriate speech detection results

[0795] Step 16:

[0796] Generate a feedback message if inappropriate language is detected

[0797] Input: Inappropriate speech detection results

[0798] Behavior: The server generates a feedback message (problem and remediation) about the inappropriate comment.

[0799] Output: Feedback message

[0800] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0801] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0802] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0803] [Second embodiment]

[0804] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0805] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0806] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0807] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0808] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0809] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0810] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0811] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0812] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0813] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0814] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0815] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0816] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[0817] Below, we will create a program for this system and explain the specific processing in natural language.

[0818] Program Overview

[0819] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on this information, the server initializes the character using a generative AI model and begins a daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state. Based on the evaluation, appropriate measures are offered and feedback is generated as necessary. The system also monitors the content of the conversation using a separate generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user.

[0820] Initial Setup

[0821] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0822] 2. The device sends the information entered by the user to the server.

[0823] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[0824] 4. The server sends the initialized virtual character to the device.

[0825] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[0826] Daily conversation

[0827] 1. The user uses the device to have everyday conversations with a virtual character.

[0828] 2. The device sends the message entered by the user to the server.

[0829] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[0830] 4. The server generates a response and sends it to the terminal.

[0831] 5. The terminal displays the generated response to the user.

[0832] Analysis of mental status

[0833] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[0834] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[0835] 3. The server generates appropriate measures according to the user's mental state.

[0836] Providing appropriate measures

[0837] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[0838] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[0839] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[0840] 4. The device displays the countermeasures from the server to the user.

[0841] Speech monitoring and feedback

[0842] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[0843] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[0844] 3. The device displays feedback to the user.

[0845] Specific examples

[0846] Example 1: Initial Setup

[0847] 1. The user enters their preference for a "cheerful and encouraging friend."

[0848] 2. The device sends the information to the server.

[0849] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[0850] 4. The device displays the virtual character to the user and begins a conversation.

[0851] Example 2: Analysis of everyday conversations and mental states

[0852] 1. The user types, "I just don't feel like doing anything these days."

[0853] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[0854] 3. The terminal displays the response to the user.

[0855] 4. The server analyzes the conversation and determines that the injury is minor.

[0856] Example 3: Severe cases

[0857] 1. The user types, "I'm sick of everything."

[0858] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[0859] 3. The device will display the message "Please talk to someone now."

[0860] This concludes the explanation of the specific form for carrying out the invention. With this system, users can receive mental care at any time by talking to a virtual character. In addition, if the condition is severe, a notification is sent to the appropriate contact person, so users can receive prompt and appropriate treatment without having to worry alone.

[0861] The processing flow will be explained below.

[0862] Initial Setup

[0863] Step 1:

[0864] The user accesses the terminal interface and inputs basic information (such as appearance and personality) about the virtual character they desire.

[0865] Step 2:

[0866] The device sends the basic information entered by the user to the server.

[0867] Step 3:

[0868] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[0869] Step 4:

[0870] The server transmits information about the initialized virtual character to the terminal.

[0871] Step 5:

[0872] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[0873] Daily conversation

[0874] Step 1:

[0875] The user uses the terminal to initiate a conversation with the virtual character.

[0876] Step 2:

[0877] The terminal transmits the conversation message input by the user to the server.

[0878] Step 3:

[0879] The server analyzes the received message using a generative AI model and generates an appropriate response.

[0880] Step 4:

[0881] The server sends the generated response message to the terminal.

[0882] Step 5:

[0883] The terminal displays the response message received from the server to the user.

[0884] Analysis of mental status

[0885] Step 1:

[0886] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[0887] Step 2:

[0888] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[0889] Step 3:

[0890] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[0891] Providing appropriate measures

[0892] Step 1:

[0893] The server generates appropriate measures depending on the user's mental state.

[0894] Step 2:

[0895] If the symptoms are mild, the server generates suggestions to encourage mood change (e.g., relaxation methods or activities).

[0896] Step 3:

[0897] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[0898] Step 4:

[0899] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[0900] Step 5:

[0901] The device displays the countermeasure information received from the server to the user.

[0902] Speech monitoring and feedback

[0903] Step 1:

[0904] The server monitors the conversation with the user in real time using a separate generative model.

[0905] Step 2:

[0906] The server analyzes and determines whether the content contains inappropriate comments.

[0907] Step 3:

[0908] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[0909] Step 4:

[0910] The server sends the generated feedback message to the terminal.

[0911] Step 5:

[0912] The device displays a feedback message to the user.

[0913] The above is a detailed description of the specific operations performed in each processing step. This system allows users to receive constant mental care and take appropriate measures.

[0914] Example 1

[0915] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0916] Mental health care has become an important issue in modern society. However, there are limited systems that allow individual users to easily receive mental health care at any time. There is also a lack of mechanisms that allow users in serious mental health conditions to quickly receive appropriate support. Furthermore, there is a need for a system that can immediately point out inappropriate comments made during conversations with users and provide appropriate feedback.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0918] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to evaluate the user's mental state, means for providing appropriate measures based on the evaluation results, and means for monitoring the content of the conversation using a separate generative model and generating feedback if inappropriate comments are made. This allows the user to receive mental care through conversations with the virtual character, and in severe cases, appropriate measures are taken promptly, allowing the user to receive support without feeling isolated. In addition, immediate feedback is provided for inappropriate comments made during conversations, encouraging the user to improve.

[0919] "Basic information" is information about the appearance and temperament of the virtual character set by the user.

[0920] A "virtual character" is an interactable character that is initialized based on basic information about the user using a generative AI model.

[0921] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input and prompts.

[0922] "Means for daily conversation" refers to a system function that allows users and virtual characters to have ongoing dialogue.

[0923] "Means for analyzing conversation content" refers to technology for analyzing the content of conversation between a virtual character and a user and assessing the user's mental state.

[0924] "Mental state" refers to the user's psychological and emotional state, and is classified as mild, moderate, severe, etc.

[0925] "Means for providing appropriate measures" refers to a system function that provides measures based on the user's mental state, such as diversion, consultation with a specialist, or in some cases notification to emergency contacts.

[0926] A "separate generative model" is a separate artificial intelligence model designed to monitor conversation content and detect inappropriate remarks.

[0927] "Means for generating feedback" refers to a system function that provides users with information about the problem and possible solutions when inappropriate comments are made.

[0928] This invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made. Specific embodiments of this system are described below.

[0929] Hardware and software configuration used

[0930] The server is a high-performance computing environment for running generative AI models (e.g., GPT-3) and mental care AI models. The server sends and receives data to and from the user's device via the HTTP protocol.

[0931] The terminal is the device (e.g., smartphone, tablet, or PC) through which the user operates the interface. The terminal is responsible for collecting user input and sending it to the server. It also displays virtual characters and response messages received from the server to the user.

[0932] Program processing

[0933] Initial Setup

[0934] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0935] 2. The device sends the information entered by the user to the server.

[0936] 3. Based on the received user information, the server customizes the generative AI model and generates a virtual character.

[0937] 4. The server sends the generated virtual character to the device.

[0938] 5. The device displays the virtual character to the user, allowing them to start a daily conversation.

[0939] Daily conversation

[0940] 1. The user uses the device to converse with a virtual character.

[0941] 2. The device sends the message entered by the user to the server.

[0942] 3. The server analyzes the message sent using a generative AI model and generates an appropriate response.

[0943] 4. The server generates a response and sends it to the terminal.

[0944] 5. The terminal displays the response to the user.

[0945] Analysis of mental status

[0946] 1. The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[0947] 2. Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe.

[0948] 3. The server generates appropriate countermeasures based on the classification results.

[0949] Providing appropriate measures

[0950] 1. The server generates suggestions for users who are judged to have mild symptoms, such as "Why don't you take a walk to change your mood?"

[0951] 2. The server generates a message for users who are judged to be in moderate condition, such as "Please consult a specialist."

[0952] 3. The server automatically sends notifications to pre-defined contacts for users who are deemed to be in a serious condition.

[0953] 4. The device displays the countermeasures sent from the server to the user.

[0954] Speech monitoring and feedback

[0955] 1. The server uses a separate generative model to monitor the conversation in real time.

[0956] 2. When the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[0957] 3. The device displays this feedback to the user.

[0958] Examples of concrete examples and prompts

[0959] Example 1: Initial Setup

[0960] 1. The user enters their preference for a "cheerful and encouraging friend."

[0961] 2. The device sends the information to the server.

[0962] 3. The server trains a generative AI model to generate a "cheerful and encouraging" character.

[0963] 4. The device displays the virtual character to the user and begins a conversation.

[0964] Example prompt sentence:

[0965] "Create a character that is cheerful and encouraging."

[0966] Example 2: Analysis of everyday conversations and mental states

[0967] 1. The user types, "I just don't feel like doing anything these days."

[0968] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[0969] 3. The terminal displays the response to the user.

[0970] 4. The server analyzes the conversation and determines that the injury is minor.

[0971] Example 3: Severe cases

[0972] 1. The user types, "I'm sick of everything."

[0973] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[0974] 3. The device will display the message "Please talk to someone now."

[0975] This system allows users to receive mental health care through conversations with virtual characters at any time, and in severe cases, prompt and appropriate support is provided. It also provides immediate feedback on inappropriate remarks made during conversations, encouraging users to work on self-improvement.

[0976] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0977] Step 1:

[0978] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[0979] Input: Basic information about the virtual character (appearance, personality) entered by the user.

[0980] Output: Basic information data entered into the terminal.

[0981] Step 2:

[0982] The terminal sends the information entered by the user to the server.

[0983] Input: Basic information data entered by the user.

[0984] Data processing: Convert the input data into JSON format.

[0985] Output: JSON data sent as an HTTP POST request to the server.

[0986] Step 3:

[0987] Based on the received user information, the server customizes the generative AI model to generate a virtual character.

[0988] Input: Basic user information data received by the server.

[0989] Data computation: Using prompts from a generative AI model (e.g., GPT-3) to generate a virtual character based on basic information.

[0990] Output: Data of the generated virtual character.

[0991] Step 4:

[0992] The server transmits the generated virtual character to the terminal.

[0993] Input: Data of the generated virtual character.

[0994] Data processing: Encode the virtual character data.

[0995] Output: The encoded data that is sent as an HTTP response to the device.

[0996] Step 5:

[0997] The device displays a virtual character to the user, allowing them to start a daily conversation.

[0998] Input: Encoded virtual character data received from the server.

[0999] Data processing: Decoded virtual character data.

[1000] Output: A virtual character displayed in the user interface.

[1001] Step 6:

[1002] The user uses the terminal to converse with the virtual character.

[1003] Input: The conversation message that the user types.

[1004] Output: A conversation message with information entered on the device.

[1005] Step 7:

[1006] The terminal sends the message entered by the user to the server.

[1007] Input: The message entered by the user.

[1008] Data processing: Convert messages to JSON format.

[1009] Output: JSON data sent as an HTTP POST request to the server.

[1010] Step 8:

[1011] The server analyzes the message sent using a generative AI model and generates an appropriate response.

[1012] Input: The user's message data as received by the server.

[1013] Data Computation: Using generative AI models to analyze messages and generate appropriate responses.

[1014] Output: The generated response data.

[1015] Step 9:

[1016] The server sends the generated response to the terminal.

[1017] Input: The generated response data.

[1018] Data processing: Encode the response data.

[1019] Output: The encoded data that is sent as an HTTP response to the device.

[1020] Step 10:

[1021] The terminal displays the generated response to the user.

[1022] Input: The encoded response data received from the server.

[1023] Data processing: Decoded response data.

[1024] Output: The response message that is displayed in the user interface.

[1025] Step 11:

[1026] The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[1027] Input: Data from everyday conversations.

[1028] Data calculation: A mental state assessment process using a mental care AI model.

[1029] Output: Assessment result (mild, moderate, severe).

[1030] Step 12:

[1031] Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe and generates appropriate countermeasures.

[1032] Input: Mental status assessment results.

[1033] Data calculation: Generation of countermeasures according to mental state.

[1034] Output: Generated countermeasure data.

[1035] Step 13:

[1036] The server generates appropriate messages and notifications for mild, moderate, and severe cases and sends them to the device or to configured contacts.

[1037] Input: Mental state classification results and generated measures data.

[1038] Data processing: Encode notification data as needed.

[1039] Output: Notification data to user device or contacts.

[1040] Step 14:

[1041] The device displays the countermeasures sent from the server to the user.

[1042] Input: Countermeasure data received from the server.

[1043] Data processing: Decoded countermeasure data.

[1044] Output: The action message that is displayed in the user interface.

[1045] Step 15:

[1046] The server uses another generative model to monitor conversations in real time and detect inappropriate remarks.

[1047] Input: Real-time conversation data.

[1048] Data computation: Detecting profanity using alternative generative models.

[1049] Output: Detection result (whether or not there is any inappropriate speech).

[1050] Step 16:

[1051] If the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[1052] Input: Profanity detection result data.

[1053] Data computation: Speech content analysis and feedback generation.

[1054] Output: Feedback data including issues and improvements.

[1055] Step 17:

[1056] The device displays feedback to the user.

[1057] Input: Feedback data received from the server.

[1058] Data processing: Decoded feedback data.

[1059] Output: Feedback message displayed in the user interface.

[1060] (Application example 1)

[1061] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1062] In modern society, stress and mental health problems are on the rise, and companies are placing importance on supporting the mental health of their employees. However, it is difficult to provide appropriate measures in a timely manner using conventional care methods, and there is a need for a system that can respond quickly, especially in emergencies.

[1063] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1064] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate measures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, and means for automatically sending a notification to a set emergency contact if the mental state is determined to be serious, thereby enabling real-time care for employee mental health and rapid response to emergencies.

[1065] "User" refers to a person who uses the system.

[1066] "Basic information" refers to the initial setting information such as the appearance and personality of the virtual character entered by the user.

[1067] A "virtual character" is a character generated by a generative AI model based on the user's basic information.

[1068] "Initialization" refers to the process of generating a virtual character based on basic information set by the user.

[1069] "Means for daily conversation" refers to a system that allows users and virtual characters to have daily conversations.

[1070] "Conversation content" refers to the text and audio data of the dialogue between the user and the virtual character.

[1071] "Mental state" refers to the user's psychological health and mood.

[1072] "Analysis" refers to the process of evaluating the content of a conversation and determining the user's mental state.

[1073] "Appropriate measures" refer to countermeasures such as advice and suggested actions provided to users based on the analysis results.

[1074] A "generative model" refers to an AI algorithm that has been trained to accomplish a specific task.

[1075] "Inappropriate comments" refer to comments that may have a negative impact on the user's mental health.

[1076] "Feedback" refers to advice provided in response to inappropriate comments for correction or improvement.

[1077] "Emergency Contacts" refers to configured contacts to whom automatic notifications are sent if a user's mental condition is deemed severe.

[1078] "Automatic Notification" refers to a message that the system automatically sends when certain conditions are met.

[1079] This system generates a virtual character based on basic information provided by the user, analyzes the user's mental state through daily conversations, and provides appropriate measures. The system also includes a function to monitor the conversation content using a separate generative model and generate feedback if inappropriate remarks are made. It also has a function to automatically send a notification to emergency contacts if the user's mental state is determined to be severe.

[1080] System configuration

[1081] The system for implementing the present invention is mainly composed of the following hardware and software.

[1082] Hardware: Smartphone (Android or iOS), Server (Cloud service such as AWS, Google Cloud, Microsoft Azure, etc.)

[1083] Software: Smartphone applications (e.g., React Native, Swift, Kotlin), generative AI models (e.g., OpenAI's GPT-4), and mental health AI models (e.g., BERT-based filtering technology).

[1084] Program Overview

[1085] Initial Setup

[1086] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1087] 2. The device sends the information entered by the user to the server.

[1088] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[1089] 4. The server sends the initialized virtual character to the device.

[1090] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[1091] Daily conversation

[1092] 1. The user uses the device to have everyday conversations with a virtual character.

[1093] 2. The device sends the message entered by the user to the server.

[1094] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[1095] 4. The server generates a response and sends it to the terminal.

[1096] 5. The terminal displays the generated response to the user.

[1097] Analysis of mental status

[1098] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[1099] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[1100] 3. The server generates appropriate measures according to the user's mental state.

[1101] Providing appropriate measures

[1102] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[1103] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[1104] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[1105] 4. The device displays the countermeasures from the server to the user.

[1106] Speech monitoring and feedback

[1107] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[1108] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[1109] 3. The device displays feedback to the user.

[1110] Description and examples

[1111] Hardware and software used

[1112] The system begins by sending basic information from the user via a smartphone application to a server, which then uses a generative AI model (such as OpenAI's GPT-4) to generate a virtual character and analyzes the conversation using a mental health AI model (such as BERT-based filtering technology).

[1113] Examples of concrete examples and prompts

[1114] 1. The user inputs their preference for a "character they can talk to casually, like a friend."

[1115] 2. The app will display the initialized character and begin the conversation.

[1116] 3. The user types, "I've been feeling stressed lately."

[1117] 4. The server generates a response saying, "Take a break and relax."

[1118] 5. The server analyzes the conversation and determines that the injury is minor.

[1119] 6. The app will display suggestions for relaxation to the user.

[1120] Example prompts

[1121] "Generate encouraging responses based on user-entered messages."

[1122] "Evaluate the user's mental state based on this text and classify it as mild, moderate, or severe."

[1123] "Depending on the user's mental state, generate appropriate measures, such as suggestions for relaxation, recommendations for medical attention, or notification to emergency contacts."

[1124] This will enable real-time care for employee mental health and rapid response to emergencies.

[1125] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1126] Step 1:

[1127] A user launches a smartphone application and inputs basic information (appearance and personality) about the desired virtual character. The input information is necessary to generate the virtual character, providing specific data (appearance, personality traits, etc.) according to the user's preferences.

[1128] Step 2:

[1129] The terminal sends the basic information entered by the user to the server. The input here is the basic information provided by the user, and by sending this to the server, data initialization is performed. The terminal performs the process of sending the user information.

[1130] Step 3:

[1131] The server customizes the generative AI model based on the received basic information and initializes the virtual character. At this time, the server inputs the user's basic information into the virtual character generation algorithm, and obtains the virtual character's profile as the output. Specifically, the data is input into the generative AI model to generate the character's appearance and personality.

[1132] Step 4:

[1133] The server transmits the initialized virtual character profile to the terminal. The input here is the generated virtual character profile, and executes a process to transmit the profile to the terminal.

[1134] Step 5:

[1135] The terminal displays the virtual character to the user, enabling the user to start daily conversation with the virtual character. The user begins to interact with the virtual character displayed on the terminal screen.

[1136] Step 6:

[1137] The user can have everyday conversations with the virtual character and input messages, which are then saved as conversation content and sent to the virtual character.

[1138] Step 7:

[1139] The terminal sends the message entered by the user to the server. The input here is the user's dialogue message, and sending this to the server starts processing the conversation content.

[1140] Step 8:

[1141] The server analyzes the input message and generates an appropriate response using a generative AI model. The server inputs the user's message into the analysis algorithm and generates an appropriate response as the output. Specific operations include the processes of text analysis and response generation.

[1142] Step 9:

[1143] The server sends the generated response to the terminal. The input here is the generated response message, and processing to send it to the terminal is executed.

[1144] Step 10:

[1145] The terminal displays the generated response to the user, where the user can confirm the response message from the virtual character.

[1146] Step 11:

[1147] The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model. The analyzed conversation content is used as input, and the mental state evaluation result is obtained as output. Specific operations include analyzing the conversation content and classifying the mental state.

[1148] Step 12:

[1149] Based on the evaluation results, the server classifies the user's mental condition as mild, moderate, or severe, which serves as the basis for providing appropriate measures.

[1150] Step 13:

[1151] The server generates appropriate measures according to the user's mental state and sends them to the device. In mild cases, it suggests a change of mood, in moderate cases it generates a message urging a specialist to examine the patient, and in severe cases it automatically sends a notification to an emergency contact. The content of the generated measures is input, and data to be sent to the user or emergency contact is obtained as output.

[1152] Step 14:

[1153] The device will display the countermeasures from the server to the user, and if necessary, will notify emergency contacts.

[1154] Step 15:

[1155] The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made. The input conversation data is fed into the monitoring algorithm, which performs an error check for inappropriate comments.

[1156] Step 16:

[1157] If the server detects an inappropriate comment, it generates feedback that presents the problem with the comment and how to improve it to the user. The generated feedback is used as input, and data to be sent to the user is obtained as output.

[1158] Step 17:

[1159] The device will display the feedback to the user, who can then review the feedback and take appropriate action.

[1160] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1161] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. In addition, it combines an emotion engine to recognize the user's emotions and respond individually. This system also includes a function that monitors the content of conversations with a separate generative model and generates feedback if inappropriate remarks are made.

[1162] Below, we will create a program for this system and explain the specific processing in natural language.

[1163] Program Overview

[1164] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on the obtained information, the server initializes the character using a generative AI model and begins daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state and emotions. Based on the evaluation, appropriate countermeasures are provided and feedback is generated as necessary. The conversation content is also monitored using another generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user. The emotion engine recognizes the user's emotions from the content of the conversation, facial expressions, voice, etc., and adjusts the response content and countermeasures.

[1165] Initial Setup

[1166] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1167] 2. The device sends the information entered by the user to the server.

[1168] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[1169] 4. The server sends information about the initialized virtual character to the terminal.

[1170] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[1171] Daily conversation

[1172] 1. The user uses the device to start a conversation with a virtual character.

[1173] 2. The terminal sends the conversation message entered by the user to the server.

[1174] 3. The server analyzes the received message using a generative AI model and generates an appropriate response.

[1175] 4. The server sends the generated response message to the terminal.

[1176] 5. The terminal displays the generated response to the user.

[1177] Emotion recognition

[1178] 1. The server uses an emotion engine to recognize emotions from the content of the conversation, the user's facial expressions, and their voice.

[1179] 2. The server adjusts the response content of the generative AI model based on the recognized emotion data.

[1180] 3. The device displays the adjusted response to the user.

[1181] Analysis of mental status

[1182] 1. The server collects and stores the conversation content with the user and inputs it into the mental care AI model.

[1183] 2. The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[1184] 3. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[1185] Providing appropriate measures

[1186] 1. The server generates appropriate measures according to the user's mental state.

[1187] 2. If the symptoms are mild, the server generates suggestions to encourage relaxation (e.g., relaxation methods or activities).

[1188] 3. If the condition is moderate, the server generates a message urging the patient to seek medical advice from a specialist.

[1189] 4. In case of serious illness, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[1190] 5. The device displays the countermeasure information received from the server to the user.

[1191] Speech monitoring and feedback

[1192] 1. The server monitors the conversation with the user in real time using a separate generative model.

[1193] 2. The server analyzes and determines whether the content contains inappropriate comments.

[1194] 3. If the server detects an inappropriate comment, it generates a feedback message containing the problem with the comment and suggestions for improvement.

[1195] 4. The server sends the generated feedback message to the terminal.

[1196] 5. The device displays a feedback message to the user.

[1197] Specific examples

[1198] Example 1: Initial Setup

[1199] 1. The user enters their preference for a "cheerful and encouraging friend."

[1200] 2. The device sends the information to the server.

[1201] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[1202] 4. The device displays the virtual character to the user and begins a conversation.

[1203] Example 2: Everyday conversation and emotion recognition

[1204] 1. The user types, "I just don't feel like doing anything these days."

[1205] 2. The server analyzes the content of the conversation and the user's tone, and uses an emotion engine to recognize "melancholy."

[1206] 3. The server generates a response and adjusts the tone: "Why don't you take a break and refresh yourself?"

[1207] 4. The terminal displays the response to the user.

[1208] Example 3: Severe cases

[1209] 1. The user types, "I'm sick of everything."

[1210] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[1211] 3. The device will display the message "Please talk to someone now."

[1212] This concludes the description of a specific embodiment for carrying out the invention. This system allows users to receive constant mental health care and take appropriate measures. The addition of an emotion engine provides more personalized and effective responses and support.

[1213] The processing flow will be explained below.

[1214] Initial Setup

[1215] Step 1:

[1216] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the virtual character they desire.

[1217] Step 2:

[1218] The device sends the basic information entered by the user to the server.

[1219] Step 3:

[1220] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[1221] Step 4:

[1222] The server transmits information about the initialized virtual character to the terminal.

[1223] Step 5:

[1224] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[1225] Daily conversation

[1226] Step 1:

[1227] The user uses the terminal to initiate a conversation with the virtual character.

[1228] Step 2:

[1229] The terminal transmits the conversation message input by the user to the server.

[1230] Step 3:

[1231] The server analyzes the received message using a generative AI model and generates an appropriate response.

[1232] Step 4:

[1233] The server sends the generated response message to the terminal.

[1234] Step 5:

[1235] The terminal displays the generated response to the user.

[1236] Emotion recognition

[1237] Step 1:

[1238] During a conversation with a user, the server uses an emotion engine to recognize the user's emotions from input text, voice, and facial expression data.

[1239] Step 2:

[1240] The server adjusts the response content of the generative AI model based on the recognized emotion.

[1241] Step 3:

[1242] The server sends the adjusted response to the terminal.

[1243] Step 4:

[1244] The terminal displays the adjusted response to the user.

[1245] Analysis of mental status

[1246] Step 1:

[1247] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[1248] Step 2:

[1249] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[1250] Step 3:

[1251] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[1252] Providing appropriate measures

[1253] Step 1:

[1254] The server generates appropriate measures depending on the user's mental state.

[1255] Step 2:

[1256] If the symptoms are mild, the server generates suggestions to encourage relaxation (e.g., relaxation methods or activities).

[1257] Step 3:

[1258] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[1259] Step 4:

[1260] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[1261] Step 5:

[1262] The device displays the countermeasure information received from the server to the user.

[1263] Speech monitoring and feedback

[1264] Step 1:

[1265] The server monitors the conversation with the user in real time using a separate generative model.

[1266] Step 2:

[1267] The server analyzes and determines whether the content contains inappropriate comments.

[1268] Step 3:

[1269] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[1270] Step 4:

[1271] The server sends the generated feedback message to the terminal.

[1272] Step 5:

[1273] The device displays a feedback message to the user.

[1274] Specific examples

[1275] Initial Setup

[1276] Step 1:

[1277] The user inputs desired information about a "cheerful and encouraging friend" into the terminal interface.

[1278] Step 2:

[1279] The terminal sends this information to the server.

[1280] Step 3:

[1281] The server trains a generative AI model and initializes a virtual character based on the user's preferences.

[1282] Step 4:

[1283] The terminal displays the initialized virtual character to the user.

[1284] Everyday conversation and emotion recognition

[1285] Step 1:

[1286] A user types, "I feel very depressed today."

[1287] Step 2:

[1288] The terminal sends the input message to the server.

[1289] Step 3:

[1290] The server analyzes the message and uses an emotion engine to recognize the user's "depressed" emotion.

[1291] Step 4:

[1292] The server generates a response based on the recognized emotion: "How about a walk to change your mood?"

[1293] Step 5:

[1294] The terminal displays the generated response to the user.

[1295] Severe cases

[1296] Step 1:

[1297] The user types, "I'm sick of everything."

[1298] Step 2:

[1299] The device sends a message to the server.

[1300] Step 3:

[1301] The server determines this to be a sign of serious illness and automatically sends a notification to pre-set emergency contacts.

[1302] Step 4:

[1303] The device displays the message, "Let's talk to someone now."

[1304] The above are the detailed processing steps of a specific embodiment for carrying out the invention. With this system, users can receive mental care by conversing with a virtual character at any time. In addition, the emotion engine enables personalized responses based on the user's emotions, providing more effective support.

[1305] Example 2

[1306] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1307] In modern society, mental health management is becoming increasingly important due to busy daily lives and increased stress. In particular, the number of individuals suffering from loneliness and mental health problems is increasing, while there is a lack of mental health care systems that can provide appropriate measures. In this situation, there is a need for a system that is easily accessible to users, analyzes each individual's mental state, and can take appropriate measures.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1309] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate countermeasures based on the analysis results, means for monitoring the content of the conversation using a separate generative model and generating feedback when inappropriate remarks are made, and means for recognizing the user's emotions using an emotion engine and adjusting the response content. This allows the user to easily monitor their own mental state and receive appropriate feedback and countermeasures.

[1310] A "virtual character" is a pseudo-being that is generated based on basic information set by the user and provides emotional support to the user through dialogue.

[1311] "Basic information" refers to personal setting data such as appearance and personality that a user provides when generating a virtual character.

[1312] A "generative AI model" is an artificial intelligence technology used to generate virtual characters based on basic information provided by users, and to provide conversation and feedback.

[1313] "Means of everyday conversation" refers to the processes and functions that allow users and virtual characters to communicate on an ongoing basis.

[1314] "Means for analyzing mental state" refers to methods or systems for analyzing the content of conversations with users and assessing their psychological and emotional state.

[1315] "Means for providing appropriate measures" refers to functions and mechanisms that provide necessary actions and suggestions to users based on the results of an analysis of their mental state.

[1316] "Means of monitoring using generative models" refers to a mechanism for analyzing conversation content in real time and detecting and monitoring inappropriate remarks.

[1317] "Means for generating feedback" refers to a function that reports inappropriate comments to users when they are detected and suggests ways to improve them.

[1318] An "emotion engine" is a technology that analyzes a user's facial expressions, voice, and conversation content to recognize their emotions.

[1319] The "means for adjusting response content" is a function for appropriately changing the response of the virtual character based on the recognized emotions of the user.

[1320] "Mild" refers to a state in which the user's mental state shows relatively mild stress or anxiety.

[1321] "Moderate" refers to a state in which the user's mental state indicates moderate stress or anxiety and may require specialized care.

[1322] "Severe" refers to a user's mental health condition that is severe enough to require immediate professional intervention.

[1323] "Emergency Contacts" refers to pre-defined contacts such as family and friends who will be automatically notified in the event of a serious illness.

[1324] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through everyday conversations with the virtual character, and provides appropriate countermeasures. It also includes a function that recognizes the user's emotions using an emotion engine and adjusts the response content. It also has a function that monitors the conversation content with a separate generative model and generates feedback if inappropriate remarks are made.

[1325] Initial Setup

[1326] 1. The user inputs basic information about the desired virtual character through the terminal interface. This basic information includes the virtual character's appearance and personality. For example, the user may input their desire for a "cheerful and encouraging friend."

[1327] 2. The device sends the information entered by the user to the server in the form of an HTTP request.

[1328] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character. A general artificial intelligence model can be used as a specific example of the generative AI model used here. The server generates the virtual character based on the prompt provided by the user.

[1329] 4. The server sends information about the virtual character that has been initialized to the device. This data is often sent in JSON format.

[1330] 5. The device displays the virtual character to the user, ready for daily conversation. Specifically, the virtual character's avatar and initial message are displayed on the screen.

[1331] Everyday conversation and emotion recognition

[1332] 1. The user starts a conversation with a virtual character using the terminal. For example, the user may type, "I haven't felt motivated to do anything recently."

[1333] 2. The device sends the conversation message entered by the user to the server. This data is also sent via an HTTP request.

[1334] 3. The server analyzes the received message using a generative AI model and generates an appropriate response. The generative model uses prompts to generate natural-sounding dialogue.

[1335] 4. The server sends the generated response message to the terminal. The generated data is sent again in JSON format.

[1336] 5. The terminal displays the generated response message to the user, adjusting the displayed message to fit the flow of the conversation.

[1337] Recognizing emotions and regulating responses

[1338] 1. The server uses an emotion engine to recognize emotions from the conversation content, the user's facial expressions, and their voice. For example, it can use Google Cloud's Speech-to-Text API or Emotion Detection API.

[1339] 2. The server adjusts the response of the generative AI model based on the recognized emotion data. For example, if the user is recognized as "depressed," the server generates a response such as "Why don't you take a break and refresh yourself?"

[1340] 3. The device displays the tailored response to the user. The displayed message is appropriately tailored to match the user's emotions.

[1341] Analysis of mental health conditions and provision of appropriate measures

[1342] 1. The server collects and stores the content of conversations with users and inputs it into the mental care AI model, including conversation logs and emotional data.

[1343] 2. The server analyzes the user's mental state from the conversation content using a mental care AI model, such as IBM Watson or Azure Cognitive Services.

[1344] 3. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[1345] 4. The server generates appropriate measures based on the user's mental state. For example, if the condition is mild, it suggests relaxation methods or activities, if the condition is moderate, it generates a message urging the user to seek medical advice, and if the condition is severe, it sends an automatic notification to pre-defined emergency contacts.

[1346] 5. The device displays the countermeasure information received from the server to the user.

[1347] Speech monitoring and feedback

[1348] 1. The server monitors the conversation with the user in real time using a separate generative model, which checks for inappropriate comments throughout the conversation.

[1349] 2. The server analyzes and determines whether the message contains inappropriate content. This analysis uses natural language processing technology.

[1350] 3. If the server detects inappropriate comments, it generates a feedback message containing the problem with the comment and suggestions for improvement.

[1351] 4. The server sends the generated feedback message to the terminal.

[1352] 5. The terminal displays a feedback message to the user.

[1353] Through the above process, the system allows users to easily monitor their own mental state and receive appropriate measures and feedback, allowing users to receive daily mental care, and the addition of an emotion engine provides more personalized and effective support.

[1354] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1355] Step 1:

[1356] The user inputs basic information about the desired virtual character through the device interface. The input information includes the virtual character's concept, appearance, personality, etc. As for specific actions, the user sets the character as a "cheerful and encouraging friend." The input data is then sent to the device.

[1357] Input: Basic information about the virtual character (name, appearance, personality)

[1358] Output: Basic information sent to the terminal

[1359] Step 2:

[1360] The device sends the basic information entered by the user to the server. The device generates an HTTP request and includes the data entered by the user. The information is sent to the server.

[1361] Input: Basic information from the user

[1362] Output: HTTP request sent to the server

[1363] Step 3:

[1364] The server customizes the generated AI model and initializes the virtual character based on the received basic information. The server uses prompts to give instructions to the generated AI model and configure the character. Data for the generated virtual character is obtained.

[1365] Input: Basic information (prompt statement)

[1366] Output: Data of the generated virtual character

[1367] Step 4:

[1368] The server sends information about the initialized virtual character in JSON format to the device. The server generates an HTTP response and sends it to the device, including the initialized character data.

[1369] Input: Data of the generated virtual character

[1370] Output: JSON formatted character data sent to the terminal

[1371] Step 5:

[1372] The terminal displays the virtual character to the user and prepares the user for daily conversation. The terminal analyzes the received character data and displays the character's avatar and initial message on the screen.

[1373] Input: Character data from the server

[1374] Output: The virtual character and initial message displayed to the user.

[1375] Step 6:

[1376] The user starts a conversation with the virtual character using the terminal. For example, the user inputs a message such as "I haven't been feeling motivated to do anything lately." This message is input into the terminal.

[1377] Input: User's message

[1378] Output: Message typed into the terminal

[1379] Step 7:

[1380] The terminal sends the message entered by the user to the server. The terminal generates an HTTP request and sends it to the server including the message data.

[1381] Input: User's message

[1382] Output: HTTP request sent to the server

[1383] Step 8:

[1384] The server analyzes the received message using a generative AI model and generates an appropriate response. The generative AI model creates a prompt based on the received message and generates an appropriate response. Response data is generated.

[1385] Input: User's message

[1386] Output: The generated response data

[1387] Step 9:

[1388] The server sends the generated response message to the terminal. The server converts the response data into JSON format, generates an HTTP response, and sends it to the terminal.

[1389] Input: Generated response data

[1390] Output: JSON-formatted response data sent to the terminal

[1391] Step 10:

[1392] The terminal displays the generated response to the user, analyzes the received response data, and displays it on the screen.

[1393] Input: Response data from the server

[1394] Output: The response message displayed to the user

[1395] Step 11:

[1396] The server uses an emotion engine to recognize emotions from the content of the conversation, the user's facial expressions, and their voice. For example, it runs an emotion recognition algorithm based on data input by the user through a camera or microphone, and evaluates the user's emotions.

[1397] Input: User's facial expressions and voice data

[1398] Output: Recognized emotion data

[1399] Step 12:

[1400] The server adjusts the response content of the generative AI model based on the recognized emotion data, and uses the emotion data to modify the response content and generate a message that is appropriate for the user's emotion.

[1401] Input: Recognized emotion data

[1402] Output: Adjusted response data

[1403] Step 13:

[1404] The terminal displays the adjusted response to the user. The adjusted message is parsed and displayed on the screen.

[1405] Input: Reconciled response data from the server

[1406] Output: The adjusted response message displayed to the user

[1407] Step 14:

[1408] The server collects and stores the content of conversations with users and inputs it into the mental care AI model. The server also stores the conversation log in a database and provides the content of the conversation to the mental care model.

[1409] Input: conversation log

[1410] Output: Conversation data input into the mental care model

[1411] Step 15:

[1412] The server uses a mental care AI model to analyze the user's mental state from the conversation content, executes the analysis algorithm, and evaluates the user's mental state.

[1413] Input: Conversation data

[1414] Output: Analyzed mental state data

[1415] Step 16:

[1416] Based on the analysis results, the server classifies the user's mental state into mild, moderate, or severe. Based on the analyzed data, the server classifies the user into the appropriate category.

[1417] Input: Parsed mental state

[1418] Output: Classified mental state data

[1419] Step 17:

[1420] The server generates appropriate measures based on the user's mental state, such as a suggestion to encourage relaxation in mild cases, a message to seek medical advice in moderate cases, and a notification to emergency contacts in severe cases.

[1421] Input: Classified mental state data

[1422] Output: Generated countermeasure data

[1423] Step 18:

[1424] The terminal displays the countermeasure information received from the server to the user, analyzes the generated countermeasure message, and displays it on the screen.

[1425] Input: Countermeasure data from the server

[1426] Output: The action message displayed to the user

[1427] Step 19:

[1428] The server monitors the conversation with the user in real time using a separate generative model, checking for inappropriate comments throughout the conversation.

[1429] Input: User conversation

[1430] Output: Inappropriate remark detection results

[1431] Step 20:

[1432] The server analyzes and determines whether the remarks contain inappropriate content, using natural language processing technology to evaluate the content of the remarks.

[1433] Input: User conversation

[1434] Output: Inappropriate remarks judgement result

[1435] Step 21:

[1436] When an inappropriate comment is detected, the server generates a feedback message including the problem with the comment and a remedy for the comment. Feedback is generated for the user based on the content of the inappropriate comment.

[1437] Input: Inappropriate comment judgment result

[1438] Output: The generated feedback message

[1439] Step 22:

[1440] The server sends the generated feedback message to the device. The feedback data is converted into JSON format and sent to the device.

[1441] Input: The generated feedback message

[1442] Output: Feedback data sent to the device

[1443] Step 23:

[1444] The terminal displays the feedback message to the user, parses the feedback message, and displays it on the screen.

[1445] Input: Feedback data from the server

[1446] Output: The feedback message displayed to the user

[1447] (Application example 2)

[1448] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1449] In modern society, there is a demand for real-time recognition of changes in a user's mental state and emotions and for prompt provision of appropriate measures. However, conventional systems are unable to perform a sufficient detailed analysis of a user's mental state and emotions, which often delays appropriate measures. In particular, there is a lack of means for providing prompt and effective feedback when inappropriate comments are made. The problem that this invention aims to solve is to comprehensively solve these problems and provide more personalized mental care and feedback to users.

[1450] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1451] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the user's mental state, means for providing appropriate countermeasures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, means for classifying the condition as mild, moderate, or severe based on the analysis results, means for recognizing the user's emotions from the content of the conversation, facial expressions, and voice, and means for generating a feedback message containing problems with the remarks and measures for improving them if inappropriate remarks are detected. This makes it possible to analyze the user's mental state and emotions in detail in real time and quickly provide appropriate countermeasures and feedback.

[1452] Below we create a definition sentence for each important word:

[1453] "User" refers to a user who interacts with a virtual character.

[1454] "Basic information" refers to information about appearance and personality provided by the user to generate a virtual character.

[1455] A "virtual character" is a character that is generated based on basic information set by the user and that engages in conversation.

[1456] "Initializing" means using a generative AI model to set the appearance and personality of a virtual character and generate that character.

[1457] "Conversation" refers to communication between a user and a virtual character through text or voice.

[1458] "Analyzing" means processing data such as conversation content, facial expressions, and voice, and extracting specific information.

[1459] "Mental state" refers to a user's psychological health and emotional state.

[1460] An "emotion engine" refers to a software component that recognizes a user's emotions from conversation content, voice, facial expressions, etc.

[1461] "Mild, moderate, severe" refers to different levels of psychological health classified based on the analysis of the user's mental state.

[1462] "Appropriate measures" refer to mental care and feedback provided according to the user's mental state.

[1463] A "generative model" refers to an algorithm or system for generating natural language using AI.

[1464] "Inappropriate comments" refers to comments made in a user's conversation that may be offensive or dangerous.

[1465] "Feedback" refers to responses or advice provided to users by virtual characters or systems.

[1466] A "feedback message" is a message that includes solutions or suggestions for improvement to inappropriate comments.

[1467] This system initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system includes a function that recognizes the user's emotions by combining it with an emotion engine and responds individually. It also has a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[1468] Program Overview

[1469] 1. Initial Setup:

[1470] The user accesses the device's interface and inputs basic information (e.g., appearance, personality) about the desired virtual character. The device then sends the information the user entered to the server. The server customizes the generative AI model based on the received user data and initializes the virtual character. The server then sends the initialized virtual character information to the device, and the device displays the virtual character to the user and begins daily conversation.

[1471] 2. Daily conversation:

[1472] The user initiates a conversation with a virtual character using the device. The device sends the conversation message entered by the user to the server. The server analyzes the received message using a generative AI model and generates an appropriate response. The server then sends the generated response message to the device, which then displays the generated response to the user.

[1473] 3. Emotion Recognition:

[1474] The server uses an emotion engine to recognize emotions from the conversation content, the user's facial expressions, and their voice. The server then adjusts the response content of the generative AI model based on the recognized emotion data. The device then displays the adjusted response to the user.

[1475] 4. Mental State Analysis:

[1476] The server inputs the conversation content with the user into the mental care AI model based on the data. The server then analyzes the user's mental state from the conversation content using the mental care AI model. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[1477] 5. Providing appropriate measures:

[1478] The server generates appropriate measures according to the user's mental state. In mild cases, it generates suggestions to encourage mood changes (e.g., relaxation methods or activities). In moderate cases, it generates a message urging the user to see a specialist. In severe cases, it sends an automatic notification to pre-set emergency contacts (family or friends). The device displays the measures information received from the server to the user.

[1479] 6. Speech monitoring and feedback:

[1480] The server monitors the conversation with the user in real time using a separate generative model. It analyzes and determines whether any inappropriate remarks are included. If an inappropriate remark is detected, it generates a feedback message containing the problem with the remark and a suggestion for improvement. The server sends the generated feedback message to the device, which then displays it to the user.

[1481] Hardware and software used

[1482] The system of the present invention uses the following hardware and software:

[1483] Smartphone: A device that provides a user interface

[1484] Server: Server for data processing and running the generative model (e.g., Amazon Web Services, Microsoft Azure)

[1485] Generative AI models: models for natural language processing (e.g., OpenAI GPT-3, ChatGPT)

[1486] Emotion engine: Software for emotion recognition (e.g., IBM Watson, Microsoft Azure Emotion API)

[1487] Profanity detection engine: Software for monitoring and analyzing profanity (e.g., Perspective API)

[1488] Specific examples

[1489] The following are specific examples of how this system can be used:

[1490] Example 1: Initial Setup

[1491] The user inputs their preference for a "cheerful and encouraging friend." The device sends this information to the server. The server trains a generative AI model and initializes a "cheerful and encouraging" character. The device then displays the virtual character to the user and begins a conversation.

[1492] Example 2: Everyday conversation and emotion recognition

[1493] The user types, "I've been feeling unmotivated lately." The server analyzes the conversation and the user's tone, and recognizes "depression" using an emotion engine. The server generates a response, such as, "Why don't you take a break and refresh yourself?" and adjusts the tone accordingly. The device displays the response to the user.

[1494] Example 3: Severe cases

[1495] The user types, "I've had enough of everything." The server determines that the condition is serious and automatically sends a notification to pre-defined family members. The device then displays the message, "Try to talk to someone right now."

[1496] Example prompt sentence:

[1497] "I'm looking for a friend who is cheerful and good at encouraging others."

[1498] "I just can't seem to get motivated to do anything these days. What should I do?"

[1499] In this way, the present invention allows users to receive constant mental health care and take appropriate measures. The addition of an emotion engine provides more personal and effective responses and support.

[1500] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1501] Program processing steps

[1502] Step 1:

[1503] The user enters basic information about the virtual character.

[1504] Input: Basic information about the virtual character the user desires (appearance, personality, etc.)

[1505] How it works: Through a smartphone app, users input information about the appearance and personality of their desired virtual character.

[1506] Output: The basic information of the virtual character entered is saved on the device.

[1507] Step 2:

[1508] The device sends the user's basic information to the server

[1509] Input: Basic information of the saved virtual character

[1510] How it works: The device sends the information entered by the user to the server using an HTTP request or similar.

[1511] Output: Basic information about the virtual character is sent to the server.

[1512] Step 3:

[1513] The server initializes the virtual character.

[1514] Input: User basic information

[1515] How it works: The server initializes a virtual character using a generative AI model (e.g. ChatGPT) and the basic information mentioned above. The generative AI model customizes the character's appearance and personality, and generates the virtual character.

[1516] Output: Initialized virtual character data

[1517] Step 4:

[1518] The server sends the initialized virtual character to the device.

[1519] Input: Initialized virtual character data

[1520] Operation: The server sends the virtual character data to the device.

[1521] Output: The virtual character data is transferred to the device.

[1522] Step 5:

[1523] The device displays the virtual character to the user.

[1524] Input: Virtual character data

[1525] How it works: The device displays an initialized virtual character on the user's screen and allows the user to initiate a conversation with the virtual character.

[1526] Output: The user sees a virtual character on the screen.

[1527] Step 6:

[1528] The user initiates a conversation with a virtual character

[1529] Input: User's conversation message

[1530] How it works: Users use a smartphone app to type messages to a virtual character.

[1531] Output: Conversation messages entered by the user are saved on the device.

[1532] Step 7:

[1533] The device sends the user's conversation messages to the server

[1534] Input: Saved conversation messages

[1535] Operation: The device sends the user's conversation message to the server.

[1536] Output: Conversation messages forwarded to the server

[1537] Step 8:

[1538] The server analyzes conversation messages using a generative AI model

[1539] Input: User's conversation message

[1540] How it works: The server uses a generative AI model (e.g. ChatGPT) to analyze conversational messages and generate appropriate responses.

[1541] Output: The generated response message

[1542] Step 9:

[1543] Sends the server-generated response message to the terminal

[1544] Input: Response message

[1545] Operation: The server generates a response message and sends it to the terminal.

[1546] Output: The response message is delivered to the terminal.

[1547] Step 10:

[1548] The terminal displays the generated response to the user

[1549] Input: Response message

[1550] Action: The terminal generates a response message and displays it on the user's screen.

[1551] Output: A response message that the user can see on their screen.

[1552] Step 11:

[1553] The server uses an emotion engine to analyze the content of the conversation, the user's facial expressions, and the voice.

[1554] Input: Conversation messages, user facial expression data, voice data

[1555] How it works: The server uses an emotion engine (e.g. IBM Watson) to analyze the conversation, facial expressions, and voice to recognize the user's emotions.

[1556] Output: User emotion data (e.g., sadness, joy)

[1557] Step 12:

[1558] The server adjusts the response based on the emotional data.

[1559] Input: Emotion data, response message

[1560] How it works: The server adjusts the response of the generative AI model based on the emotion data.

[1561] Output: Reconciled response message

[1562] Step 13:

[1563] The server inputs the conversation content into a mental care AI model to analyze the mental state.

[1564] Input: Conversation data

[1565] How it works: The server inputs the conversation content into a mental care AI model and analyzes the user's mental state.

[1566] Output: Mental status data (mild, moderate, severe)

[1567] Step 14:

[1568] The server generates countermeasures according to the mental state.

[1569] Input: Mental state data

[1570] How it works: The server generates corresponding measures based on the mental state (suggestions for relaxation, facilitating professional consultation, notifying emergency contacts).

[1571] Output: Solution message

[1572] Step 15:

[1573] The server monitors inappropriate comments using a separate generative model.

[1574] Input: Conversation data

[1575] How it works: The server uses another generative model (e.g., Perspective API) to monitor conversations in real time and detect inappropriate comments.

[1576] Output: Inappropriate speech detection results

[1577] Step 16:

[1578] Generate a feedback message if inappropriate language is detected

[1579] Input: Inappropriate speech detection results

[1580] Behavior: The server generates a feedback message (problem and remediation) about the inappropriate comment.

[1581] Output: Feedback message

[1582] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1583] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1584] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1585] [Third embodiment]

[1586] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1587] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1588] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1589] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1590] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1591] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1592] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1593] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1594] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1595] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1596] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1597] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1598] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[1599] Below, we will create a program for this system and explain the specific processing in natural language.

[1600] Program Overview

[1601] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on this information, the server initializes the character using a generative AI model and begins a daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state. Based on the evaluation, appropriate measures are offered and feedback is generated as necessary. The system also monitors the content of the conversation using a separate generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user.

[1602] Initial Setup

[1603] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1604] 2. The device sends the information entered by the user to the server.

[1605] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[1606] 4. The server sends the initialized virtual character to the device.

[1607] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[1608] Daily conversation

[1609] 1. The user uses the device to have everyday conversations with a virtual character.

[1610] 2. The device sends the message entered by the user to the server.

[1611] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[1612] 4. The server generates a response and sends it to the terminal.

[1613] 5. The terminal displays the generated response to the user.

[1614] Analysis of mental status

[1615] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[1616] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[1617] 3. The server generates appropriate measures according to the user's mental state.

[1618] Providing appropriate measures

[1619] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[1620] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[1621] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[1622] 4. The device displays the countermeasures from the server to the user.

[1623] Speech monitoring and feedback

[1624] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[1625] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[1626] 3. The device displays feedback to the user.

[1627] Specific examples

[1628] Example 1: Initial Setup

[1629] 1. The user enters their preference for a "cheerful and encouraging friend."

[1630] 2. The device sends the information to the server.

[1631] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[1632] 4. The device displays the virtual character to the user and begins a conversation.

[1633] Example 2: Analysis of everyday conversations and mental states

[1634] 1. The user types, "I just don't feel like doing anything these days."

[1635] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[1636] 3. The terminal displays the response to the user.

[1637] 4. The server analyzes the conversation and determines that the injury is minor.

[1638] Example 3: Severe cases

[1639] 1. The user types, "I'm sick of everything."

[1640] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[1641] 3. The device will display the message "Please talk to someone now."

[1642] This concludes the explanation of the specific form for carrying out the invention. With this system, users can receive mental care at any time by talking to a virtual character. In addition, if the condition is severe, a notification is sent to the appropriate contact person, so users can receive prompt and appropriate treatment without having to worry alone.

[1643] The processing flow will be explained below.

[1644] Initial Setup

[1645] Step 1:

[1646] The user accesses the terminal interface and inputs basic information (such as appearance and personality) about the virtual character they desire.

[1647] Step 2:

[1648] The device sends the basic information entered by the user to the server.

[1649] Step 3:

[1650] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[1651] Step 4:

[1652] The server transmits information about the initialized virtual character to the terminal.

[1653] Step 5:

[1654] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[1655] Daily conversation

[1656] Step 1:

[1657] The user uses the terminal to initiate a conversation with the virtual character.

[1658] Step 2:

[1659] The terminal transmits the conversation message input by the user to the server.

[1660] Step 3:

[1661] The server analyzes the received message using a generative AI model and generates an appropriate response.

[1662] Step 4:

[1663] The server sends the generated response message to the terminal.

[1664] Step 5:

[1665] The terminal displays the response message received from the server to the user.

[1666] Analysis of mental status

[1667] Step 1:

[1668] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[1669] Step 2:

[1670] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[1671] Step 3:

[1672] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[1673] Providing appropriate measures

[1674] Step 1:

[1675] The server generates appropriate measures depending on the user's mental state.

[1676] Step 2:

[1677] If the symptoms are mild, the server generates suggestions to encourage mood change (e.g., relaxation methods or activities).

[1678] Step 3:

[1679] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[1680] Step 4:

[1681] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[1682] Step 5:

[1683] The device displays the countermeasure information received from the server to the user.

[1684] Speech monitoring and feedback

[1685] Step 1:

[1686] The server monitors the conversation with the user in real time using a separate generative model.

[1687] Step 2:

[1688] The server analyzes and determines whether the content contains inappropriate comments.

[1689] Step 3:

[1690] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[1691] Step 4:

[1692] The server sends the generated feedback message to the terminal.

[1693] Step 5:

[1694] The device displays a feedback message to the user.

[1695] The above is a detailed description of the specific operations performed in each processing step. This system allows users to receive constant mental care and take appropriate measures.

[1696] Example 1

[1697] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1698] Mental health care has become an important issue in modern society. However, there are limited systems that allow individual users to easily receive mental health care at any time. There is also a lack of mechanisms that allow users in serious mental health conditions to quickly receive appropriate support. Furthermore, there is a need for a system that can immediately point out inappropriate comments made during conversations with users and provide appropriate feedback.

[1699] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1700] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to evaluate the user's mental state, means for providing appropriate measures based on the evaluation results, and means for monitoring the content of the conversation using a separate generative model and generating feedback if inappropriate comments are made. This allows the user to receive mental care through conversations with the virtual character, and in severe cases, appropriate measures are taken promptly, allowing the user to receive support without feeling isolated. In addition, immediate feedback is provided for inappropriate comments made during conversations, encouraging the user to improve.

[1701] "Basic information" is information about the appearance and temperament of the virtual character set by the user.

[1702] A "virtual character" is an interactable character that is initialized based on basic information about the user using a generative AI model.

[1703] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input and prompts.

[1704] "Means for daily conversation" refers to a system function that allows users and virtual characters to have ongoing dialogue.

[1705] "Means for analyzing conversation content" refers to technology for analyzing the content of conversation between a virtual character and a user and assessing the user's mental state.

[1706] "Mental state" refers to the user's psychological and emotional state, and is classified as mild, moderate, severe, etc.

[1707] "Means for providing appropriate measures" refers to a system function that provides measures based on the user's mental state, such as diversion, consultation with a specialist, or in some cases notification to emergency contacts.

[1708] A "separate generative model" is a separate artificial intelligence model designed to monitor conversation content and detect inappropriate remarks.

[1709] "Means for generating feedback" refers to a system function that provides users with information about the problem and possible solutions when inappropriate comments are made.

[1710] This invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made. Specific embodiments of this system are described below.

[1711] Hardware and software configuration used

[1712] The server is a high-performance computing environment for running generative AI models (e.g., GPT-3) and mental care AI models. The server sends and receives data to and from the user's device via the HTTP protocol.

[1713] The terminal is the device (e.g., smartphone, tablet, or PC) through which the user operates the interface. The terminal is responsible for collecting user input and sending it to the server. It also displays virtual characters and response messages received from the server to the user.

[1714] Program processing

[1715] Initial Setup

[1716] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1717] 2. The device sends the information entered by the user to the server.

[1718] 3. Based on the received user information, the server customizes the generative AI model and generates a virtual character.

[1719] 4. The server sends the generated virtual character to the device.

[1720] 5. The device displays the virtual character to the user, allowing them to start a daily conversation.

[1721] Daily conversation

[1722] 1. The user uses the device to converse with a virtual character.

[1723] 2. The device sends the message entered by the user to the server.

[1724] 3. The server analyzes the message sent using a generative AI model and generates an appropriate response.

[1725] 4. The server generates a response and sends it to the terminal.

[1726] 5. The terminal displays the response to the user.

[1727] Analysis of mental status

[1728] 1. The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[1729] 2. Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe.

[1730] 3. The server generates appropriate countermeasures based on the classification results.

[1731] Providing appropriate measures

[1732] 1. The server generates suggestions for users who are judged to have mild symptoms, such as "Why don't you take a walk to change your mood?"

[1733] 2. The server generates a message for users who are judged to be in moderate condition, such as "Please consult a specialist."

[1734] 3. The server automatically sends notifications to pre-defined contacts for users who are deemed to be in a serious condition.

[1735] 4. The device displays the countermeasures sent from the server to the user.

[1736] Speech monitoring and feedback

[1737] 1. The server uses a separate generative model to monitor the conversation in real time.

[1738] 2. When the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[1739] 3. The device displays this feedback to the user.

[1740] Examples of concrete examples and prompts

[1741] Example 1: Initial Setup

[1742] 1. The user enters their preference for a "cheerful and encouraging friend."

[1743] 2. The device sends the information to the server.

[1744] 3. The server trains a generative AI model to generate a "cheerful and encouraging" character.

[1745] 4. The device displays the virtual character to the user and begins a conversation.

[1746] Example prompt sentence:

[1747] "Create a character that is cheerful and encouraging."

[1748] Example 2: Analysis of everyday conversations and mental states

[1749] 1. The user types, "I just don't feel like doing anything these days."

[1750] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[1751] 3. The terminal displays the response to the user.

[1752] 4. The server analyzes the conversation and determines that the injury is minor.

[1753] Example 3: Severe cases

[1754] 1. The user types, "I'm sick of everything."

[1755] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[1756] 3. The device will display the message "Please talk to someone now."

[1757] This system allows users to receive mental health care through conversations with virtual characters at any time, and in severe cases, prompt and appropriate support is provided. It also provides immediate feedback on inappropriate remarks made during conversations, encouraging users to work on self-improvement.

[1758] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1759] Step 1:

[1760] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1761] Input: Basic information about the virtual character (appearance, personality) entered by the user.

[1762] Output: Basic information data entered into the terminal.

[1763] Step 2:

[1764] The terminal sends the information entered by the user to the server.

[1765] Input: Basic information data entered by the user.

[1766] Data processing: Convert the input data into JSON format.

[1767] Output: JSON data sent as an HTTP POST request to the server.

[1768] Step 3:

[1769] Based on the received user information, the server customizes the generative AI model to generate a virtual character.

[1770] Input: Basic user information data received by the server.

[1771] Data computation: Using prompts from a generative AI model (e.g., GPT-3) to generate a virtual character based on basic information.

[1772] Output: Data of the generated virtual character.

[1773] Step 4:

[1774] The server transmits the generated virtual character to the terminal.

[1775] Input: Data of the generated virtual character.

[1776] Data processing: Encode the virtual character data.

[1777] Output: The encoded data that is sent as an HTTP response to the device.

[1778] Step 5:

[1779] The device displays a virtual character to the user, allowing them to start a daily conversation.

[1780] Input: Encoded virtual character data received from the server.

[1781] Data processing: Decoded virtual character data.

[1782] Output: A virtual character displayed in the user interface.

[1783] Step 6:

[1784] The user uses the terminal to converse with the virtual character.

[1785] Input: The conversation message that the user types.

[1786] Output: A conversation message with information entered on the device.

[1787] Step 7:

[1788] The terminal sends the message entered by the user to the server.

[1789] Input: The message entered by the user.

[1790] Data processing: Convert messages to JSON format.

[1791] Output: JSON data sent as an HTTP POST request to the server.

[1792] Step 8:

[1793] The server analyzes the message sent using a generative AI model and generates an appropriate response.

[1794] Input: The user's message data as received by the server.

[1795] Data Computation: Using generative AI models to analyze messages and generate appropriate responses.

[1796] Output: The generated response data.

[1797] Step 9:

[1798] The server sends the generated response to the terminal.

[1799] Input: The generated response data.

[1800] Data processing: Encode the response data.

[1801] Output: The encoded data that is sent as an HTTP response to the device.

[1802] Step 10:

[1803] The terminal displays the generated response to the user.

[1804] Input: The encoded response data received from the server.

[1805] Data processing: Decoded response data.

[1806] Output: The response message that is displayed in the user interface.

[1807] Step 11:

[1808] The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[1809] Input: Data from everyday conversations.

[1810] Data calculation: A mental state assessment process using a mental care AI model.

[1811] Output: Assessment result (mild, moderate, severe).

[1812] Step 12:

[1813] Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe and generates appropriate countermeasures.

[1814] Input: Mental status assessment results.

[1815] Data calculation: Generation of countermeasures according to mental state.

[1816] Output: Generated countermeasure data.

[1817] Step 13:

[1818] The server generates appropriate messages and notifications for mild, moderate, and severe cases and sends them to the device or to configured contacts.

[1819] Input: Mental state classification results and generated measures data.

[1820] Data processing: Encode notification data as needed.

[1821] Output: Notification data to user device or contacts.

[1822] Step 14:

[1823] The device displays the countermeasures sent from the server to the user.

[1824] Input: Countermeasure data received from the server.

[1825] Data processing: Decoded countermeasure data.

[1826] Output: The action message that is displayed in the user interface.

[1827] Step 15:

[1828] The server uses another generative model to monitor conversations in real time and detect inappropriate remarks.

[1829] Input: Real-time conversation data.

[1830] Data computation: Detecting profanity using alternative generative models.

[1831] Output: Detection result (whether or not there is any inappropriate speech).

[1832] Step 16:

[1833] If the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[1834] Input: Profanity detection result data.

[1835] Data computation: Speech content analysis and feedback generation.

[1836] Output: Feedback data including issues and improvements.

[1837] Step 17:

[1838] The device displays feedback to the user.

[1839] Input: Feedback data received from the server.

[1840] Data processing: Decoded feedback data.

[1841] Output: Feedback message displayed in the user interface.

[1842] (Application example 1)

[1843] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1844] In modern society, stress and mental health problems are on the rise, and companies are placing importance on supporting the mental health of their employees. However, it is difficult to provide appropriate measures in a timely manner using conventional care methods, and there is a need for a system that can respond quickly, especially in emergencies.

[1845] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1846] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate measures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, and means for automatically sending a notification to a set emergency contact if the mental state is determined to be serious, thereby enabling real-time care for employee mental health and rapid response to emergencies.

[1847] "User" refers to a person who uses the system.

[1848] "Basic information" refers to the initial setting information such as the appearance and personality of the virtual character entered by the user.

[1849] A "virtual character" is a character generated by a generative AI model based on the user's basic information.

[1850] "Initialization" refers to the process of generating a virtual character based on basic information set by the user.

[1851] "Means for daily conversation" refers to a system that allows users and virtual characters to have daily conversations.

[1852] "Conversation content" refers to the text and audio data of the dialogue between the user and the virtual character.

[1853] "Mental state" refers to the user's psychological health and mood.

[1854] "Analysis" refers to the process of evaluating the content of a conversation and determining the user's mental state.

[1855] "Appropriate measures" refer to countermeasures such as advice and suggested actions provided to users based on the analysis results.

[1856] A "generative model" refers to an AI algorithm that has been trained to accomplish a specific task.

[1857] "Inappropriate comments" refer to comments that may have a negative impact on the user's mental health.

[1858] "Feedback" refers to advice provided in response to inappropriate comments for correction or improvement.

[1859] "Emergency Contacts" refers to configured contacts to whom automatic notifications are sent if a user's mental condition is deemed severe.

[1860] "Automatic Notification" refers to a message that the system automatically sends when certain conditions are met.

[1861] This system generates a virtual character based on basic information provided by the user, analyzes the user's mental state through daily conversations, and provides appropriate measures. The system also includes a function to monitor the conversation content using a separate generative model and generate feedback if inappropriate remarks are made. It also has a function to automatically send a notification to emergency contacts if the user's mental state is determined to be severe.

[1862] System configuration

[1863] The system for implementing the present invention is mainly composed of the following hardware and software.

[1864] Hardware: Smartphone (Android or iOS), Server (Cloud service such as AWS, Google Cloud, Microsoft Azure, etc.)

[1865] Software: Smartphone applications (e.g., React Native, Swift, Kotlin), generative AI models (e.g., OpenAI's GPT-4), and mental health AI models (e.g., BERT-based filtering technology).

[1866] Program Overview

[1867] Initial Setup

[1868] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1869] 2. The device sends the information entered by the user to the server.

[1870] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[1871] 4. The server sends the initialized virtual character to the device.

[1872] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[1873] Daily conversation

[1874] 1. The user uses the device to have everyday conversations with a virtual character.

[1875] 2. The device sends the message entered by the user to the server.

[1876] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[1877] 4. The server generates a response and sends it to the terminal.

[1878] 5. The terminal displays the generated response to the user.

[1879] Analysis of mental status

[1880] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[1881] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[1882] 3. The server generates appropriate measures according to the user's mental state.

[1883] Providing appropriate measures

[1884] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[1885] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[1886] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[1887] 4. The device displays the countermeasures from the server to the user.

[1888] Speech monitoring and feedback

[1889] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[1890] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[1891] 3. The device displays feedback to the user.

[1892] Description and examples

[1893] Hardware and software used

[1894] The system begins by sending basic information from the user via a smartphone application to a server, which then uses a generative AI model (such as OpenAI's GPT-4) to generate a virtual character and analyzes the conversation using a mental health AI model (such as BERT-based filtering technology).

[1895] Examples of concrete examples and prompts

[1896] 1. The user inputs their preference for a "character they can talk to casually, like a friend."

[1897] 2. The app will display the initialized character and begin the conversation.

[1898] 3. The user types, "I've been feeling stressed lately."

[1899] 4. The server generates a response saying, "Take a break and relax."

[1900] 5. The server analyzes the conversation and determines that the injury is minor.

[1901] 6. The app will display suggestions for relaxation to the user.

[1902] Example prompts

[1903] "Generate encouraging responses based on user-entered messages."

[1904] "Evaluate the user's mental state based on this text and classify it as mild, moderate, or severe."

[1905] "Depending on the user's mental state, generate appropriate measures, such as suggestions for relaxation, recommendations for medical attention, or notification to emergency contacts."

[1906] This will enable real-time care for employee mental health and rapid response to emergencies.

[1907] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1908] Step 1:

[1909] A user launches a smartphone application and inputs basic information (appearance and personality) about the desired virtual character. The input information is necessary to generate the virtual character, providing specific data (appearance, personality traits, etc.) according to the user's preferences.

[1910] Step 2:

[1911] The terminal sends the basic information entered by the user to the server. The input here is the basic information provided by the user, and by sending this to the server, data initialization is performed. The terminal performs the process of sending the user information.

[1912] Step 3:

[1913] The server customizes the generative AI model based on the received basic information and initializes the virtual character. At this time, the server inputs the user's basic information into the virtual character generation algorithm, and obtains the virtual character's profile as the output. Specifically, the data is input into the generative AI model to generate the character's appearance and personality.

[1914] Step 4:

[1915] The server transmits the initialized virtual character profile to the terminal. The input here is the generated virtual character profile, and executes a process to transmit the profile to the terminal.

[1916] Step 5:

[1917] The terminal displays the virtual character to the user, enabling the user to start daily conversation with the virtual character. The user begins to interact with the virtual character displayed on the terminal screen.

[1918] Step 6:

[1919] The user can have everyday conversations with the virtual character and input messages, which are then saved as conversation content and sent to the virtual character.

[1920] Step 7:

[1921] The terminal sends the message entered by the user to the server. The input here is the user's dialogue message, and sending this to the server starts processing the conversation content.

[1922] Step 8:

[1923] The server analyzes the input message and generates an appropriate response using a generative AI model. The server inputs the user's message into the analysis algorithm and generates an appropriate response as the output. Specific operations include the processes of text analysis and response generation.

[1924] Step 9:

[1925] The server sends the generated response to the terminal. The input here is the generated response message, and processing to send it to the terminal is executed.

[1926] Step 10:

[1927] The terminal displays the generated response to the user, where the user can confirm the response message from the virtual character.

[1928] Step 11:

[1929] The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model. The analyzed conversation content is used as input, and the mental state evaluation result is obtained as output. Specific operations include analyzing the conversation content and classifying the mental state.

[1930] Step 12:

[1931] Based on the evaluation results, the server classifies the user's mental condition as mild, moderate, or severe, which serves as the basis for providing appropriate measures.

[1932] Step 13:

[1933] The server generates appropriate measures according to the user's mental state and sends them to the device. In mild cases, it suggests a change of mood, in moderate cases it generates a message urging a specialist to examine the patient, and in severe cases it automatically sends a notification to an emergency contact. The content of the generated measures is input, and data to be sent to the user or emergency contact is obtained as output.

[1934] Step 14:

[1935] The device will display the countermeasures from the server to the user, and if necessary, will notify emergency contacts.

[1936] Step 15:

[1937] The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made. The input conversation data is fed into the monitoring algorithm, which performs an error check for inappropriate comments.

[1938] Step 16:

[1939] If the server detects an inappropriate comment, it generates feedback that presents the problem with the comment and how to improve it to the user. The generated feedback is used as input, and data to be sent to the user is obtained as output.

[1940] Step 17:

[1941] The device will display the feedback to the user, who can then review the feedback and take appropriate action.

[1942] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1943] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. In addition, it combines an emotion engine to recognize the user's emotions and respond individually. This system also includes a function that monitors the content of conversations with a separate generative model and generates feedback if inappropriate remarks are made.

[1944] Below, we will create a program for this system and explain the specific processing in natural language.

[1945] Program Overview

[1946] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on the obtained information, the server initializes the character using a generative AI model and begins daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state and emotions. Based on the evaluation, appropriate countermeasures are provided and feedback is generated as necessary. The conversation content is also monitored using another generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user. The emotion engine recognizes the user's emotions from the content of the conversation, facial expressions, voice, etc., and adjusts the response content and countermeasures.

[1947] Initial Setup

[1948] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[1949] 2. The device sends the information entered by the user to the server.

[1950] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[1951] 4. The server sends information about the initialized virtual character to the terminal.

[1952] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[1953] Daily conversation

[1954] 1. The user uses the device to start a conversation with a virtual character.

[1955] 2. The terminal sends the conversation message entered by the user to the server.

[1956] 3. The server analyzes the received message using a generative AI model and generates an appropriate response.

[1957] 4. The server sends the generated response message to the terminal.

[1958] 5. The terminal displays the generated response to the user.

[1959] Emotion recognition

[1960] 1. The server uses an emotion engine to recognize emotions from the content of the conversation, the user's facial expressions, and their voice.

[1961] 2. The server adjusts the response content of the generative AI model based on the recognized emotion data.

[1962] 3. The device displays the adjusted response to the user.

[1963] Analysis of mental status

[1964] 1. The server collects and stores the conversation content with the user and inputs it into the mental care AI model.

[1965] 2. The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[1966] 3. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[1967] Providing appropriate measures

[1968] 1. The server generates appropriate measures according to the user's mental state.

[1969] 2. If the symptoms are mild, the server generates suggestions to encourage relaxation (e.g., relaxation methods or activities).

[1970] 3. If the condition is moderate, the server generates a message urging the patient to seek medical advice from a specialist.

[1971] 4. In case of serious illness, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[1972] 5. The device displays the countermeasure information received from the server to the user.

[1973] Speech monitoring and feedback

[1974] 1. The server monitors the conversation with the user in real time using a separate generative model.

[1975] 2. The server analyzes and determines whether the content contains inappropriate comments.

[1976] 3. If the server detects an inappropriate comment, it generates a feedback message containing the problem with the comment and suggestions for improvement.

[1977] 4. The server sends the generated feedback message to the terminal.

[1978] 5. The device displays a feedback message to the user.

[1979] Specific examples

[1980] Example 1: Initial Setup

[1981] 1. The user enters their preference for a "cheerful and encouraging friend."

[1982] 2. The device sends the information to the server.

[1983] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[1984] 4. The device displays the virtual character to the user and begins a conversation.

[1985] Example 2: Everyday conversation and emotion recognition

[1986] 1. The user types, "I just don't feel like doing anything these days."

[1987] 2. The server analyzes the content of the conversation and the user's tone, and uses an emotion engine to recognize "melancholy."

[1988] 3. The server generates a response and adjusts the tone: "Why don't you take a break and refresh yourself?"

[1989] 4. The terminal displays the response to the user.

[1990] Example 3: Severe cases

[1991] 1. The user types, "I'm sick of everything."

[1992] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[1993] 3. The device will display the message "Please talk to someone now."

[1994] This concludes the description of a specific embodiment for carrying out the invention. This system allows users to receive constant mental health care and take appropriate measures. The addition of an emotion engine provides more personalized and effective responses and support.

[1995] The processing flow will be explained below.

[1996] Initial Setup

[1997] Step 1:

[1998] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the virtual character they desire.

[1999] Step 2:

[2000] The device sends the basic information entered by the user to the server.

[2001] Step 3:

[2002] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[2003] Step 4:

[2004] The server transmits information about the initialized virtual character to the terminal.

[2005] Step 5:

[2006] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[2007] Daily conversation

[2008] Step 1:

[2009] The user uses the terminal to initiate a conversation with the virtual character.

[2010] Step 2:

[2011] The terminal transmits the conversation message input by the user to the server.

[2012] Step 3:

[2013] The server analyzes the received message using a generative AI model and generates an appropriate response.

[2014] Step 4:

[2015] The server sends the generated response message to the terminal.

[2016] Step 5:

[2017] The terminal displays the generated response to the user.

[2018] Emotion recognition

[2019] Step 1:

[2020] During a conversation with a user, the server uses an emotion engine to recognize the user's emotions from input text, voice, and facial expression data.

[2021] Step 2:

[2022] The server adjusts the response content of the generative AI model based on the recognized emotion.

[2023] Step 3:

[2024] The server sends the adjusted response to the terminal.

[2025] Step 4:

[2026] The terminal displays the adjusted response to the user.

[2027] Analysis of mental status

[2028] Step 1:

[2029] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[2030] Step 2:

[2031] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[2032] Step 3:

[2033] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[2034] Providing appropriate measures

[2035] Step 1:

[2036] The server generates appropriate measures depending on the user's mental state.

[2037] Step 2:

[2038] If the symptoms are mild, the server generates suggestions to encourage relaxation (e.g., relaxation methods or activities).

[2039] Step 3:

[2040] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[2041] Step 4:

[2042] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[2043] Step 5:

[2044] The device displays the countermeasure information received from the server to the user.

[2045] Speech monitoring and feedback

[2046] Step 1:

[2047] The server monitors the conversation with the user in real time using a separate generative model.

[2048] Step 2:

[2049] The server analyzes and determines whether the content contains inappropriate comments.

[2050] Step 3:

[2051] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[2052] Step 4:

[2053] The server sends the generated feedback message to the terminal.

[2054] Step 5:

[2055] The device displays a feedback message to the user.

[2056] Specific examples

[2057] Initial Setup

[2058] Step 1:

[2059] The user inputs desired information about a "cheerful and encouraging friend" into the terminal interface.

[2060] Step 2:

[2061] The terminal sends this information to the server.

[2062] Step 3:

[2063] The server trains a generative AI model and initializes a virtual character based on the user's preferences.

[2064] Step 4:

[2065] The terminal displays the initialized virtual character to the user.

[2066] Everyday conversation and emotion recognition

[2067] Step 1:

[2068] A user types, "I feel very depressed today."

[2069] Step 2:

[2070] The terminal sends the input message to the server.

[2071] Step 3:

[2072] The server analyzes the message and uses an emotion engine to recognize the user's "depressed" emotion.

[2073] Step 4:

[2074] The server generates a response based on the recognized emotion: "How about a walk to change your mood?"

[2075] Step 5:

[2076] The terminal displays the generated response to the user.

[2077] Severe cases

[2078] Step 1:

[2079] The user types, "I'm sick of everything."

[2080] Step 2:

[2081] The device sends a message to the server.

[2082] Step 3:

[2083] The server determines this to be a sign of serious illness and automatically sends a notification to pre-set emergency contacts.

[2084] Step 4:

[2085] The device displays the message, "Let's talk to someone now."

[2086] The above are the detailed processing steps of a specific embodiment for carrying out the invention. With this system, users can receive mental care by conversing with a virtual character at any time. In addition, the emotion engine enables personalized responses based on the user's emotions, providing more effective support.

[2087] Example 2

[2088] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2089] In modern society, mental health management is becoming increasingly important due to busy daily lives and increased stress. In particular, the number of individuals suffering from loneliness and mental health problems is increasing, while there is a lack of mental health care systems that can provide appropriate measures. In this situation, there is a need for a system that is easily accessible to users, analyzes each individual's mental state, and can take appropriate measures.

[2090] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2091] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate countermeasures based on the analysis results, means for monitoring the content of the conversation using a separate generative model and generating feedback when inappropriate remarks are made, and means for recognizing the user's emotions using an emotion engine and adjusting the response content. This allows the user to easily monitor their own mental state and receive appropriate feedback and countermeasures.

[2092] A "virtual character" is a pseudo-being that is generated based on basic information set by the user and provides emotional support to the user through dialogue.

[2093] "Basic information" refers to personal setting data such as appearance and personality that a user provides when generating a virtual character.

[2094] A "generative AI model" is an artificial intelligence technology used to generate virtual characters based on basic information provided by users, and to provide conversation and feedback.

[2095] "Means of everyday conversation" refers to the processes and functions that allow users and virtual characters to communicate on an ongoing basis.

[2096] "Means for analyzing mental state" refers to methods or systems for analyzing the content of conversations with users and assessing their psychological and emotional state.

[2097] "Means for providing appropriate measures" refers to functions and mechanisms that provide necessary actions and suggestions to users based on the results of an analysis of their mental state.

[2098] "Means of monitoring using generative models" refers to a mechanism for analyzing conversation content in real time and detecting and monitoring inappropriate remarks.

[2099] "Means for generating feedback" refers to a function that reports inappropriate comments to users when they are detected and suggests ways to improve them.

[2100] An "emotion engine" is a technology that analyzes a user's facial expressions, voice, and conversation content to recognize their emotions.

[2101] The "means for adjusting response content" is a function for appropriately changing the response of the virtual character based on the recognized emotions of the user.

[2102] "Mild" refers to a state in which the user's mental state shows relatively mild stress or anxiety.

[2103] "Moderate" refers to a state in which the user's mental state indicates moderate stress or anxiety and may require specialized care.

[2104] "Severe" refers to a user's mental health condition that is severe enough to require immediate professional intervention.

[2105] "Emergency Contacts" refers to pre-defined contacts such as family and friends who will be automatically notified in the event of a serious illness.

[2106] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through everyday conversations with the virtual character, and provides appropriate countermeasures. It also includes a function that recognizes the user's emotions using an emotion engine and adjusts the response content. It also has a function that monitors the conversation content with a separate generative model and generates feedback if inappropriate remarks are made.

[2107] Initial Setup

[2108] 1. The user inputs basic information about the desired virtual character through the terminal interface. This basic information includes the virtual character's appearance and personality. For example, the user may input their desire for a "cheerful and encouraging friend."

[2109] 2. The device sends the information entered by the user to the server in the form of an HTTP request.

[2110] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character. A general artificial intelligence model can be used as a specific example of the generative AI model used here. The server generates the virtual character based on the prompt provided by the user.

[2111] 4. The server sends information about the virtual character that has been initialized to the device. This data is often sent in JSON format.

[2112] 5. The device displays the virtual character to the user, ready for daily conversation. Specifically, the virtual character's avatar and initial message are displayed on the screen.

[2113] Everyday conversation and emotion recognition

[2114] 1. The user starts a conversation with a virtual character using the terminal. For example, the user may type, "I haven't felt motivated to do anything recently."

[2115] 2. The device sends the conversation message entered by the user to the server. This data is also sent via an HTTP request.

[2116] 3. The server analyzes the received message using a generative AI model and generates an appropriate response. The generative model uses prompts to generate natural-sounding dialogue.

[2117] 4. The server sends the generated response message to the terminal. The generated data is sent again in JSON format.

[2118] 5. The terminal displays the generated response message to the user, adjusting the displayed message to fit the flow of the conversation.

[2119] Recognizing emotions and regulating responses

[2120] 1. The server uses an emotion engine to recognize emotions from the conversation content, the user's facial expressions, and their voice. For example, it can use Google Cloud's Speech-to-Text API or Emotion Detection API.

[2121] 2. The server adjusts the response of the generative AI model based on the recognized emotion data. For example, if the user is recognized as "depressed," the server generates a response such as "Why don't you take a break and refresh yourself?"

[2122] 3. The device displays the tailored response to the user. The displayed message is appropriately tailored to match the user's emotions.

[2123] Analysis of mental health conditions and provision of appropriate measures

[2124] 1. The server collects and stores the content of conversations with users and inputs it into the mental care AI model, including conversation logs and emotional data.

[2125] 2. The server analyzes the user's mental state from the conversation content using a mental care AI model, such as IBM Watson or Azure Cognitive Services.

[2126] 3. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[2127] 4. The server generates appropriate measures based on the user's mental state. For example, if the condition is mild, it suggests relaxation methods or activities, if the condition is moderate, it generates a message urging the user to seek medical advice, and if the condition is severe, it sends an automatic notification to pre-defined emergency contacts.

[2128] 5. The device displays the countermeasure information received from the server to the user.

[2129] Speech monitoring and feedback

[2130] 1. The server monitors the conversation with the user in real time using a separate generative model, which checks for inappropriate comments throughout the conversation.

[2131] 2. The server analyzes and determines whether the message contains inappropriate content. This analysis uses natural language processing technology.

[2132] 3. If the server detects inappropriate comments, it generates a feedback message containing the problem with the comment and suggestions for improvement.

[2133] 4. The server sends the generated feedback message to the terminal.

[2134] 5. The terminal displays a feedback message to the user.

[2135] Through the above process, the system allows users to easily monitor their own mental state and receive appropriate measures and feedback, allowing users to receive daily mental care, and the addition of an emotion engine provides more personalized and effective support.

[2136] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2137] Step 1:

[2138] The user inputs basic information about the desired virtual character through the device interface. The input information includes the virtual character's concept, appearance, personality, etc. As for specific actions, the user sets the character as a "cheerful and encouraging friend." The input data is then sent to the device.

[2139] Input: Basic information about the virtual character (name, appearance, personality)

[2140] Output: Basic information sent to the terminal

[2141] Step 2:

[2142] The device sends the basic information entered by the user to the server. The device generates an HTTP request and includes the data entered by the user. The information is sent to the server.

[2143] Input: Basic information from the user

[2144] Output: HTTP request sent to the server

[2145] Step 3:

[2146] The server customizes the generated AI model and initializes the virtual character based on the received basic information. The server uses prompts to give instructions to the generated AI model and configure the character. Data for the generated virtual character is obtained.

[2147] Input: Basic information (prompt statement)

[2148] Output: Data of the generated virtual character

[2149] Step 4:

[2150] The server sends information about the initialized virtual character in JSON format to the device. The server generates an HTTP response and sends it to the device, including the initialized character data.

[2151] Input: Data of the generated virtual character

[2152] Output: JSON formatted character data sent to the terminal

[2153] Step 5:

[2154] The terminal displays the virtual character to the user and prepares the user for daily conversation. The terminal analyzes the received character data and displays the character's avatar and initial message on the screen.

[2155] Input: Character data from the server

[2156] Output: The virtual character and initial message displayed to the user.

[2157] Step 6:

[2158] The user starts a conversation with the virtual character using the terminal. For example, the user inputs a message such as "I haven't been feeling motivated to do anything lately." This message is input into the terminal.

[2159] Input: User's message

[2160] Output: Message typed into the terminal

[2161] Step 7:

[2162] The terminal sends the message entered by the user to the server. The terminal generates an HTTP request and sends it to the server including the message data.

[2163] Input: User's message

[2164] Output: HTTP request sent to the server

[2165] Step 8:

[2166] The server analyzes the received message using a generative AI model and generates an appropriate response. The generative AI model creates a prompt based on the received message and generates an appropriate response. Response data is generated.

[2167] Input: User's message

[2168] Output: The generated response data

[2169] Step 9:

[2170] The server sends the generated response message to the terminal. The server converts the response data into JSON format, generates an HTTP response, and sends it to the terminal.

[2171] Input: Generated response data

[2172] Output: JSON-formatted response data sent to the terminal

[2173] Step 10:

[2174] The terminal displays the generated response to the user, analyzes the received response data, and displays it on the screen.

[2175] Input: Response data from the server

[2176] Output: The response message displayed to the user

[2177] Step 11:

[2178] The server uses an emotion engine to recognize emotions from the content of the conversation, the user's facial expressions, and their voice. For example, it runs an emotion recognition algorithm based on data input by the user through a camera or microphone, and evaluates the user's emotions.

[2179] Input: User's facial expressions and voice data

[2180] Output: Recognized emotion data

[2181] Step 12:

[2182] The server adjusts the response content of the generative AI model based on the recognized emotion data, and uses the emotion data to modify the response content and generate a message that is appropriate for the user's emotion.

[2183] Input: Recognized emotion data

[2184] Output: Adjusted response data

[2185] Step 13:

[2186] The terminal displays the adjusted response to the user. The adjusted message is parsed and displayed on the screen.

[2187] Input: Reconciled response data from the server

[2188] Output: The adjusted response message displayed to the user

[2189] Step 14:

[2190] The server collects and stores the content of conversations with users and inputs it into the mental care AI model. The server also stores the conversation log in a database and provides the content of the conversation to the mental care model.

[2191] Input: conversation log

[2192] Output: Conversation data input into the mental care model

[2193] Step 15:

[2194] The server uses a mental care AI model to analyze the user's mental state from the conversation content, executes the analysis algorithm, and evaluates the user's mental state.

[2195] Input: Conversation data

[2196] Output: Analyzed mental state data

[2197] Step 16:

[2198] Based on the analysis results, the server classifies the user's mental state into mild, moderate, or severe. Based on the analyzed data, the server classifies the user into the appropriate category.

[2199] Input: Parsed mental state

[2200] Output: Classified mental state data

[2201] Step 17:

[2202] The server generates appropriate measures based on the user's mental state, such as a suggestion to encourage relaxation in mild cases, a message to seek medical advice in moderate cases, and a notification to emergency contacts in severe cases.

[2203] Input: Classified mental state data

[2204] Output: Generated countermeasure data

[2205] Step 18:

[2206] The terminal displays the countermeasure information received from the server to the user, analyzes the generated countermeasure message, and displays it on the screen.

[2207] Input: Countermeasure data from the server

[2208] Output: The action message displayed to the user

[2209] Step 19:

[2210] The server monitors the conversation with the user in real time using a separate generative model, checking for inappropriate comments throughout the conversation.

[2211] Input: User conversation

[2212] Output: Inappropriate remark detection results

[2213] Step 20:

[2214] The server analyzes and determines whether the remarks contain inappropriate content, using natural language processing technology to evaluate the content of the remarks.

[2215] Input: User conversation

[2216] Output: Inappropriate remarks judgement result

[2217] Step 21:

[2218] When an inappropriate comment is detected, the server generates a feedback message including the problem with the comment and a remedy for the comment. Feedback is generated for the user based on the content of the inappropriate comment.

[2219] Input: Inappropriate comment judgment result

[2220] Output: The generated feedback message

[2221] Step 22:

[2222] The server sends the generated feedback message to the device. The feedback data is converted into JSON format and sent to the device.

[2223] Input: The generated feedback message

[2224] Output: Feedback data sent to the device

[2225] Step 23:

[2226] The terminal displays the feedback message to the user, parses the feedback message, and displays it on the screen.

[2227] Input: Feedback data from the server

[2228] Output: The feedback message displayed to the user

[2229] (Application example 2)

[2230] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2231] In modern society, there is a demand for real-time recognition of changes in a user's mental state and emotions and for prompt provision of appropriate measures. However, conventional systems are unable to perform a sufficient detailed analysis of a user's mental state and emotions, which often delays appropriate measures. In particular, there is a lack of means for providing prompt and effective feedback when inappropriate comments are made. The problem that this invention aims to solve is to comprehensively solve these problems and provide more personalized mental care and feedback to users.

[2232] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2233] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the user's mental state, means for providing appropriate countermeasures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, means for classifying the condition as mild, moderate, or severe based on the analysis results, means for recognizing the user's emotions from the content of the conversation, facial expressions, and voice, and means for generating a feedback message containing problems with the remarks and measures for improving them if inappropriate remarks are detected. This makes it possible to analyze the user's mental state and emotions in detail in real time and quickly provide appropriate countermeasures and feedback.

[2234] Below we create a definition sentence for each important word:

[2235] "User" refers to a user who interacts with a virtual character.

[2236] "Basic information" refers to information about appearance and personality provided by the user to generate a virtual character.

[2237] A "virtual character" is a character that is generated based on basic information set by the user and that engages in conversation.

[2238] "Initializing" means using a generative AI model to set the appearance and personality of a virtual character and generate that character.

[2239] "Conversation" refers to communication between a user and a virtual character through text or voice.

[2240] "Analyzing" means processing data such as conversation content, facial expressions, and voice, and extracting specific information.

[2241] "Mental state" refers to a user's psychological health and emotional state.

[2242] An "emotion engine" refers to a software component that recognizes a user's emotions from conversation content, voice, facial expressions, etc.

[2243] "Mild, moderate, severe" refers to different levels of psychological health classified based on the analysis of the user's mental state.

[2244] "Appropriate measures" refer to mental care and feedback provided according to the user's mental state.

[2245] A "generative model" refers to an algorithm or system for generating natural language using AI.

[2246] "Inappropriate comments" refers to comments made in a user's conversation that may be offensive or dangerous.

[2247] "Feedback" refers to responses or advice provided to users by virtual characters or systems.

[2248] A "feedback message" is a message that includes solutions or suggestions for improvement to inappropriate comments.

[2249] This system initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system includes a function that recognizes the user's emotions by combining it with an emotion engine and responds individually. It also has a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[2250] Program Overview

[2251] 1. Initial Setup:

[2252] The user accesses the device's interface and inputs basic information (e.g., appearance, personality) about the desired virtual character. The device then sends the information the user entered to the server. The server customizes the generative AI model based on the received user data and initializes the virtual character. The server then sends the initialized virtual character information to the device, and the device displays the virtual character to the user and begins daily conversation.

[2253] 2. Daily conversation:

[2254] The user initiates a conversation with a virtual character using the device. The device sends the conversation message entered by the user to the server. The server analyzes the received message using a generative AI model and generates an appropriate response. The server then sends the generated response message to the device, which then displays the generated response to the user.

[2255] 3. Emotion Recognition:

[2256] The server uses an emotion engine to recognize emotions from the conversation content, the user's facial expressions, and their voice. The server then adjusts the response content of the generative AI model based on the recognized emotion data. The device then displays the adjusted response to the user.

[2257] 4. Mental State Analysis:

[2258] The server inputs the conversation content with the user into the mental care AI model based on the data. The server then analyzes the user's mental state from the conversation content using the mental care AI model. Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[2259] 5. Providing appropriate measures:

[2260] The server generates appropriate measures according to the user's mental state. In mild cases, it generates suggestions to encourage mood changes (e.g., relaxation methods or activities). In moderate cases, it generates a message urging the user to see a specialist. In severe cases, it sends an automatic notification to pre-set emergency contacts (family or friends). The device displays the measures information received from the server to the user.

[2261] 6. Speech monitoring and feedback:

[2262] The server monitors the conversation with the user in real time using a separate generative model. It analyzes and determines whether any inappropriate remarks are included. If an inappropriate remark is detected, it generates a feedback message containing the problem with the remark and a suggestion for improvement. The server sends the generated feedback message to the device, which then displays it to the user.

[2263] Hardware and software used

[2264] The system of the present invention uses the following hardware and software:

[2265] Smartphone: A device that provides a user interface

[2266] Server: Server for data processing and running the generative model (e.g., Amazon Web Services, Microsoft Azure)

[2267] Generative AI models: models for natural language processing (e.g., OpenAI GPT-3, ChatGPT)

[2268] Emotion engine: Software for emotion recognition (e.g., IBM Watson, Microsoft Azure Emotion API)

[2269] Profanity detection engine: Software for monitoring and analyzing profanity (e.g., Perspective API)

[2270] Specific examples

[2271] The following are specific examples of how this system can be used:

[2272] Example 1: Initial Setup

[2273] The user inputs their preference for a "cheerful and encouraging friend." The device sends this information to the server. The server trains a generative AI model and initializes a "cheerful and encouraging" character. The device then displays the virtual character to the user and begins a conversation.

[2274] Example 2: Everyday conversation and emotion recognition

[2275] The user types, "I've been feeling unmotivated lately." The server analyzes the conversation and the user's tone, and recognizes "depression" using an emotion engine. The server generates a response, such as, "Why don't you take a break and refresh yourself?" and adjusts the tone accordingly. The device displays the response to the user.

[2276] Example 3: Severe cases

[2277] The user types, "I've had enough of everything." The server determines that the condition is serious and automatically sends a notification to pre-defined family members. The device then displays the message, "Try to talk to someone right now."

[2278] Example prompt sentence:

[2279] "I'm looking for a friend who is cheerful and good at encouraging others."

[2280] "I just can't seem to get motivated to do anything these days. What should I do?"

[2281] In this way, the present invention allows users to receive constant mental health care and take appropriate measures. The addition of an emotion engine provides more personal and effective responses and support.

[2282] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2283] Program processing steps

[2284] Step 1:

[2285] The user enters basic information about the virtual character.

[2286] Input: Basic information about the virtual character the user desires (appearance, personality, etc.)

[2287] How it works: Through a smartphone app, users input information about the appearance and personality of their desired virtual character.

[2288] Output: The basic information of the virtual character entered is saved on the device.

[2289] Step 2:

[2290] The device sends the user's basic information to the server

[2291] Input: Basic information of the saved virtual character

[2292] How it works: The device sends the information entered by the user to the server using an HTTP request or similar.

[2293] Output: Basic information about the virtual character is sent to the server.

[2294] Step 3:

[2295] The server initializes the virtual character.

[2296] Input: User basic information

[2297] How it works: The server initializes a virtual character using a generative AI model (e.g. ChatGPT) and the basic information mentioned above. The generative AI model customizes the character's appearance and personality, and generates the virtual character.

[2298] Output: Initialized virtual character data

[2299] Step 4:

[2300] The server sends the initialized virtual character to the device.

[2301] Input: Initialized virtual character data

[2302] Operation: The server sends the virtual character data to the device.

[2303] Output: The virtual character data is transferred to the device.

[2304] Step 5:

[2305] The device displays the virtual character to the user.

[2306] Input: Virtual character data

[2307] How it works: The device displays an initialized virtual character on the user's screen and allows the user to initiate a conversation with the virtual character.

[2308] Output: The user sees a virtual character on the screen.

[2309] Step 6:

[2310] The user initiates a conversation with a virtual character

[2311] Input: User's conversation message

[2312] How it works: Users use a smartphone app to type messages to a virtual character.

[2313] Output: Conversation messages entered by the user are saved on the device.

[2314] Step 7:

[2315] The device sends the user's conversation messages to the server

[2316] Input: Saved conversation messages

[2317] Operation: The device sends the user's conversation message to the server.

[2318] Output: Conversation messages forwarded to the server

[2319] Step 8:

[2320] The server analyzes conversation messages using a generative AI model

[2321] Input: User's conversation message

[2322] How it works: The server uses a generative AI model (e.g. ChatGPT) to analyze conversational messages and generate appropriate responses.

[2323] Output: The generated response message

[2324] Step 9:

[2325] Sends the server-generated response message to the terminal

[2326] Input: Response message

[2327] Operation: The server generates a response message and sends it to the terminal.

[2328] Output: The response message is delivered to the terminal.

[2329] Step 10:

[2330] The terminal displays the generated response to the user

[2331] Input: Response message

[2332] Action: The terminal generates a response message and displays it on the user's screen.

[2333] Output: A response message that the user can see on their screen.

[2334] Step 11:

[2335] The server uses an emotion engine to analyze the content of the conversation, the user's facial expressions, and the voice.

[2336] Input: Conversation messages, user facial expression data, voice data

[2337] How it works: The server uses an emotion engine (e.g. IBM Watson) to analyze the conversation, facial expressions, and voice to recognize the user's emotions.

[2338] Output: User emotion data (e.g., sadness, joy)

[2339] Step 12:

[2340] The server adjusts the response based on the emotional data.

[2341] Input: Emotion data, response message

[2342] How it works: The server adjusts the response of the generative AI model based on the emotion data.

[2343] Output: Reconciled response message

[2344] Step 13:

[2345] The server inputs the conversation content into a mental care AI model to analyze the mental state.

[2346] Input: Conversation data

[2347] How it works: The server inputs the conversation content into a mental care AI model and analyzes the user's mental state.

[2348] Output: Mental status data (mild, moderate, severe)

[2349] Step 14:

[2350] The server generates countermeasures according to the mental state.

[2351] Input: Mental state data

[2352] How it works: The server generates corresponding measures based on the mental state (suggestions for relaxation, facilitating professional consultation, notifying emergency contacts).

[2353] Output: Solution message

[2354] Step 15:

[2355] The server monitors inappropriate comments using a separate generative model.

[2356] Input: Conversation data

[2357] How it works: The server uses another generative model (e.g., Perspective API) to monitor conversations in real time and detect inappropriate comments.

[2358] Output: Inappropriate speech detection results

[2359] Step 16:

[2360] Generate a feedback message if inappropriate language is detected

[2361] Input: Inappropriate speech detection results

[2362] Behavior: The server generates a feedback message (problem and remediation) about the inappropriate comment.

[2363] Output: Feedback message

[2364] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2365] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2366] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2367] [Fourth embodiment]

[2368] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2369] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2370] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2371] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2372] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2373] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2374] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2375] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2376] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2377] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2378] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2379] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2380] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2381] The present invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made.

[2382] Below, we will create a program for this system and explain the specific processing in natural language.

[2383] Program Overview

[2384] The system first obtains basic information (appearance and personality) about the virtual character desired by the user. Based on this information, the server initializes the character using a generative AI model and begins a daily conversation with the user. The content of the conversation is analyzed in real time to evaluate the user's mental state. Based on the evaluation, appropriate measures are offered and feedback is generated as necessary. The system also monitors the content of the conversation using a separate generative model, and if inappropriate remarks are detected, feedback is promptly provided to the user.

[2385] Initial Setup

[2386] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[2387] 2. The device sends the information entered by the user to the server.

[2388] 3. The server customizes the generative AI model based on the received user data and initializes the virtual character.

[2389] 4. The server sends the initialized virtual character to the device.

[2390] 5. The device displays the virtual character to the user, allowing them to start daily conversations.

[2391] Daily conversation

[2392] 1. The user uses the device to have everyday conversations with a virtual character.

[2393] 2. The device sends the message entered by the user to the server.

[2394] 3. The server parses the message and uses a generative AI model to generate an appropriate response.

[2395] 4. The server generates a response and sends it to the terminal.

[2396] 5. The terminal displays the generated response to the user.

[2397] Analysis of mental status

[2398] 1. The server analyzes the conversation content and evaluates the user's mental state using a mental care AI model.

[2399] 2. Based on the evaluation results, the server will classify the condition as mild, moderate, or severe.

[2400] 3. The server generates appropriate measures according to the user's mental state.

[2401] Providing appropriate measures

[2402] 1. If the symptoms are mild, the server generates suggestions to encourage the user to change their mood (e.g., take a walk or find a way to relax).

[2403] 2. If the condition is moderate, the server generates a message urging the user to seek medical advice.

[2404] 3. The server will automatically send notifications to pre-defined contacts (family and friends) in case of serious illness.

[2405] 4. The device displays the countermeasures from the server to the user.

[2406] Speech monitoring and feedback

[2407] 1. The server monitors the conversation in real time using a separate generative model to determine whether any inappropriate comments have been made.

[2408] 2. If the server detects inappropriate comments, it generates feedback to the user indicating the problem with the comment and how to improve it.

[2409] 3. The device displays feedback to the user.

[2410] Specific examples

[2411] Example 1: Initial Setup

[2412] 1. The user enters their preference for a "cheerful and encouraging friend."

[2413] 2. The device sends the information to the server.

[2414] 3. The server trains the generative AI model and initializes a "cheerful and encouraging" character.

[2415] 4. The device displays the virtual character to the user and begins a conversation.

[2416] Example 2: Analysis of everyday conversations and mental states

[2417] 1. The user types, "I just don't feel like doing anything these days."

[2418] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[2419] 3. The terminal displays the response to the user.

[2420] 4. The server analyzes the conversation and determines that the injury is minor.

[2421] Example 3: Severe cases

[2422] 1. The user types, "I'm sick of everything."

[2423] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[2424] 3. The device will display the message "Please talk to someone now."

[2425] This concludes the explanation of the specific form for carrying out the invention. With this system, users can receive mental care at any time by talking to a virtual character. In addition, if the condition is severe, a notification is sent to the appropriate contact person, so users can receive prompt and appropriate treatment without having to worry alone.

[2426] The processing flow will be explained below.

[2427] Initial Setup

[2428] Step 1:

[2429] The user accesses the terminal interface and inputs basic information (such as appearance and personality) about the virtual character they desire.

[2430] Step 2:

[2431] The device sends the basic information entered by the user to the server.

[2432] Step 3:

[2433] The server customizes the generated AI model based on the received user data and initializes the virtual character.

[2434] Step 4:

[2435] The server transmits information about the initialized virtual character to the terminal.

[2436] Step 5:

[2437] The terminal displays the virtual character to the user, and the user is then ready to start daily conversations with the character.

[2438] Daily conversation

[2439] Step 1:

[2440] The user uses the terminal to initiate a conversation with the virtual character.

[2441] Step 2:

[2442] The terminal transmits the conversation message input by the user to the server.

[2443] Step 3:

[2444] The server analyzes the received message using a generative AI model and generates an appropriate response.

[2445] Step 4:

[2446] The server sends the generated response message to the terminal.

[2447] Step 5:

[2448] The terminal displays the response message received from the server to the user.

[2449] Analysis of mental status

[2450] Step 1:

[2451] The server collects and stores the content of conversations with users and inputs it into the mental care AI model.

[2452] Step 2:

[2453] The server uses a mental care AI model to analyze the user's mental state from the content of the conversation.

[2454] Step 3:

[2455] Based on the analysis results, the server classifies the user's mental state as mild, moderate, or severe.

[2456] Providing appropriate measures

[2457] Step 1:

[2458] The server generates appropriate measures depending on the user's mental state.

[2459] Step 2:

[2460] If the symptoms are mild, the server generates suggestions to encourage mood change (e.g., relaxation methods or activities).

[2461] Step 3:

[2462] In moderate cases, the server generates a message urging the patient to seek professional medical attention.

[2463] Step 4:

[2464] In the event of a serious condition, the server will automatically send a notification to pre-defined emergency contacts (family and friends).

[2465] Step 5:

[2466] The device displays the countermeasure information received from the server to the user.

[2467] Speech monitoring and feedback

[2468] Step 1:

[2469] The server monitors the conversation with the user in real time using a separate generative model.

[2470] Step 2:

[2471] The server analyzes and determines whether the content contains inappropriate comments.

[2472] Step 3:

[2473] If an inappropriate comment is detected, the server generates a feedback message containing the problem with the comment and a remedy.

[2474] Step 4:

[2475] The server sends the generated feedback message to the terminal.

[2476] Step 5:

[2477] The device displays a feedback message to the user.

[2478] The above is a detailed description of the specific operations performed in each processing step. This system allows users to receive constant mental care and take appropriate measures.

[2479] Example 1

[2480] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2481] Mental health care has become an important issue in modern society. However, there are limited systems that allow individual users to easily receive mental health care at any time. There is also a lack of mechanisms that allow users in serious mental health conditions to quickly receive appropriate support. Furthermore, there is a need for a system that can immediately point out inappropriate comments made during conversations with users and provide appropriate feedback.

[2482] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2483] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to evaluate the user's mental state, means for providing appropriate measures based on the evaluation results, and means for monitoring the content of the conversation using a separate generative model and generating feedback if inappropriate comments are made. This allows the user to receive mental care through conversations with the virtual character, and in severe cases, appropriate measures are taken promptly, allowing the user to receive support without feeling isolated. In addition, immediate feedback is provided for inappropriate comments made during conversations, encouraging the user to improve.

[2484] "Basic information" is information about the appearance and temperament of the virtual character set by the user.

[2485] A "virtual character" is an interactable character that is initialized based on basic information about the user using a generative AI model.

[2486] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input and prompts.

[2487] "Means for daily conversation" refers to a system function that allows users and virtual characters to have ongoing dialogue.

[2488] "Means for analyzing conversation content" refers to technology for analyzing the content of conversation between a virtual character and a user and assessing the user's mental state.

[2489] "Mental state" refers to the user's psychological and emotional state, and is classified as mild, moderate, severe, etc.

[2490] "Means for providing appropriate measures" refers to a system function that provides measures based on the user's mental state, such as diversion, consultation with a specialist, or in some cases notification to emergency contacts.

[2491] A "separate generative model" is a separate artificial intelligence model designed to monitor conversation content and detect inappropriate remarks.

[2492] "Means for generating feedback" refers to a system function that provides users with information about the problem and possible solutions when inappropriate comments are made.

[2493] This invention is a system that initializes a virtual character based on basic information set by the user, analyzes the user's mental state through daily conversations, and provides appropriate countermeasures. This system also includes a function that monitors the content of the conversation using a separate generative model and generates feedback if inappropriate remarks are made. Specific embodiments of this system are described below.

[2494] Hardware and software configuration used

[2495] The server is a high-performance computing environment for running generative AI models (e.g., GPT-3) and mental care AI models. The server sends and receives data to and from the user's device via the HTTP protocol.

[2496] The terminal is the device (e.g., smartphone, tablet, or PC) through which the user operates the interface. The terminal is responsible for collecting user input and sending it to the server. It also displays virtual characters and response messages received from the server to the user.

[2497] Program processing

[2498] Initial Setup

[2499] 1. The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[2500] 2. The device sends the information entered by the user to the server.

[2501] 3. Based on the received user information, the server customizes the generative AI model and generates a virtual character.

[2502] 4. The server sends the generated virtual character to the device.

[2503] 5. The device displays the virtual character to the user, allowing them to start a daily conversation.

[2504] Daily conversation

[2505] 1. The user uses the device to converse with a virtual character.

[2506] 2. The device sends the message entered by the user to the server.

[2507] 3. The server analyzes the message sent using a generative AI model and generates an appropriate response.

[2508] 4. The server generates a response and sends it to the terminal.

[2509] 5. The terminal displays the response to the user.

[2510] Analysis of mental status

[2511] 1. The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[2512] 2. Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe.

[2513] 3. The server generates appropriate countermeasures based on the classification results.

[2514] Providing appropriate measures

[2515] 1. The server generates suggestions for users who are judged to have mild symptoms, such as "Why don't you take a walk to change your mood?"

[2516] 2. The server generates a message for users who are judged to be in moderate condition, such as "Please consult a specialist."

[2517] 3. The server automatically sends notifications to pre-defined contacts for users who are deemed to be in a serious condition.

[2518] 4. The device displays the countermeasures sent from the server to the user.

[2519] Speech monitoring and feedback

[2520] 1. The server uses a separate generative model to monitor the conversation in real time.

[2521] 2. When the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[2522] 3. The device displays this feedback to the user.

[2523] Examples of concrete examples and prompts

[2524] Example 1: Initial Setup

[2525] 1. The user enters their preference for a "cheerful and encouraging friend."

[2526] 2. The device sends the information to the server.

[2527] 3. The server trains a generative AI model to generate a "cheerful and encouraging" character.

[2528] 4. The device displays the virtual character to the user and begins a conversation.

[2529] Example prompt sentence:

[2530] "Create a character that is cheerful and encouraging."

[2531] Example 2: Analysis of everyday conversations and mental states

[2532] 1. The user types, "I just don't feel like doing anything these days."

[2533] 2. The server generates a response saying, "Why don't you take a break and refresh yourself?"

[2534] 3. The terminal displays the response to the user.

[2535] 4. The server analyzes the conversation and determines that the injury is minor.

[2536] Example 3: Severe cases

[2537] 1. The user types, "I'm sick of everything."

[2538] 2. The server determines that the patient is seriously ill and automatically sends a notification to pre-defined family members.

[2539] 3. The device will display the message "Please talk to someone now."

[2540] This system allows users to receive mental health care through conversations with virtual characters at any time, and in severe cases, prompt and appropriate support is provided. It also provides immediate feedback on inappropriate remarks made during conversations, encouraging users to work on self-improvement.

[2541] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2542] Step 1:

[2543] The user accesses the terminal interface and inputs basic information (e.g., appearance, personality) of the desired virtual character.

[2544] Input: Basic information about the virtual character (appearance, personality) entered by the user.

[2545] Output: Basic information data entered into the terminal.

[2546] Step 2:

[2547] The terminal sends the information entered by the user to the server.

[2548] Input: Basic information data entered by the user.

[2549] Data processing: Convert the input data into JSON format.

[2550] Output: JSON data sent as an HTTP POST request to the server.

[2551] Step 3:

[2552] Based on the received user information, the server customizes the generative AI model to generate a virtual character.

[2553] Input: Basic user information data received by the server.

[2554] Data computation: Using prompts from a generative AI model (e.g., GPT-3) to generate a virtual character based on basic information.

[2555] Output: Data of the generated virtual character.

[2556] Step 4:

[2557] The server transmits the generated virtual character to the terminal.

[2558] Input: Data of the generated virtual character.

[2559] Data processing: Encode the virtual character data.

[2560] Output: The encoded data that is sent as an HTTP response to the device.

[2561] Step 5:

[2562] The device displays a virtual character to the user, allowing them to start a daily conversation.

[2563] Input: Encoded virtual character data received from the server.

[2564] Data processing: Decoded virtual character data.

[2565] Output: A virtual character displayed in the user interface.

[2566] Step 6:

[2567] The user uses the terminal to converse with the virtual character.

[2568] Input: The conversation message that the user types.

[2569] Output: A conversation message with information entered on the device.

[2570] Step 7:

[2571] The terminal sends the message entered by the user to the server.

[2572] Input: The message entered by the user.

[2573] Data processing: Convert messages to JSON format.

[2574] Output: JSON data sent as an HTTP POST request to the server.

[2575] Step 8:

[2576] The server analyzes the message sent using a generative AI model and generates an appropriate response.

[2577] Input: The user's message data as received by the server.

[2578] Data Computation: Using generative AI models to analyze messages and generate appropriate responses.

[2579] Output: The generated response data.

[2580] Step 9:

[2581] The server sends the generated response to the terminal.

[2582] Input: The generated response data.

[2583] Data processing: Encode the response data.

[2584] Output: The encoded data that is sent as an HTTP response to the device.

[2585] Step 10:

[2586] The terminal displays the generated response to the user.

[2587] Input: The encoded response data received from the server.

[2588] Data processing: Decoded response data.

[2589] Output: The response message that is displayed in the user interface.

[2590] Step 11:

[2591] The server analyzes daily conversation data and evaluates the user's mental state using a mental care AI model.

[2592] Input: Data from everyday conversations.

[2593] Data calculation: A mental state assessment process using a mental care AI model.

[2594] Output: Assessment result (mild, moderate, severe).

[2595] Step 12:

[2596] Based on the evaluation results, the server classifies the user's mental state as mild, moderate, or severe and generates appropriate countermeasures.

[2597] Input: Mental status assessment results.

[2598] Data calculation: Generation of countermeasures according to mental state.

[2599] Output: Generated countermeasure data.

[2600] Step 13:

[2601] The server generates appropriate messages and notifications for mild, moderate, and severe cases and sends them to the device or to configured contacts.

[2602] Input: Mental state classification results and generated measures data.

[2603] Data processing: Encode notification data as needed.

[2604] Output: Notification data to user device or contacts.

[2605] Step 14:

[2606] The device displays the countermeasures sent from the server to the user.

[2607] Input: Countermeasure data received from the server.

[2608] Data processing: Decoded countermeasure data.

[2609] Output: The action message that is displayed in the user interface.

[2610] Step 15:

[2611] The server uses another generative model to monitor conversations in real time and detect inappropriate remarks.

[2612] Input: Real-time conversation data.

[2613] Data computation: Detecting profanity using alternative generative models.

[2614] Output: Detection result (whether or not there is any inappropriate speech).

[2615] Step 16:

[2616] If the server detects inappropriate comments, it analyzes the content and generates feedback including specific problems and suggestions for improvement.

[2617] Input: Profanity detection result data.

[2618] Data computation: Speech content analysis and feedback generation.

[2619] Output: Feedback data including issues and improvements.

[2620] Step 17:

[2621] The device displays feedback to the user.

[2622] Input: Feedback data received from the server.

[2623] Data processing: Decoded feedback data.

[2624] Output: Feedback message displayed in the user interface.

[2625] (Application example 1)

[2626] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2627] In modern society, stress and mental health problems are on the rise, and companies are placing importance on supporting the mental health of their employees. However, it is difficult to provide appropriate measures in a timely manner using conventional care methods, and there is a need for a system that can respond quickly, especially in emergencies.

[2628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2629] In this invention, the server includes means for initializing a virtual character generated based on basic information set by the user, means for having daily conversations with the generated virtual character, means for analyzing the content of the conversation to analyze the mental state of the user, means for providing appropriate measures based on the analysis results, means for monitoring the content of the conversation using another generative model and generating feedback if inappropriate remarks are made, and means f...

Claims

1. A means for initializing a virtual character generated based on basic information set by a user; A means for engaging in daily conversation with the generated virtual character; A means for analyzing the user's mental state by analyzing the content of the conversation; A means of providing appropriate countermeasures based on the analysis results; A means of monitoring the conversation using a separate generative model and generating feedback when inappropriate comments are made; A system including:

2. A means for customizing the virtual character's basic information based on appearance and personality; A means for classifying the mental state into mild, moderate, or severe based on the analysis result of the mental state; The feedback on the inappropriate comments includes the reason for the comment and a means to suggest improvements. The system of claim 1 .

3. Based on the analysis results, measures to encourage mood change in mild cases and In moderate cases, measures to encourage specialist consultation; Including the means to automatically send notifications to pre-defined contacts in case of severe illness; The system of claim 1 .

Images