System

Abstract

To provide a system for developing a positive feeling by analyzing the feeling state of a user through communication between the user and a generated character.SOLUTION: A means for receiving information from a user and generating an account, a means for generating a user-customizable character based on the generated account, a voice recognition means and a natural language processing means for realizing voice communication between the user and the character, an emotion analysis means for generating a response of the character, a means for giving points based on a positive emotion of the user, a means for growing the character according to accumulation of the points, a matching means and a chat means for anonymously communicating with other users in the same situation or emotional state, and a purchase means for realizing communication with a designated avatar as a paid option.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, the problem of lonely deaths is becoming more serious due to the increasing number of elderly people and people who feel isolated. To address this problem, it is necessary to provide continuous emotional support and opportunities for communication to those who feel isolated. It is also important to improve mental health conditions such as dementia and depression. Conventional methods have difficulty in properly analyzing individuals' emotions and providing appropriate support, so effective solutions are needed. [Means for solving the problem]

[0005] The present invention provides a system that analyzes a user's emotional state through communication between the user and a generated character using an application installed on a terminal, fostering positive emotions. When a user makes positive comments, points are awarded, and the character develops based on those points. Furthermore, by providing an opportunity to anonymously communicate with other users who are in the same situation or emotional state, feelings of loneliness are reduced and mental health is improved. Furthermore, communication with specific celebrity avatars is available as a paid option, providing further motivation.

[0006] Specifically, it includes a means for receiving information from users and creating an account, a means for creating a character that can be customized by the user, a voice recognition means and natural language processing means for realizing voice communication between the user and the character, an emotion analysis means for generating the character's responses, a means for awarding points based on the user's positive emotions, a means for character development according to the accumulation of points, a matching means and chat means for anonymously communicating with other users in the same circumstances or emotional state, and a means for purchasing a paid option to communicate with a designated avatar. This can reduce the risk of dying alone and improve mental health.

[0007] "Terminal" refers to an electronic device used by a user that can install and run applications. Examples include smartphones, tablets, and PCs.

[0008] "Application" refers to a program installed on a terminal and software that provides specific functions. In this invention, it refers to an application that has functions for analyzing user emotions, communicating with users, a point system, and character creation.

[0009] "User" refers to an individual who uses the application, specifically including seniors and people who feel lonely.

[0010] "Account" refers to a data set used to register and individually identify a user.

[0011] "Character" refers to a virtual entity that can be customized by the user and that is used to communicate with the user within the application.

[0012] "Speech recognition means" refers to technology that converts a user's voice into text data.

[0013] "Natural language processing means" refers to technology that analyzes text data obtained by speech recognition means and understands the user's emotions and intentions.

[0014] "Emotion analysis means" refers to technology that analyzes the user's emotional state using speech recognition means and natural language processing means.

[0015] "Points" refer to rewards given to users for their positive comments and actions.

[0016] "Accumulated points" refers to the total number of points a User has earned by using the Application.

[0017] "Character growth" refers to the character's growth by acquiring new skills and changing their appearance based on accumulated points.

[0018] "Matching methods" refer to technologies that connect users to search for other users in the same circumstances or emotional state and communicate anonymously.

[0019] "Chat means" refers to a communication means for users to exchange text messages with each other.

[0020] "Purchase means" refers to the procedures and processes for users to purchase paid options.

[0021] "Paid Option" refers to a special feature that is available to the User in addition to the basic feature by paying an additional fee. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] MODE FOR CARRYING OUT THE INVENTION

[0044] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[0045] System configuration

[0046] The system mainly consists of the following components:

[0047] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[0048] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[0049] 3. Application: Software installed on the device that has functions such as communication between users and characters, emotion analysis, point awarding, and matching.

[0050] System Operation

[0051] 1. User Registration

[0052] The user downloads and installs the application onto the terminal.

[0053] Users enter required information such as name, age, and gender to create an account.

[0054] The terminal transmits the entered information to a server, which generates a user profile.

[0055] 2. Character Creation and Customization

[0056] The server uses a generative AI to generate virtual characters that can be customized by users.

[0057] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[0058] Once customization is complete, the device sends the information to the server, which then stores the character information.

[0059] 3. Audio data collection and emotion analysis

[0060] The user speaks to the character and begins a conversation.

[0061] The terminal records the user's voice and transmits this voice data to the server.

[0062] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[0063] 4. Character response generation

[0064] The server generates an appropriate character response based on the analyzed emotional data.

[0065] The terminal displays or reads the generated response to the user.

[0066] 5. Cultivating positive emotions

[0067] The device records the user's positive comments and sends them to the server.

[0068] The server analyzes positive comments and awards points to the user.

[0069] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[0070] 6. Communication with Anonymous Users

[0071] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[0072] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[0073] 7. Paid options available

[0074] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[0075] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[0076] Specific examples

[0077] 1. User registration and character creation

[0078] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[0079] The server receives this information and creates a user profile.

[0080] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[0081] 2. Emotion Analysis and Character Response

[0082] The user speaks to the character, saying, "It's a nice day today."

[0083] The device records the user's voice and sends it to the server.

[0084] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[0085] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[0086] The device displays or reads this response to the user.

[0087] 3. Points awarded and character growth

[0088] The user continues to make positive comments, saying, "I've recently started jogging."

[0089] The device records this audio and sends it to the server.

[0090] The server analyzes positive comments and awards points to the user.

[0091] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[0092] The terminal notifies the user of the character's growth.

[0093] 4. Communication with Anonymous Users

[0094] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[0095] The server generates the chat room and the device displays it to the user.

[0096] Users anonymously exchange jogging information with other users.

[0097] 5. Use of paid options

[0098] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[0099] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[0100] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[0101] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health.

[0102] The processing flow will be explained below.

[0103] Program processing steps

[0104] User Registration and Initial Setup

[0105] Step 1:

[0106] The user downloads and installs the application on the device.

[0107] Step 2:

[0108] Users launch the application and create an account by entering required information such as name, age, and gender.

[0109] Step 3:

[0110] The terminal transmits the input information to the server.

[0111] Step 4:

[0112] The server generates and stores a user profile based on the received information.

[0113] Step 5:

[0114] The server sends an initialization success message to the terminal.

[0115] Step 6:

[0116] The device will notify the user that the initial setup is complete and open the character creation screen.

[0117] Step 7:

[0118] Users customize their character's appearance and name.

[0119] Step 8:

[0120] The terminal transmits the customized character information to the server.

[0121] Step 9:

[0122] The server stores the character information in a user profile.

[0123] Emotion analysis and character generation

[0124] Step 10:

[0125] The user speaks to the character and begins a conversation.

[0126] Step 11:

[0127] The terminal records the user's voice and transmits the voice data to the server.

[0128] Step 12:

[0129] The server uses voice recognition technology to convert the voice data into text.

[0130] Step 13:

[0131] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[0132] Step 14:

[0133] The server generates a response message for the character based on the result of the emotion analysis.

[0134] Step 15:

[0135] The server sends the response of the generated character to the terminal.

[0136] Step 16:

[0137] The terminal displays or reads out the character's response to the user.

[0138] Cultivating positive emotions and awarding points

[0139] Step 17:

[0140] The user makes positive comments during a conversation with the character.

[0141] Step 18:

[0142] The device records positive comments and sends the audio data to a server.

[0143] Step 19:

[0144] The server analyzes the positive comments.

[0145] Step 20:

[0146] The server awards points to the user based on the analysis results.

[0147] Step 21:

[0148] The server determines character growth (gaining new skills or changing appearance) based on the accumulated points.

[0149] Step 22:

[0150] The server sends character growth information to the terminal.

[0151] Step 23:

[0152] The terminal notifies the user of the character's growth and point allocation.

[0153] Communicating with Anonymous Users

[0154] Step 24:

[0155] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[0156] Step 25:

[0157] The terminal transmits the user's selection to the server.

[0158] Step 26:

[0159] The server searches for other users with the same emotional state or circumstances and performs matching.

[0160] Step 27:

[0161] The server creates anonymous chat rooms for matched users.

[0162] Step 28:

[0163] The server transmits chat room information to the terminal.

[0164] Step 29:

[0165] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[0166] Step 30:

[0167] Users communicate with other users anonymously.

[0168] Paid options available

[0169] Step 31:

[0170] The user selects the communication function with the celebrity avatar from a menu within the application.

[0171] Step 32:

[0172] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[0173] Step 33:

[0174] The user enters the necessary information and presses the "Purchase" button.

[0175] Step 34:

[0176] The terminal transmits the purchase information to the server.

[0177] Step 35:

[0178] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[0179] Step 36:

[0180] The server sends a notification of purchase completion to the terminal.

[0181] Step 37:

[0182] The terminal notifies the user that the paid option is now available.

[0183] Step 38:

[0184] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[0185] Example 1

[0186] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0187] In modern society, many people face the problem of feeling lonely. The lack of opportunities to receive emotional support in daily life is a particular challenge for the elderly and those who tend to be isolated. Under these circumstances, maintaining mental health becomes difficult, increasing the risk of serious mental and physical problems. Furthermore, existing emotional support systems have difficulty responding flexibly to the emotional state of individual users. There is a need for a system that can solve these issues and provide users with continuous, personalized emotional support.

[0188] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0189] In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a virtual character that the user can customize based on the generated account; speech recognition and natural language processing means for enabling voice communication between the user and the virtual character; emotion analysis means for generating responses from the virtual character; means for awarding points based on the user's positive emotions; means for upgrading the virtual character according to the accumulated points; matching and messaging means for anonymously communicating with other users in the same circumstances or emotional state; purchase means for realizing communication with a specific avatar as a paid option; display or audio output means on the terminal for notifying the user of the generated responses; means for generating responses from the virtual character using a generative AI model; means for generating prompt sentences to be input to the generative AI model based on the emotion analysis results; and means for generating responses from the virtual character using the generated prompt sentences. This makes it possible to provide continuous and personalized emotional support to users who feel lonely and improve their mental health.

[0190] A "terminal" is an electronic device used by a user and on which software is installed.

[0191] "Software" refers to a program that is installed and executed on a terminal, and provides various functions through interaction with the user.

[0192] A "user" is a person who uses the system and inputs information and communicates via the application.

[0193] "Account" means a digital management unit that contains user-specific information and is required to use the Software.

[0194] A "virtual character" is a user-customizable digital agent that interacts with the user to provide emotional support.

[0195] "Speech recognition means" is a technology that converts voice data into text data.

[0196] "Natural language processing means" is a technology that analyzes text data and understands meaning and emotions.

[0197] "Emotion analysis means" is a technology that estimates a user's emotional state from their statements and text.

[0198] "Points" are digital evaluation units awarded based on users' actions and comments.

[0199] "Growth methods" are techniques that change the skills and appearance of a virtual character according to the accumulation of points.

[0200] "Matching methods" are technologies that connect users with similar circumstances or emotional states.

[0201] "Messaging means" refers to technology that allows anonymous messaging.

[0202] "Paid Options" are additional features or services that are available for an additional fee.

[0203] "Purchase Instrument" means a payment technique for trading paid options.

[0204] "Display means" refers to a technique for displaying the generated response on the screen of the terminal.

[0205] The "audio output means" is a technique for outputting the generated response as audio.

[0206] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and responses.

[0207] A "prompt" is text data that can be input into a generative AI model to elicit a specific response.

[0208] MODE FOR CARRYING OUT THE INVENTION

[0209] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[0210] System configuration

[0211] The system consists of the following elements:

[0212] 1. Device: An electronic device used by a user, such as a smartphone, tablet, or PC. Dedicated software is installed on the device.

[0213] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[0214] 3. Software: A program installed on the device that has functions such as communication between the user and virtual characters, emotion analysis, point awarding, and matching.

[0215] System Operation

[0216] 1. User Registration

[0217] The user downloads and installs the application onto the terminal.

[0218] Users enter required information such as name, age, and gender to create an account.

[0219] The terminal transmits the entered information to a server, which generates a user profile.

[0220] 2. Character Creation and Customization

[0221] The server uses a generative AI model to generate virtual characters that can be customized by users.

[0222] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[0223] Once customization is complete, the device sends the information to the server, which then stores the character information.

[0224] 3. Audio data collection and emotion analysis

[0225] The user speaks to the character and begins a conversation.

[0226] The terminal records the user's voice and transmits this voice data to the server.

[0227] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[0228] 4. Character response generation

[0229] The server generates an appropriate character response based on the analyzed emotion data. A generative AI model (e.g., ChatGPT) is used to generate a prompt. An example of a prompt is, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[0230] The terminal displays or reads the generated response to the user.

[0231] 5. Cultivating positive emotions

[0232] The device records the user's positive comments and sends them to the server.

[0233] The server analyzes positive comments and awards points to the user.

[0234] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[0235] 6. Communication with Anonymous Users

[0236] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[0237] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[0238] 7. Paid options available

[0239] When a user selects a paid option and completes the purchase procedure, the server receives the purchase information and adds a communication function with a specific avatar to the account.

[0240] The device notifies the user that paid options are available and displays conversations with specific avatars.

[0241] Specific examples

[0242] For example, if a user says to a character, "It's a nice day today," the following happens:

[0243] The user speaks to the character, saying, "It's a nice day today."

[0244] The device uses a built-in microphone to record the user's voice and transmits this voice data to the server.

[0245] The server converts the speech into text using the Google Cloud Speech-to-Text API, and then analyzes the emotional state of the text using natural language processing technology (e.g., IBM Watson NLU).

[0246] The server sends a prompt to a generative AI model (e.g., ChatGPT) to generate a response text. Example prompt: "The user said, 'It's a nice day today.' Please generate a positive, constructive response."

[0247] The server generates a response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[0248] The device will either display the text or convert it into speech using a speech synthesis API and respond to the user audibly.

[0249] This allows users to receive continuous and personalized emotional support, which can help reduce feelings of loneliness and improve mental health.

[0250] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0251] Program processing flow

[0252] Registering Users

[0253] Step 1:

[0254] Input: The user downloads and installs the application on their device.

[0255] Output: The installed application starts running on the device.

[0256] What happens: A user downloads and installs an app from the app store.

[0257] Step 2:

[0258] Input: The user enters the required information (name, age, gender, etc.) on the account registration screen.

[0259] Output: User input information is temporarily stored on the device and sent to the server.

[0260] Specific behavior: The device displays an input form, and the user enters information using a keyboard or on-screen keyboard.

[0261] Step 3:

[0262] Input: User information sent from the device.

[0263] Output: The server generates a user profile and stores it in the database.

[0264] Specific operation: The device sends an HTTPS request and the server saves the user information in the database.

[0265] Character Generation and Customization

[0266] Step 4:

[0267] Input: The server generates basic information about the virtual character using a generative AI model.

[0268] Output: Basic setting data of the virtual character is generated.

[0269] Specific operation: The server sends prompts to the generative AI model (e.g., ChatGPT) to generate the initial settings for the virtual character.

[0270] Step 5:

[0271] Input: The user interacts with the character creation screen on their device.

[0272] Output: User-customized character information is saved on the device.

[0273] Specific behavior: The device displays a user interface (UI), and the user operates drop-down menus and sliders.

[0274] Step 6:

[0275] Input: The user completes the customization and the device sends the information to the server.

[0276] Output: The server saves the character information to the database.

[0277] Specific operation: The device sends JSON data containing customization information to the server, and the server stores it in a database.

[0278] Voice data collection and sentiment analysis

[0279] Step 7:

[0280] Input: The user speaks to the character.

[0281] Output: The user's voice data is recorded on the device.

[0282] Specific action: The user speaks into the device's microphone.

[0283] Step 8:

[0284] Input: Recorded audio data.

[0285] Output: The device sends the audio data to the server.

[0286] Specific operation: The device starts the voice recording function and sends the recorded data to the server.

[0287] Step 9:

[0288] Input: The audio data received by the server.

[0289] Output: The server parses the user's utterance as text data.

[0290] Specific operation: The server calls a speech recognition API (e.g., Google Cloud Speech-to-Text) and sends the resulting text data to a natural language processing API (e.g., IBM Watson NLU).

[0291] Character response generation

[0292] Step 10:

[0293] Input: Server parsed emotion data.

[0294] Output: The server generates an appropriate response text using the generative AI model.

[0295] Specific operation: The server sends a prompt to the generative AI model to generate a response text. For example, it sends the prompt sentence, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[0296] Step 11:

[0297] Input: The generated response text.

[0298] Output: The device displays or reads the text.

[0299] Specific behavior: The device displays the generated response text on the screen or converts it into speech using a speech synthesis API (e.g., Amazon Polly) and reads it to the user.

[0300] Cultivating positive emotions

[0301] Step 12:

[0302] Input: User makes a positive statement.

[0303] Output: The device records what you say and sends it to the server.

[0304] Specific operation: The device records what the user says and sends the recording data to the server.

[0305] Step 13:

[0306] Input: The server receives positive utterance data.

[0307] Output: Positive comments are analyzed and points are awarded to the user.

[0308] What it does: The server uses natural language processing to detect positive words and phrases and adds points to the user's profile.

[0309] Step 14:

[0310] Input: The points accumulated by the server.

[0311] Output: Reflects character growth (gaining new skills and changing appearance).

[0312] Specific operation: The server updates the character information based on the growth algorithm and reflects it on the device.

[0313] Communicating with Anonymous Users

[0314] Step 15:

[0315] Input: The user selects the anonymous communication feature.

[0316] Output: The server matches users with the same circumstances and emotional state.

[0317] Specific operation: The server compares user profiles and selects other users with high matching scores.

[0318] Step 16:

[0319] Input: Matched user information.

[0320] Output: The server creates an anonymous chat room and sends a link to the device.

[0321] Specific operation: The server generates a chat room URL and sends it to the device, which displays the link.

[0322] Paid options available

[0323] Step 17:

[0324] Input: The user selects a paid option and completes the purchase.

[0325] Output: The purchase is completed and the ability to communicate with the specified avatar is added to your account.

[0326] Specific operation: The device displays a list of paid options, and the user selects one. The purchase is completed using a payment API (e.g., Stripe or PayPal).

[0327] Step 18:

[0328] Input: The server verifies the purchase information.

[0329] Output: Paid options become available and are notified on the device.

[0330] What happens: The server confirms the purchase and adds the new feature to the user's profile. The device displays a pop-up notification informing the user of the paid option and showing the celebrity avatar's conversation screen.

[0331] (Application example 1)

[0332] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] In recent years, the number of individuals experiencing loneliness has increased, creating a need for emotional support. However, there is a lack of effective methods for improving emotional and shopping experiences in physical stores. Conventional systems struggle to provide an environment that alleviates users' feelings of loneliness and fosters positive emotions. Furthermore, they do not recommend services or products based on the user's real-time emotional state, making it impossible to provide an optimized experience for each individual user. Therefore, a new system is needed to alleviate loneliness, foster positive emotions, and provide a personalized experience in physical stores.

[0334] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0335] In this invention, the server includes means for receiving information from a user and generating an account, means for generating a character that the user can customize based on the generated account, speech recognition means and natural language processing means for enabling voice communication between the user and the character, emotion analysis means for generating responses from the character, means for awarding points based on the user's positive emotions, means for enhancing the character according to the accumulated points, matching means and chat means for anonymously communicating with other users in the same circumstances or emotional state, means for recommending products and services based on the user's emotional state in a physical store, and purchasing means for communicating with a specified avatar. This allows users to reduce feelings of loneliness, foster positive emotions, and enjoy a personalized experience in a physical store.

[0336] A "terminal" is an electronic device used by a user, including a smartphone, tablet, or PC.

[0337] An "application" is a software program that is installed on a terminal and has functions such as communication between the user and a virtual character and emotion analysis.

[0338] "User" refers to a person who uses the system and is an individual who wishes to receive emotional support.

[0339] An "account" is a record containing a user's identifying information, and is used to identify an individual user within the system.

[0340] A "character" is a user-customizable virtual entity that provides emotional support through communication with the user.

[0341] "Speech recognition means" refers to a technology that converts a user's voice into text data, and is used to analyze the content of what the user says.

[0342] "Natural language processing means" is a technology for understanding text data obtained by speech recognition means and generating an appropriate response.

[0343] "Emotion analysis means" is a technology that analyzes the emotional state of a user from their statements and actions.

[0344] "Points" are numerical values ​​that the system awards to users for their positive comments and actions, and are used to develop their characters and receive special benefits.

[0345] "Growth" means that the character's skills and appearance improve or change depending on the user's accumulated points.

[0346] "Matching means" is a technology that automatically finds other users who are in the same circumstances or emotional state, and enables them to communicate with each other.

[0347] "Chat means" is a function that allows users to send and receive text messages.

[0348] "Physical store" refers to a physical commercial facility or service location.

[0349] "Means for recommending products and services" refers to technology for presenting products and services that are considered optimal based on the user's emotional state.

[0350] "Purchase means" is a function that allows a user to select and purchase a paid option.

[0351] "Communicating anonymously" means exchanging messages with other users without using their real names.

[0352] The present invention relates to a system for providing emotional support to users and improving their mental health, which is realized through an application installed on a terminal, a central server, and a network connecting them.

[0353] System configuration

[0354] 1. Terminal

[0355] A terminal is an electronic device used by a user, and includes a smartphone, tablet, PC, etc. An application that enables communication between the user and a virtual character is installed on this terminal.

[0356] 2. Server

[0357] The server acts as a central server, processing, storing, and analyzing data sent by users. The server uses emotion analysis tools and generative AI models to analyze the user's emotional state and generate appropriate responses.

[0358] 3. Application

[0359] An application is a software program that is installed on a terminal and has functions such as communication between the user and a virtual character and emotion analysis.

[0360] System Operation

[0361] 1. User Registration

[0362] Users download and install the application on their device, create an account by entering information such as their name, age, and gender, and the device then sends this information to a server to generate a user profile.

[0363] 2. Character Creation and Customization

[0364] The server uses a generative AI model to generate virtual characters that users can customize. Users select and customize the character's appearance and name, and the information is stored on the server.

[0365] 3. Speech Communication and Emotion Analysis

[0366] When a user speaks to a character, the device records the user's voice and sends it to the server, which converts the voice data into text using speech recognition and natural language processing, and analyzes it using emotion analysis.

[0367] 4. Response Generation

[0368] The server generates an appropriate character response based on the analysis results and sends it to the device, which then displays or reads the generated response to the user.

[0369] Example

[0370] 1. Examples of applications in physical stores

[0371] In a physical store, users communicate with a virtual character via their smartphone or smart glasses. The character recommends relaxation areas and products based on the user's emotional state. If the character asks, "How are you feeling today?" and the user replies, "I'm a little tired," the character will recommend, "There's a relaxation area in this area."

[0372] 2. Points awarded and character growth

[0373] When a user engages in positive conversation, points are awarded and the character grows. For example, if a user says, "I went to a cafe with my friends yesterday and had fun," points are awarded and the character acquires a new skill.

[0374] Program processing overview

[0375] Hardware and software used:

[0376] Hardware: Smartphones, smart glasses

[0377] Software: Flask (web application framework), SpeechRecognition (voice recognition library), openai (generative AI model)

[0378] Data processing and calculation:

[0379] The server manages user profiles, analyzes emotions, and generates responses using generative AI. For example, the following prompts are generated:

[0380] The user said 'I'm a bit tired'. Their emotion is 'tired'. Give an appropriate response.

[0381] The system allows users to reduce feelings of loneliness, foster positive emotions, and enjoy personalized experiences in physical stores.

[0382] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0383] Step 1:

[0384] The user downloads the application and installs it on their device.

[0385] Input: User's internet-connected device

[0386] Output: Installed applications

[0387] Specific operation: The user downloads the application from the official store and installs it on their device.

[0388] Step 2:

[0389] To create an account, a user enters information such as name, age, and gender.

[0390] Input: User personal information

[0391] Output: The information entered by the user is sent to the server and a user profile is generated.

[0392] Specific operation: The user enters the required information into a form within the application and presses the "Submit" button.

[0393] Step 3:

[0394] The server uses the generative AI model to generate a virtual character that can be customized by the user.

[0395] Input: User profile information

[0396] Output: Generated virtual character

[0397] Specific operation: The server generates a virtual character using a generative AI model based on the user's profile information.

[0398] Step 4:

[0399] The device displays a character creation screen where the user can customize the character's appearance and name.

[0400] Input: Generated character and user customization settings

[0401] Output:Customized Character

[0402] Specific operation: A character builder will appear on the device screen, and the user can adjust the appearance and name and save it.

[0403] Step 5:

[0404] The user speaks to the character and begins a conversation.

[0405] Input: User's voice

[0406] Output: Audio data

[0407] Specific actions: The user speaks to the character through the microphone.

[0408] Step 6:

[0409] The device records the user's voice and sends it to the server.

[0410] Input: User's voice data

[0411] Output: Audio data sent to the server

[0412] Specific operation: The device sends the audio recorded by the microphone to the server as digital data.

[0413] Step 7:

[0414] The server converts the voice data into text using a voice recognition means, and analyzes the text data using a natural language processing means.

[0415] Input: User's voice data

[0416] Output: Text data and analysis results

[0417] Specific operation: The speech recognition library converts the speech data into a string of characters, and the natural language processing means analyzes the string of characters.

[0418] Step 8:

[0419] An emotion analysis means analyzes the user's emotional state.

[0420] Input: Parsed text data

[0421] Output: User's emotional state

[0422] What it does: Sentiment analysis algorithms extract emotional states from text data.

[0423] Step 9:

[0424] The server generates a response for the virtual character based on the emotion analysis results.

[0425] Input: User emotional state and parsed text data

[0426] Output: The generated response

[0427] Specific behavior: The generative AI model generates an appropriate response based on emotional state and text data.

[0428] Step 10:

[0429] The terminal displays or reads the generated response to the user.

[0430] Input: The generated response from the server

[0431] Output: The response that is displayed or read to the user

[0432] Specific behavior: Display the response text on the device screen or read it aloud through the speaker.

[0433] Step 11:

[0434] Points are awarded for positive comments made by users.

[0435] Input: Analysis results of positive comments

[0436] Output: Updated points

[0437] What it does: The server recognizes positive emotions and adds them to a points system.

[0438] Step 12:

[0439] Characters grow according to the accumulated points.

[0440] Input: Accumulated points

[0441] Output: Grown-up character

[0442] What it does: The server checks the accumulated points and adds new skills and cosmetic changes to the character.

[0443] Step 13:

[0444] It provides a chat function and matches users with similar circumstances and emotional states to communicate anonymously.

[0445] Input: User's emotional state and situation information

[0446] Output: Matching results and chat rooms

[0447] What it does: The server finds other suitable users and creates an anonymous chat room.

[0448] Step 14:

[0449] Recommend products and services based on the user's emotional state in a physical store.

[0450] Input: User's emotional state

[0451] Output: Recommended products and services

[0452] Specific operation: The server generates prompt sentences that suggest appropriate products and services based on the emotional state.

[0453] The user said 'I'm a bit tired'. Their emotion is 'tired'. Give an appropriate response.

[0454] Step 15:

[0455] The user performs a purchase procedure to realize communication with a designated avatar as a paid option.

[0456] Input: User's purchase intent

[0457] Output: Purchase completed and function added

[0458] Specific operation: The server completes the purchase process and adds the ability to communicate with the specified avatar.

[0459] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0460] MODE FOR CARRYING OUT THE INVENTION

[0461] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The system aims to foster positive emotions through communication between users and virtual characters using emotion analysis technology, generative AI technology, and an emotion engine.

[0462] System configuration

[0463] The system mainly consists of the following components:

[0464] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[0465] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[0466] 3. Application: Software installed on the device that has various functions such as communication between the user and the character, emotion analysis, point awarding, matching, and an emotion engine.

[0467] 4. Emotion engine: An engine that analyzes the user's emotions in real time and dynamically adjusts the character's responses and actions based on that information.

[0468] System Operation

[0469] 1. User Registration

[0470] The user downloads and installs the application on the device.

[0471] Users enter required information such as name, age, and gender to create an account.

[0472] The terminal transmits the entered information to a server, which generates a user profile.

[0473] 2. Character Creation and Customization

[0474] The server uses a generative AI to generate virtual characters that can be customized by users.

[0475] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[0476] Once customization is complete, the device sends the information to the server, which then stores the character information.

[0477] 3. Audio data collection and emotion analysis

[0478] The user speaks to the character and begins a conversation.

[0479] The terminal records the user's voice and transmits this voice data to the server.

[0480] The server converts the voice data into text using speech recognition technology and analyzes the emotional state using natural language processing means.

[0481] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[0482] 4. Character response generation

[0483] The emotion engine dynamically generates character responses based on the analyzed emotion data.

[0484] The server sends the generated response to the terminal.

[0485] The terminal displays or reads out the character's response to the user.

[0486] 5. Cultivating positive emotions

[0487] The device records the user's positive comments and sends them to the server.

[0488] The server analyzes positive comments and awards points to the user.

[0489] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[0490] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[0491] 6. Communication with Anonymous Users

[0492] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[0493] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[0494] 7. Paid options available

[0495] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[0496] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[0497] Specific examples

[0498] 1. User registration and character creation

[0499] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[0500] The server receives this information and creates a user profile.

[0501] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[0502] 2. Emotion Analysis and Character Response

[0503] The user speaks to the character, saying, "It's a nice day today."

[0504] The device records the user's voice and sends it to the server.

[0505] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[0506] Using the recorded emotion data, the emotion engine analyzes the user's real-time emotional state.

[0507] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[0508] The device displays or reads this response to the user.

[0509] 3. Points awarded and character growth

[0510] The user continues to make positive comments, saying, "I've recently started jogging."

[0511] The device records this audio and sends it to the server.

[0512] The server analyzes positive comments and awards points to the user.

[0513] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[0514] The terminal notifies the user of the character's growth.

[0515] The emotion engine tracks the user's long-term emotional changes and responds appropriately.

[0516] 4. Communication with Anonymous Users

[0517] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[0518] The server generates the chat room and the device displays it to the user.

[0519] Users anonymously exchange jogging information with other users.

[0520] 5. Use of paid options

[0521] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[0522] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[0523] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[0524] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health. By combining real-time emotion analysis by the emotion engine and its feedback function, it is possible to maintain a better long-term mental state for users.

[0525] The processing flow will be explained below.

[0526] Program processing steps

[0527] User Registration and Initial Setup

[0528] Step 1:

[0529] The user downloads and installs the application on the device.

[0530] Step 2:

[0531] Users launch the application and create an account by entering required information such as name, age, and gender.

[0532] Step 3:

[0533] The terminal transmits the input information to the server.

[0534] Step 4:

[0535] The server generates and stores a user profile based on the received information.

[0536] Step 5:

[0537] The server sends an initialization success message to the terminal.

[0538] Step 6:

[0539] The device will notify the user that the initial setup is complete and open the character creation screen.

[0540] Step 7:

[0541] Users customize their character's appearance and name.

[0542] Step 8:

[0543] The terminal transmits the customized character information to the server.

[0544] Step 9:

[0545] The server stores the character information in a user profile.

[0546] Emotion analysis and character generation

[0547] Step 10:

[0548] The user speaks to the character and begins a conversation.

[0549] Step 11:

[0550] The terminal records the user's voice and transmits the voice data to the server.

[0551] Step 12:

[0552] The server uses voice recognition technology to convert the voice data into text.

[0553] Step 13:

[0554] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[0555] Step 14:

[0556] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[0557] Step 15:

[0558] The emotion engine dynamically generates character responses based on the analyzed data.

[0559] Step 16:

[0560] The server sends the response of the generated character to the terminal.

[0561] Step 17:

[0562] The terminal displays or reads out the character's response to the user.

[0563] Cultivating positive emotions and awarding points

[0564] Step 18:

[0565] The user makes positive comments during a conversation with the character.

[0566] Step 19:

[0567] The device records positive comments and sends the audio data to a server.

[0568] Step 20:

[0569] The server analyzes the positive comments.

[0570] Step 21:

[0571] The server awards points to the user based on the analysis results.

[0572] Step 22:

[0573] The server determines the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[0574] Step 23:

[0575] The server sends character growth information to the terminal.

[0576] Step 24:

[0577] The terminal notifies the user of the character's growth and point allocation.

[0578] Step 25:

[0579] The emotion engine tracks the user's long-term emotional changes and dynamically adjusts the character's response.

[0580] Communicating with Anonymous Users

[0581] Step 26:

[0582] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[0583] Step 27:

[0584] The terminal transmits the user's selection to the server.

[0585] Step 28:

[0586] The server searches for other users with the same emotional state or circumstances and performs matching.

[0587] Step 29:

[0588] The server creates anonymous chat rooms for matched users.

[0589] Step 30:

[0590] The server transmits chat room information to the terminal.

[0591] Step 31:

[0592] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[0593] Step 32:

[0594] Users communicate with other users anonymously.

[0595] Paid options available

[0596] Step 33:

[0597] The user selects the communication function with the celebrity avatar from a menu within the application.

[0598] Step 34:

[0599] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[0600] Step 35:

[0601] The user enters the necessary information and presses the "Purchase" button.

[0602] Step 36:

[0603] The terminal transmits the purchase information to the server.

[0604] Step 37:

[0605] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[0606] Step 38:

[0607] The server sends a notification of purchase completion to the terminal.

[0608] Step 39:

[0609] The terminal notifies the user that the paid option is now available.

[0610] Step 40:

[0611] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[0612] Example 2

[0613] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0614] Conventional emotional support systems have been unable to provide sufficient emotional support to users who feel lonely, and have been insufficient to improve the user's long-term mental health. Furthermore, communication with the user is one-way, making it difficult to provide immediate feedback based on the user's emotional state. As a result, users are less likely to be satisfied with the system, making it difficult to promote continued use.

[0615] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving information from a user and generating an account, a means for generating a virtual character that the user can customize based on the generated account, a voice recognition means and natural language processing means for enabling voice communication between the user and the character, a means for converting voice data into text and analyzing the emotional state, an emotion engine for generating the character's responses, a means for awarding points based on the user's positive emotions, a means for improving the character according to the accumulated points, a matching means and chat means for anonymously communicating with other users in the same situation or emotional state, and a purchasing means for realizing communication with a specified character as a paid option. This enables immediate feedback based on the user's emotional state and provides continuous emotional support. Furthermore, the user can experience a reduction in loneliness and an improvement in mental health.

[0616] A "terminal" is an electronic device used by a user, and includes a smartphone, tablet, PC, etc.

[0617] An "application" is a software program installed on a terminal that manages the interaction between the user and the character.

[0618] An "account" is a collection of data that includes a user's identification information and is used to manage an individual user's operations within the system.

[0619] A "virtual character" is a digital character that can be customized by the user using a generative AI model and that communicates with the user.

[0620] "Speech recognition means" refers to technology that records a user's voice and converts that voice into text data.

[0621] "Natural language processing means" refers to technology that analyzes text data and extracts linguistic meanings and emotional states.

[0622] "Emotional state" refers to the emotional state extracted from a user's speech or text, and includes, for example, joy, sadness, anger, etc.

[0623] The "emotion engine" is an engine that analyzes the user's emotional state and generates responses from the digital character based on the results.

[0624] The "means of awarding points" refers to a mechanism that adds points within the system based on users' positive actions and comments.

[0625] "Means for character development" refers to a system that allows you to evolve your virtual character's skills and appearance based on accumulated points.

[0626] "Matching means" refers to technology that identifies other users with the same emotional state or interests on the server side and connects them in anonymous chat rooms.

[0627] "Chat vehicle" refers to the interface and functionality for exchanging text messages between users.

[0628] "Purchase method" refers to the operation by which a user selects a paid option and makes additional features available through payment.

[0629] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The system aims to foster positive emotions through interactive communication between users and virtual characters using emotion analysis technology, generative AI technology, and an emotion engine.

[0630] System configuration

[0631] This system consists of the following elements:

[0632] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[0633] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[0634] 3. Application: Software installed on the device that has various functions such as communication between the user and the character, emotion analysis, point awarding, matching, and an emotion engine.

[0635] 4. Emotion engine: An engine that analyzes the user's emotions in real time and dynamically adjusts the character's responses and actions based on that information.

[0636] System Operation

[0637] 1. User Registration

[0638] The user downloads and installs the application on the device.

[0639] Users enter required information such as name, age, and gender to create an account.

[0640] The terminal transmits the entered information to a server, which generates a user profile.

[0641] 2. Character Creation and Customization

[0642] The server uses generative AI to generate virtual characters that users can customize. Specifically, it uses a generative AI model (e.g., GPT-4).

[0643] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[0644] Once customization is complete, the device sends the information to the server, which then stores the character information.

[0645] 3. Audio data collection and emotion analysis

[0646] The user speaks to the character and begins a conversation.

[0647] The terminal records the user's voice and transmits this voice data to the server.

[0648] The server converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[0649] The server analyzes the emotional state using natural language processing means (NLP library).

[0650] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[0651] 4. Character response generation

[0652] The emotion engine dynamically generates character responses based on the analyzed emotion data.

[0653] The server sends the generated response to the terminal.

[0654] The terminal displays or reads out the character's response to the user.

[0655] 5. Cultivating positive emotions

[0656] The device records the user's positive comments and sends them to the server.

[0657] The server analyzes positive comments and awards points to the user.

[0658] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[0659] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[0660] 6. Communication with Anonymous Users

[0661] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[0662] The terminal displays a chat room, allowing users to exchange messages with other users.

[0663] 7. Paid options available

[0664] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific character to the account.

[0665] The terminal notifies the user that a paid option is available and displays a conversation with a specific character.

[0666] Specific examples

[0667] 1. User registration and character creation

[0668] Elderly users install the application on their devices and create an account by entering information such as their name, age, and gender.

[0669] The user's name is "Tanaka Ichiro," his age is "70," and his gender is "male." Based on this information, please generate a character that this user would like.

[0670] The server receives this information and creates a user profile.

[0671] The device displays a screen that allows the user to customize the character's appearance and name.

[0672] 2. Emotion Analysis and Character Response

[0673] The user speaks to the character, saying, "It's a nice day today."

[0674] The user says, "It's a beautiful day today." Create a response that emphasizes positive sentiment.

[0675] The device records the user's voice and sends it to the server.

[0676] The server converts the speech into text using speech recognition and natural language processing technology, and sends the analyzed emotional data to the emotion engine.

[0677] The emotion engine analyzes the user's real-time emotional state and generates an appropriate response.

[0678] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[0679] The device displays or reads this response to the user.

[0680] 3. Points awarded and character growth

[0681] The user continues to make positive comments, saying, "I've recently started jogging."

[0682] The device records this audio and sends it to the server.

[0683] The server analyzes positive comments and awards points to the user.

[0684] Based on the accumulated points, the server adds new skills to the character and sends that information to the terminal.

[0685] The terminal notifies the user of the character's growth.

[0686] The emotion engine can track the user's long-term emotional changes and respond appropriately.

[0687] 4. Communication with Anonymous Users

[0688] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[0689] The server generates the chat room and the terminal displays it to the user.

[0690] Users anonymously exchange jogging information with other users.

[0691] 5. Use of paid options

[0692] The user selects a special communication function with the character and completes the purchase procedure.

[0693] The server will verify the purchase information and add the communication function for the specific character to the user's account.

[0694] The terminal notifies the user that a paid option has become available and displays a conversation with a specific character.

[0695] This system allows users to receive emotional support and foster positive emotions. It is also expected to reduce feelings of loneliness and improve mental health. By combining real-time emotion analysis by the emotion engine and its feedback function, it is possible to maintain a better mental state in the long term.

[0696] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0697] Step 1: Registering a user

[0698] The user downloads and installs the application on the device.

[0699] The user enters the required information such as name, age, and gender to create an account.

[0700] Input: User information (name, age, gender)

[0701] Data processing: Format verification and hashing of input information

[0702] Output: Account data including user information

[0703] The terminal transmits the input information to the server.

[0704] Input: User information

[0705] Data processing: Encryption

[0706] Output: Data packet with encrypted user information

[0707] The server generates a user profile based on the received information and stores it in a database.

[0708] Input: Encrypted user information

[0709] Data processing: Decryption, storing in database

[0710] Output: User profile added to database entry

[0711] Step 2: Character Generation and Customization

[0712] The server uses a generative AI to generate virtual characters that can be customized by users.

[0713] Input: User Profile

[0714] Data processing: Character generation using generative AI models (e.g., GPT-4)

[0715] Output: Virtual character data

[0716] The device displays a character creation screen where the user can select and customize the character's appearance and name.

[0717] Input: Virtual character data

[0718] Data processing: Drawing character customization UI

[0719] Output: User-selected customization data

[0720] The terminal transmits the customization information to the server.

[0721] Input: Customization data

[0722] Data processing: Encryption

[0723] Output: Encrypted customization data

[0724] The server stores the character information.

[0725] Input: Encrypted customization data

[0726] Data processing: Decryption, storing in database

[0727] Output: Updated character information in the database entry

[0728] Step 3: Collecting audio data and analyzing emotions

[0729] The user speaks to the character and begins a conversation.

[0730] Input: Speech

[0731] Data processing: generating audio streams

[0732] Output: Audio stream data

[0733] The terminal records the user's voice and transmits this voice data to the server.

[0734] Input: Audio stream data

[0735] Data processing: compression and encryption of audio data

[0736] Output: Encrypted audio data file

[0737] The server uses voice recognition technology to convert the voice data into text.

[0738] Input: Encrypted audio data file

[0739] Data processing: decoding, speech recognition (e.g., Google Cloud Speech-to-Text API)

[0740] Output: Text data

[0741] The server analyzes the text data using natural language processing means and extracts the emotional state.

[0742] Input: Text data

[0743] Data processing: Sentiment analysis using natural language processing (NLP library)

[0744] Output: Emotional state data

[0745] The emotion engine analyzes the user's emotions in real time based on the emotion data.

[0746] Input: Emotional state data

[0747] Data processing: Real-time emotion analysis using an emotion engine

[0748] Output: User's emotional state

[0749] Step 4: Generate character responses

[0750] An emotion engine dynamically generates character responses based on the analyzed emotion data.

[0751] Input: User's emotional state

[0752] Data processing: prompt generation, querying generative AI models (e.g., GPT-4)

[0753] Output: Response text

[0754] The server sends the generated response to the terminal.

[0755] Input: Response text

[0756] Data processing: Encryption

[0757] Output: Encrypted response data packet

[0758] The device displays or reads the character's response to the user.

[0759] Input: Encrypted response data packet

[0760] Data processing: decoding, text display or speech synthesis

[0761] Output: Feedback to the user

[0762] Step 5: Cultivating positive emotions

[0763] The device records the user's positive comments and sends them to the server.

[0764] Input: Positive speech

[0765] Data processing: compression and encryption of audio data

[0766] Output: Encrypted audio data file

[0767] The server analyzes positive comments and awards points to the user.

[0768] Input: Encrypted audio data file

[0769] Data processing: decoding, speech recognition, sentiment analysis, point calculation

[0770] Output: Update user point data

[0771] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[0772] Input: User point data

[0773] Data processing: Character growth calculation

[0774] Output: Updated character information

[0775] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[0776] Input: User's emotional history data

[0777] Data processing: Long-term sentiment analysis, character adjustment

[0778] Output: Updated character parameters

[0779] Step 6: Communicating with Anonymous Users

[0780] The user selects the anonymous communication feature.

[0781] Input: User's choice

[0782] Data processing: Request generation

[0783] Output: Anonymous chat request

[0784] The server matches users with the same emotional state or circumstances and creates anonymous chat rooms.

[0785] Input: Anonymous chat request, user profile data

[0786] Data processing: Anonymous user matching, chat room generation

[0787] Output: Chat room ID

[0788] The terminal displays a chat room, allowing users to exchange messages with other users.

[0789] Input: Chat room ID

[0790] Data processing: Drawing chat UI

[0791] Output: Message exchange between users

[0792] Step 7: Offering paid options

[0793] The user selects a paid option and completes the purchase procedure.

[0794] Input: User selection, payment information

[0795] Data processing: encryption, sending to payment gateway

[0796] Output: Payment completion notification

[0797] The server receives the purchase information and adds the ability to communicate with specific characters to the account.

[0798] Input: Payment completion notification

[0799] Data processing: User account updates

[0800] Output: Updated account information

[0801] The device will notify you that a paid option is available and display a conversation with a specific character.

[0802] Input: Updated account information

[0803] Data processing: Notification generation

[0804] Output: User notification and conversation UI with specific characters

[0805] (Application example 2)

[0806] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0807] In recent years, technologies that reduce loneliness and improve mental health by allowing users to receive real-time emotional support using smart devices have become increasingly important. However, existing technologies have not been sufficient in analyzing emotions and providing personalized services to individual customers in physical stores. Furthermore, it has been difficult to instantly provide relevant information when a user shows interest in a product, which has led to a poor user experience. Furthermore, it has been difficult to respond appropriately to store staff's emotions when communicating with them. As a result, user satisfaction often declines, which can negatively impact store sales.

[0808] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a character that the user can customize based on the generated account; speech recognition means and natural language processing means for enabling voice communication between the user and the character; emotion analysis means for generating responses for the character; means for awarding points based on the user's positive emotions; means for improving the character according to the accumulated points; matching means and chat means for anonymously communicating with other users in the same situation or emotional state; purchase means for communicating with a specified avatar as a paid option; gaze tracking means and display means for analyzing the user's dynamic gaze and speech content in real time in a physical store and displaying related information; and generative AI model and prompt sentence utilization means for analyzing the emotions of staff and generating appropriate responses. This allows users to receive personalized emotional support in real time even in a physical store, not only stimulating their interest in products but also enabling them to enjoy high-quality support in communication with staff.

[0809] The "account generation means" is a function that generates a unique user account based on information provided by the user.

[0810] "Customizable character generation means" is a function that allows the user to change the appearance and attributes of the character according to their own preferences.

[0811] The "voice recognition means" is a function that analyzes the voice spoken by the user and converts it into text data.

[0812] "Natural language processing means" is a technology for analyzing text data acquired by speech recognition means and understanding its meaning and context.

[0813] The "emotion analysis means" is a function that analyzes the user's emotional state from their speech and text, and generates an appropriate response based on the results.

[0814] The "point giving means" is a function for assigning points based on the user's actions and comments.

[0815] "Character growth means" is a function that changes a character's skills and appearance according to accumulated points.

[0816] "Matching methods" are functions that anonymously connect users with other users who have the same emotional state or circumstances.

[0817] "Chat means" is a function that allows matched users to exchange text messages.

[0818] The "paid option purchasing means" is a function that allows the user to purchase additional paid services.

[0819] "Eye tracking means" is a technology that detects the user's gaze in real time and identifies the object that the gaze is directed at.

[0820] The "display means" is a function that visually displays appropriate information based on eye tracking and speech content.

[0821] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on the user's emotions and situation.

[0822] "Prompt sentence utilization means" is a function that generates a response using a prompt sentence that gives appropriate instructions to the generative AI model.

[0823] MODE FOR CARRYING OUT THE INVENTION

[0824] The present invention provides a system for providing emotional support to users in real time and improving their mental health. This system aims to improve the customer experience, particularly in physical stores. The specific configuration and operation of the system are described in detail below.

[0825] System configuration

[0826] 1. Device:

[0827] Smart glasses worn by the user that include eye tracking and display capabilities.

[0828] It has a built-in voice recording device that records what the user says.

[0829] 2. Server:

[0830] It recognizes and analyzes voice data and generates responses using generative AI.

[0831] Emotion analysis technology is used to analyze the emotions of users and store staff and provide appropriate services.

[0832] 3. Application:

[0833] It features voice recognition, natural language processing, emotion analysis, point awarding, character growth, matching and chat functions, and the ability to purchase paid options.

[0834] How it works

[0835] User registration and character creation

[0836] 1. The user puts on the smart glasses and launches the application.

[0837] 2. The user creates an account by entering personal information such as name and age.

[0838] 3. The server generates a user profile based on this information using an account generation means.

[0839] 4. The server uses the generative AI model to generate a customizable character that can then be customized by the user.

[0840] Emotion analysis and character responses

[0841] 1. When a user speaks to a character, the device records the voice using a recording device and sends it to the server.

[0842] 2. The server converts the voice data into text using speech recognition and natural language processing means, and performs sentiment analysis.

[0843] 3. Based on the emotional data analyzed by the emotion engine, the generative AI model generates an appropriate response.

[0844] 4. The device displays the response on the smart glasses display or reads it out loud.

[0845] Points awarded and character growth

[0846] 1. When a user makes a positive comment, the server analyzes it and awards points.

[0847] 2. The server will develop the character and add new skills according to the accumulated points.

[0848] Improving user experience in physical stores

[0849] 1. When a user enters a physical store, the device uses eye tracking to identify products that interest the user.

[0850] 2. The device analyzes the speech in real time and displays relevant information on the screen.

[0851] Dialogue with staff

[0852] 1. The device records the conversation with the staff and sends it to the server.

[0853] 2. The server analyzes the staff member's emotions, generates an appropriate response, sends it to the terminal, and displays or reads it aloud.

[0854] Specific examples

[0855] 1. Imagine a scenario where a user enters a brick-and-mortar store and puts on a pair of smart glasses.

[0856] The staff greets you with "Welcome!"

[0857] The device records this audio and sends it to the server.

[0858] The server performs speech recognition and emotion analysis and generates a response such as, "That's very kind of you. I see you like our products."

[0859] The terminal displays this response on its display and the user confirms it.

[0860] Example prompt sentence:

[0861] Scenario: Inside the store, a staff member is heard saying, "Welcome!"

[0862] The user's sentiment is encouraging.

[0863] Generate an appropriate response.

[0864] Response: That's very kind of you, I see you like our products.

[0865] This allows users to receive personalized emotional support in real time even in physical stores, not only stimulating their interest in products but also enabling them to enjoy high-quality support when communicating with staff.

[0866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0867] Specific explanations divided into processing steps

[0868] Step 1:

[0869] A user puts on the smart glasses and launches the application. The user creates an account by entering personal information such as name and age. The device sends this information to the server. Based on the entered information, the server generates a user profile and creates a unique account using the account generation means. The output is the user profile and account information.

[0870] Step 2:

[0871] The server uses the generative AI model to generate a character that can be customized by the user. The user customizes the character's appearance and attributes through the smart glasses display and sends that information to the server. The output is the customized character information.

[0872] Step 3:

[0873] When a user speaks to a character, the device records the voice and transmits the voice data to the server in real time. The server converts the voice data into text data using a voice recognition means, and obtains the voice data as input and the text data as output.

[0874] Step 4:

[0875] The server uses natural language processing means to analyze the converted text data, obtaining the text data as input. This analysis allows the server to understand the meaning and context of the user's speech, and then uses emotion analysis means to obtain analyzed emotion data as output.

[0876] Step 5:

[0877] The emotion engine uses a generative AI model to generate an appropriate response based on the analyzed emotion data. The prompt text is used to instruct the generative AI model to generate an appropriate response. The generated response text is obtained as output.

[0878] Step 6:

[0879] The device displays the generated response on the smart glasses display or reads it out loud, allowing the user to receive a response from the character in real time. The displayed or spoken response is obtained as output.

[0880] Step 7:

[0881] When a user makes a positive comment, the device records the voice and sends it to the server. The server analyzes the comment and awards points to the user. The input is the voice data of the positive comment, and the output is point information.

[0882] Step 8:

[0883] As points accumulate, the server uses character development tools to develop the character, including acquiring new skills and changing appearance. The output is information about the developed character.

[0884] Step 9:

[0885] When a user enters a physical store, the smart glasses' eye tracking function catches the user's gaze. The device uses the eye tracking means to transmit product information about the user's gaze to the server in real time. Eye tracking data is obtained as input, and product information is obtained as output.

[0886] Step 10:

[0887] The server uses a generative AI model based on the gaze data and the user's speech to generate appropriate information and send it to the device. The device then provides this information to the user via a display, providing product-related information as output.

[0888] Step 11:

[0889] When a user interacts with a store staff member, the device records the voice and sends it to the server. The server then analyzes the staff member's speech and generates an appropriate response using emotion analysis. The staff member's speech is obtained as input, and the response text is obtained as output.

[0890] Step 12:

[0891] The terminal generates a response based on the interaction with the staff and displays it on the user's smart glasses or reads it out loud, allowing the user to smoothly interact with the staff. The displayed or spoken response is obtained as an output.

[0892] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0893] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0894] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0895] [Second embodiment]

[0896] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0897] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0898] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0899] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0900] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0901] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0902] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0903] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0904] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0905] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0906] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0907] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0908] MODE FOR CARRYING OUT THE INVENTION

[0909] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[0910] System configuration

[0911] The system mainly consists of the following components:

[0912] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[0913] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[0914] 3. Application: Software installed on the device that has functions such as communication between users and characters, emotion analysis, point awarding, and matching.

[0915] System Operation

[0916] 1. User Registration

[0917] The user downloads and installs the application onto the terminal.

[0918] Users enter required information such as name, age, and gender to create an account.

[0919] The terminal transmits the entered information to a server, which generates a user profile.

[0920] 2. Character Creation and Customization

[0921] The server uses a generative AI to generate virtual characters that can be customized by users.

[0922] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[0923] Once customization is complete, the device sends the information to the server, which then stores the character information.

[0924] 3. Audio data collection and emotion analysis

[0925] The user speaks to the character and begins a conversation.

[0926] The terminal records the user's voice and transmits this voice data to the server.

[0927] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[0928] 4. Character response generation

[0929] The server generates an appropriate character response based on the analyzed emotional data.

[0930] The terminal displays or reads the generated response to the user.

[0931] 5. Cultivating positive emotions

[0932] The device records the user's positive comments and sends them to the server.

[0933] The server analyzes positive comments and awards points to the user.

[0934] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[0935] 6. Communication with Anonymous Users

[0936] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[0937] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[0938] 7. Paid options available

[0939] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[0940] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[0941] Specific examples

[0942] 1. User registration and character creation

[0943] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[0944] The server receives this information and creates a user profile.

[0945] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[0946] 2. Emotion Analysis and Character Response

[0947] The user speaks to the character, saying, "It's a nice day today."

[0948] The device records the user's voice and sends it to the server.

[0949] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[0950] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[0951] The device displays or reads this response to the user.

[0952] 3. Points awarded and character growth

[0953] The user continues to make positive comments, saying, "I've recently started jogging."

[0954] The device records this audio and sends it to the server.

[0955] The server analyzes positive comments and awards points to the user.

[0956] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[0957] The terminal notifies the user of the character's growth.

[0958] 4. Communication with Anonymous Users

[0959] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[0960] The server generates the chat room and the device displays it to the user.

[0961] Users anonymously exchange jogging information with other users.

[0962] 5. Use of paid options

[0963] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[0964] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[0965] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[0966] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health.

[0967] The processing flow will be explained below.

[0968] Program processing steps

[0969] User Registration and Initial Setup

[0970] Step 1:

[0971] The user downloads and installs the application on the device.

[0972] Step 2:

[0973] Users launch the application and create an account by entering required information such as name, age, and gender.

[0974] Step 3:

[0975] The terminal transmits the input information to the server.

[0976] Step 4:

[0977] The server generates and stores a user profile based on the received information.

[0978] Step 5:

[0979] The server sends an initialization success message to the terminal.

[0980] Step 6:

[0981] The device will notify the user that the initial setup is complete and open the character creation screen.

[0982] Step 7:

[0983] Users customize their character's appearance and name.

[0984] Step 8:

[0985] The terminal transmits the customized character information to the server.

[0986] Step 9:

[0987] The server stores the character information in a user profile.

[0988] Emotion analysis and character generation

[0989] Step 10:

[0990] The user speaks to the character and begins a conversation.

[0991] Step 11:

[0992] The terminal records the user's voice and transmits the voice data to the server.

[0993] Step 12:

[0994] The server uses voice recognition technology to convert the voice data into text.

[0995] Step 13:

[0996] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[0997] Step 14:

[0998] The server generates a response message for the character based on the result of the emotion analysis.

[0999] Step 15:

[1000] The server sends the response of the generated character to the terminal.

[1001] Step 16:

[1002] The terminal displays or reads out the character's response to the user.

[1003] Cultivating positive emotions and awarding points

[1004] Step 17:

[1005] The user makes positive comments during a conversation with the character.

[1006] Step 18:

[1007] The device records positive comments and sends the audio data to a server.

[1008] Step 19:

[1009] The server analyzes the positive comments.

[1010] Step 20:

[1011] The server awards points to the user based on the analysis results.

[1012] Step 21:

[1013] The server determines character growth (gaining new skills or changing appearance) based on the accumulated points.

[1014] Step 22:

[1015] The server sends character growth information to the terminal.

[1016] Step 23:

[1017] The terminal notifies the user of the character's growth and point allocation.

[1018] Communicating with Anonymous Users

[1019] Step 24:

[1020] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[1021] Step 25:

[1022] The terminal transmits the user's selection to the server.

[1023] Step 26:

[1024] The server searches for other users with the same emotional state or circumstances and performs matching.

[1025] Step 27:

[1026] The server creates anonymous chat rooms for matched users.

[1027] Step 28:

[1028] The server transmits chat room information to the terminal.

[1029] Step 29:

[1030] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[1031] Step 30:

[1032] Users communicate with other users anonymously.

[1033] Paid options available

[1034] Step 31:

[1035] The user selects the communication function with the celebrity avatar from a menu within the application.

[1036] Step 32:

[1037] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[1038] Step 33:

[1039] The user enters the necessary information and presses the "Purchase" button.

[1040] Step 34:

[1041] The terminal transmits the purchase information to the server.

[1042] Step 35:

[1043] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[1044] Step 36:

[1045] The server sends a notification of purchase completion to the terminal.

[1046] Step 37:

[1047] The terminal notifies the user that the paid option is now available.

[1048] Step 38:

[1049] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[1050] Example 1

[1051] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1052] In modern society, many people face the problem of feeling lonely. The lack of opportunities to receive emotional support in daily life is a particular challenge for the elderly and those who tend to be isolated. Under these circumstances, maintaining mental health becomes difficult, increasing the risk of serious mental and physical problems. Furthermore, existing emotional support systems have difficulty responding flexibly to the emotional state of individual users. There is a need for a system that can solve these issues and provide users with continuous, personalized emotional support.

[1053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1054] In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a virtual character that the user can customize based on the generated account; speech recognition and natural language processing means for enabling voice communication between the user and the virtual character; emotion analysis means for generating responses from the virtual character; means for awarding points based on the user's positive emotions; means for upgrading the virtual character according to the accumulated points; matching and messaging means for anonymously communicating with other users in the same circumstances or emotional state; purchase means for realizing communication with a specific avatar as a paid option; display or audio output means on the terminal for notifying the user of the generated responses; means for generating responses from the virtual character using a generative AI model; means for generating prompt sentences to be input to the generative AI model based on the emotion analysis results; and means for generating responses from the virtual character using the generated prompt sentences. This makes it possible to provide continuous and personalized emotional support to users who feel lonely and improve their mental health.

[1055] A "terminal" is an electronic device used by a user and on which software is installed.

[1056] "Software" refers to a program that is installed and executed on a terminal, and provides various functions through interaction with the user.

[1057] A "user" is a person who uses the system and inputs information and communicates via the application.

[1058] "Account" means a digital management unit that contains user-specific information and is required to use the Software.

[1059] A "virtual character" is a user-customizable digital agent that interacts with the user to provide emotional support.

[1060] "Speech recognition means" is a technology that converts voice data into text data.

[1061] "Natural language processing means" is a technology that analyzes text data and understands meaning and emotions.

[1062] "Emotion analysis means" is a technology that estimates a user's emotional state from their statements and text.

[1063] "Points" are digital evaluation units awarded based on users' actions and comments.

[1064] "Growth methods" are techniques that change the skills and appearance of a virtual character according to the accumulation of points.

[1065] "Matching methods" are technologies that connect users with similar circumstances or emotional states.

[1066] "Messaging means" refers to technology that allows anonymous messaging.

[1067] "Paid Options" are additional features or services that are available for an additional fee.

[1068] "Purchase Instrument" means a payment technique for trading paid options.

[1069] "Display means" refers to a technique for displaying the generated response on the screen of the terminal.

[1070] The "audio output means" is a technique for outputting the generated response as audio.

[1071] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and responses.

[1072] A "prompt" is text data that can be input into a generative AI model to elicit a specific response.

[1073] MODE FOR CARRYING OUT THE INVENTION

[1074] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[1075] System configuration

[1076] The system consists of the following elements:

[1077] 1. Device: An electronic device used by a user, such as a smartphone, tablet, or PC. Dedicated software is installed on the device.

[1078] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[1079] 3. Software: A program installed on the device that has functions such as communication between the user and virtual characters, emotion analysis, point awarding, and matching.

[1080] System Operation

[1081] 1. User Registration

[1082] The user downloads and installs the application onto the terminal.

[1083] Users enter required information such as name, age, and gender to create an account.

[1084] The terminal transmits the entered information to a server, which generates a user profile.

[1085] 2. Character Creation and Customization

[1086] The server uses a generative AI model to generate virtual characters that can be customized by users.

[1087] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[1088] Once customization is complete, the device sends the information to the server, which then stores the character information.

[1089] 3. Audio data collection and emotion analysis

[1090] The user speaks to the character and begins a conversation.

[1091] The terminal records the user's voice and transmits this voice data to the server.

[1092] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[1093] 4. Character response generation

[1094] The server generates an appropriate character response based on the analyzed emotion data. A generative AI model (e.g., ChatGPT) is used to generate a prompt. An example of a prompt is, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[1095] The terminal displays or reads the generated response to the user.

[1096] 5. Cultivating positive emotions

[1097] The device records the user's positive comments and sends them to the server.

[1098] The server analyzes positive comments and awards points to the user.

[1099] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[1100] 6. Communication with Anonymous Users

[1101] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[1102] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[1103] 7. Paid options available

[1104] When a user selects a paid option and completes the purchase procedure, the server receives the purchase information and adds a communication function with a specific avatar to the account.

[1105] The device notifies the user that paid options are available and displays conversations with specific avatars.

[1106] Specific examples

[1107] For example, if a user says to a character, "It's a nice day today," the following happens:

[1108] The user speaks to the character, saying, "It's a nice day today."

[1109] The device uses a built-in microphone to record the user's voice and transmits this voice data to the server.

[1110] The server converts the speech into text using the Google Cloud Speech-to-Text API, and then analyzes the emotional state of the text using natural language processing technology (e.g., IBM Watson NLU).

[1111] The server sends a prompt to a generative AI model (e.g., ChatGPT) to generate a response text. Example prompt: "The user said, 'It's a nice day today.' Please generate a positive, constructive response."

[1112] The server generates a response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[1113] The device will either display the text or convert it into speech using a speech synthesis API and respond to the user audibly.

[1114] This allows users to receive continuous and personalized emotional support, which can help reduce feelings of loneliness and improve mental health.

[1115] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1116] Program processing flow

[1117] Registering Users

[1118] Step 1:

[1119] Input: The user downloads and installs the application on their device.

[1120] Output: The installed application starts running on the device.

[1121] What happens: A user downloads and installs an app from the app store.

[1122] Step 2:

[1123] Input: The user enters the required information (name, age, gender, etc.) on the account registration screen.

[1124] Output: User input information is temporarily stored on the device and sent to the server.

[1125] Specific behavior: The device displays an input form, and the user enters information using a keyboard or on-screen keyboard.

[1126] Step 3:

[1127] Input: User information sent from the device.

[1128] Output: The server generates a user profile and stores it in the database.

[1129] Specific operation: The device sends an HTTPS request and the server saves the user information in the database.

[1130] Character Generation and Customization

[1131] Step 4:

[1132] Input: The server generates basic information about the virtual character using a generative AI model.

[1133] Output: Basic setting data of the virtual character is generated.

[1134] Specific operation: The server sends prompts to the generative AI model (e.g., ChatGPT) to generate the initial settings for the virtual character.

[1135] Step 5:

[1136] Input: The user interacts with the character creation screen on their device.

[1137] Output: User-customized character information is saved on the device.

[1138] Specific behavior: The device displays a user interface (UI), and the user operates drop-down menus and sliders.

[1139] Step 6:

[1140] Input: The user completes the customization and the device sends the information to the server.

[1141] Output: The server saves the character information to the database.

[1142] Specific operation: The device sends JSON data containing customization information to the server, and the server stores it in a database.

[1143] Voice data collection and sentiment analysis

[1144] Step 7:

[1145] Input: The user speaks to the character.

[1146] Output: The user's voice data is recorded on the device.

[1147] Specific action: The user speaks into the device's microphone.

[1148] Step 8:

[1149] Input: Recorded audio data.

[1150] Output: The device sends the audio data to the server.

[1151] Specific operation: The device starts the voice recording function and sends the recorded data to the server.

[1152] Step 9:

[1153] Input: The audio data received by the server.

[1154] Output: The server parses the user's utterance as text data.

[1155] Specific operation: The server calls a speech recognition API (e.g., Google Cloud Speech-to-Text) and sends the resulting text data to a natural language processing API (e.g., IBM Watson NLU).

[1156] Character response generation

[1157] Step 10:

[1158] Input: Server parsed emotion data.

[1159] Output: The server generates an appropriate response text using the generative AI model.

[1160] Specific operation: The server sends a prompt to the generative AI model to generate a response text. For example, it sends the prompt sentence, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[1161] Step 11:

[1162] Input: The generated response text.

[1163] Output: The device displays or reads the text.

[1164] Specific behavior: The device displays the generated response text on the screen or converts it into speech using a speech synthesis API (e.g., Amazon Polly) and reads it to the user.

[1165] Cultivating positive emotions

[1166] Step 12:

[1167] Input: User makes a positive statement.

[1168] Output: The device records what you say and sends it to the server.

[1169] Specific operation: The device records what the user says and sends the recording data to the server.

[1170] Step 13:

[1171] Input: The server receives positive utterance data.

[1172] Output: Positive comments are analyzed and points are awarded to the user.

[1173] What it does: The server uses natural language processing to detect positive words and phrases and adds points to the user's profile.

[1174] Step 14:

[1175] Input: The points accumulated by the server.

[1176] Output: Reflects character growth (gaining new skills and changing appearance).

[1177] Specific operation: The server updates the character information based on the growth algorithm and reflects it on the device.

[1178] Communicating with Anonymous Users

[1179] Step 15:

[1180] Input: The user selects the anonymous communication feature.

[1181] Output: The server matches users with the same circumstances and emotional state.

[1182] Specific operation: The server compares user profiles and selects other users with high matching scores.

[1183] Step 16:

[1184] Input: Matched user information.

[1185] Output: The server creates an anonymous chat room and sends a link to the device.

[1186] Specific operation: The server generates a chat room URL and sends it to the device, which displays the link.

[1187] Paid options available

[1188] Step 17:

[1189] Input: The user selects a paid option and completes the purchase.

[1190] Output: The purchase is completed and the ability to communicate with the specified avatar is added to your account.

[1191] Specific operation: The device displays a list of paid options, and the user selects one. The purchase is completed using a payment API (e.g., Stripe or PayPal).

[1192] Step 18:

[1193] Input: The server verifies the purchase information.

[1194] Output: Paid options become available and are notified on the device.

[1195] What happens: The server confirms the purchase and adds the new feature to the user's profile. The device displays a pop-up notification informing the user of the paid option and showing the celebrity avatar's conversation screen.

[1196] (Application example 1)

[1197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1198] In recent years, the number of individuals experiencing loneliness has increased, creating a need for emotional support. However, there is a lack of effective methods for improving emotional and shopping experiences in physical stores. Conventional systems struggle to provide an environment that alleviates users' feelings of loneliness and fosters positive emotions. Furthermore, they do not recommend services or products based on the user's real-time emotional state, making it impossible to provide an optimized experience for each individual user. Therefore, a new system is needed to alleviate loneliness, foster positive emotions, and provide a personalized experience in physical stores.

[1199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1200] In this invention, the server includes means for receiving information from a user and generating an account, means for generating a character that the user can customize based on the generated account, speech recognition means and natural language processing means for enabling voice communication between the user and the character, emotion analysis means for generating responses from the character, means for awarding points based on the user's positive emotions, means for enhancing the character according to the accumulated points, matching means and chat means for anonymously communicating with other users in the same circumstances or emotional state, means for recommending products and services based on the user's emotional state in a physical store, and purchasing means for communicating with a specified avatar. This allows users to reduce feelings of loneliness, foster positive emotions, and enjoy a personalized experience in a physical store.

[1201] A "terminal" is an electronic device used by a user, including a smartphone, tablet, or PC.

[1202] An "application" is a software program that is installed on a terminal and has functions such as communication between the user and a virtual character and emotion analysis.

[1203] "User" refers to a person who uses the system and is an individual who wishes to receive emotional support.

[1204] An "account" is a record containing a user's identifying information, and is used to identify an individual user within the system.

[1205] A "character" is a user-customizable virtual entity that provides emotional support through communication with the user.

[1206] "Speech recognition means" refers to a technology that converts a user's voice into text data, and is used to analyze the content of what the user says.

[1207] "Natural language processing means" is a technology for understanding text data obtained by speech recognition means and generating an appropriate response.

[1208] "Emotion analysis means" is a technology that analyzes the emotional state of a user from their statements and actions.

[1209] "Points" are numerical values ​​that the system awards to users for their positive comments and actions, and are used to develop their characters and receive special benefits.

[1210] "Growth" means that the character's skills and appearance improve or change depending on the user's accumulated points.

[1211] "Matching means" is a technology that automatically finds other users who are in the same circumstances or emotional state, and enables them to communicate with each other.

[1212] "Chat means" is a function that allows users to send and receive text messages.

[1213] "Physical store" refers to a physical commercial facility or service location.

[1214] "Means for recommending products and services" refers to technology for presenting products and services that are considered optimal based on the user's emotional state.

[1215] "Purchase means" is a function that allows a user to select and purchase a paid option.

[1216] "Communicating anonymously" means exchanging messages with other users without using their real names.

[1217] The present invention relates to a system for providing emotional support to users and improving their mental health, which is realized through an application installed on a terminal, a central server, and a network connecting them.

[1218] System configuration

[1219] 1. Terminal

[1220] A terminal is an electronic device used by a user, and includes a smartphone, tablet, PC, etc. An application that enables communication between the user and a virtual character is installed on this terminal.

[1221] 2. Server

[1222] The server acts as a central server, processing, storing, and analyzing data sent by users. The server uses emotion analysis tools and generative AI models to analyze the user's emotional state and generate appropriate responses.

[1223] 3. Application

[1224] An application is a software program that is installed on a terminal and has functions such as communication between the user and a virtual character and emotion analysis.

[1225] System Operation

[1226] 1. User Registration

[1227] Users download and install the application on their device, create an account by entering information such as their name, age, and gender, and the device then sends this information to a server to generate a user profile.

[1228] 2. Character Creation and Customization

[1229] The server uses a generative AI model to generate virtual characters that users can customize. Users select and customize the character's appearance and name, and the information is stored on the server.

[1230] 3. Speech Communication and Emotion Analysis

[1231] When a user speaks to a character, the device records the user's voice and sends it to the server, which converts the voice data into text using speech recognition and natural language processing, and analyzes it using emotion analysis.

[1232] 4. Response Generation

[1233] The server generates an appropriate character response based on the analysis results and sends it to the device, which then displays or reads the generated response to the user.

[1234] Example

[1235] 1. Examples of applications in physical stores

[1236] In a physical store, users communicate with a virtual character via their smartphone or smart glasses. The character recommends relaxation areas and products based on the user's emotional state. If the character asks, "How are you feeling today?" and the user replies, "I'm a little tired," the character will recommend, "There's a relaxation area in this area."

[1237] 2. Points awarded and character growth

[1238] When a user engages in positive conversation, points are awarded and the character grows. For example, if a user says, "I went to a cafe with my friends yesterday and had fun," points are awarded and the character acquires a new skill.

[1239] Program processing overview

[1240] Hardware and software used:

[1241] Hardware: Smartphones, smart glasses

[1242] Software: Flask (web application framework), SpeechRecognition (voice recognition library), openai (generative AI model)

[1243] Data processing and calculation:

[1244] The server manages user profiles, analyzes emotions, and generates responses using generative AI. For example, the following prompts are generated:

[1245] The user said 'I'm a bit tired'. Their emotion is 'tired'. Give an appropriate response.

[1246] The system allows users to reduce feelings of loneliness, foster positive emotions, and enjoy personalized experiences in physical stores.

[1247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1248] Step 1:

[1249] The user downloads the application and installs it on their device.

[1250] Input: User's internet-connected device

[1251] Output: Installed applications

[1252] Specific operation: The user downloads the application from the official store and installs it on their device.

[1253] Step 2:

[1254] To create an account, a user enters information such as name, age, and gender.

[1255] Input: User personal information

[1256] Output: The information entered by the user is sent to the server and a user profile is generated.

[1257] Specific operation: The user enters the required information into a form within the application and presses the "Submit" button.

[1258] Step 3:

[1259] The server uses the generative AI model to generate a virtual character that can be customized by the user.

[1260] Input: User profile information

[1261] Output: Generated virtual character

[1262] Specific operation: The server generates a virtual character using a generative AI model based on the user's profile information.

[1263] Step 4:

[1264] The device displays a character creation screen where the user can customize the character's appearance and name.

[1265] Input: Generated character and user customization settings

[1266] Output:Customized Character

[1267] Specific operation: A character builder will appear on the device screen, and the user can adjust the appearance and name and save it.

[1268] Step 5:

[1269] The user speaks to the character and begins a conversation.

[1270] Input: User's voice

[1271] Output: Audio data

[1272] Specific actions: The user speaks to the character through the microphone.

[1273] Step 6:

[1274] The device records the user's voice and sends it to the server.

[1275] Input: User's voice data

[1276] Output: Audio data sent to the server

[1277] Specific operation: The device sends the audio recorded by the microphone to the server as digital data.

[1278] Step 7:

[1279] The server converts the voice data into text using a voice recognition means, and analyzes the text data using a natural language processing means.

[1280] Input: User's voice data

[1281] Output: Text data and analysis results

[1282] Specific operation: The speech recognition library converts the speech data into a string of characters, and the natural language processing means analyzes the string of characters.

[1283] Step 8:

[1284] An emotion analysis means analyzes the user's emotional state.

[1285] Input: Parsed text data

[1286] Output: User's emotional state

[1287] What it does: Sentiment analysis algorithms extract emotional states from text data.

[1288] Step 9:

[1289] The server generates a response for the virtual character based on the emotion analysis results.

[1290] Input: User emotional state and parsed text data

[1291] Output: The generated response

[1292] Specific behavior: The generative AI model generates an appropriate response based on emotional state and text data.

[1293] Step 10:

[1294] The terminal displays or reads the generated response to the user.

[1295] Input: The generated response from the server

[1296] Output: The response that is displayed or read to the user

[1297] Specific behavior: Display the response text on the device screen or read it aloud through the speaker.

[1298] Step 11:

[1299] Points are awarded for positive comments made by users.

[1300] Input: Analysis results of positive comments

[1301] Output: Updated points

[1302] What it does: The server recognizes positive emotions and adds them to a points system.

[1303] Step 12:

[1304] Characters grow according to the accumulated points.

[1305] Input: Accumulated points

[1306] Output: Grown-up character

[1307] What it does: The server checks the accumulated points and adds new skills and cosmetic changes to the character.

[1308] Step 13:

[1309] It provides a chat function and matches users with similar circumstances and emotional states to communicate anonymously.

[1310] Input: User's emotional state and situation information

[1311] Output: Matching results and chat rooms

[1312] What it does: The server finds other suitable users and creates an anonymous chat room.

[1313] Step 14:

[1314] Recommend products and services based on the user's emotional state in a physical store.

[1315] Input: User's emotional state

[1316] Output: Recommended products and services

[1317] Specific operation: The server generates prompt sentences that suggest appropriate products and services based on the emotional state.

[1318] The user said 'I'm a bit tired'. Their emotion is 'tired'. Give an appropriate response.

[1319] Step 15:

[1320] The user performs a purchase procedure to realize communication with a designated avatar as a paid option.

[1321] Input: User's purchase intent

[1322] Output: Purchase completed and function added

[1323] Specific operation: The server completes the purchase process and adds the ability to communicate with the specified avatar.

[1324] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1325] MODE FOR CARRYING OUT THE INVENTION

[1326] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The system aims to foster positive emotions through communication between users and virtual characters using emotion analysis technology, generative AI technology, and an emotion engine.

[1327] System configuration

[1328] The system mainly consists of the following components:

[1329] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[1330] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[1331] 3. Application: Software installed on the device that has various functions such as communication between the user and the character, emotion analysis, point awarding, matching, and an emotion engine.

[1332] 4. Emotion engine: An engine that analyzes the user's emotions in real time and dynamically adjusts the character's responses and actions based on that information.

[1333] System Operation

[1334] 1. User Registration

[1335] The user downloads and installs the application on the device.

[1336] Users enter required information such as name, age, and gender to create an account.

[1337] The terminal transmits the entered information to a server, which generates a user profile.

[1338] 2. Character Creation and Customization

[1339] The server uses a generative AI to generate virtual characters that can be customized by users.

[1340] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[1341] Once customization is complete, the device sends the information to the server, which then stores the character information.

[1342] 3. Audio data collection and emotion analysis

[1343] The user speaks to the character and begins a conversation.

[1344] The terminal records the user's voice and transmits this voice data to the server.

[1345] The server converts the voice data into text using speech recognition technology and analyzes the emotional state using natural language processing means.

[1346] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[1347] 4. Character response generation

[1348] The emotion engine dynamically generates character responses based on the analyzed emotion data.

[1349] The server sends the generated response to the terminal.

[1350] The terminal displays or reads out the character's response to the user.

[1351] 5. Cultivating positive emotions

[1352] The device records the user's positive comments and sends them to the server.

[1353] The server analyzes positive comments and awards points to the user.

[1354] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[1355] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[1356] 6. Communication with Anonymous Users

[1357] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[1358] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[1359] 7. Paid options available

[1360] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[1361] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[1362] Specific examples

[1363] 1. User registration and character creation

[1364] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[1365] The server receives this information and creates a user profile.

[1366] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[1367] 2. Emotion Analysis and Character Response

[1368] The user speaks to the character, saying, "It's a nice day today."

[1369] The device records the user's voice and sends it to the server.

[1370] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[1371] Using the recorded emotion data, the emotion engine analyzes the user's real-time emotional state.

[1372] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[1373] The device displays or reads this response to the user.

[1374] 3. Points awarded and character growth

[1375] The user continues to make positive comments, saying, "I've recently started jogging."

[1376] The device records this audio and sends it to the server.

[1377] The server analyzes positive comments and awards points to the user.

[1378] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[1379] The terminal notifies the user of the character's growth.

[1380] The emotion engine tracks the user's long-term emotional changes and responds appropriately.

[1381] 4. Communication with Anonymous Users

[1382] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[1383] The server generates the chat room and the device displays it to the user.

[1384] Users anonymously exchange jogging information with other users.

[1385] 5. Use of paid options

[1386] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[1387] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[1388] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[1389] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health. By combining real-time emotion analysis by the emotion engine and its feedback function, it is possible to maintain a better long-term mental state for users.

[1390] The processing flow will be explained below.

[1391] Program processing steps

[1392] User Registration and Initial Setup

[1393] Step 1:

[1394] The user downloads and installs the application on the device.

[1395] Step 2:

[1396] Users launch the application and create an account by entering required information such as name, age, and gender.

[1397] Step 3:

[1398] The terminal transmits the input information to the server.

[1399] Step 4:

[1400] The server generates and stores a user profile based on the received information.

[1401] Step 5:

[1402] The server sends an initialization success message to the terminal.

[1403] Step 6:

[1404] The device will notify the user that the initial setup is complete and open the character creation screen.

[1405] Step 7:

[1406] Users customize their character's appearance and name.

[1407] Step 8:

[1408] The terminal transmits the customized character information to the server.

[1409] Step 9:

[1410] The server stores the character information in a user profile.

[1411] Emotion analysis and character generation

[1412] Step 10:

[1413] The user speaks to the character and begins a conversation.

[1414] Step 11:

[1415] The terminal records the user's voice and transmits the voice data to the server.

[1416] Step 12:

[1417] The server uses voice recognition technology to convert the voice data into text.

[1418] Step 13:

[1419] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[1420] Step 14:

[1421] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[1422] Step 15:

[1423] The emotion engine dynamically generates character responses based on the analyzed data.

[1424] Step 16:

[1425] The server sends the response of the generated character to the terminal.

[1426] Step 17:

[1427] The terminal displays or reads out the character's response to the user.

[1428] Cultivating positive emotions and awarding points

[1429] Step 18:

[1430] The user makes positive comments during a conversation with the character.

[1431] Step 19:

[1432] The device records positive comments and sends the audio data to a server.

[1433] Step 20:

[1434] The server analyzes the positive comments.

[1435] Step 21:

[1436] The server awards points to the user based on the analysis results.

[1437] Step 22:

[1438] The server determines the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[1439] Step 23:

[1440] The server sends character growth information to the terminal.

[1441] Step 24:

[1442] The terminal notifies the user of the character's growth and point allocation.

[1443] Step 25:

[1444] The emotion engine tracks the user's long-term emotional changes and dynamically adjusts the character's response.

[1445] Communicating with Anonymous Users

[1446] Step 26:

[1447] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[1448] Step 27:

[1449] The terminal transmits the user's selection to the server.

[1450] Step 28:

[1451] The server searches for other users with the same emotional state or circumstances and performs matching.

[1452] Step 29:

[1453] The server creates anonymous chat rooms for matched users.

[1454] Step 30:

[1455] The server transmits chat room information to the terminal.

[1456] Step 31:

[1457] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[1458] Step 32:

[1459] Users communicate with other users anonymously.

[1460] Paid options available

[1461] Step 33:

[1462] The user selects the communication function with the celebrity avatar from a menu within the application.

[1463] Step 34:

[1464] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[1465] Step 35:

[1466] The user enters the necessary information and presses the "Purchase" button.

[1467] Step 36:

[1468] The terminal transmits the purchase information to the server.

[1469] Step 37:

[1470] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[1471] Step 38:

[1472] The server sends a notification of purchase completion to the terminal.

[1473] Step 39:

[1474] The terminal notifies the user that the paid option is now available.

[1475] Step 40:

[1476] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[1477] Example 2

[1478] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1479] Conventional emotional support systems have been unable to provide sufficient emotional support to users who feel lonely, and have been insufficient to improve the user's long-term mental health. Furthermore, communication with the user is one-way, making it difficult to provide immediate feedback based on the user's emotional state. As a result, users are less likely to be satisfied with the system, making it difficult to promote continued use.

[1480] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving information from a user and generating an account, a means for generating a virtual character that the user can customize based on the generated account, a voice recognition means and natural language processing means for enabling voice communication between the user and the character, a means for converting voice data into text and analyzing the emotional state, an emotion engine for generating the character's responses, a means for awarding points based on the user's positive emotions, a means for improving the character according to the accumulated points, a matching means and chat means for anonymously communicating with other users in the same situation or emotional state, and a purchasing means for realizing communication with a specified character as a paid option. This enables immediate feedback based on the user's emotional state and provides continuous emotional support. Furthermore, the user can experience a reduction in loneliness and an improvement in mental health.

[1481] A "terminal" is an electronic device used by a user, and includes a smartphone, tablet, PC, etc.

[1482] An "application" is a software program installed on a terminal that manages the interaction between the user and the character.

[1483] An "account" is a collection of data that includes a user's identification information and is used to manage an individual user's operations within the system.

[1484] A "virtual character" is a digital character that can be customized by the user using a generative AI model and that communicates with the user.

[1485] "Speech recognition means" refers to technology that records a user's voice and converts that voice into text data.

[1486] "Natural language processing means" refers to technology that analyzes text data and extracts linguistic meanings and emotional states.

[1487] "Emotional state" refers to the emotional state extracted from a user's speech or text, and includes, for example, joy, sadness, anger, etc.

[1488] The "emotion engine" is an engine that analyzes the user's emotional state and generates responses from the digital character based on the results.

[1489] The "means of awarding points" refers to a mechanism that adds points within the system based on users' positive actions and comments.

[1490] "Means for character development" refers to a system that allows you to evolve your virtual character's skills and appearance based on accumulated points.

[1491] "Matching means" refers to technology that identifies other users with the same emotional state or interests on the server side and connects them in anonymous chat rooms.

[1492] "Chat vehicle" refers to the interface and functionality for exchanging text messages between users.

[1493] "Purchase method" refers to the operation by which a user selects a paid option and makes additional features available through payment.

[1494] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The system aims to foster positive emotions through interactive communication between users and virtual characters using emotion analysis technology, generative AI technology, and an emotion engine.

[1495] System configuration

[1496] This system consists of the following elements:

[1497] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[1498] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[1499] 3. Application: Software installed on the device that has various functions such as communication between the user and the character, emotion analysis, point awarding, matching, and an emotion engine.

[1500] 4. Emotion engine: An engine that analyzes the user's emotions in real time and dynamically adjusts the character's responses and actions based on that information.

[1501] System Operation

[1502] 1. User Registration

[1503] The user downloads and installs the application on the device.

[1504] Users enter required information such as name, age, and gender to create an account.

[1505] The terminal transmits the entered information to a server, which generates a user profile.

[1506] 2. Character Creation and Customization

[1507] The server uses generative AI to generate virtual characters that users can customize. Specifically, it uses a generative AI model (e.g., GPT-4).

[1508] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[1509] Once customization is complete, the device sends the information to the server, which then stores the character information.

[1510] 3. Audio data collection and emotion analysis

[1511] The user speaks to the character and begins a conversation.

[1512] The terminal records the user's voice and transmits this voice data to the server.

[1513] The server converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[1514] The server analyzes the emotional state using natural language processing means (NLP library).

[1515] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[1516] 4. Character response generation

[1517] The emotion engine dynamically generates character responses based on the analyzed emotion data.

[1518] The server sends the generated response to the terminal.

[1519] The terminal displays or reads out the character's response to the user.

[1520] 5. Cultivating positive emotions

[1521] The device records the user's positive comments and sends them to the server.

[1522] The server analyzes positive comments and awards points to the user.

[1523] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[1524] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[1525] 6. Communication with Anonymous Users

[1526] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[1527] The terminal displays a chat room, allowing users to exchange messages with other users.

[1528] 7. Paid options available

[1529] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific character to the account.

[1530] The terminal notifies the user that a paid option is available and displays a conversation with a specific character.

[1531] Specific examples

[1532] 1. User registration and character creation

[1533] Elderly users install the application on their devices and create an account by entering information such as their name, age, and gender.

[1534] The user's name is "Tanaka Ichiro," his age is "70," and his gender is "male." Based on this information, please generate a character that this user would like.

[1535] The server receives this information and creates a user profile.

[1536] The device displays a screen that allows the user to customize the character's appearance and name.

[1537] 2. Emotion Analysis and Character Response

[1538] The user speaks to the character, saying, "It's a nice day today."

[1539] The user says, "It's a beautiful day today." Create a response that emphasizes positive sentiment.

[1540] The device records the user's voice and sends it to the server.

[1541] The server converts the speech into text using speech recognition and natural language processing technology, and sends the analyzed emotional data to the emotion engine.

[1542] The emotion engine analyzes the user's real-time emotional state and generates an appropriate response.

[1543] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[1544] The device displays or reads this response to the user.

[1545] 3. Points awarded and character growth

[1546] The user continues to make positive comments, saying, "I've recently started jogging."

[1547] The device records this audio and sends it to the server.

[1548] The server analyzes positive comments and awards points to the user.

[1549] Based on the accumulated points, the server adds new skills to the character and sends that information to the terminal.

[1550] The terminal notifies the user of the character's growth.

[1551] The emotion engine can track the user's long-term emotional changes and respond appropriately.

[1552] 4. Communication with Anonymous Users

[1553] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[1554] The server generates the chat room and the terminal displays it to the user.

[1555] Users anonymously exchange jogging information with other users.

[1556] 5. Use of paid options

[1557] The user selects a special communication function with the character and completes the purchase procedure.

[1558] The server will verify the purchase information and add the communication function for the specific character to the user's account.

[1559] The terminal notifies the user that a paid option has become available and displays a conversation with a specific character.

[1560] This system allows users to receive emotional support and foster positive emotions. It is also expected to reduce feelings of loneliness and improve mental health. By combining real-time emotion analysis by the emotion engine and its feedback function, it is possible to maintain a better mental state in the long term.

[1561] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1562] Step 1: Registering a user

[1563] The user downloads and installs the application on the device.

[1564] The user enters the required information such as name, age, and gender to create an account.

[1565] Input: User information (name, age, gender)

[1566] Data processing: Format verification and hashing of input information

[1567] Output: Account data including user information

[1568] The terminal transmits the input information to the server.

[1569] Input: User information

[1570] Data processing: Encryption

[1571] Output: Data packet with encrypted user information

[1572] The server generates a user profile based on the received information and stores it in a database.

[1573] Input: Encrypted user information

[1574] Data processing: Decryption, storing in database

[1575] Output: User profile added to database entry

[1576] Step 2: Character Generation and Customization

[1577] The server uses a generative AI to generate virtual characters that can be customized by users.

[1578] Input: User Profile

[1579] Data processing: Character generation using generative AI models (e.g., GPT-4)

[1580] Output: Virtual character data

[1581] The device displays a character creation screen where the user can select and customize the character's appearance and name.

[1582] Input: Virtual character data

[1583] Data processing: Drawing character customization UI

[1584] Output: User-selected customization data

[1585] The terminal transmits the customization information to the server.

[1586] Input: Customization data

[1587] Data processing: Encryption

[1588] Output: Encrypted customization data

[1589] The server stores the character information.

[1590] Input: Encrypted customization data

[1591] Data processing: Decryption, storing in database

[1592] Output: Updated character information in the database entry

[1593] Step 3: Collecting audio data and analyzing emotions

[1594] The user speaks to the character and begins a conversation.

[1595] Input: Speech

[1596] Data processing: generating audio streams

[1597] Output: Audio stream data

[1598] The terminal records the user's voice and transmits this voice data to the server.

[1599] Input: Audio stream data

[1600] Data processing: compression and encryption of audio data

[1601] Output: Encrypted audio data file

[1602] The server uses voice recognition technology to convert the voice data into text.

[1603] Input: Encrypted audio data file

[1604] Data processing: decoding, speech recognition (e.g., Google Cloud Speech-to-Text API)

[1605] Output: Text data

[1606] The server analyzes the text data using natural language processing means and extracts the emotional state.

[1607] Input: Text data

[1608] Data processing: Sentiment analysis using natural language processing (NLP library)

[1609] Output: Emotional state data

[1610] The emotion engine analyzes the user's emotions in real time based on the emotion data.

[1611] Input: Emotional state data

[1612] Data processing: Real-time emotion analysis using an emotion engine

[1613] Output: User's emotional state

[1614] Step 4: Generate character responses

[1615] An emotion engine dynamically generates character responses based on the analyzed emotion data.

[1616] Input: User's emotional state

[1617] Data processing: prompt generation, querying generative AI models (e.g., GPT-4)

[1618] Output: Response text

[1619] The server sends the generated response to the terminal.

[1620] Input: Response text

[1621] Data processing: Encryption

[1622] Output: Encrypted response data packet

[1623] The device displays or reads the character's response to the user.

[1624] Input: Encrypted response data packet

[1625] Data processing: decoding, text display or speech synthesis

[1626] Output: Feedback to the user

[1627] Step 5: Cultivating positive emotions

[1628] The device records the user's positive comments and sends them to the server.

[1629] Input: Positive speech

[1630] Data processing: compression and encryption of audio data

[1631] Output: Encrypted audio data file

[1632] The server analyzes positive comments and awards points to the user.

[1633] Input: Encrypted audio data file

[1634] Data processing: decoding, speech recognition, sentiment analysis, point calculation

[1635] Output: Update user point data

[1636] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[1637] Input: User point data

[1638] Data processing: Character growth calculation

[1639] Output: Updated character information

[1640] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[1641] Input: User's emotional history data

[1642] Data processing: Long-term sentiment analysis, character adjustment

[1643] Output: Updated character parameters

[1644] Step 6: Communicating with Anonymous Users

[1645] The user selects the anonymous communication feature.

[1646] Input: User's choice

[1647] Data processing: Request generation

[1648] Output: Anonymous chat request

[1649] The server matches users with the same emotional state or circumstances and creates anonymous chat rooms.

[1650] Input: Anonymous chat request, user profile data

[1651] Data processing: Anonymous user matching, chat room generation

[1652] Output: Chat room ID

[1653] The terminal displays a chat room, allowing users to exchange messages with other users.

[1654] Input: Chat room ID

[1655] Data processing: Drawing chat UI

[1656] Output: Message exchange between users

[1657] Step 7: Offering paid options

[1658] The user selects a paid option and completes the purchase procedure.

[1659] Input: User selection, payment information

[1660] Data processing: encryption, sending to payment gateway

[1661] Output: Payment completion notification

[1662] The server receives the purchase information and adds the ability to communicate with specific characters to the account.

[1663] Input: Payment completion notification

[1664] Data processing: User account updates

[1665] Output: Updated account information

[1666] The device will notify you that a paid option is available and display a conversation with a specific character.

[1667] Input: Updated account information

[1668] Data processing: Notification generation

[1669] Output: User notification and conversation UI with specific characters

[1670] (Application example 2)

[1671] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1672] In recent years, technologies that reduce loneliness and improve mental health by allowing users to receive real-time emotional support using smart devices have become increasingly important. However, existing technologies have not been sufficient in analyzing emotions and providing personalized services to individual customers in physical stores. Furthermore, it has been difficult to instantly provide relevant information when a user shows interest in a product, which has led to a poor user experience. Furthermore, it has been difficult to respond appropriately to store staff's emotions when communicating with them. As a result, user satisfaction often declines, which can negatively impact store sales.

[1673] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a character that the user can customize based on the generated account; speech recognition means and natural language processing means for enabling voice communication between the user and the character; emotion analysis means for generating responses for the character; means for awarding points based on the user's positive emotions; means for improving the character according to the accumulated points; matching means and chat means for anonymously communicating with other users in the same situation or emotional state; purchase means for communicating with a specified avatar as a paid option; gaze tracking means and display means for analyzing the user's dynamic gaze and speech content in real time in a physical store and displaying related information; and generative AI model and prompt sentence utilization means for analyzing the emotions of staff and generating appropriate responses. This allows users to receive personalized emotional support in real time even in a physical store, not only stimulating their interest in products but also enabling them to enjoy high-quality support in communication with staff.

[1674] The "account generation means" is a function that generates a unique user account based on information provided by the user.

[1675] "Customizable character generation means" is a function that allows the user to change the appearance and attributes of the character according to their own preferences.

[1676] The "voice recognition means" is a function that analyzes the voice spoken by the user and converts it into text data.

[1677] "Natural language processing means" is a technology for analyzing text data acquired by speech recognition means and understanding its meaning and context.

[1678] The "emotion analysis means" is a function that analyzes the user's emotional state from their speech and text, and generates an appropriate response based on the results.

[1679] The "point giving means" is a function for assigning points based on the user's actions and comments.

[1680] "Character growth means" is a function that changes a character's skills and appearance according to accumulated points.

[1681] "Matching methods" are functions that anonymously connect users with other users who have the same emotional state or circumstances.

[1682] "Chat means" is a function that allows matched users to exchange text messages.

[1683] The "paid option purchasing means" is a function that allows the user to purchase additional paid services.

[1684] "Eye tracking means" is a technology that detects the user's gaze in real time and identifies the object that the gaze is directed at.

[1685] The "display means" is a function that visually displays appropriate information based on eye tracking and speech content.

[1686] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on the user's emotions and situation.

[1687] "Prompt sentence utilization means" is a function that generates a response using a prompt sentence that gives appropriate instructions to the generative AI model.

[1688] MODE FOR CARRYING OUT THE INVENTION

[1689] The present invention provides a system for providing emotional support to users in real time and improving their mental health. This system aims to improve the customer experience, particularly in physical stores. The specific configuration and operation of the system are described in detail below.

[1690] System configuration

[1691] 1. Device:

[1692] Smart glasses worn by the user that include eye tracking and display capabilities.

[1693] It has a built-in voice recording device that records what the user says.

[1694] 2. Server:

[1695] It recognizes and analyzes voice data and generates responses using generative AI.

[1696] Emotion analysis technology is used to analyze the emotions of users and store staff and provide appropriate services.

[1697] 3. Application:

[1698] It features voice recognition, natural language processing, emotion analysis, point awarding, character growth, matching and chat functions, and the ability to purchase paid options.

[1699] How it works

[1700] User registration and character creation

[1701] 1. The user puts on the smart glasses and launches the application.

[1702] 2. The user creates an account by entering personal information such as name and age.

[1703] 3. The server generates a user profile based on this information using an account generation means.

[1704] 4. The server uses the generative AI model to generate a customizable character that can then be customized by the user.

[1705] Emotion analysis and character responses

[1706] 1. When a user speaks to a character, the device records the voice using a recording device and sends it to the server.

[1707] 2. The server converts the voice data into text using speech recognition and natural language processing means, and performs sentiment analysis.

[1708] 3. Based on the emotional data analyzed by the emotion engine, the generative AI model generates an appropriate response.

[1709] 4. The device displays the response on the smart glasses display or reads it out loud.

[1710] Points awarded and character growth

[1711] 1. When a user makes a positive comment, the server analyzes it and awards points.

[1712] 2. The server will develop the character and add new skills according to the accumulated points.

[1713] Improving user experience in physical stores

[1714] 1. When a user enters a physical store, the device uses eye tracking to identify products that interest the user.

[1715] 2. The device analyzes the speech in real time and displays relevant information on the screen.

[1716] Dialogue with staff

[1717] 1. The device records the conversation with the staff and sends it to the server.

[1718] 2. The server analyzes the staff member's emotions, generates an appropriate response, sends it to the terminal, and displays or reads it aloud.

[1719] Specific examples

[1720] 1. Imagine a scenario where a user enters a brick-and-mortar store and puts on a pair of smart glasses.

[1721] The staff greets you with "Welcome!"

[1722] The device records this audio and sends it to the server.

[1723] The server performs speech recognition and emotion analysis and generates a response such as, "That's very kind of you. I see you like our products."

[1724] The terminal displays this response on its display and the user confirms it.

[1725] Example prompt sentence:

[1726] Scenario: Inside the store, a staff member is heard saying, "Welcome!"

[1727] The user's sentiment is encouraging.

[1728] Generate an appropriate response.

[1729] Response: That's very kind of you, I see you like our products.

[1730] This allows users to receive personalized emotional support in real time even in physical stores, not only stimulating their interest in products but also enabling them to enjoy high-quality support when communicating with staff.

[1731] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1732] Specific explanations divided into processing steps

[1733] Step 1:

[1734] A user puts on the smart glasses and launches the application. The user creates an account by entering personal information such as name and age. The device sends this information to the server. Based on the entered information, the server generates a user profile and creates a unique account using the account generation means. The output is the user profile and account information.

[1735] Step 2:

[1736] The server uses the generative AI model to generate a character that can be customized by the user. The user customizes the character's appearance and attributes through the smart glasses display and sends that information to the server. The output is the customized character information.

[1737] Step 3:

[1738] When a user speaks to a character, the device records the voice and transmits the voice data to the server in real time. The server converts the voice data into text data using a voice recognition means, and obtains the voice data as input and the text data as output.

[1739] Step 4:

[1740] The server uses natural language processing means to analyze the converted text data, obtaining the text data as input. This analysis allows the server to understand the meaning and context of the user's speech, and then uses emotion analysis means to obtain analyzed emotion data as output.

[1741] Step 5:

[1742] The emotion engine uses a generative AI model to generate an appropriate response based on the analyzed emotion data. The prompt text is used to instruct the generative AI model to generate an appropriate response. The generated response text is obtained as output.

[1743] Step 6:

[1744] The device displays the generated response on the smart glasses display or reads it out loud, allowing the user to receive a response from the character in real time. The displayed or spoken response is obtained as output.

[1745] Step 7:

[1746] When a user makes a positive comment, the device records the voice and sends it to the server. The server analyzes the comment and awards points to the user. The input is the voice data of the positive comment, and the output is point information.

[1747] Step 8:

[1748] As points accumulate, the server uses character development tools to develop the character, including acquiring new skills and changing appearance. The output is information about the developed character.

[1749] Step 9:

[1750] When a user enters a physical store, the smart glasses' eye tracking function catches the user's gaze. The device uses the eye tracking means to transmit product information about the user's gaze to the server in real time. Eye tracking data is obtained as input, and product information is obtained as output.

[1751] Step 10:

[1752] The server uses a generative AI model based on the gaze data and the user's speech to generate appropriate information and send it to the device. The device then provides this information to the user via a display, providing product-related information as output.

[1753] Step 11:

[1754] When a user interacts with a store staff member, the device records the voice and sends it to the server. The server then analyzes the staff member's speech and generates an appropriate response using emotion analysis. The staff member's speech is obtained as input, and the response text is obtained as output.

[1755] Step 12:

[1756] The terminal generates a response based on the interaction with the staff and displays it on the user's smart glasses or reads it out loud, allowing the user to smoothly interact with the staff. The displayed or spoken response is obtained as an output.

[1757] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1758] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1759] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1760] [Third embodiment]

[1761] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1762] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1763] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1764] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1765] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1766] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1767] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1768] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1769] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1770] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1771] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1772] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1773] MODE FOR CARRYING OUT THE INVENTION

[1774] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[1775] System configuration

[1776] The system mainly consists of the following components:

[1777] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[1778] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[1779] 3. Application: Software installed on the device that has functions such as communication between users and characters, emotion analysis, point awarding, and matching.

[1780] System Operation

[1781] 1. User Registration

[1782] The user downloads and installs the application onto the terminal.

[1783] Users enter required information such as name, age, and gender to create an account.

[1784] The terminal transmits the entered information to a server, which generates a user profile.

[1785] 2. Character Creation and Customization

[1786] The server uses a generative AI to generate virtual characters that can be customized by users.

[1787] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[1788] Once customization is complete, the device sends the information to the server, which then stores the character information.

[1789] 3. Audio data collection and emotion analysis

[1790] The user speaks to the character and begins a conversation.

[1791] The terminal records the user's voice and transmits this voice data to the server.

[1792] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[1793] 4. Character response generation

[1794] The server generates an appropriate character response based on the analyzed emotional data.

[1795] The terminal displays or reads the generated response to the user.

[1796] 5. Cultivating positive emotions

[1797] The device records the user's positive comments and sends them to the server.

[1798] The server analyzes positive comments and awards points to the user.

[1799] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[1800] 6. Communication with Anonymous Users

[1801] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[1802] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[1803] 7. Paid options available

[1804] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[1805] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[1806] Specific examples

[1807] 1. User registration and character creation

[1808] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[1809] The server receives this information and creates a user profile.

[1810] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[1811] 2. Emotion Analysis and Character Response

[1812] The user speaks to the character, saying, "It's a nice day today."

[1813] The device records the user's voice and sends it to the server.

[1814] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[1815] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[1816] The device displays or reads this response to the user.

[1817] 3. Points awarded and character growth

[1818] The user continues to make positive comments, saying, "I've recently started jogging."

[1819] The device records this audio and sends it to the server.

[1820] The server analyzes positive comments and awards points to the user.

[1821] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[1822] The terminal notifies the user of the character's growth.

[1823] 4. Communication with Anonymous Users

[1824] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[1825] The server generates the chat room and the device displays it to the user.

[1826] Users anonymously exchange jogging information with other users.

[1827] 5. Use of paid options

[1828] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[1829] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[1830] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[1831] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health.

[1832] The processing flow will be explained below.

[1833] Program processing steps

[1834] User Registration and Initial Setup

[1835] Step 1:

[1836] The user downloads and installs the application on the device.

[1837] Step 2:

[1838] Users launch the application and create an account by entering required information such as name, age, and gender.

[1839] Step 3:

[1840] The terminal transmits the input information to the server.

[1841] Step 4:

[1842] The server generates and stores a user profile based on the received information.

[1843] Step 5:

[1844] The server sends an initialization success message to the terminal.

[1845] Step 6:

[1846] The device will notify the user that the initial setup is complete and open the character creation screen.

[1847] Step 7:

[1848] Users customize their character's appearance and name.

[1849] Step 8:

[1850] The terminal transmits the customized character information to the server.

[1851] Step 9:

[1852] The server stores the character information in a user profile.

[1853] Emotion analysis and character generation

[1854] Step 10:

[1855] The user speaks to the character and begins a conversation.

[1856] Step 11:

[1857] The terminal records the user's voice and transmits the voice data to the server.

[1858] Step 12:

[1859] The server uses voice recognition technology to convert the voice data into text.

[1860] Step 13:

[1861] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[1862] Step 14:

[1863] The server generates a response message for the character based on the result of the emotion analysis.

[1864] Step 15:

[1865] The server sends the response of the generated character to the terminal.

[1866] Step 16:

[1867] The terminal displays or reads out the character's response to the user.

[1868] Cultivating positive emotions and awarding points

[1869] Step 17:

[1870] The user makes positive comments during a conversation with the character.

[1871] Step 18:

[1872] The device records positive comments and sends the audio data to a server.

[1873] Step 19:

[1874] The server analyzes the positive comments.

[1875] Step 20:

[1876] The server awards points to the user based on the analysis results.

[1877] Step 21:

[1878] The server determines character growth (gaining new skills or changing appearance) based on the accumulated points.

[1879] Step 22:

[1880] The server sends character growth information to the terminal.

[1881] Step 23:

[1882] The terminal notifies the user of the character's growth and point allocation.

[1883] Communicating with Anonymous Users

[1884] Step 24:

[1885] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[1886] Step 25:

[1887] The terminal transmits the user's selection to the server.

[1888] Step 26:

[1889] The server searches for other users with the same emotional state or circumstances and performs matching.

[1890] Step 27:

[1891] The server creates anonymous chat rooms for matched users.

[1892] Step 28:

[1893] The server transmits chat room information to the terminal.

[1894] Step 29:

[1895] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[1896] Step 30:

[1897] Users communicate with other users anonymously.

[1898] Paid options available

[1899] Step 31:

[1900] The user selects the communication function with the celebrity avatar from a menu within the application.

[1901] Step 32:

[1902] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[1903] Step 33:

[1904] The user enters the necessary information and presses the "Purchase" button.

[1905] Step 34:

[1906] The terminal transmits the purchase information to the server.

[1907] Step 35:

[1908] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[1909] Step 36:

[1910] The server sends a notification of purchase completion to the terminal.

[1911] Step 37:

[1912] The terminal notifies the user that the paid option is now available.

[1913] Step 38:

[1914] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[1915] Example 1

[1916] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1917] In modern society, many people face the problem of feeling lonely. The lack of opportunities to receive emotional support in daily life is a particular challenge for the elderly and those who tend to be isolated. Under these circumstances, maintaining mental health becomes difficult, increasing the risk of serious mental and physical problems. Furthermore, existing emotional support systems have difficulty responding flexibly to the emotional state of individual users. There is a need for a system that can solve these issues and provide users with continuous, personalized emotional support.

[1918] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1919] In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a virtual character that the user can customize based on the generated account; speech recognition and natural language processing means for enabling voice communication between the user and the virtual character; emotion analysis means for generating responses from the virtual character; means for awarding points based on the user's positive emotions; means for upgrading the virtual character according to the accumulated points; matching and messaging means for anonymously communicating with other users in the same circumstances or emotional state; purchase means for realizing communication with a specific avatar as a paid option; display or audio output means on the terminal for notifying the user of the generated responses; means for generating responses from the virtual character using a generative AI model; means for generating prompt sentences to be input to the generative AI model based on the emotion analysis results; and means for generating responses from the virtual character using the generated prompt sentences. This makes it possible to provide continuous and personalized emotional support to users who feel lonely and improve their mental health.

[1920] A "terminal" is an electronic device used by a user and on which software is installed.

[1921] "Software" refers to a program that is installed and executed on a terminal, and provides various functions through interaction with the user.

[1922] A "user" is a person who uses the system and inputs information and communicates via the application.

[1923] "Account" means a digital management unit that contains user-specific information and is required to use the Software.

[1924] A "virtual character" is a user-customizable digital agent that interacts with the user to provide emotional support.

[1925] "Speech recognition means" is a technology that converts voice data into text data.

[1926] "Natural language processing means" is a technology that analyzes text data and understands meaning and emotions.

[1927] "Emotion analysis means" is a technology that estimates a user's emotional state from their statements and text.

[1928] "Points" are digital evaluation units awarded based on users' actions and comments.

[1929] "Growth methods" are techniques that change the skills and appearance of a virtual character according to the accumulation of points.

[1930] "Matching methods" are technologies that connect users with similar circumstances or emotional states.

[1931] "Messaging means" refers to technology that allows anonymous messaging.

[1932] "Paid Options" are additional features or services that are available for an additional fee.

[1933] "Purchase Instrument" means a payment technique for trading paid options.

[1934] "Display means" refers to a technique for displaying the generated response on the screen of the terminal.

[1935] The "audio output means" is a technique for outputting the generated response as audio.

[1936] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and responses.

[1937] A "prompt" is text data that can be input into a generative AI model to elicit a specific response.

[1938] MODE FOR CARRYING OUT THE INVENTION

[1939] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[1940] System configuration

[1941] The system consists of the following elements:

[1942] 1. Device: An electronic device used by a user, such as a smartphone, tablet, or PC. Dedicated software is installed on the device.

[1943] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[1944] 3. Software: A program installed on the device that has functions such as communication between the user and virtual characters, emotion analysis, point awarding, and matching.

[1945] System Operation

[1946] 1. User Registration

[1947] The user downloads and installs the application onto the terminal.

[1948] Users enter required information such as name, age, and gender to create an account.

[1949] The terminal transmits the entered information to a server, which generates a user profile.

[1950] 2. Character Creation and Customization

[1951] The server uses a generative AI model to generate virtual characters that can be customized by users.

[1952] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[1953] Once customization is complete, the device sends the information to the server, which then stores the character information.

[1954] 3. Audio data collection and emotion analysis

[1955] The user speaks to the character and begins a conversation.

[1956] The terminal records the user's voice and transmits this voice data to the server.

[1957] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[1958] 4. Character response generation

[1959] The server generates an appropriate character response based on the analyzed emotion data. A generative AI model (e.g., ChatGPT) is used to generate a prompt. An example of a prompt is, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[1960] The terminal displays or reads the generated response to the user.

[1961] 5. Cultivating positive emotions

[1962] The device records the user's positive comments and sends them to the server.

[1963] The server analyzes positive comments and awards points to the user.

[1964] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[1965] 6. Communication with Anonymous Users

[1966] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[1967] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[1968] 7. Paid options available

[1969] When a user selects a paid option and completes the purchase procedure, the server receives the purchase information and adds a communication function with a specific avatar to the account.

[1970] The device notifies the user that paid options are available and displays conversations with specific avatars.

[1971] Specific examples

[1972] For example, if a user says to a character, "It's a nice day today," the following happens:

[1973] The user speaks to the character, saying, "It's a nice day today."

[1974] The device uses a built-in microphone to record the user's voice and transmits this voice data to the server.

[1975] The server converts the speech into text using the Google Cloud Speech-to-Text API, and then analyzes the emotional state of the text using natural language processing technology (e.g., IBM Watson NLU).

[1976] The server sends a prompt to a generative AI model (e.g., ChatGPT) to generate a response text. Example prompt: "The user said, 'It's a nice day today.' Please generate a positive, constructive response."

[1977] The server generates a response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[1978] The device will either display the text or convert it into speech using a speech synthesis API and respond to the user audibly.

[1979] This allows users to receive continuous and personalized emotional support, which can help reduce feelings of loneliness and improve mental health.

[1980] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1981] Program processing flow

[1982] Registering Users

[1983] Step 1:

[1984] Input: The user downloads and installs the application on their device.

[1985] Output: The installed application starts running on the device.

[1986] What happens: A user downloads and installs an app from the app store.

[1987] Step 2:

[1988] Input: The user enters the required information (name, age, gender, etc.) on the account registration screen.

[1989] Output: User input information is temporarily stored on the device and sent to the server.

[1990] Specific behavior: The device displays an input form, and the user enters information using a keyboard or on-screen keyboard.

[1991] Step 3:

[1992] Input: User information sent from the device.

[1993] Output: The server generates a user profile and stores it in the database.

[1994] Specific operation: The device sends an HTTPS request and the server saves the user information in the database.

[1995] Character Generation and Customization

[1996] Step 4:

[1997] Input: The server generates basic information about the virtual character using a generative AI model.

[1998] Output: Basic setting data of the virtual character is generated.

[1999] Specific operation: The server sends prompts to the generative AI model (e.g., ChatGPT) to generate the initial settings for the virtual character.

[2000] Step 5:

[2001] Input: The user interacts with the character creation screen on their device.

[2002] Output: User-customized character information is saved on the device.

[2003] Specific behavior: The device displays a user interface (UI), and the user operates drop-down menus and sliders.

[2004] Step 6:

[2005] Input: The user completes the customization and the device sends the information to the server.

[2006] Output: The server saves the character information to the database.

[2007] Specific operation: The device sends JSON data containing customization information to the server, and the server stores it in a database.

[2008] Voice data collection and sentiment analysis

[2009] Step 7:

[2010] Input: The user speaks to the character.

[2011] Output: The user's voice data is recorded on the device.

[2012] Specific action: The user speaks into the device's microphone.

[2013] Step 8:

[2014] Input: Recorded audio data.

[2015] Output: The device sends the audio data to the server.

[2016] Specific operation: The device starts the voice recording function and sends the recorded data to the server.

[2017] Step 9:

[2018] Input: The audio data received by the server.

[2019] Output: The server parses the user's utterance as text data.

[2020] Specific operation: The server calls a speech recognition API (e.g., Google Cloud Speech-to-Text) and sends the resulting text data to a natural language processing API (e.g., IBM Watson NLU).

[2021] Character response generation

[2022] Step 10:

[2023] Input: Server parsed emotion data.

[2024] Output: The server generates an appropriate response text using the generative AI model.

[2025] Specific operation: The server sends a prompt to the generative AI model to generate a response text. For example, it sends the prompt sentence, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[2026] Step 11:

[2027] Input: The generated response text.

[2028] Output: The device displays or reads the text.

[2029] Specific behavior: The device displays the generated response text on the screen or converts it into speech using a speech synthesis API (e.g., Amazon Polly) and reads it to the user.

[2030] Cultivating positive emotions

[2031] Step 12:

[2032] Input: User makes a positive statement.

[2033] Output: The device records what you say and sends it to the server.

[2034] Specific operation: The device records what the user says and sends the recording data to the server.

[2035] Step 13:

[2036] Input: The server receives positive utterance data.

[2037] Output: Positive comments are analyzed and points are awarded to the user.

[2038] What it does: The server uses natural language processing to detect positive words and phrases and adds points to the user's profile.

[2039] Step 14:

[2040] Input: The points accumulated by the server.

[2041] Output: Reflects character growth (gaining new skills and changing appearance).

[2042] Specific operation: The server updates the character information based on the growth algorithm and reflects it on the device.

[2043] Communicating with Anonymous Users

[2044] Step 15:

[2045] Input: The user selects the anonymous communication feature.

[2046] Output: The server matches users with the same circumstances and emotional state.

[2047] Specific operation: The server compares user profiles and selects other users with high matching scores.

[2048] Step 16:

[2049] Input: Matched user information.

[2050] Output: The server creates an anonymous chat room and sends a link to the device.

[2051] Specific operation: The server generates a chat room URL and sends it to the device, which displays the link.

[2052] Paid options available

[2053] Step 17:

[2054] Input: The user selects a paid option and completes the purchase.

[2055] Output: The purchase is completed and the ability to communicate with the specified avatar is added to your account.

[2056] Specific operation: The device displays a list of paid options, and the user selects one. The purchase is completed using a payment API (e.g., Stripe or PayPal).

[2057] Step 18:

[2058] Input: The server verifies the purchase information.

[2059] Output: Paid options become available and are notified on the device.

[2060] What happens: The server confirms the purchase and adds the new feature to the user's profile. The device displays a pop-up notification informing the user of the paid option and showing the celebrity avatar's conversation screen.

[2061] (Application example 1)

[2062] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2063] In recent years, the number of individuals experiencing loneliness has increased, creating a need for emotional support. However, there is a lack of effective methods for improving emotional and shopping experiences in physical stores. Conventional systems struggle to provide an environment that alleviates users' feelings of loneliness and fosters positive emotions. Furthermore, they do not recommend services or products based on the user's real-time emotional state, making it impossible to provide an optimized experience for each individual user. Therefore, a new system is needed to alleviate loneliness, foster positive emotions, and provide a personalized experience in physical stores.

[2064] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2065] In this invention, the server includes means for receiving information from a user and generating an account, means for generating a character that the user can customize based on the generated account, speech recognition means and natural language processing means for enabling voice communication between the user and the character, emotion analysis means for generating responses from the character, means for awarding points based on the user's positive emotions, means for enhancing the character according to the accumulated points, matching means and chat means for anonymously communicating with other users in the same circumstances or emotional state, means for recommending products and services based on the user's emotional state in a physical store, and purchasing means for communicating with a specified avatar. This allows users to reduce feelings of loneliness, foster positive emotions, and enjoy a personalized experience in a physical store.

[2066] A "terminal" is an electronic device used by a user, including a smartphone, tablet, or PC.

[2067] An "application" is a software program that is installed on a terminal and has functions such as communication between the user and a virtual character and emotion analysis.

[2068] "User" refers to a person who uses the system and is an individual who wishes to receive emotional support.

[2069] An "account" is a record containing a user's identifying information, and is used to identify an individual user within the system.

[2070] A "character" is a user-customizable virtual entity that provides emotional support through communication with the user.

[2071] "Speech recognition means" refers to a technology that converts a user's voice into text data, and is used to analyze the content of what the user says.

[2072] "Natural language processing means" is a technology for understanding text data obtained by speech recognition means and generating an appropriate response.

[2073] "Emotion analysis means" is a technology that analyzes the emotional state of a user from their statements and actions.

[2074] "Points" are numerical values ​​that the system awards to users for their positive comments and actions, and are used to develop their characters and receive special benefits.

[2075] "Growth" means that the character's skills and appearance improve or change depending on the user's accumulated points.

[2076] "Matching means" is a technology that automatically finds other users who are in the same circumstances or emotional state, and enables them to communicate with each other.

[2077] "Chat means" is a function that allows users to send and receive text messages.

[2078] "Physical store" refers to a physical commercial facility or service location.

[2079] "Means for recommending products and services" refers to technology for presenting products and services that are considered optimal based on the user's emotional state.

[2080] "Purchase means" is a function that allows a user to select and purchase a paid option.

[2081] "Communicating anonymously" means exchanging messages with other users without using their real names.

[2082] The present invention relates to a system for providing emotional support to users and improving their mental health, which is realized through an application installed on a terminal, a central server, and a network connecting them.

[2083] System configuration

[2084] 1. Terminal

[2085] A terminal is an electronic device used by a user, and includes a smartphone, tablet, PC, etc. An application that enables communication between the user and a virtual character is installed on this terminal.

[2086] 2. Server

[2087] The server acts as a central server, processing, storing, and analyzing data sent by users. The server uses emotion analysis tools and generative AI models to analyze the user's emotional state and generate appropriate responses.

[2088] 3. Application

[2089] An application is a software program that is installed on a terminal and has functions such as communication between the user and a virtual character and emotion analysis.

[2090] System Operation

[2091] 1. User Registration

[2092] Users download and install the application on their device, create an account by entering information such as their name, age, and gender, and the device then sends this information to a server to generate a user profile.

[2093] 2. Character Creation and Customization

[2094] The server uses a generative AI model to generate virtual characters that users can customize. Users select and customize the character's appearance and name, and the information is stored on the server.

[2095] 3. Speech Communication and Emotion Analysis

[2096] When a user speaks to a character, the device records the user's voice and sends it to the server, which converts the voice data into text using speech recognition and natural language processing, and analyzes it using emotion analysis.

[2097] 4. Response Generation

[2098] The server generates an appropriate character response based on the analysis results and sends it to the device, which then displays or reads the generated response to the user.

[2099] Example

[2100] 1. Examples of applications in physical stores

[2101] In a physical store, users communicate with a virtual character via their smartphone or smart glasses. The character recommends relaxation areas and products based on the user's emotional state. If the character asks, "How are you feeling today?" and the user replies, "I'm a little tired," the character will recommend, "There's a relaxation area in this area."

[2102] 2. Points awarded and character growth

[2103] When a user engages in positive conversation, points are awarded and the character grows. For example, if a user says, "I went to a cafe with my friends yesterday and had fun," points are awarded and the character acquires a new skill.

[2104] Program processing overview

[2105] Hardware and software used:

[2106] Hardware: Smartphones, smart glasses

[2107] Software: Flask (web application framework), SpeechRecognition (voice recognition library), openai (generative AI model)

[2108] Data processing and calculation:

[2109] The server manages user profiles, analyzes emotions, and generates responses using generative AI. For example, the following prompts are generated:

[2110] The user said 'I'm a bit tired'. Their emotion is 'tired'. Give an appropriate response.

[2111] The system allows users to reduce feelings of loneliness, foster positive emotions, and enjoy personalized experiences in physical stores.

[2112] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2113] Step 1:

[2114] The user downloads the application and installs it on their device.

[2115] Input: User's internet-connected device

[2116] Output: Installed applications

[2117] Specific operation: The user downloads the application from the official store and installs it on their device.

[2118] Step 2:

[2119] To create an account, a user enters information such as name, age, and gender.

[2120] Input: User personal information

[2121] Output: The information entered by the user is sent to the server and a user profile is generated.

[2122] Specific operation: The user enters the required information into a form within the application and presses the "Submit" button.

[2123] Step 3:

[2124] The server uses the generative AI model to generate a virtual character that can be customized by the user.

[2125] Input: User profile information

[2126] Output: Generated virtual character

[2127] Specific operation: The server generates a virtual character using a generative AI model based on the user's profile information.

[2128] Step 4:

[2129] The device displays a character creation screen where the user can customize the character's appearance and name.

[2130] Input: Generated character and user customization settings

[2131] Output:Customized Character

[2132] Specific operation: A character builder will appear on the device screen, and the user can adjust the appearance and name and save it.

[2133] Step 5:

[2134] The user speaks to the character and begins a conversation.

[2135] Input: User's voice

[2136] Output: Audio data

[2137] Specific actions: The user speaks to the character through the microphone.

[2138] Step 6:

[2139] The device records the user's voice and sends it to the server.

[2140] Input: User's voice data

[2141] Output: Audio data sent to the server

[2142] Specific operation: The device sends the audio recorded by the microphone to the server as digital data.

[2143] Step 7:

[2144] The server converts the voice data into text using a voice recognition means, and analyzes the text data using a natural language processing means.

[2145] Input: User's voice data

[2146] Output: Text data and analysis results

[2147] Specific operation: The speech recognition library converts the speech data into a string of characters, and the natural language processing means analyzes the string of characters.

[2148] Step 8:

[2149] An emotion analysis means analyzes the user's emotional state.

[2150] Input: Parsed text data

[2151] Output: User's emotional state

[2152] What it does: Sentiment analysis algorithms extract emotional states from text data.

[2153] Step 9:

[2154] The server generates a response for the virtual character based on the emotion analysis results.

[2155] Input: User emotional state and parsed text data

[2156] Output: The generated response

[2157] Specific behavior: The generative AI model generates an appropriate response based on emotional state and text data.

[2158] Step 10:

[2159] The terminal displays or reads the generated response to the user.

[2160] Input: The generated response from the server

[2161] Output: The response that is displayed or read to the user

[2162] Specific behavior: Display the response text on the device screen or read it aloud through the speaker.

[2163] Step 11:

[2164] Points are awarded for positive comments made by users.

[2165] Input: Analysis results of positive comments

[2166] Output: Updated points

[2167] What it does: The server recognizes positive emotions and adds them to a points system.

[2168] Step 12:

[2169] Characters grow according to the accumulated points.

[2170] Input: Accumulated points

[2171] Output: Grown-up character

[2172] What it does: The server checks the accumulated points and adds new skills and cosmetic changes to the character.

[2173] Step 13:

[2174] It provides a chat function and matches users with similar circumstances and emotional states to communicate anonymously.

[2175] Input: User's emotional state and situation information

[2176] Output: Matching results and chat rooms

[2177] What it does: The server finds other suitable users and creates an anonymous chat room.

[2178] Step 14:

[2179] Recommend products and services based on the user's emotional state in a physical store.

[2180] Input: User's emotional state

[2181] Output: Recommended products and services

[2182] Specific operation: The server generates prompt sentences that suggest appropriate products and services based on the emotional state.

[2183] The user said 'I'm a bit tired'. Their emotion is 'tired'. Give an appropriate response.

[2184] Step 15:

[2185] The user performs a purchase procedure to realize communication with a designated avatar as a paid option.

[2186] Input: User's purchase intent

[2187] Output: Purchase completed and function added

[2188] Specific operation: The server completes the purchase process and adds the ability to communicate with the specified avatar.

[2189] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2190] MODE FOR CARRYING OUT THE INVENTION

[2191] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The system aims to foster positive emotions through communication between users and virtual characters using emotion analysis technology, generative AI technology, and an emotion engine.

[2192] System configuration

[2193] The system mainly consists of the following components:

[2194] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[2195] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[2196] 3. Application: Software installed on the device that has various functions such as communication between the user and the character, emotion analysis, point awarding, matching, and an emotion engine.

[2197] 4. Emotion engine: An engine that analyzes the user's emotions in real time and dynamically adjusts the character's responses and actions based on that information.

[2198] System Operation

[2199] 1. User Registration

[2200] The user downloads and installs the application on the device.

[2201] Users enter required information such as name, age, and gender to create an account.

[2202] The terminal transmits the entered information to a server, which generates a user profile.

[2203] 2. Character Creation and Customization

[2204] The server uses a generative AI to generate virtual characters that can be customized by users.

[2205] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[2206] Once customization is complete, the device sends the information to the server, which then stores the character information.

[2207] 3. Audio data collection and emotion analysis

[2208] The user speaks to the character and begins a conversation.

[2209] The terminal records the user's voice and transmits this voice data to the server.

[2210] The server converts the voice data into text using speech recognition technology and analyzes the emotional state using natural language processing means.

[2211] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[2212] 4. Character response generation

[2213] The emotion engine dynamically generates character responses based on the analyzed emotion data.

[2214] The server sends the generated response to the terminal.

[2215] The terminal displays or reads out the character's response to the user.

[2216] 5. Cultivating positive emotions

[2217] The device records the user's positive comments and sends them to the server.

[2218] The server analyzes positive comments and awards points to the user.

[2219] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[2220] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[2221] 6. Communication with Anonymous Users

[2222] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[2223] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[2224] 7. Paid options available

[2225] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[2226] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[2227] Specific examples

[2228] 1. User registration and character creation

[2229] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[2230] The server receives this information and creates a user profile.

[2231] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[2232] 2. Emotion Analysis and Character Response

[2233] The user speaks to the character, saying, "It's a nice day today."

[2234] The device records the user's voice and sends it to the server.

[2235] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[2236] Using the recorded emotion data, the emotion engine analyzes the user's real-time emotional state.

[2237] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[2238] The device displays or reads this response to the user.

[2239] 3. Points awarded and character growth

[2240] The user continues to make positive comments, saying, "I've recently started jogging."

[2241] The device records this audio and sends it to the server.

[2242] The server analyzes positive comments and awards points to the user.

[2243] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[2244] The terminal notifies the user of the character's growth.

[2245] The emotion engine tracks the user's long-term emotional changes and responds appropriately.

[2246] 4. Communication with Anonymous Users

[2247] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[2248] The server generates the chat room and the device displays it to the user.

[2249] Users anonymously exchange jogging information with other users.

[2250] 5. Use of paid options

[2251] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[2252] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[2253] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[2254] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health. By combining real-time emotion analysis by the emotion engine and its feedback function, it is possible to maintain a better long-term mental state for users.

[2255] The processing flow will be explained below.

[2256] Program processing steps

[2257] User Registration and Initial Setup

[2258] Step 1:

[2259] The user downloads and installs the application on the device.

[2260] Step 2:

[2261] Users launch the application and create an account by entering required information such as name, age, and gender.

[2262] Step 3:

[2263] The terminal transmits the input information to the server.

[2264] Step 4:

[2265] The server generates and stores a user profile based on the received information.

[2266] Step 5:

[2267] The server sends an initialization success message to the terminal.

[2268] Step 6:

[2269] The device will notify the user that the initial setup is complete and open the character creation screen.

[2270] Step 7:

[2271] Users customize their character's appearance and name.

[2272] Step 8:

[2273] The terminal transmits the customized character information to the server.

[2274] Step 9:

[2275] The server stores the character information in a user profile.

[2276] Emotion analysis and character generation

[2277] Step 10:

[2278] The user speaks to the character and begins a conversation.

[2279] Step 11:

[2280] The terminal records the user's voice and transmits the voice data to the server.

[2281] Step 12:

[2282] The server uses voice recognition technology to convert the voice data into text.

[2283] Step 13:

[2284] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[2285] Step 14:

[2286] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[2287] Step 15:

[2288] The emotion engine dynamically generates character responses based on the analyzed data.

[2289] Step 16:

[2290] The server sends the response of the generated character to the terminal.

[2291] Step 17:

[2292] The terminal displays or reads out the character's response to the user.

[2293] Cultivating positive emotions and awarding points

[2294] Step 18:

[2295] The user makes positive comments during a conversation with the character.

[2296] Step 19:

[2297] The device records positive comments and sends the audio data to a server.

[2298] Step 20:

[2299] The server analyzes the positive comments.

[2300] Step 21:

[2301] The server awards points to the user based on the analysis results.

[2302] Step 22:

[2303] The server determines the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[2304] Step 23:

[2305] The server sends character growth information to the terminal.

[2306] Step 24:

[2307] The terminal notifies the user of the character's growth and point allocation.

[2308] Step 25:

[2309] The emotion engine tracks the user's long-term emotional changes and dynamically adjusts the character's response.

[2310] Communicating with Anonymous Users

[2311] Step 26:

[2312] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[2313] Step 27:

[2314] The terminal transmits the user's selection to the server.

[2315] Step 28:

[2316] The server searches for other users with the same emotional state or circumstances and performs matching.

[2317] Step 29:

[2318] The server creates anonymous chat rooms for matched users.

[2319] Step 30:

[2320] The server transmits chat room information to the terminal.

[2321] Step 31:

[2322] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[2323] Step 32:

[2324] Users communicate with other users anonymously.

[2325] Paid options available

[2326] Step 33:

[2327] The user selects the communication function with the celebrity avatar from a menu within the application.

[2328] Step 34:

[2329] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[2330] Step 35:

[2331] The user enters the necessary information and presses the "Purchase" button.

[2332] Step 36:

[2333] The terminal transmits the purchase information to the server.

[2334] Step 37:

[2335] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[2336] Step 38:

[2337] The server sends a notification of purchase completion to the terminal.

[2338] Step 39:

[2339] The terminal notifies the user that the paid option is now available.

[2340] Step 40:

[2341] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[2342] Example 2

[2343] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2344] Conventional emotional support systems have been unable to provide sufficient emotional support to users who feel lonely, and have been insufficient to improve the user's long-term mental health. Furthermore, communication with the user is one-way, making it difficult to provide immediate feedback based on the user's emotional state. As a result, users are less likely to be satisfied with the system, making it difficult to promote continued use.

[2345] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving information from a user and generating an account, a means for generating a virtual character that the user can customize based on the generated account, a voice recognition means and natural language processing means for enabling voice communication between the user and the character, a means for converting voice data into text and analyzing the emotional state, an emotion engine for generating the character's responses, a means for awarding points based on the user's positive emotions, a means for improving the character according to the accumulated points, a matching means and chat means for anonymously communicating with other users in the same situation or emotional state, and a purchasing means for realizing communication with a specified character as a paid option. This enables immediate feedback based on the user's emotional state and provides continuous emotional support. Furthermore, the user can experience a reduction in loneliness and an improvement in mental health.

[2346] A "terminal" is an electronic device used by a user, and includes a smartphone, tablet, PC, etc.

[2347] An "application" is a software program installed on a terminal that manages the interaction between the user and the character.

[2348] An "account" is a collection of data that includes a user's identification information and is used to manage an individual user's operations within the system.

[2349] A "virtual character" is a digital character that can be customized by the user using a generative AI model and that communicates with the user.

[2350] "Speech recognition means" refers to technology that records a user's voice and converts that voice into text data.

[2351] "Natural language processing means" refers to technology that analyzes text data and extracts linguistic meanings and emotional states.

[2352] "Emotional state" refers to the emotional state extracted from a user's speech or text, and includes, for example, joy, sadness, anger, etc.

[2353] The "emotion engine" is an engine that analyzes the user's emotional state and generates responses from the digital character based on the results.

[2354] The "means of awarding points" refers to a mechanism that adds points within the system based on users' positive actions and comments.

[2355] "Means for character development" refers to a system that allows you to evolve your virtual character's skills and appearance based on accumulated points.

[2356] "Matching means" refers to technology that identifies other users with the same emotional state or interests on the server side and connects them in anonymous chat rooms.

[2357] "Chat vehicle" refers to the interface and functionality for exchanging text messages between users.

[2358] "Purchase method" refers to the operation by which a user selects a paid option and makes additional features available through payment.

[2359] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The system aims to foster positive emotions through interactive communication between users and virtual characters using emotion analysis technology, generative AI technology, and an emotion engine.

[2360] System configuration

[2361] This system consists of the following elements:

[2362] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[2363] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[2364] 3. Application: Software installed on the device that has various functions such as communication between the user and the character, emotion analysis, point awarding, matching, and an emotion engine.

[2365] 4. Emotion engine: An engine that analyzes the user's emotions in real time and dynamically adjusts the character's responses and actions based on that information.

[2366] System Operation

[2367] 1. User Registration

[2368] The user downloads and installs the application on the device.

[2369] Users enter required information such as name, age, and gender to create an account.

[2370] The terminal transmits the entered information to a server, which generates a user profile.

[2371] 2. Character Creation and Customization

[2372] The server uses generative AI to generate virtual characters that users can customize. Specifically, it uses a generative AI model (e.g., GPT-4).

[2373] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[2374] Once customization is complete, the device sends the information to the server, which then stores the character information.

[2375] 3. Audio data collection and emotion analysis

[2376] The user speaks to the character and begins a conversation.

[2377] The terminal records the user's voice and transmits this voice data to the server.

[2378] The server converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[2379] The server analyzes the emotional state using natural language processing means (NLP library).

[2380] The emotion engine receives the analyzed emotional state and recognizes the user's emotions in real time.

[2381] 4. Character response generation

[2382] The emotion engine dynamically generates character responses based on the analyzed emotion data.

[2383] The server sends the generated response to the terminal.

[2384] The terminal displays or reads out the character's response to the user.

[2385] 5. Cultivating positive emotions

[2386] The device records the user's positive comments and sends them to the server.

[2387] The server analyzes positive comments and awards points to the user.

[2388] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[2389] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[2390] 6. Communication with Anonymous Users

[2391] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[2392] The terminal displays a chat room, allowing users to exchange messages with other users.

[2393] 7. Paid options available

[2394] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific character to the account.

[2395] The terminal notifies the user that a paid option is available and displays a conversation with a specific character.

[2396] Specific examples

[2397] 1. User registration and character creation

[2398] Elderly users install the application on their devices and create an account by entering information such as their name, age, and gender.

[2399] The user's name is "Tanaka Ichiro," his age is "70," and his gender is "male." Based on this information, please generate a character that this user would like.

[2400] The server receives this information and creates a user profile.

[2401] The device displays a screen that allows the user to customize the character's appearance and name.

[2402] 2. Emotion Analysis and Character Response

[2403] The user speaks to the character, saying, "It's a nice day today."

[2404] The user says, "It's a beautiful day today." Create a response that emphasizes positive sentiment.

[2405] The device records the user's voice and sends it to the server.

[2406] The server converts the speech into text using speech recognition and natural language processing technology, and sends the analyzed emotional data to the emotion engine.

[2407] The emotion engine analyzes the user's real-time emotional state and generates an appropriate response.

[2408] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[2409] The device displays or reads this response to the user.

[2410] 3. Points awarded and character growth

[2411] The user continues to make positive comments, saying, "I've recently started jogging."

[2412] The device records this audio and sends it to the server.

[2413] The server analyzes positive comments and awards points to the user.

[2414] Based on the accumulated points, the server adds new skills to the character and sends that information to the terminal.

[2415] The terminal notifies the user of the character's growth.

[2416] The emotion engine can track the user's long-term emotional changes and respond appropriately.

[2417] 4. Communication with Anonymous Users

[2418] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[2419] The server generates the chat room and the terminal displays it to the user.

[2420] Users anonymously exchange jogging information with other users.

[2421] 5. Use of paid options

[2422] The user selects a special communication function with the character and completes the purchase procedure.

[2423] The server will verify the purchase information and add the communication function for the specific character to the user's account.

[2424] The terminal notifies the user that a paid option has become available and displays a conversation with a specific character.

[2425] This system allows users to receive emotional support and foster positive emotions. It is also expected to reduce feelings of loneliness and improve mental health. By combining real-time emotion analysis by the emotion engine and its feedback function, it is possible to maintain a better mental state in the long term.

[2426] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2427] Step 1: Registering a user

[2428] The user downloads and installs the application on the device.

[2429] The user enters the required information such as name, age, and gender to create an account.

[2430] Input: User information (name, age, gender)

[2431] Data processing: Format verification and hashing of input information

[2432] Output: Account data including user information

[2433] The terminal transmits the input information to the server.

[2434] Input: User information

[2435] Data processing: Encryption

[2436] Output: Data packet with encrypted user information

[2437] The server generates a user profile based on the received information and stores it in a database.

[2438] Input: Encrypted user information

[2439] Data processing: Decryption, storing in database

[2440] Output: User profile added to database entry

[2441] Step 2: Character Generation and Customization

[2442] The server uses a generative AI to generate virtual characters that can be customized by users.

[2443] Input: User Profile

[2444] Data processing: Character generation using generative AI models (e.g., GPT-4)

[2445] Output: Virtual character data

[2446] The device displays a character creation screen where the user can select and customize the character's appearance and name.

[2447] Input: Virtual character data

[2448] Data processing: Drawing character customization UI

[2449] Output: User-selected customization data

[2450] The terminal transmits the customization information to the server.

[2451] Input: Customization data

[2452] Data processing: Encryption

[2453] Output: Encrypted customization data

[2454] The server stores the character information.

[2455] Input: Encrypted customization data

[2456] Data processing: Decryption, storing in database

[2457] Output: Updated character information in the database entry

[2458] Step 3: Collecting audio data and analyzing emotions

[2459] The user speaks to the character and begins a conversation.

[2460] Input: Speech

[2461] Data processing: generating audio streams

[2462] Output: Audio stream data

[2463] The terminal records the user's voice and transmits this voice data to the server.

[2464] Input: Audio stream data

[2465] Data processing: compression and encryption of audio data

[2466] Output: Encrypted audio data file

[2467] The server uses voice recognition technology to convert the voice data into text.

[2468] Input: Encrypted audio data file

[2469] Data processing: decoding, speech recognition (e.g., Google Cloud Speech-to-Text API)

[2470] Output: Text data

[2471] The server analyzes the text data using natural language processing means and extracts the emotional state.

[2472] Input: Text data

[2473] Data processing: Sentiment analysis using natural language processing (NLP library)

[2474] Output: Emotional state data

[2475] The emotion engine analyzes the user's emotions in real time based on the emotion data.

[2476] Input: Emotional state data

[2477] Data processing: Real-time emotion analysis using an emotion engine

[2478] Output: User's emotional state

[2479] Step 4: Generate character responses

[2480] An emotion engine dynamically generates character responses based on the analyzed emotion data.

[2481] Input: User's emotional state

[2482] Data processing: prompt generation, querying generative AI models (e.g., GPT-4)

[2483] Output: Response text

[2484] The server sends the generated response to the terminal.

[2485] Input: Response text

[2486] Data processing: Encryption

[2487] Output: Encrypted response data packet

[2488] The device displays or reads the character's response to the user.

[2489] Input: Encrypted response data packet

[2490] Data processing: decoding, text display or speech synthesis

[2491] Output: Feedback to the user

[2492] Step 5: Cultivating positive emotions

[2493] The device records the user's positive comments and sends them to the server.

[2494] Input: Positive speech

[2495] Data processing: compression and encryption of audio data

[2496] Output: Encrypted audio data file

[2497] The server analyzes positive comments and awards points to the user.

[2498] Input: Encrypted audio data file

[2499] Data processing: decoding, speech recognition, sentiment analysis, point calculation

[2500] Output: Update user point data

[2501] The server manages the character's growth (acquiring new skills and changing appearance) based on the accumulated points.

[2502] Input: User point data

[2503] Data processing: Character growth calculation

[2504] Output: Updated character information

[2505] The emotion engine tracks the user's long-term emotional changes and makes corresponding adjustments to the character.

[2506] Input: User's emotional history data

[2507] Data processing: Long-term sentiment analysis, character adjustment

[2508] Output: Updated character parameters

[2509] Step 6: Communicating with Anonymous Users

[2510] The user selects the anonymous communication feature.

[2511] Input: User's choice

[2512] Data processing: Request generation

[2513] Output: Anonymous chat request

[2514] The server matches users with the same emotional state or circumstances and creates anonymous chat rooms.

[2515] Input: Anonymous chat request, user profile data

[2516] Data processing: Anonymous user matching, chat room generation

[2517] Output: Chat room ID

[2518] The terminal displays a chat room, allowing users to exchange messages with other users.

[2519] Input: Chat room ID

[2520] Data processing: Drawing chat UI

[2521] Output: Message exchange between users

[2522] Step 7: Offering paid options

[2523] The user selects a paid option and completes the purchase procedure.

[2524] Input: User selection, payment information

[2525] Data processing: encryption, sending to payment gateway

[2526] Output: Payment completion notification

[2527] The server receives the purchase information and adds the ability to communicate with specific characters to the account.

[2528] Input: Payment completion notification

[2529] Data processing: User account updates

[2530] Output: Updated account information

[2531] The device will notify you that a paid option is available and display a conversation with a specific character.

[2532] Input: Updated account information

[2533] Data processing: Notification generation

[2534] Output: User notification and conversation UI with specific characters

[2535] (Application example 2)

[2536] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2537] In recent years, technologies that reduce loneliness and improve mental health by allowing users to receive real-time emotional support using smart devices have become increasingly important. However, existing technologies have not been sufficient in analyzing emotions and providing personalized services to individual customers in physical stores. Furthermore, it has been difficult to instantly provide relevant information when a user shows interest in a product, which has led to a poor user experience. Furthermore, it has been difficult to respond appropriately to store staff's emotions when communicating with them. As a result, user satisfaction often declines, which can negatively impact store sales.

[2538] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a character that the user can customize based on the generated account; speech recognition means and natural language processing means for enabling voice communication between the user and the character; emotion analysis means for generating responses for the character; means for awarding points based on the user's positive emotions; means for improving the character according to the accumulated points; matching means and chat means for anonymously communicating with other users in the same situation or emotional state; purchase means for communicating with a specified avatar as a paid option; gaze tracking means and display means for analyzing the user's dynamic gaze and speech content in real time in a physical store and displaying related information; and generative AI model and prompt sentence utilization means for analyzing the emotions of staff and generating appropriate responses. This allows users to receive personalized emotional support in real time even in a physical store, not only stimulating their interest in products but also enabling them to enjoy high-quality support in communication with staff.

[2539] The "account generation means" is a function that generates a unique user account based on information provided by the user.

[2540] "Customizable character generation means" is a function that allows the user to change the appearance and attributes of the character according to their own preferences.

[2541] The "voice recognition means" is a function that analyzes the voice spoken by the user and converts it into text data.

[2542] "Natural language processing means" is a technology for analyzing text data acquired by speech recognition means and understanding its meaning and context.

[2543] The "emotion analysis means" is a function that analyzes the user's emotional state from their speech and text, and generates an appropriate response based on the results.

[2544] The "point giving means" is a function for assigning points based on the user's actions and comments.

[2545] "Character growth means" is a function that changes a character's skills and appearance according to accumulated points.

[2546] "Matching methods" are functions that anonymously connect users with other users who have the same emotional state or circumstances.

[2547] "Chat means" is a function that allows matched users to exchange text messages.

[2548] The "paid option purchasing means" is a function that allows the user to purchase additional paid services.

[2549] "Eye tracking means" is a technology that detects the user's gaze in real time and identifies the object that the gaze is directed at.

[2550] The "display means" is a function that visually displays appropriate information based on eye tracking and speech content.

[2551] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on the user's emotions and situation.

[2552] "Prompt sentence utilization means" is a function that generates a response using a prompt sentence that gives appropriate instructions to the generative AI model.

[2553] MODE FOR CARRYING OUT THE INVENTION

[2554] The present invention provides a system for providing emotional support to users in real time and improving their mental health. This system aims to improve the customer experience, particularly in physical stores. The specific configuration and operation of the system are described in detail below.

[2555] System configuration

[2556] 1. Device:

[2557] Smart glasses worn by the user that include eye tracking and display capabilities.

[2558] It has a built-in voice recording device that records what the user says.

[2559] 2. Server:

[2560] It recognizes and analyzes voice data and generates responses using generative AI.

[2561] Emotion analysis technology is used to analyze the emotions of users and store staff and provide appropriate services.

[2562] 3. Application:

[2563] It features voice recognition, natural language processing, emotion analysis, point awarding, character growth, matching and chat functions, and the ability to purchase paid options.

[2564] How it works

[2565] User registration and character creation

[2566] 1. The user puts on the smart glasses and launches the application.

[2567] 2. The user creates an account by entering personal information such as name and age.

[2568] 3. The server generates a user profile based on this information using an account generation means.

[2569] 4. The server uses the generative AI model to generate a customizable character that can then be customized by the user.

[2570] Emotion analysis and character responses

[2571] 1. When a user speaks to a character, the device records the voice using a recording device and sends it to the server.

[2572] 2. The server converts the voice data into text using speech recognition and natural language processing means, and performs sentiment analysis.

[2573] 3. Based on the emotional data analyzed by the emotion engine, the generative AI model generates an appropriate response.

[2574] 4. The device displays the response on the smart glasses display or reads it out loud.

[2575] Points awarded and character growth

[2576] 1. When a user makes a positive comment, the server analyzes it and awards points.

[2577] 2. The server will develop the character and add new skills according to the accumulated points.

[2578] Improving user experience in physical stores

[2579] 1. When a user enters a physical store, the device uses eye tracking to identify products that interest the user.

[2580] 2. The device analyzes the speech in real time and displays relevant information on the screen.

[2581] Dialogue with staff

[2582] 1. The device records the conversation with the staff and sends it to the server.

[2583] 2. The server analyzes the staff member's emotions, generates an appropriate response, sends it to the terminal, and displays or reads it aloud.

[2584] Specific examples

[2585] 1. Imagine a scenario where a user enters a brick-and-mortar store and puts on a pair of smart glasses.

[2586] The staff greets you with "Welcome!"

[2587] The device records this audio and sends it to the server.

[2588] The server performs speech recognition and emotion analysis and generates a response such as, "That's very kind of you. I see you like our products."

[2589] The terminal displays this response on its display and the user confirms it.

[2590] Example prompt sentence:

[2591] Scenario: Inside the store, a staff member is heard saying, "Welcome!"

[2592] The user's sentiment is encouraging.

[2593] Generate an appropriate response.

[2594] Response: That's very kind of you, I see you like our products.

[2595] This allows users to receive personalized emotional support in real time even in physical stores, not only stimulating their interest in products but also enabling them to enjoy high-quality support when communicating with staff.

[2596] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2597] Specific explanations divided into processing steps

[2598] Step 1:

[2599] A user puts on the smart glasses and launches the application. The user creates an account by entering personal information such as name and age. The device sends this information to the server. Based on the entered information, the server generates a user profile and creates a unique account using the account generation means. The output is the user profile and account information.

[2600] Step 2:

[2601] The server uses the generative AI model to generate a character that can be customized by the user. The user customizes the character's appearance and attributes through the smart glasses display and sends that information to the server. The output is the customized character information.

[2602] Step 3:

[2603] When a user speaks to a character, the device records the voice and transmits the voice data to the server in real time. The server converts the voice data into text data using a voice recognition means, and obtains the voice data as input and the text data as output.

[2604] Step 4:

[2605] The server uses natural language processing means to analyze the converted text data, obtaining the text data as input. This analysis allows the server to understand the meaning and context of the user's speech, and then uses emotion analysis means to obtain analyzed emotion data as output.

[2606] Step 5:

[2607] The emotion engine uses a generative AI model to generate an appropriate response based on the analyzed emotion data. The prompt text is used to instruct the generative AI model to generate an appropriate response. The generated response text is obtained as output.

[2608] Step 6:

[2609] The device displays the generated response on the smart glasses display or reads it out loud, allowing the user to receive a response from the character in real time. The displayed or spoken response is obtained as output.

[2610] Step 7:

[2611] When a user makes a positive comment, the device records the voice and sends it to the server. The server analyzes the comment and awards points to the user. The input is the voice data of the positive comment, and the output is point information.

[2612] Step 8:

[2613] As points accumulate, the server uses character development tools to develop the character, including acquiring new skills and changing appearance. The output is information about the developed character.

[2614] Step 9:

[2615] When a user enters a physical store, the smart glasses' eye tracking function catches the user's gaze. The device uses the eye tracking means to transmit product information about the user's gaze to the server in real time. Eye tracking data is obtained as input, and product information is obtained as output.

[2616] Step 10:

[2617] The server uses a generative AI model based on the gaze data and the user's speech to generate appropriate information and send it to the device. The device then provides this information to the user via a display, providing product-related information as output.

[2618] Step 11:

[2619] When a user interacts with a store staff member, the device records the voice and sends it to the server. The server then analyzes the staff member's speech and generates an appropriate response using emotion analysis. The staff member's speech is obtained as input, and the response text is obtained as output.

[2620] Step 12:

[2621] The terminal generates a response based on the interaction with the staff and displays it on the user's smart glasses or reads it out loud, allowing the user to smoothly interact with the staff. The displayed or spoken response is obtained as an output.

[2622] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2623] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2624] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2625] [Fourth embodiment]

[2626] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2627] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2628] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2629] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2630] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2631] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2632] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2633] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2634] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2635] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2636] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2637] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2638] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2639] MODE FOR CARRYING OUT THE INVENTION

[2640] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[2641] System configuration

[2642] The system mainly consists of the following components:

[2643] 1. Device: The electronic device used by the user, including a smartphone, tablet, or PC. The application is installed on the device.

[2644] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[2645] 3. Application: Software installed on the device that has functions such as communication between users and characters, emotion analysis, point awarding, and matching.

[2646] System Operation

[2647] 1. User Registration

[2648] The user downloads and installs the application onto the terminal.

[2649] Users enter required information such as name, age, and gender to create an account.

[2650] The terminal transmits the entered information to a server, which generates a user profile.

[2651] 2. Character Creation and Customization

[2652] The server uses a generative AI to generate virtual characters that can be customized by users.

[2653] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[2654] Once customization is complete, the device sends the information to the server, which then stores the character information.

[2655] 3. Audio data collection and emotion analysis

[2656] The user speaks to the character and begins a conversation.

[2657] The terminal records the user's voice and transmits this voice data to the server.

[2658] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[2659] 4. Character response generation

[2660] The server generates an appropriate character response based on the analyzed emotional data.

[2661] The terminal displays or reads the generated response to the user.

[2662] 5. Cultivating positive emotions

[2663] The device records the user's positive comments and sends them to the server.

[2664] The server analyzes positive comments and awards points to the user.

[2665] The server manages the character's growth (acquiring new skills and changing appearance) as points accumulate.

[2666] 6. Communication with Anonymous Users

[2667] When a user selects the anonymous communication function, the server matches other users with the same emotional state or circumstances and creates an anonymous chat room.

[2668] The terminal displays a chat room to the user, allowing the user to exchange messages with other users.

[2669] 7. Paid options available

[2670] When a user selects a paid option and completes the purchase process, the server receives the purchase information and adds a communication function with a specific celebrity avatar to the user's account.

[2671] The device will notify the user that paid options are available and display a conversation with a celebrity avatar.

[2672] Specific examples

[2673] 1. User registration and character creation

[2674] An elderly user installs the application on their device and creates an account by entering information such as their name "Ichiro Tanaka," age "70," and gender "male."

[2675] The server receives this information and creates a user profile.

[2676] The device displays a screen that allows the user to customize the appearance and name of their favorite character.

[2677] 2. Emotion Analysis and Character Response

[2678] The user speaks to the character, saying, "It's a nice day today."

[2679] The device records the user's voice and sends it to the server.

[2680] The server uses speech recognition and natural language processing technology to convert the speech into text and analyze the positive emotion of "good weather."

[2681] The server generates the character's response, "That's true! It would be nice to go for a walk!" and sends it to the device.

[2682] The device displays or reads this response to the user.

[2683] 3. Points awarded and character growth

[2684] The user continues to make positive comments, saying, "I've recently started jogging."

[2685] The device records this audio and sends it to the server.

[2686] The server analyzes positive comments and awards points to the user.

[2687] Based on the accumulated points, the server adds new skills to the character and sends that information to the device.

[2688] The terminal notifies the user of the character's growth.

[2689] 4. Communication with Anonymous Users

[2690] Users can select the anonymous communication feature and be matched with other users who also enjoy jogging.

[2691] The server generates the chat room and the device displays it to the user.

[2692] Users anonymously exchange jogging information with other users.

[2693] 5. Use of paid options

[2694] The user selects the communication function with the celebrity avatar and proceeds with the purchase.

[2695] The server verifies the purchase information and adds the celebrity avatar communication feature to the user's account.

[2696] The device notifies the user that paid options are available and displays a special conversation with a celebrity avatar.

[2697] This system allows users to receive emotional support and develop positive emotions, which is expected to reduce feelings of loneliness and improve mental health.

[2698] The processing flow will be explained below.

[2699] Program processing steps

[2700] User Registration and Initial Setup

[2701] Step 1:

[2702] The user downloads and installs the application on the device.

[2703] Step 2:

[2704] Users launch the application and create an account by entering required information such as name, age, and gender.

[2705] Step 3:

[2706] The terminal transmits the input information to the server.

[2707] Step 4:

[2708] The server generates and stores a user profile based on the received information.

[2709] Step 5:

[2710] The server sends an initialization success message to the terminal.

[2711] Step 6:

[2712] The device will notify the user that the initial setup is complete and open the character creation screen.

[2713] Step 7:

[2714] Users customize their character's appearance and name.

[2715] Step 8:

[2716] The terminal transmits the customized character information to the server.

[2717] Step 9:

[2718] The server stores the character information in a user profile.

[2719] Emotion analysis and character generation

[2720] Step 10:

[2721] The user speaks to the character and begins a conversation.

[2722] Step 11:

[2723] The terminal records the user's voice and transmits the voice data to the server.

[2724] Step 12:

[2725] The server uses voice recognition technology to convert the voice data into text.

[2726] Step 13:

[2727] The server uses natural language processing techniques to analyze the user's emotional state from the text.

[2728] Step 14:

[2729] The server generates a response message for the character based on the result of the emotion analysis.

[2730] Step 15:

[2731] The server sends the response of the generated character to the terminal.

[2732] Step 16:

[2733] The terminal displays or reads out the character's response to the user.

[2734] Cultivating positive emotions and awarding points

[2735] Step 17:

[2736] The user makes positive comments during a conversation with the character.

[2737] Step 18:

[2738] The device records positive comments and sends the audio data to a server.

[2739] Step 19:

[2740] The server analyzes the positive comments.

[2741] Step 20:

[2742] The server awards points to the user based on the analysis results.

[2743] Step 21:

[2744] The server determines character growth (gaining new skills or changing appearance) based on the accumulated points.

[2745] Step 22:

[2746] The server sends character growth information to the terminal.

[2747] Step 23:

[2748] The terminal notifies the user of the character's growth and point allocation.

[2749] Communicating with Anonymous Users

[2750] Step 24:

[2751] The user selects the "Communicate with Anonymous Users" function from a menu within the application.

[2752] Step 25:

[2753] The terminal transmits the user's selection to the server.

[2754] Step 26:

[2755] The server searches for other users with the same emotional state or circumstances and performs matching.

[2756] Step 27:

[2757] The server creates anonymous chat rooms for matched users.

[2758] Step 28:

[2759] The server transmits chat room information to the terminal.

[2760] Step 29:

[2761] The terminal displays anonymous chat rooms to the user, allowing them to exchange messages with other users.

[2762] Step 30:

[2763] Users communicate with other users anonymously.

[2764] Paid options available

[2765] Step 31:

[2766] The user selects the communication function with the celebrity avatar from a menu within the application.

[2767] Step 32:

[2768] The terminal displays a purchase confirmation screen for the paid option and prompts the user to enter the necessary information.

[2769] Step 33:

[2770] The user enters the necessary information and presses the "Purchase" button.

[2771] Step 34:

[2772] The terminal transmits the purchase information to the server.

[2773] Step 35:

[2774] The server verifies the purchase information and adds the celebrity avatar's communication capabilities to the user's account.

[2775] Step 36:

[2776] The server sends a notification of purchase completion to the terminal.

[2777] Step 37:

[2778] The terminal notifies the user that the paid option is now available.

[2779] Step 38:

[2780] The device will display a conversation screen with a celebrity avatar, allowing users to use this feature.

[2781] Example 1

[2782] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2783] In modern society, many people face the problem of feeling lonely. The lack of opportunities to receive emotional support in daily life is a particular challenge for the elderly and those who tend to be isolated. Under these circumstances, maintaining mental health becomes difficult, increasing the risk of serious mental and physical problems. Furthermore, existing emotional support systems have difficulty responding flexibly to the emotional state of individual users. There is a need for a system that can solve these issues and provide users with continuous, personalized emotional support.

[2784] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2785] In this invention, the server includes: means for receiving information from a user and generating an account; means for generating a virtual character that the user can customize based on the generated account; speech recognition and natural language processing means for enabling voice communication between the user and the virtual character; emotion analysis means for generating responses from the virtual character; means for awarding points based on the user's positive emotions; means for upgrading the virtual character according to the accumulated points; matching and messaging means for anonymously communicating with other users in the same circumstances or emotional state; purchase means for realizing communication with a specific avatar as a paid option; display or audio output means on the terminal for notifying the user of the generated responses; means for generating responses from the virtual character using a generative AI model; means for generating prompt sentences to be input to the generative AI model based on the emotion analysis results; and means for generating responses from the virtual character using the generated prompt sentences. This makes it possible to provide continuous and personalized emotional support to users who feel lonely and improve their mental health.

[2786] A "terminal" is an electronic device used by a user and on which software is installed.

[2787] "Software" refers to a program that is installed and executed on a terminal, and provides various functions through interaction with the user.

[2788] A "user" is a person who uses the system and inputs information and communicates via the application.

[2789] "Account" means a digital management unit that contains user-specific information and is required to use the Software.

[2790] A "virtual character" is a user-customizable digital agent that interacts with the user to provide emotional support.

[2791] "Speech recognition means" is a technology that converts voice data into text data.

[2792] "Natural language processing means" is a technology that analyzes text data and understands meaning and emotions.

[2793] "Emotion analysis means" is a technology that estimates a user's emotional state from their statements and text.

[2794] "Points" are digital evaluation units awarded based on users' actions and comments.

[2795] "Growth methods" are techniques that change the skills and appearance of a virtual character according to the accumulation of points.

[2796] "Matching methods" are technologies that connect users with similar circumstances or emotional states.

[2797] "Messaging means" refers to technology that allows anonymous messaging.

[2798] "Paid Options" are additional features or services that are available for an additional fee.

[2799] "Purchase Instrument" means a payment technique for trading paid options.

[2800] "Display means" refers to a technique for displaying the generated response on the screen of the terminal.

[2801] The "audio output means" is a technique for outputting the generated response as audio.

[2802] A "generative AI model" is an algorithm that uses artificial intelligence to generate text and responses.

[2803] A "prompt" is text data that can be input into a generative AI model to elicit a specific response.

[2804] MODE FOR CARRYING OUT THE INVENTION

[2805] This invention is a system for providing continuous emotional support to users who feel lonely and improving their mental health. The purpose of this invention is to foster positive emotions through communication between users and virtual characters using emotion analysis technology and generative AI technology.

[2806] System configuration

[2807] The system consists of the following elements:

[2808] 1. Device: An electronic device used by a user, such as a smartphone, tablet, or PC. Dedicated software is installed on the device.

[2809] 2. Server: Acts as a central server, processes, stores and analyzes data sent by users.

[2810] 3. Software: A program installed on the device that has functions such as communication between the user and virtual characters, emotion analysis, point awarding, and matching.

[2811] System Operation

[2812] 1. User Registration

[2813] The user downloads and installs the application onto the terminal.

[2814] Users enter required information such as name, age, and gender to create an account.

[2815] The terminal transmits the entered information to a server, which generates a user profile.

[2816] 2. Character Creation and Customization

[2817] The server uses a generative AI model to generate virtual characters that can be customized by users.

[2818] The device will then display a character creation screen where the user can select and customize their character's appearance and name.

[2819] Once customization is complete, the device sends the information to the server, which then stores the character information.

[2820] 3. Audio data collection and emotion analysis

[2821] The user speaks to the character and begins a conversation.

[2822] The terminal records the user's voice and transmits this voice data to the server.

[2823] The server converts the voice data into text using speech recognition means and analyzes the emotional state using natural language processing means.

[2824] 4. Character response generation

[2825] The server generates an appropriate character response based on the analyzed emotion data. A generative AI model (e.g., ChatGPT) is used to generate a prompt. An example of a prompt is, "The user said, 'It's a nice day today.' Please generate a positive and constructive response."

[2826] The terminal dis...

Claims

1. It is an application that is installed on the device. means for receiving information from a user and generating an account; means for generating a user-customizable character based on the generated account; a voice recognition means and a natural language processing means for realizing voice communication between a user and a character; emotion analysis means for generating responses for the character; A means for awarding points based on the user's positive emotions; A way to grow your character according to the accumulated points, A matching and chatting means for anonymously communicating with other users in the same situation or emotional state; A system including a means for purchasing a paid option to communicate with a designated avatar.

2. 10. The system of claim 1, further comprising a terminal-provided function for recording user voice data and transmitting the data to the server in real time.

3. 2. The system according to claim 1, further comprising means for identifying the emotional state of the user based on an emotion analysis, generating a response for the character based on the result, and notifying the user of the response.

4. 2. The system according to claim 1, further comprising means for awarding points to users for positive comments and changing the character's growth status.

5. The system according to claim 1, further comprising a matching means and a chat means for anonymously communicating with other users in the same situation.

Images