System

Abstract

A system is provided.SOLUTION: A system includes a means for generating a customizable avatar based on personal information input by a user, a means for forming a personality of the generated avatar by an artificial intelligence model, a means for providing a reminder or information in response to a daily schedule or a question by the user, a means for cooperating with an IoT device and managing and feeding back a health condition of the user, and a means for learning the avatar through continuous interaction with the user and updating the personality.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The feelings of loneliness and alienation felt by the elderly can lead to psychological and health problems. In modern society, elderly people lack support in their daily lives due to less contact with family and friends and fewer opportunities to go out. In addition, dementia and other health problems increase the need for reminders and health management. In this context, an efficient system is needed to alleviate the elderly's sense of loneliness and support their daily lives. [Means for solving the problem]

[0005] This invention provides a means for generating a customizable avatar based on personal information entered by the user, and a means for creating the avatar's personality using an artificial intelligence model. It also includes a means for providing reminders and information based on the user's daily schedule and questions, and a means for linking with IoT devices to manage and provide feedback on the user's health status. It also includes a means for the avatar to learn and update its personality through continuous interaction with the user. This allows for a system that can alleviate the sense of loneliness of the elderly and effectively support their daily lives.

[0006] "Users" refer to elderly people who use the system and interact with the avatar based on their personal information and preferences.

[0007] An "avatar" is a customizable visual and auditory character that is generated based on user input and interacts with the user and assists them in their daily lives.

[0008] An "artificial intelligence model" is a means that includes algorithms and techniques to create an avatar's personality based on the user's personality and preferences, and for the avatar to learn and evolve through continuous interaction.

[0009] "Reminders" refers to a feature that notifies users of scheduled events and tasks and reminds them as needed.

[0010] "Information provision" is a function that provides necessary data and advice in response to user questions or requests.

[0011] An "IoT device" is an internet-connected device that is used to collect and transmit data about a user's health and daily life.

[0012] "Health management" is a function that analyzes the user's health status based on data acquired by IoT devices and provides necessary feedback and advice.

[0013] "Dialogue" refers to the voice and text communication between the user and the avatar, including the avatar's learning and personality updates through this.

[0014] "Learning" refers to the process by which the avatar continuously evolves using an artificial intelligence model based on interaction data with the user, providing more appropriate responses. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. This system is realized by linking terminals, servers, and IoT devices.

[0037] System configuration

[0038] 1. User registration and avatar customization

[0039] On device: The user installs the app and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, old).

[0040] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[0041] 2. Avatar personality development

[0042] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[0043] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[0044] 3. Daily life support

[0045] Reminder function:

[0046] User: Instructs the avatar about daily schedules and tasks (such as doctor's appointments and shopping lists).

[0047] Device: Sends the requested information to the server and sets the reminder.

[0048] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[0049] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[0050] Health management function:

[0051] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[0052] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[0053] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0054] 4. Continuous interaction and learning

[0055] Interactive features:

[0056] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[0057] Terminal: Sends the user's questions and conversation content to the server.

[0058] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[0059] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[0060] Learning features:

[0061] Server: Continuously collects user interaction data and trains the AI ​​model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0062] Device: Receive updated personality data and keep your avatar settings up to date.

[0063] Specific examples

[0064] 1. User Registration and Customization:

[0065] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[0066] Server: Stores user information, generates initial avatar data, and sends it to the device.

[0067] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[0068] 2. Reminders and Health Management:

[0069] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[0070] Terminal: Sends a request to the server.

[0071] Server: Set reminders and send confirmation messages to your device.

[0072] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[0073] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[0074] In this way, the system of the present invention not only alleviates the sense of loneliness of the elderly, but also provides support in various aspects of daily life. Through continuous learning and updates, it is possible to provide optimal assistance to users.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[0078] Step 2:

[0079] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[0080] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[0081] Step 3:

[0082] Device: Presents the user with options for the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[0083] User: Choose your avatar's appearance and voice to complete the customization.

[0084] Step 4:

[0085] Terminal: Sends the information entered and selected by the user to the server.

[0086] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[0087] Step 5:

[0088] Server: Based on the user's personal information and customization information, an artificial intelligence model is used to create an avatar's personality.

[0089] Server: Sends the created personality data to the terminal.

[0090] Step 6:

[0091] Device: Based on the received avatar personality data, the initial avatar settings are performed. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[0092] Step 7:

[0093] User: Enter daily schedules and tasks into the avatar (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?").

[0094] Terminal: Sends input requests to the server.

[0095] Step 8:

[0096] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[0097] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0098] Step 9:

[0099] IoT devices: Collect user health data (heart rate, steps, etc.).

[0100] Server: Periodically receives data collected from IoT devices and analyzes their health status.

[0101] Server: Sends the analysis results to the device and generates appropriate feedback and advice.

[0102] Step 10:

[0103] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[0104] Step 11:

[0105] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[0106] Terminal: Sends the user's query to the server.

[0107] Step 12:

[0108] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[0109] Device: The avatar tells the user, "Today's weather is sunny. The temperature is 25 degrees."

[0110] Step 13:

[0111] Server: Continuously collects user interaction data and updates the avatar's personality using an artificial intelligence model.

[0112] Server: Sends updated personality data to devices, keeping avatar settings up to date.

[0113] Example 1

[0114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] This invention relates to a system for supporting the daily lives of elderly people. Elderly people often experience feelings of loneliness and difficulty managing their health, and therefore support is needed to solve these problems and enrich their daily lives. While existing technologies exist for avatar systems that interact with users, they face challenges in providing optimal support tailored to the user's behavior and preferences through continuous learning and customization based on personal information.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0117] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating a personality for the generated avatar using an artificial intelligence model; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT devices; means for the avatar to learn and update its personality through continuous dialogue with the user; means for the avatar to understand the content of the dialogue using a natural language processing algorithm based on the user's personal information and daily dialogue data and generate appropriate responses; and means for the server to analyze data collected from the user and learn the user's behavioral patterns and preferences. This provides comprehensive support for the daily lives of the elderly, alleviating feelings of loneliness, and improving the efficiency of health management.

[0118] "User" refers to an individual or elderly person who uses the system and receives assistance in daily life through interaction with an avatar.

[0119] "Personal Information" refers to information provided by users, including data such as name, birth date, gender, hobbies, and preferences.

[0120] A "customizable avatar" refers to a pseudo-personality whose appearance, voice, etc. can be set based on personal information entered by the user.

[0121] "Artificial intelligence model" refers to the machine learning algorithms and natural language processing technologies used to create the avatar's personality and generate responses.

[0122] "Reminders" refers to a feature that notifies users of their daily schedules and tasks.

[0123] An "IoT device" refers to a sensor device that is connected to the Internet and collects user health data, etc.

[0124] "Health status" refers to a user's physical data, such as heart rate, steps taken, and blood pressure.

[0125] "Feedback" refers to the act of providing advice or notifications to users based on collected data.

[0126] "Continuous dialogue" refers to long-term interaction between the user and the avatar.

[0127] "Natural language processing algorithms" refer to technologies that understand user speech or text input and generate appropriate responses.

[0128] "Behavioral patterns" refer to the user's daily life tendencies and habits.

[0129] "Preferences" refers to a user's hobbies and tendencies.

[0130] This is a system designed to support the daily lives of elderly people. It generates a customizable avatar based on personal information entered by the user, and supports the user's daily life through interaction with this avatar. This system is realized mainly by terminals, servers, and IoT devices.

[0131] First, the user installs and launches the application on their device. This application provides an interface for entering personal information (e.g., name, birthday, gender, hobbies, preferences, etc.). Data such as name, birthday, and hobbies are entered into each input field.

[0132] Next, the user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, old), and this customization information is also sent from the device to the server.

[0133] The server generates a user profile based on the received personal information and customization information. It then uses an artificial intelligence model (generative AI model) to generate a personality for the avatar. This personality data is also sent from the server to the device. Specifically, it uses a natural language processing algorithm to generate dialogue based on the user's interests and behavioral patterns.

[0134] The generated avatar will be displayed on the device and greet the user for the first time. For example, the avatar may say to the user, "Hello, I'm your new lifemate."

[0135] This system has a reminder function, and the user can instruct the avatar about daily schedules and tasks, which are then managed and notified. When the user asks the avatar to "set a reminder to take my medicine at 9:00 tomorrow," the device sends this request to the server, and the reminder is set. At the specified time, the avatar will notify the user, "It's time to take your medicine now."

[0136] It also works with IoT devices as a health management function, collecting user health data (e.g., heart rate, number of steps, blood pressure, etc.). The collected data is sent to a server where it is analyzed. For example, if a user walks 5,000 steps in a day, the server will generate health advice such as "You walked 5,000 steps today. You're very healthy," and notify the user through an avatar on the device.

[0137] Furthermore, this system has the ability to allow the avatar to learn through continuous dialogue and continuously update its personality. When a user asks a question such as "What's in the news today?", the device sends the question to the server, which analyzes the dialogue data, generates appropriate information, and sends it to the device. This allows the avatar to provide information to the user, such as "Here's today's news." This dialogue data is then learned by the server, allowing the avatar to understand the user's preferences and behavioral patterns and provide more appropriate responses.

[0138] For example, if a user inputs the name "Yamada Taro," the hobby "reading," and sets the appearance as "dog" and the voice as "calm female voice," the server will generate a profile for "Yamada Taro" and set the avatar as "a dog who loves reading and has a laid-back personality." Based on this setting, the avatar will greet the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[0139] Examples of prompts include "Set a reminder to take my medicine at 9:00 tomorrow" and "What's the news today?" This allows the present invention to comprehensively support the daily lives of elderly people, alleviating their feelings of loneliness and improving the efficiency of health management.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1:

[0142] The user installs the app on their device and launches it. The user enters personal information (such as name, birthday, gender, hobbies, and preferences) through the app's interface. The input data includes "Name: Yamada Taro," "Birthday: January 1, 1945," and "Hobbies: Reading." This input data is sent from the device to the server. The server receives the input data, converts it to JSON format data, and sends it to the server.

[0143] Step 2:

[0144] The server generates a user profile based on the personal information received from the device. The server stores the data in a personal information database and creates a user profile. This user profile includes the user's name, birthday, gender, hobbies, preferences, etc. The output is the initial setting data for the user profile.

[0145] Step 3:

[0146] The user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, elderly). This customization information is sent from the device to the server. Input data includes, for example, "appearance: dog" and "voice: calm female voice." The device sends the customized information to the server.

[0147] Step 4:

[0148] Based on the customization information received by the server, a generative AI model is used to generate initial avatar settings. Specifically, a natural language processing algorithm is used to generate dialogue data based on the user's hobbies and preferences. The output is a customized and personalized avatar settings setup.

[0149] Step 5:

[0150] The server sends the generated avatar personality data to the device. The device receives this personality data and displays the avatar. Based on the initial setup data, the avatar greets the user for the first time. Specifically, the avatar greets the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[0151] Step 6:

[0152] The user instructs the avatar about daily schedules and tasks. For example, the user might instruct the avatar to "set a reminder to take medicine at 9:00 tomorrow." This instruction is sent from the device to the server. The input data includes "a reminder to take medicine at 9:00 tomorrow."

[0153] Step 7:

[0154] The server sets a reminder based on the schedule data it receives. It saves the data in the reminder setting database and notifies the device that the setting is complete. As an output, it generates a message indicating that the reminder setting is complete and sends it to the device.

[0155] Step 8:

[0156] When the set time arrives, the server sends a reminder notification to the device. The device receives this notification, and an avatar notifies the user, "It's time to take your medicine now." Specifically, the avatar notifies the user by voice.

[0157] Step 9:

[0158] IoT devices collect user health data (e.g., heart rate, step count, blood pressure, etc.) and transmit the collected data to a terminal or server in real time. Input data includes heart rate, step count, and blood pressure data.

[0159] Step 10:

[0160] The server analyzes the received health data and evaluates the user's health condition. For example, if the number of steps is low, it generates advice such as "You should walk a little more." This advice data is sent to the device. The generated health advice is included as an output.

[0161] Step 11:

[0162] The device notifies the user through an avatar based on the health advice received from the server. Specifically, the avatar will say to the user, "You walked 5,000 steps today. That's very healthy."

[0163] Step 12:

[0164] The user asks the avatar a question of everyday life, such as "What's in the news today?" This question is sent from the device to the server. The input data includes the question itself.

[0165] Step 13:

[0166] The server generates appropriate information based on the question data it receives. It extracts relevant news information from the dialogue database and sends it to the terminal. As an output, news information is generated and sent to the terminal.

[0167] Step 14:

[0168] Based on the information received from the server, the device uses an avatar to provide news to the user. Specifically, the avatar will say, "Here's today's news."

[0169] Step 15:

[0170] The server continuously collects user interaction data, and the AI ​​model learns from it. The data is stored in an interaction database and the avatar's personality data is updated. The output is the updated personality data.

[0171] Step 16:

[0172] The device keeps the avatar's settings up to date based on the updated personality data it receives from the server. Specifically, the avatar will ask the user, "How did you like the history book you read recently?"

[0173] (Application example 1)

[0174] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0175] When seniors shop in brick-and-mortar stores, they often face challenges in finding the exact product they need and selecting the right product for their health condition. It's also difficult to quickly respond appropriately in emergencies. These challenges can make the shopping experience stressful for seniors, making it difficult for them to receive optimal health care.

[0176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0177] In this invention, the server includes a means for generating a customizable avatar based on personal information entered by the user, a means for creating a personality for the generated avatar using an artificial intelligence model, a means for providing reminders and information in response to the user's daily schedule and questions, a means for managing and providing feedback on the user's health status in cooperation with IoT devices, a means for navigating the user in a store and providing product information in cooperation with smart glasses, a means for suggesting products according to the user's health status, and a means for contacting store staff in an emergency. This allows elderly people to smoothly shop in physical stores and receive appropriate health care.

[0178] A "customizable avatar" is a virtual character that can interact with the user and whose appearance, voice, etc. can be freely set based on personal information entered by the user.

[0179] An "artificial intelligence model" is a technology that uses machine learning and natural language processing to generate appropriate responses and actions in response to user interactions and instructions.

[0180] "Reminder" is a function that notifies the user of scheduled events or tasks at a pre-set time.

[0181] An "IoT device" is a device that has the ability to collect, analyze, and transmit data via the Internet. Examples include health management devices and smart home devices.

[0182] "Smart glasses" are a wearable eyeglass-type device that can display augmented reality and information when worn by the user.

[0183] "Health Status Management" is a system that continuously collects and analyzes users' health data and provides appropriate feedback and advice.

[0184] "In-store navigation" is a function that guides users to the location of the product they are looking for within a physical store.

[0185] "Product Information Provision" is a function that provides users with detailed information about a specified product and related suggestions.

[0186] "Product suggestions based on health status" is a function that recommends products suitable for a user's condition based on their health data.

[0187] "Emergency Contact" is a function that quickly sends alerts to store staff and relevant institutions if there is any change in the user's health condition.

[0188] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. The system also works in conjunction with smart glasses to support shopping in brick-and-mortar stores.

[0189] System configuration

[0190] 1. User registration and avatar customization

[0191] Device: The user installs a dedicated app and enters personal information such as name, birthday, gender, hobbies, and preferences. Next, the user customizes the avatar's appearance (e.g., human or animal) and voice (e.g., male or female). This information is sent to the server.

[0192] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[0193] 2. Avatar personality development

[0194] Server: Based on the user's personal information and input data, a generative AI model is used to create an avatar personality. This model uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[0195] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[0196] 3. Daily life support

[0197] Reminder function:

[0198] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[0199] Device: Sends the requested information to the server and sets the reminder.

[0200] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[0201] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[0202] Health management function:

[0203] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[0204] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[0205] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0206] 4. Shopping support in physical stores

[0207] Shopping Navigation:

[0208] Smart glasses: Navigate users through physical stores and guide them to the location of the product they are looking for. For example, an avatar will guide them, saying, "Milk is in Aisle 3."

[0209] Product information provided:

[0210] Server: Generates detailed product information and related suggestions and sends them to the smart glasses.

[0211] Smart glasses: Avatars present product information to users.

[0212] Product suggestions based on health condition:

[0213] Server: Based on the user's health data, the server recommends suitable products. For example, it generates advice such as "Choose low-salt products."

[0214] Smart glasses: An avatar suggests products based on your health condition.

[0215] 5. Emergency Response

[0216] Smart glasses: Send emergency alerts to store staff if there are any abnormalities in the user's health.

[0217] Server: Records the abnormality in the emergency response database and prompts emergency response.

[0218] Specific examples

[0219] 1. Shopping Navigation:

[0220] User: Asks smart glasses, "Where is the milk?"

[0221] Smart glasses: The avatar navigates by saying "Milk is in Aisle 3."

[0222] 2. Product recommendations based on health conditions:

[0223] User: Asks smart glasses, "What healthy food would you like to eat today?"

[0224] Smart glasses: An avatar suggests, "Choose low-salt products."

[0225] Prompt Sentence Examples

[0226] User: "Where's the milk?"

[0227] Avatar: "Milk is in Aisle 3."

[0228] This system allows elderly people to shop smoothly in brick-and-mortar stores and receive appropriate health care.

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] User registration and avatar customization

[0232] Input: The user enters personal information such as name, birthday, gender, hobbies, and preferences into the device.

[0233] How it works: The user launches the app and enters the required information into a form.

[0234] Data processing: The terminal sends the entered personal information to the server.

[0235] Output: User personal information and customization information is stored on the server.

[0236] Step 2:

[0237] Avatar initial settings

[0238] Input: Your personal and customization information stored on our servers.

[0239] How it works: The server uses this information to generate a user profile and create initial avatar data.

[0240] Data calculation: Generate initial setting data for the avatar using a generative AI model.

[0241] Output: Initialization data is sent to the terminal.

[0242] Step 3:

[0243] Avatar personality development

[0244] Input: Initial setting data for the server-generated avatar.

[0245] How it works: The device configures an avatar based on the data it receives and uses a generative AI model to create the avatar's personality.

[0246] Data calculation: Using a natural language processing algorithm, the avatar's initial setting data is analyzed and personality data is generated.

[0247] Output: The formed personality data is saved on the device.

[0248] Step 4:

[0249] Reminder function

[0250] Input: The user tells the device about their daily schedule and tasks.

[0251] How it works: The user sets a reminder through the app and sends the information to the server.

[0252] Data processing: The server stores the information in the reminder setting database and sends a notification to the device at the specified time.

[0253] Output: The device will notify the user of the reminder via the avatar.

[0254] Step 5:

[0255] Health management function

[0256] Input: User health data collected by IoT devices.

[0257] How it works: The server stores data received from IoT devices and analyzes their health status.

[0258] Data calculation: Generates advice to users based on the results of health data analysis.

[0259] Output: The device provides health advice via an avatar.

[0260] Step 6:

[0261] Shopping support in physical stores - Navigation function

[0262] Input: The user inputs information about the product they are looking for through the smart glasses.

[0263] How it works: The smart glasses send the input information to the server.

[0264] Data calculation: The server references the store database and obtains product location information.

[0265] Output: The smart glasses guide the user to the product location via an avatar.

[0266] Step 7:

[0267] Shopping support in physical stores - Product information provision

[0268] Input: Server-generated product details.

[0269] How it works: The server sends detailed product information and related suggestions to the smart glasses.

[0270] Data processing: Generative AI models select appropriate product information for users.

[0271] Output: The smart glasses present product information to the user via an avatar.

[0272] Step 8:

[0273] Product suggestions based on health status

[0274] Input: User health data and related product information.

[0275] How it works: The server analyzes health data and suggests products suitable for the user.

[0276] Data calculation: Generate optimal product recommendations based on health status data.

[0277] Output: The smart glasses display product suggestions to the user through an avatar.

[0278] Step 9:

[0279] Emergency response

[0280] Input: Anomalous health data detected by IoT devices.

[0281] How it works: The smart glasses send anomaly data to the server.

[0282] Data calculation: The server refers to the emergency response database and generates the necessary response measures.

[0283] Output: An emergency notification is sent to store staff and an avatar informs the user about emergency procedures.

[0284] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0285] This invention is a system that generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized by linking terminals, servers, and IoT devices.

[0286] System configuration

[0287] 1. User registration and avatar customization

[0288] Device: The user installs the application and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[0289] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[0290] 2. Avatar personality development

[0291] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[0292] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[0293] 3. Introducing the Emotion Engine

[0294] On-device: The device is equipped with an emotion engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected.

[0295] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[0296] 4. Daily life support

[0297] Reminder function:

[0298] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[0299] Device: Sends the requested information to the server and sets the reminder.

[0300] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[0301] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0302] Health management function:

[0303] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[0304] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[0305] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0306] 5. Continuous interaction and learning

[0307] Interactive features:

[0308] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[0309] Terminal: Sends the user's questions and conversation content to the server.

[0310] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[0311] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[0312] Learning features:

[0313] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0314] Device: Receive updated personality data and keep your avatar settings up to date.

[0315] Specific examples

[0316] 1. User Registration and Customization:

[0317] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[0318] Server: Stores user information, generates initial avatar data, and sends it to the device.

[0319] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[0320] 2. Reminders and Health Management:

[0321] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[0322] Terminal: Sends a request to the server.

[0323] Server: Set reminders and send confirmation messages to your device.

[0324] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[0325] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[0326] 3. Leveraging Emotion Recognition:

[0327] User: After eating, tell your avatar, "I'm feeling a bit tired today."

[0328] Device: The emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[0329] Server: Based on the emotional data, adjust the content and tone of the avatar's response. "I understand. Please get plenty of rest today," the avatar says in a comforting voice.

[0330] On the device: The avatar provides tailored responses to the user, providing emotionally sensitive support.

[0331] In this way, by combining the emotion engine, the system of the present invention not only alleviates the sense of loneliness felt by the elderly, but also provides emotionally sensitive responses, allowing users to experience richer interactions and receive assistance in their daily lives.

[0332] The processing flow will be explained below.

[0333] Step 1:

[0334] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[0335] Step 2:

[0336] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[0337] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[0338] Step 3:

[0339] On the device: Users are presented with options for avatar appearance (human, animal, etc.) and voice (male, female, young, adult) and can customize their avatar.

[0340] User: Choose your avatar's appearance and voice to complete the customization.

[0341] Step 4:

[0342] Terminal: Sends the information entered and selected by the user to the server.

[0343] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[0344] Step 5:

[0345] Server: Based on the user's personal information and customization information, an AI model is used to create an avatar personality, and the created personality data is sent to the device.

[0346] Step 6:

[0347] Device: Based on the received avatar personality data, the avatar is initialized. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[0348] Step 7:

[0349] User: Enters daily schedules and tasks (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?") and gives instructions to the avatar.

[0350] Terminal: Sends input requests to the server.

[0351] Step 8:

[0352] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[0353] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0354] Step 9:

[0355] IoT devices: Collect user health data (heart rate, steps, etc.).

[0356] Server: Periodically receives data collected from IoT devices and analyzes health status. Sends the analysis results to the device. For example, if the number of steps is low, the avatar will advise the user, "You should walk a little more."

[0357] Step 10:

[0358] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[0359] Step 11:

[0360] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[0361] Terminal: Sends the user's query to the server.

[0362] Step 12:

[0363] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[0364] Device: The avatar explains to the user, "Today's weather is sunny. The temperature is 25 degrees."

[0365] Step 13:

[0366] Device: The emotion engine analyzes the user's voice and facial expressions. For example, if a user says, "I'm a little tired today," the voice tone and facial expression data are collected.

[0367] Step 14:

[0368] Server: Adjusts the avatar's response content and tone based on the emotional data analyzed by the emotion engine. "I understand. Please get plenty of rest today," the avatar says comfortingly.

[0369] Step 15:

[0370] Device: The avatar provides tailored responses to the user, providing appropriate feedback based on emotional data and providing support that is sensitive to the user's emotions.

[0371] Step 16:

[0372] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an artificial intelligence model.

[0373] Device: Receives the latest personality data and updates avatar settings.

[0374] Example 2

[0375] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0376] There is a need for an effective system that can support the daily lives of elderly people while alleviating their feelings of loneliness. Conventional support systems are limited to one-way information provision or simple reminder functions, and do not adequately address the emotions and individual needs of users. This has led to issues such as limited user satisfaction and utilization effectiveness.

[0377] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0378] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating the personality of the generated avatar using an artificial intelligence model; means for analyzing the user's voice and facial expressions to acquire emotional data; means for adjusting the avatar's response content and tone based on the emotional data; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT sensors; and means for the avatar to learn and update its personality through continuous dialogue with the user. This makes it possible to respond to the user's emotions and individual needs and provide comprehensive support for the elderly in their daily lives.

[0379] "User" refers to the individual who uses the system, who is often a senior citizen.

[0380] "Personal information" refers to information that identifies a user, such as the user's name, birthday, gender, hobbies, and preferences.

[0381] A "customizable avatar" is a virtual character whose appearance and voice can be changed based on information entered by the user.

[0382] "Artificial intelligence model" refers to the algorithms and machine learning models used to generate avatar personalities and responses based on user input data.

[0383] "Emotional data" is data that indicates the emotional state of a user, obtained by analyzing the user's voice and facial expressions.

[0384] An "IoT sensor" is a device that is connected to the Internet and collects data on a user's health and environment.

[0385] "Health Guidance" means advice or suggestions to improve a user's health.

[0386] "Reminder" is a feature that notifies users of their schedules and tasks on time.

[0387] "Dialogue" refers to verbal exchanges between a user and an avatar.

[0388] "Continuous interaction" refers to interaction between a user and an avatar that occurs regularly over an extended period of time.

[0389] "Personality updating" refers to the process of improving an avatar's responses and behavioral patterns based on interaction data and emotional data with the user.

[0390] MODE FOR CARRYING OUT THE INVENTION

[0391] This system generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized through collaboration between terminals, servers, and IoT devices.

[0392] System Configuration

[0393] 1. User registration and avatar customization

[0394] Device: The user installs the application and enters the necessary personal information (such as name, birthday, gender, and hobbies). The user then customizes the avatar's appearance and voice. Specific hardware examples include smartphones and tablets.

[0395] Server: Receives the entered personal information and customization information and generates a user profile. This is done using a database server and an AI server. Furthermore, based on this information, it generates initial setting data for the avatar and sends it to the device.

[0396] Examples:

[0397] The user enters their name "Taro Tanaka," their date of birth "March 15, 1950," their hobby "travel," and sets their avatar's appearance to "cat" and voice to "young man's voice."

[0398] 2. Avatar personality development

[0399] Server: Based on the user's personal information and input data, an artificial intelligence model (e.g., GPT-4 (registered trademark)) is used to create an avatar personality. Specific software examples include a machine learning platform and a natural language processing engine. The created personality data is sent to the device.

[0400] Device: Initial settings are made based on the received avatar personality data. For example, the avatar may say, "Hello, I'm your new lifemate."

[0401] Examples:

[0402] The avatar greets, "Hello, Tanaka-san. How was your day?"

[0403] 3. Introducing the Emotion Engine

[0404] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions. Examples of specifications include a camera-equipped device and a voice recognition microphone.

[0405] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[0406] Examples:

[0407] The user says, "I'm feeling a bit tired today." The emotion engine interprets this and the avatar responds, "That must be tough. Why don't you take a break?"

[0408] 4. Daily life support

[0409] Reminder function:

[0410] User: Instructs the avatar on daily schedules and tasks.

[0411] Server: Stores the information in the reminder setting database and sends a notification to the device at the specified time.

[0412] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0413] Health management function:

[0414] IoT devices: Collect user health data (e.g., heart rate, steps, blood pressure, etc.). Specific examples of such devices include fitness bands and smartwatches.

[0415] Server: Receives and analyzes the collected data. For example, it generates advice such as "If your step count is low, you should walk a little more."

[0416] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0417] Examples:

[0418] If a user requests, "Set a reminder to take my medicine at 9:00 tomorrow," the server will set this reminder and the device will notify them at the specified time.

[0419] 5. Continuous interaction and learning

[0420] Interactive features:

[0421] Users: Ask the avatar everyday topics and questions.

[0422] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[0423] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[0424] Learning features:

[0425] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0426] Device: Receive updated personality data and keep your avatar settings up to date.

[0427] Example prompt sentence:

[0428] "What news is there today?"

[0429] "Tell me about the weather lately."

[0430] "When's your next doctor's appointment?"

[0431] In this way, this system generates and operates customizable avatars based on personal information and emotional data entered by the user, providing comprehensive support for the user's daily life.

[0432] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0433] Step 1:

[0434] User Registration and Customization

[0435] Input: Users install applications on their devices and enter personal information such as name, birthday, gender, hobbies, etc. They also customize the appearance and voice of their avatar.

[0436] Operation: The device sends the entered personal information and customization information to the server. Specifically, the user fills in the information in the input form and presses the send button.

[0437] Output: The server generates a user profile based on the received data and creates initial avatar data. This initial data is then sent to the device.

[0438] Step 2:

[0439] Avatar personality development

[0440] Input: The server receives the user's personal information and customization data and inputs it into an artificial intelligence model (e.g., GPT-4).

[0441] How it works: The server uses an artificial intelligence model to generate the avatar's personality, passing the incoming data to a natural language processing engine and processing the output from the model.

[0442] Output: The generated avatar's personality data is sent to the device. The device configures the avatar based on the received data and displays it on the interface.

[0443] Step 3:

[0444] Completing the initial setup and first greetings

[0445] Input: The device performs initial settings based on the avatar's personality data received from the server.

[0446] How it works: The device applies the avatar's appearance, voice, and personality data to display the avatar for the first time. The avatar then greets the user for the first time.

[0447] Output: The avatar will start a customized interaction with the user, such as "Hello, I'm your new lifemate."

[0448] Step 4:

[0449] Emotion engine analysis and response adjustment

[0450] Input: The device collects the user's voice and facial expressions through the camera and microphone.

[0451] How it works: The emotion engine analyzes the collected data, extracts emotion data, and sends this data to the server.

[0452] Output: The server adjusts the avatar's response content and tone based on the emotional data and sends the data to the device. The avatar on the device then provides the adjusted response to the user.

[0453] Step 5:

[0454] Reminder settings and notifications

[0455] Input: The user gives the avatar instructions for daily schedules and tasks, such as "Set a reminder to take my medicine tomorrow at 9 AM."

[0456] How it works: The device sends instructions from the user to the server, which stores the information in the reminder settings database. When the specified time arrives, the server sends the notification data to the device.

[0457] Output: The device receives notification data from the server, and the avatar notifies the user, such as "It's time to take your medicine now."

[0458] Step 6:

[0459] Health management features for data collection and feedback

[0460] Input: IoT devices collect user health data (e.g., heart rate, steps, blood pressure, etc.).

[0461] How it works: The IoT device sends the collected data to a server, which analyzes the data and generates advice such as "You should walk a little more" if the number of steps is low.

[0462] Output: The analysis results are sent to the device, and the avatar tells the user, "You walked 5,000 steps today. You're very healthy."

[0463] Step 7:

[0464] Continuous interaction and learning

[0465] Input: The user asks the avatar everyday topics and questions, such as "What's in the news today?"

[0466] How it works: The device sends the user's question data to the server, which then generates appropriate information based on the received data and sends it to the device. Furthermore, the server continuously collects conversation data and emotional data with the user, and updates the avatar's personality using an artificial intelligence model.

[0467] Output: The device receives information from the server, the avatar tells the user "Here's today's news" and provides the appropriate information, and keeps the avatar's settings up to date.

[0468] As described above, the program of the system of the present invention provides a function to support the daily lives of elderly people and alleviate their feelings of loneliness through a series of processing steps.

[0469] (Application example 2)

[0470] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0471] To support the daily lives of elderly people while alleviating their feelings of loneliness, simple reminder and health management functions are insufficient. Support tailored to individual needs through continuous interaction with the elderly and flexible response to emotional changes is necessary. However, current systems lack the ability to read the user's unique emotions and respond appropriately based on them, and therefore are unable to provide sufficient psychological satisfaction and security to the elderly.

[0472] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0473] In this invention, the server includes means for generating a customizable virtual character based on personal information entered by the user, means for forming the personality of the generated virtual character using an artificial intelligence model, and means for adjusting the content and tone of the virtual character's responses based on emotional data from an emotion analysis engine. This enables personalized responses that are sensitive to the emotions of the elderly, providing them with a greater sense of satisfaction and security.

[0474] "user"

[0475] refers to individuals who use the system, often specifically elderly people.

[0476] "personal information"

[0477] This refers to data entered by the user to identify the user, such as name, birthday, gender, hobbies, and preferences.

[0478] "Virtual Character"

[0479] is a digital character that is customized based on a user's personal information, with visual and audio elements including appearance and voice.

[0480] "Artificial Intelligence Model"

[0481] refers to algorithms and learning models that use user input data to shape the personality of a virtual character.

[0482] Emotion analysis engine

[0483] This refers to a system that analyzes a user's voice, facial expressions, etc. to detect their emotional state.

[0484] Reminders

[0485] This refers to a function that notifies users of scheduled events and tasks in a timely manner.

[0486] "IoT devices"

[0487] refers to a physical device that is connected to the Internet and is capable of sending and receiving sensor data, including wearable devices for health management.

[0488] "Health status"

[0489] refers to biometric data such as a user's heart rate, blood pressure, number of steps, etc. These data are used to monitor the user's health.

[0490] "Response content"

[0491] refers to the responses that a virtual character provides based on the user's questions and emotional state, including information and advice.

[0492] "tone"

[0493] This refers to the tone of voice and speaking style of a virtual character, which is adjusted by an emotion analysis engine, depending on the user's emotional state.

[0494] This invention provides a system that supports the daily lives of elderly people and alleviates their feelings of loneliness by generating a customizable virtual character based on the user's personal information and combining it with an emotion analysis engine. This system is composed of a terminal, a server, and an IoT device. The details of this system are described below.

[0495] System configuration

[0496] 1. User registration and virtual character customization

[0497] The terminal provides an interface where the user installs the application and inputs necessary personal information (such as name, birthday, gender, hobbies, and preferences). The user can then customize the virtual character's appearance (human, animal, etc.) and voice (male, female, young, adult). The server receives the input personal information and customization information and generates a user profile. Based on this information, the server then generates initial setting data for the virtual character and sends it to the terminal.

[0498] 2. Personality development of virtual characters

[0499] The server uses a generative AI model to create a virtual character's personality based on the user's personal information and input data. This uses natural language processing and machine learning algorithms. The created personality data is then sent to the device. The device then performs initial setup based on the received virtual character's personality data and greets the user for the first time.

[0500] 3. Introduction of a sentiment analysis engine

[0501] The device is equipped with an emotion analysis engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected. The server receives the emotional data analyzed by the emotion analysis engine and adjusts the content and tone of the virtual character's responses.

[0502] 4. Daily life support

[0503] The device can instruct the virtual character about the user's daily schedule and tasks (such as doctor's appointments and shopping lists). It also provides reminder setting and health management functions. The server receives the information specified by the user, stores it in a reminder setting database, and sends notifications to the device at the specified time. It also analyzes health data (heart rate, number of steps, blood pressure, etc.) collected from IoT devices and generates health advice.

[0504] 5. Continuous interaction and learning

[0505] The device has a dialogue function that allows users to ask the virtual character everyday topics and questions. The server generates appropriate information based on the dialogue data received and sends it to the device. It also continuously collects dialogue data and emotional data and updates the virtual character's personality using a generative AI model. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0506] Specific examples

[0507] User Registration and Customization

[0508] The user launches the app on the device, inputs their name "General Taro," their date of birth "January 1, 1945," and their hobby "reading," selects an "animal" as the virtual character's appearance, and sets the voice to a "calm female voice." The server saves the user information, generates initial setting data for the virtual character, and sends it to the device. The virtual character greets the device by saying, "Hello, Taro. I'm your new friend."

[0509] Utilizing Emotion Recognition

[0510] When a user says, "I feel a little tired today," the device uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data. The server then adjusts the content and tone of the virtual character's response based on the emotional data, and the virtual character then comforts the user by saying, "I understand. Please take it easy today."

[0511] Example prompts to input to a generative AI model:

[0512] Username: General Taro

[0513] User input: Set a reminder to take my medicine tomorrow at 9am

[0514] Emotion engine: Analyzes user input and extracts emotional data (e.g., "I'm tired")

[0515] Virtual character response: Sets a reminder and responds to the user (e.g., "I've set a reminder to take my medicine tomorrow at 9 o'clock.")

[0516] In this way, the system of the present invention can respond to the user's emotions and individual needs, and can support the elderly in their daily lives while alleviating their sense of loneliness.

[0517] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0518] Step 1:

[0519] The user inputs personal information (such as name, birthday, gender, hobbies, and preferences) through the device. The device acquires this information and sends it to the server. The personal information is the input data, and the data sent to the server is the output data.

[0520] Step 2:

[0521] The server receives the personal information sent by the user and generates a user profile. The server stores the personal information in a database and generates initial setting data for the virtual character. This initial setting data includes customization information such as the virtual character's appearance and voice. The input data is personal information, and the output is the initial setting data, which is then sent to the terminal.

[0522] Step 3:

[0523] The device displays the virtual character and greets the user based on the initial setting data of the virtual character received from the server. Specifically, it plays a voice and text such as "Hello, Taro. I'm your new friend." The initial setting data is the input data, and the greeting is displayed as the output.

[0524] Step 4:

[0525] The user inputs daily plans and questions to the virtual character. For example, they can give instructions such as "Set a reminder to take my medicine at 9:00 tomorrow." The device acquires the user's input and sends it to the server. The input data includes the user's instructions, and the output data is sent to the server.

[0526] Step 5:

[0527] The server receives instructions sent by the user, analyzes them, and sets reminders. Specifically, it saves the information in a reminder setting database and prepares to send a reminder notification to the device at the specified time. The input data is the user's instructions, and the output is reminder setting data.

[0528] Step 6:

[0529] The device will notify the user of the reminder at the specified time based on the reminder setting data received from the server. Specifically, it will display a voice and text message saying, "It's time to take your medicine now." The input data is the reminder setting data, and the output is a reminder notification.

[0530] Step 7:

[0531] To analyze a user's emotions, the device analyzes the user's voice and facial expressions using an emotion analysis engine. The emotion analysis engine receives the user's voice and facial expression data and detects their emotional state (for example, "tired" or "energetic"). The input data is the user's voice and facial expression, and the output is emotion data.

[0532] Step 8:

[0533] The server adjusts the content and tone of the virtual character's responses based on the emotional data. For example, if a user says, "I'm a little tired today," the server generates an adjusted response, such as "I understand. Please take it easy today," and sends it to the device. Emotional data is the input data, and adjusted response data is generated as the output.

[0534] Step 9:

[0535] The device receives the adjusted response data from the server, and the virtual character responds to the user with voice and text. Specifically, the virtual character responds to the user by saying, "I understand. Please take a good rest today." The adjusted response data is input, and the response is displayed as output.

[0536] Step 10:

[0537] The server continuously collects user interaction and emotional data and uses a generative AI model to update the virtual character's personality. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses. The input data includes interaction data and emotional data, and the output is updated personality data.

[0538] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0540] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0541] [Second embodiment]

[0542] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0543] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0544] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0545] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0546] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0547] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0548] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0549] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0550] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0551] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0552] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0553] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0554] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. This system is realized by linking terminals, servers, and IoT devices.

[0555] System configuration

[0556] 1. User registration and avatar customization

[0557] On device: The user installs the app and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, old).

[0558] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[0559] 2. Avatar personality development

[0560] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[0561] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[0562] 3. Daily life support

[0563] Reminder function:

[0564] User: Instructs the avatar about daily schedules and tasks (such as doctor's appointments and shopping lists).

[0565] Device: Sends the requested information to the server and sets the reminder.

[0566] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[0567] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[0568] Health management function:

[0569] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[0570] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[0571] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0572] 4. Continuous interaction and learning

[0573] Interactive features:

[0574] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[0575] Terminal: Sends the user's questions and conversation content to the server.

[0576] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[0577] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[0578] Learning features:

[0579] Server: Continuously collects user interaction data and trains the AI ​​model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0580] Device: Receive updated personality data and keep your avatar settings up to date.

[0581] Specific examples

[0582] 1. User Registration and Customization:

[0583] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[0584] Server: Stores user information, generates initial avatar data, and sends it to the device.

[0585] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[0586] 2. Reminders and Health Management:

[0587] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[0588] Terminal: Sends a request to the server.

[0589] Server: Set reminders and send confirmation messages to your device.

[0590] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[0591] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[0592] In this way, the system of the present invention not only alleviates the sense of loneliness of the elderly, but also provides support in various aspects of daily life. Through continuous learning and updates, it is possible to provide optimal assistance to users.

[0593] The processing flow will be explained below.

[0594] Step 1:

[0595] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[0596] Step 2:

[0597] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[0598] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[0599] Step 3:

[0600] Device: Presents the user with options for the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[0601] User: Choose your avatar's appearance and voice to complete the customization.

[0602] Step 4:

[0603] Terminal: Sends the information entered and selected by the user to the server.

[0604] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[0605] Step 5:

[0606] Server: Based on the user's personal information and customization information, an artificial intelligence model is used to create an avatar's personality.

[0607] Server: Sends the created personality data to the terminal.

[0608] Step 6:

[0609] Device: Based on the received avatar personality data, the initial avatar settings are performed. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[0610] Step 7:

[0611] User: Enter daily schedules and tasks into the avatar (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?").

[0612] Terminal: Sends input requests to the server.

[0613] Step 8:

[0614] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[0615] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0616] Step 9:

[0617] IoT devices: Collect user health data (heart rate, steps, etc.).

[0618] Server: Periodically receives data collected from IoT devices and analyzes their health status.

[0619] Server: Sends the analysis results to the device and generates appropriate feedback and advice.

[0620] Step 10:

[0621] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[0622] Step 11:

[0623] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[0624] Terminal: Sends the user's query to the server.

[0625] Step 12:

[0626] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[0627] Device: The avatar tells the user, "Today's weather is sunny. The temperature is 25 degrees."

[0628] Step 13:

[0629] Server: Continuously collects user interaction data and updates the avatar's personality using an artificial intelligence model.

[0630] Server: Sends updated personality data to devices, keeping avatar settings up to date.

[0631] Example 1

[0632] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0633] This invention relates to a system for supporting the daily lives of elderly people. Elderly people often experience feelings of loneliness and difficulty managing their health, and therefore support is needed to solve these problems and enrich their daily lives. While existing technologies exist for avatar systems that interact with users, they face challenges in providing optimal support tailored to the user's behavior and preferences through continuous learning and customization based on personal information.

[0634] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0635] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating a personality for the generated avatar using an artificial intelligence model; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT devices; means for the avatar to learn and update its personality through continuous dialogue with the user; means for the avatar to understand the content of the dialogue using a natural language processing algorithm based on the user's personal information and daily dialogue data and generate appropriate responses; and means for the server to analyze data collected from the user and learn the user's behavioral patterns and preferences. This provides comprehensive support for the daily lives of the elderly, alleviating feelings of loneliness, and improving the efficiency of health management.

[0636] "User" refers to an individual or elderly person who uses the system and receives assistance in daily life through interaction with an avatar.

[0637] "Personal Information" refers to information provided by users, including data such as name, birth date, gender, hobbies, and preferences.

[0638] A "customizable avatar" refers to a pseudo-personality whose appearance, voice, etc. can be set based on personal information entered by the user.

[0639] "Artificial intelligence model" refers to the machine learning algorithms and natural language processing technologies used to create the avatar's personality and generate responses.

[0640] "Reminders" refers to a feature that notifies users of their daily schedules and tasks.

[0641] An "IoT device" refers to a sensor device that is connected to the Internet and collects user health data, etc.

[0642] "Health status" refers to a user's physical data, such as heart rate, steps taken, and blood pressure.

[0643] "Feedback" refers to the act of providing advice or notifications to users based on collected data.

[0644] "Continuous dialogue" refers to long-term interaction between the user and the avatar.

[0645] "Natural language processing algorithms" refer to technologies that understand user speech or text input and generate appropriate responses.

[0646] "Behavioral patterns" refer to the user's daily life tendencies and habits.

[0647] "Preferences" refers to a user's hobbies and tendencies.

[0648] This is a system designed to support the daily lives of elderly people. It generates a customizable avatar based on personal information entered by the user, and supports the user's daily life through interaction with this avatar. This system is realized mainly by terminals, servers, and IoT devices.

[0649] First, the user installs and launches the application on their device. This application provides an interface for entering personal information (e.g., name, birthday, gender, hobbies, preferences, etc.). Data such as name, birthday, and hobbies are entered into each input field.

[0650] Next, the user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, old), and this customization information is also sent from the device to the server.

[0651] The server generates a user profile based on the received personal information and customization information. It then uses an artificial intelligence model (generative AI model) to generate a personality for the avatar. This personality data is also sent from the server to the device. Specifically, it uses a natural language processing algorithm to generate dialogue based on the user's interests and behavioral patterns.

[0652] The generated avatar will be displayed on the device and greet the user for the first time. For example, the avatar may say to the user, "Hello, I'm your new lifemate."

[0653] This system has a reminder function, and the user can instruct the avatar about daily schedules and tasks, which are then managed and notified. When the user asks the avatar to "set a reminder to take my medicine at 9:00 tomorrow," the device sends this request to the server, and the reminder is set. At the specified time, the avatar will notify the user, "It's time to take your medicine now."

[0654] It also works with IoT devices as a health management function, collecting user health data (e.g., heart rate, number of steps, blood pressure, etc.). The collected data is sent to a server where it is analyzed. For example, if a user walks 5,000 steps in a day, the server will generate health advice such as "You walked 5,000 steps today. You're very healthy," and notify the user through an avatar on the device.

[0655] Furthermore, this system has the ability to allow the avatar to learn through continuous dialogue and continuously update its personality. When a user asks a question such as "What's in the news today?", the device sends the question to the server, which analyzes the dialogue data, generates appropriate information, and sends it to the device. This allows the avatar to provide information to the user, such as "Here's today's news." This dialogue data is then learned by the server, allowing the avatar to understand the user's preferences and behavioral patterns and provide more appropriate responses.

[0656] For example, if a user inputs the name "Yamada Taro," the hobby "reading," and sets the appearance as "dog" and the voice as "calm female voice," the server will generate a profile for "Yamada Taro" and set the avatar as "a dog who loves reading and has a laid-back personality." Based on this setting, the avatar will greet the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[0657] Examples of prompts include "Set a reminder to take my medicine at 9:00 tomorrow" and "What's the news today?" This allows the present invention to comprehensively support the daily lives of elderly people, alleviating their feelings of loneliness and improving the efficiency of health management.

[0658] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0659] Step 1:

[0660] The user installs the app on their device and launches it. The user enters personal information (such as name, birthday, gender, hobbies, and preferences) through the app's interface. The input data includes "Name: Yamada Taro," "Birthday: January 1, 1945," and "Hobbies: Reading." This input data is sent from the device to the server. The server receives the input data, converts it to JSON format data, and sends it to the server.

[0661] Step 2:

[0662] The server generates a user profile based on the personal information received from the device. The server stores the data in a personal information database and creates a user profile. This user profile includes the user's name, birthday, gender, hobbies, preferences, etc. The output is the initial setting data for the user profile.

[0663] Step 3:

[0664] The user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, elderly). This customization information is sent from the device to the server. Input data includes, for example, "appearance: dog" and "voice: calm female voice." The device sends the customized information to the server.

[0665] Step 4:

[0666] Based on the customization information received by the server, a generative AI model is used to generate initial avatar settings. Specifically, a natural language processing algorithm is used to generate dialogue data based on the user's hobbies and preferences. The output is a customized and personalized avatar settings setup.

[0667] Step 5:

[0668] The server sends the generated avatar personality data to the device. The device receives this personality data and displays the avatar. Based on the initial setup data, the avatar greets the user for the first time. Specifically, the avatar greets the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[0669] Step 6:

[0670] The user instructs the avatar about daily schedules and tasks. For example, the user might instruct the avatar to "set a reminder to take medicine at 9:00 tomorrow." This instruction is sent from the device to the server. The input data includes "a reminder to take medicine at 9:00 tomorrow."

[0671] Step 7:

[0672] The server sets a reminder based on the schedule data it receives. It saves the data in the reminder setting database and notifies the device that the setting is complete. As an output, it generates a message indicating that the reminder setting is complete and sends it to the device.

[0673] Step 8:

[0674] When the set time arrives, the server sends a reminder notification to the device. The device receives this notification, and an avatar notifies the user, "It's time to take your medicine now." Specifically, the avatar notifies the user by voice.

[0675] Step 9:

[0676] IoT devices collect user health data (e.g., heart rate, step count, blood pressure, etc.) and transmit the collected data to a terminal or server in real time. Input data includes heart rate, step count, and blood pressure data.

[0677] Step 10:

[0678] The server analyzes the received health data and evaluates the user's health condition. For example, if the number of steps is low, it generates advice such as "You should walk a little more." This advice data is sent to the device. The generated health advice is included as an output.

[0679] Step 11:

[0680] The device notifies the user through an avatar based on the health advice received from the server. Specifically, the avatar will say to the user, "You walked 5,000 steps today. That's very healthy."

[0681] Step 12:

[0682] The user asks the avatar a question of everyday life, such as "What's in the news today?" This question is sent from the device to the server. The input data includes the question itself.

[0683] Step 13:

[0684] The server generates appropriate information based on the question data it receives. It extracts relevant news information from the dialogue database and sends it to the terminal. As an output, news information is generated and sent to the terminal.

[0685] Step 14:

[0686] Based on the information received from the server, the device uses an avatar to provide news to the user. Specifically, the avatar will say, "Here's today's news."

[0687] Step 15:

[0688] The server continuously collects user interaction data, and the AI ​​model learns from it. The data is stored in an interaction database and the avatar's personality data is updated. The output is the updated personality data.

[0689] Step 16:

[0690] The device keeps the avatar's settings up to date based on the updated personality data it receives from the server. Specifically, the avatar will ask the user, "How did you like the history book you read recently?"

[0691] (Application example 1)

[0692] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0693] When seniors shop in brick-and-mortar stores, they often face challenges in finding the exact product they need and selecting the right product for their health condition. It's also difficult to quickly respond appropriately in emergencies. These challenges can make the shopping experience stressful for seniors, making it difficult for them to receive optimal health care.

[0694] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0695] In this invention, the server includes a means for generating a customizable avatar based on personal information entered by the user, a means for creating a personality for the generated avatar using an artificial intelligence model, a means for providing reminders and information in response to the user's daily schedule and questions, a means for managing and providing feedback on the user's health status in cooperation with IoT devices, a means for navigating the user in a store and providing product information in cooperation with smart glasses, a means for suggesting products according to the user's health status, and a means for contacting store staff in an emergency. This allows elderly people to smoothly shop in physical stores and receive appropriate health care.

[0696] A "customizable avatar" is a virtual character that can interact with the user and whose appearance, voice, etc. can be freely set based on personal information entered by the user.

[0697] An "artificial intelligence model" is a technology that uses machine learning and natural language processing to generate appropriate responses and actions in response to user interactions and instructions.

[0698] "Reminder" is a function that notifies the user of scheduled events or tasks at a pre-set time.

[0699] An "IoT device" is a device that has the ability to collect, analyze, and transmit data via the Internet. Examples include health management devices and smart home devices.

[0700] "Smart glasses" are a wearable eyeglass-type device that can display augmented reality and information when worn by the user.

[0701] "Health Status Management" is a system that continuously collects and analyzes users' health data and provides appropriate feedback and advice.

[0702] "In-store navigation" is a function that guides users to the location of the product they are looking for within a physical store.

[0703] "Product Information Provision" is a function that provides users with detailed information about a specified product and related suggestions.

[0704] "Product suggestions based on health status" is a function that recommends products suitable for a user's condition based on their health data.

[0705] "Emergency Contact" is a function that quickly sends alerts to store staff and relevant institutions if there is any change in the user's health condition.

[0706] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. The system also works in conjunction with smart glasses to support shopping in brick-and-mortar stores.

[0707] System configuration

[0708] 1. User registration and avatar customization

[0709] Device: The user installs a dedicated app and enters personal information such as name, birthday, gender, hobbies, and preferences. Next, the user customizes the avatar's appearance (e.g., human or animal) and voice (e.g., male or female). This information is sent to the server.

[0710] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[0711] 2. Avatar personality development

[0712] Server: Based on the user's personal information and input data, a generative AI model is used to create an avatar personality. This model uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[0713] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[0714] 3. Daily life support

[0715] Reminder function:

[0716] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[0717] Device: Sends the requested information to the server and sets the reminder.

[0718] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[0719] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[0720] Health management function:

[0721] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[0722] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[0723] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0724] 4. Shopping support in physical stores

[0725] Shopping Navigation:

[0726] Smart glasses: Navigate users through physical stores and guide them to the location of the product they are looking for. For example, an avatar will guide them, saying, "Milk is in Aisle 3."

[0727] Product information provided:

[0728] Server: Generates detailed product information and related suggestions and sends them to the smart glasses.

[0729] Smart glasses: Avatars present product information to users.

[0730] Product suggestions based on health condition:

[0731] Server: Based on the user's health data, the server recommends suitable products. For example, it generates advice such as "Choose low-salt products."

[0732] Smart glasses: An avatar suggests products based on your health condition.

[0733] 5. Emergency Response

[0734] Smart glasses: Send emergency alerts to store staff if there are any abnormalities in the user's health.

[0735] Server: Records the abnormality in the emergency response database and prompts emergency response.

[0736] Specific examples

[0737] 1. Shopping Navigation:

[0738] User: Asks smart glasses, "Where is the milk?"

[0739] Smart glasses: The avatar navigates by saying "Milk is in Aisle 3."

[0740] 2. Product recommendations based on health conditions:

[0741] User: Asks smart glasses, "What healthy food would you like to eat today?"

[0742] Smart glasses: An avatar suggests, "Choose low-salt products."

[0743] Prompt Sentence Examples

[0744] User: "Where's the milk?"

[0745] Avatar: "Milk is in Aisle 3."

[0746] This system allows elderly people to shop smoothly in brick-and-mortar stores and receive appropriate health care.

[0747] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0748] Step 1:

[0749] User registration and avatar customization

[0750] Input: The user enters personal information such as name, birthday, gender, hobbies, and preferences into the device.

[0751] How it works: The user launches the app and enters the required information into a form.

[0752] Data processing: The terminal sends the entered personal information to the server.

[0753] Output: User personal information and customization information is stored on the server.

[0754] Step 2:

[0755] Avatar initial settings

[0756] Input: Your personal and customization information stored on our servers.

[0757] How it works: The server uses this information to generate a user profile and create initial avatar data.

[0758] Data calculation: Generate initial setting data for the avatar using a generative AI model.

[0759] Output: Initialization data is sent to the terminal.

[0760] Step 3:

[0761] Avatar personality development

[0762] Input: Initial setting data for the server-generated avatar.

[0763] How it works: The device configures an avatar based on the data it receives and uses a generative AI model to create the avatar's personality.

[0764] Data calculation: Using a natural language processing algorithm, the avatar's initial setting data is analyzed and personality data is generated.

[0765] Output: The formed personality data is saved on the device.

[0766] Step 4:

[0767] Reminder function

[0768] Input: The user tells the device about their daily schedule and tasks.

[0769] How it works: The user sets a reminder through the app and sends the information to the server.

[0770] Data processing: The server stores the information in the reminder setting database and sends a notification to the device at the specified time.

[0771] Output: The device will notify the user of the reminder via the avatar.

[0772] Step 5:

[0773] Health management function

[0774] Input: User health data collected by IoT devices.

[0775] How it works: The server stores data received from IoT devices and analyzes their health status.

[0776] Data calculation: Generates advice to users based on the results of health data analysis.

[0777] Output: The device provides health advice via an avatar.

[0778] Step 6:

[0779] Shopping support in physical stores - Navigation function

[0780] Input: The user inputs information about the product they are looking for through the smart glasses.

[0781] How it works: The smart glasses send the input information to the server.

[0782] Data calculation: The server references the store database and obtains product location information.

[0783] Output: The smart glasses guide the user to the product location via an avatar.

[0784] Step 7:

[0785] Shopping support in physical stores - Product information provision

[0786] Input: Server-generated product details.

[0787] How it works: The server sends detailed product information and related suggestions to the smart glasses.

[0788] Data processing: Generative AI models select appropriate product information for users.

[0789] Output: The smart glasses present product information to the user via an avatar.

[0790] Step 8:

[0791] Product suggestions based on health status

[0792] Input: User health data and related product information.

[0793] How it works: The server analyzes health data and suggests products suitable for the user.

[0794] Data calculation: Generate optimal product recommendations based on health status data.

[0795] Output: The smart glasses display product suggestions to the user through an avatar.

[0796] Step 9:

[0797] Emergency response

[0798] Input: Anomalous health data detected by IoT devices.

[0799] How it works: The smart glasses send anomaly data to the server.

[0800] Data calculation: The server refers to the emergency response database and generates the necessary response measures.

[0801] Output: An emergency notification is sent to store staff and an avatar informs the user about emergency procedures.

[0802] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0803] This invention is a system that generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized by linking terminals, servers, and IoT devices.

[0804] System configuration

[0805] 1. User registration and avatar customization

[0806] Device: The user installs the application and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[0807] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[0808] 2. Avatar personality development

[0809] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[0810] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[0811] 3. Introducing the Emotion Engine

[0812] On-device: The device is equipped with an emotion engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected.

[0813] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[0814] 4. Daily life support

[0815] Reminder function:

[0816] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[0817] Device: Sends the requested information to the server and sets the reminder.

[0818] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[0819] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0820] Health management function:

[0821] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[0822] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[0823] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0824] 5. Continuous interaction and learning

[0825] Interactive features:

[0826] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[0827] Terminal: Sends the user's questions and conversation content to the server.

[0828] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[0829] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[0830] Learning features:

[0831] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0832] Device: Receive updated personality data and keep your avatar settings up to date.

[0833] Specific examples

[0834] 1. User Registration and Customization:

[0835] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[0836] Server: Stores user information, generates initial avatar data, and sends it to the device.

[0837] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[0838] 2. Reminders and Health Management:

[0839] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[0840] Terminal: Sends a request to the server.

[0841] Server: Set reminders and send confirmation messages to your device.

[0842] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[0843] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[0844] 3. Leveraging Emotion Recognition:

[0845] User: After eating, tell your avatar, "I'm feeling a bit tired today."

[0846] Device: The emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[0847] Server: Based on the emotional data, adjust the content and tone of the avatar's response. "I understand. Please get plenty of rest today," the avatar says in a comforting voice.

[0848] On the device: The avatar provides tailored responses to the user, providing emotionally sensitive support.

[0849] In this way, by combining the emotion engine, the system of the present invention not only alleviates the sense of loneliness felt by the elderly, but also provides emotionally sensitive responses, allowing users to experience richer interactions and receive assistance in their daily lives.

[0850] The processing flow will be explained below.

[0851] Step 1:

[0852] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[0853] Step 2:

[0854] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[0855] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[0856] Step 3:

[0857] On the device: Users are presented with options for avatar appearance (human, animal, etc.) and voice (male, female, young, adult) and can customize their avatar.

[0858] User: Choose your avatar's appearance and voice to complete the customization.

[0859] Step 4:

[0860] Terminal: Sends the information entered and selected by the user to the server.

[0861] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[0862] Step 5:

[0863] Server: Based on the user's personal information and customization information, an AI model is used to create an avatar personality, and the created personality data is sent to the device.

[0864] Step 6:

[0865] Device: Based on the received avatar personality data, the avatar is initialized. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[0866] Step 7:

[0867] User: Enters daily schedules and tasks (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?") and gives instructions to the avatar.

[0868] Terminal: Sends input requests to the server.

[0869] Step 8:

[0870] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[0871] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0872] Step 9:

[0873] IoT devices: Collect user health data (heart rate, steps, etc.).

[0874] Server: Periodically receives data collected from IoT devices and analyzes health status. Sends the analysis results to the device. For example, if the number of steps is low, the avatar will advise the user, "You should walk a little more."

[0875] Step 10:

[0876] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[0877] Step 11:

[0878] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[0879] Terminal: Sends the user's query to the server.

[0880] Step 12:

[0881] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[0882] Device: The avatar explains to the user, "Today's weather is sunny. The temperature is 25 degrees."

[0883] Step 13:

[0884] Device: The emotion engine analyzes the user's voice and facial expressions. For example, if a user says, "I'm a little tired today," the voice tone and facial expression data are collected.

[0885] Step 14:

[0886] Server: Adjusts the avatar's response content and tone based on the emotional data analyzed by the emotion engine. "I understand. Please get plenty of rest today," the avatar says comfortingly.

[0887] Step 15:

[0888] Device: The avatar provides tailored responses to the user, providing appropriate feedback based on emotional data and providing support that is sensitive to the user's emotions.

[0889] Step 16:

[0890] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an artificial intelligence model.

[0891] Device: Receives the latest personality data and updates avatar settings.

[0892] Example 2

[0893] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] There is a need for an effective system that can support the daily lives of elderly people while alleviating their feelings of loneliness. Conventional support systems are limited to one-way information provision or simple reminder functions, and do not adequately address the emotions and individual needs of users. This has led to issues such as limited user satisfaction and utilization effectiveness.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0896] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating the personality of the generated avatar using an artificial intelligence model; means for analyzing the user's voice and facial expressions to acquire emotional data; means for adjusting the avatar's response content and tone based on the emotional data; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT sensors; and means for the avatar to learn and update its personality through continuous dialogue with the user. This makes it possible to respond to the user's emotions and individual needs and provide comprehensive support for the elderly in their daily lives.

[0897] "User" refers to the individual who uses the system, who is often a senior citizen.

[0898] "Personal information" refers to information that identifies a user, such as the user's name, birthday, gender, hobbies, and preferences.

[0899] A "customizable avatar" is a virtual character whose appearance and voice can be changed based on information entered by the user.

[0900] "Artificial intelligence model" refers to the algorithms and machine learning models used to generate avatar personalities and responses based on user input data.

[0901] "Emotional data" is data that indicates the emotional state of a user, obtained by analyzing the user's voice and facial expressions.

[0902] An "IoT sensor" is a device that is connected to the Internet and collects data on a user's health and environment.

[0903] "Health Guidance" means advice or suggestions to improve a user's health.

[0904] "Reminder" is a feature that notifies users of their schedules and tasks on time.

[0905] "Dialogue" refers to verbal exchanges between a user and an avatar.

[0906] "Continuous interaction" refers to interaction between a user and an avatar that occurs regularly over an extended period of time.

[0907] "Personality updating" refers to the process of improving an avatar's responses and behavioral patterns based on interaction data and emotional data with the user.

[0908] MODE FOR CARRYING OUT THE INVENTION

[0909] This system generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized through collaboration between terminals, servers, and IoT devices.

[0910] System Configuration

[0911] 1. User registration and avatar customization

[0912] Device: The user installs the application and enters the necessary personal information (such as name, birthday, gender, and hobbies). The user then customizes the avatar's appearance and voice. Specific hardware examples include smartphones and tablets.

[0913] Server: Receives the entered personal information and customization information and generates a user profile. This is done using a database server and an AI server. Furthermore, based on this information, it generates initial setting data for the avatar and sends it to the device.

[0914] Examples:

[0915] The user enters their name "Taro Tanaka," their date of birth "March 15, 1950," their hobby "travel," and sets their avatar's appearance to "cat" and voice to "young man's voice."

[0916] 2. Avatar personality development

[0917] Server: Based on the user's personal information and input data, an artificial intelligence model (e.g., GPT-4) is used to create an avatar personality. Specific software examples include a machine learning platform and a natural language processing engine. The created personality data is sent to the device.

[0918] Device: Initial settings are made based on the received avatar personality data. For example, the avatar may say, "Hello, I'm your new lifemate."

[0919] Examples:

[0920] The avatar greets, "Hello, Tanaka-san. How was your day?"

[0921] 3. Introducing the Emotion Engine

[0922] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions. Examples of specifications include a camera-equipped device and a voice recognition microphone.

[0923] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[0924] Examples:

[0925] The user says, "I'm feeling a bit tired today." The emotion engine interprets this and the avatar responds, "That must be tough. Why don't you take a break?"

[0926] 4. Daily life support

[0927] Reminder function:

[0928] User: Instructs the avatar on daily schedules and tasks.

[0929] Server: Stores the information in the reminder setting database and sends a notification to the device at the specified time.

[0930] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[0931] Health management function:

[0932] IoT devices: Collect user health data (e.g., heart rate, steps, blood pressure, etc.). Specific examples of such devices include fitness bands and smartwatches.

[0933] Server: Receives and analyzes the collected data. For example, it generates advice such as "If your step count is low, you should walk a little more."

[0934] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[0935] Examples:

[0936] If a user requests, "Set a reminder to take my medicine at 9:00 tomorrow," the server will set this reminder and the device will notify them at the specified time.

[0937] 5. Continuous interaction and learning

[0938] Interactive features:

[0939] Users: Ask the avatar everyday topics and questions.

[0940] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[0941] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[0942] Learning features:

[0943] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[0944] Device: Receive updated personality data and keep your avatar settings up to date.

[0945] Example prompt sentence:

[0946] "What news is there today?"

[0947] "Tell me about the weather lately."

[0948] "When's your next doctor's appointment?"

[0949] In this way, this system generates and operates customizable avatars based on personal information and emotional data entered by the user, providing comprehensive support for the user's daily life.

[0950] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0951] Step 1:

[0952] User Registration and Customization

[0953] Input: Users install applications on their devices and enter personal information such as name, birthday, gender, hobbies, etc. They also customize the appearance and voice of their avatar.

[0954] Operation: The device sends the entered personal information and customization information to the server. Specifically, the user fills in the information in the input form and presses the send button.

[0955] Output: The server generates a user profile based on the received data and creates initial avatar data. This initial data is then sent to the device.

[0956] Step 2:

[0957] Avatar personality development

[0958] Input: The server receives the user's personal information and customization data and inputs it into an artificial intelligence model (e.g., GPT-4).

[0959] How it works: The server uses an artificial intelligence model to generate the avatar's personality, passing the incoming data to a natural language processing engine and processing the output from the model.

[0960] Output: The generated avatar's personality data is sent to the device. The device configures the avatar based on the received data and displays it on the interface.

[0961] Step 3:

[0962] Completing the initial setup and first greetings

[0963] Input: The device performs initial settings based on the avatar's personality data received from the server.

[0964] How it works: The device applies the avatar's appearance, voice, and personality data to display the avatar for the first time. The avatar then greets the user for the first time.

[0965] Output: The avatar will start a customized interaction with the user, such as "Hello, I'm your new lifemate."

[0966] Step 4:

[0967] Emotion engine analysis and response adjustment

[0968] Input: The device collects the user's voice and facial expressions through the camera and microphone.

[0969] How it works: The emotion engine analyzes the collected data, extracts emotion data, and sends this data to the server.

[0970] Output: The server adjusts the avatar's response content and tone based on the emotional data and sends the data to the device. The avatar on the device then provides the adjusted response to the user.

[0971] Step 5:

[0972] Reminder settings and notifications

[0973] Input: The user gives the avatar instructions for daily schedules and tasks, such as "Set a reminder to take my medicine tomorrow at 9 AM."

[0974] How it works: The device sends instructions from the user to the server, which stores the information in the reminder settings database. When the specified time arrives, the server sends the notification data to the device.

[0975] Output: The device receives notification data from the server, and the avatar notifies the user, such as "It's time to take your medicine now."

[0976] Step 6:

[0977] Health management features for data collection and feedback

[0978] Input: IoT devices collect user health data (e.g., heart rate, steps, blood pressure, etc.).

[0979] How it works: The IoT device sends the collected data to a server, which analyzes the data and generates advice such as "You should walk a little more" if the number of steps is low.

[0980] Output: The analysis results are sent to the device, and the avatar tells the user, "You walked 5,000 steps today. You're very healthy."

[0981] Step 7:

[0982] Continuous interaction and learning

[0983] Input: The user asks the avatar everyday topics and questions, such as "What's in the news today?"

[0984] How it works: The device sends the user's question data to the server, which then generates appropriate information based on the received data and sends it to the device. Furthermore, the server continuously collects conversation data and emotional data with the user, and updates the avatar's personality using an artificial intelligence model.

[0985] Output: The device receives information from the server, the avatar tells the user "Here's today's news" and provides the appropriate information, and keeps the avatar's settings up to date.

[0986] As described above, the program of the system of the present invention provides a function to support the daily lives of elderly people and alleviate their feelings of loneliness through a series of processing steps.

[0987] (Application example 2)

[0988] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0989] To support the daily lives of elderly people while alleviating their feelings of loneliness, simple reminder and health management functions are insufficient. Support tailored to individual needs through continuous interaction with the elderly and flexible response to emotional changes is necessary. However, current systems lack the ability to read the user's unique emotions and respond appropriately based on them, and therefore are unable to provide sufficient psychological satisfaction and security to the elderly.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0991] In this invention, the server includes means for generating a customizable virtual character based on personal information entered by the user, means for forming the personality of the generated virtual character using an artificial intelligence model, and means for adjusting the content and tone of the virtual character's responses based on emotional data from an emotion analysis engine. This enables personalized responses that are sensitive to the emotions of the elderly, providing them with a greater sense of satisfaction and security.

[0992] "user"

[0993] refers to individuals who use the system, often specifically elderly people.

[0994] "personal information"

[0995] This refers to data entered by the user to identify the user, such as name, birthday, gender, hobbies, and preferences.

[0996] "Virtual Character"

[0997] is a digital character that is customized based on a user's personal information, with visual and audio elements including appearance and voice.

[0998] "Artificial Intelligence Model"

[0999] refers to algorithms and learning models that use user input data to shape the personality of a virtual character.

[1000] Emotion analysis engine

[1001] This refers to a system that analyzes a user's voice, facial expressions, etc. to detect their emotional state.

[1002] Reminders

[1003] This refers to a function that notifies users of scheduled events and tasks in a timely manner.

[1004] "IoT devices"

[1005] refers to a physical device that is connected to the Internet and is capable of sending and receiving sensor data, including wearable devices for health management.

[1006] "Health status"

[1007] refers to biometric data such as a user's heart rate, blood pressure, number of steps, etc. These data are used to monitor the user's health.

[1008] "Response content"

[1009] refers to the responses that a virtual character provides based on the user's questions and emotional state, including information and advice.

[1010] "tone"

[1011] This refers to the tone of voice and speaking style of a virtual character, which is adjusted by an emotion analysis engine, depending on the user's emotional state.

[1012] This invention provides a system that supports the daily lives of elderly people and alleviates their feelings of loneliness by generating a customizable virtual character based on the user's personal information and combining it with an emotion analysis engine. This system is composed of a terminal, a server, and an IoT device. The details of this system are described below.

[1013] System configuration

[1014] 1. User registration and virtual character customization

[1015] The terminal provides an interface where the user installs the application and inputs necessary personal information (such as name, birthday, gender, hobbies, and preferences). The user can then customize the virtual character's appearance (human, animal, etc.) and voice (male, female, young, adult). The server receives the input personal information and customization information and generates a user profile. Based on this information, the server then generates initial setting data for the virtual character and sends it to the terminal.

[1016] 2. Personality development of virtual characters

[1017] The server uses a generative AI model to create a virtual character's personality based on the user's personal information and input data. This uses natural language processing and machine learning algorithms. The created personality data is then sent to the device. The device then performs initial setup based on the received virtual character's personality data and greets the user for the first time.

[1018] 3. Introduction of a sentiment analysis engine

[1019] The device is equipped with an emotion analysis engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected. The server receives the emotional data analyzed by the emotion analysis engine and adjusts the content and tone of the virtual character's responses.

[1020] 4. Daily life support

[1021] The device can instruct the virtual character about the user's daily schedule and tasks (such as doctor's appointments and shopping lists). It also provides reminder setting and health management functions. The server receives the information specified by the user, stores it in a reminder setting database, and sends notifications to the device at the specified time. It also analyzes health data (heart rate, number of steps, blood pressure, etc.) collected from IoT devices and generates health advice.

[1022] 5. Continuous interaction and learning

[1023] The device has a dialogue function that allows users to ask the virtual character everyday topics and questions. The server generates appropriate information based on the dialogue data received and sends it to the device. It also continuously collects dialogue data and emotional data and updates the virtual character's personality using a generative AI model. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1024] Specific examples

[1025] User Registration and Customization

[1026] The user launches the app on the device, inputs their name "General Taro," their date of birth "January 1, 1945," and their hobby "reading," selects an "animal" as the virtual character's appearance, and sets the voice to a "calm female voice." The server saves the user information, generates initial setting data for the virtual character, and sends it to the device. The virtual character greets the device by saying, "Hello, Taro. I'm your new friend."

[1027] Utilizing Emotion Recognition

[1028] When a user says, "I feel a little tired today," the device uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data. The server then adjusts the content and tone of the virtual character's response based on the emotional data, and the virtual character then comforts the user by saying, "I understand. Please take it easy today."

[1029] Example prompts to input to a generative AI model:

[1030] Username: General Taro

[1031] User input: Set a reminder to take my medicine tomorrow at 9am

[1032] Emotion engine: Analyzes user input and extracts emotional data (e.g., "I'm tired")

[1033] Virtual character response: Sets a reminder and responds to the user (e.g., "I've set a reminder to take my medicine tomorrow at 9 o'clock.")

[1034] In this way, the system of the present invention can respond to the user's emotions and individual needs, and can support the elderly in their daily lives while alleviating their sense of loneliness.

[1035] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1036] Step 1:

[1037] The user inputs personal information (such as name, birthday, gender, hobbies, and preferences) through the device. The device acquires this information and sends it to the server. The personal information is the input data, and the data sent to the server is the output data.

[1038] Step 2:

[1039] The server receives the personal information sent by the user and generates a user profile. The server stores the personal information in a database and generates initial setting data for the virtual character. This initial setting data includes customization information such as the virtual character's appearance and voice. The input data is personal information, and the output is the initial setting data, which is then sent to the terminal.

[1040] Step 3:

[1041] The device displays the virtual character and greets the user based on the initial setting data of the virtual character received from the server. Specifically, it plays a voice and text such as "Hello, Taro. I'm your new friend." The initial setting data is the input data, and the greeting is displayed as the output.

[1042] Step 4:

[1043] The user inputs daily plans and questions to the virtual character. For example, they can give instructions such as "Set a reminder to take my medicine at 9:00 tomorrow." The device acquires the user's input and sends it to the server. The input data includes the user's instructions, and the output data is sent to the server.

[1044] Step 5:

[1045] The server receives instructions sent by the user, analyzes them, and sets reminders. Specifically, it saves the information in a reminder setting database and prepares to send a reminder notification to the device at the specified time. The input data is the user's instructions, and the output is reminder setting data.

[1046] Step 6:

[1047] The device will notify the user of the reminder at the specified time based on the reminder setting data received from the server. Specifically, it will display a voice and text message saying, "It's time to take your medicine now." The input data is the reminder setting data, and the output is a reminder notification.

[1048] Step 7:

[1049] To analyze a user's emotions, the device analyzes the user's voice and facial expressions using an emotion analysis engine. The emotion analysis engine receives the user's voice and facial expression data and detects their emotional state (for example, "tired" or "energetic"). The input data is the user's voice and facial expression, and the output is emotion data.

[1050] Step 8:

[1051] The server adjusts the content and tone of the virtual character's responses based on the emotional data. For example, if a user says, "I'm a little tired today," the server generates an adjusted response, such as "I understand. Please take it easy today," and sends it to the device. Emotional data is the input data, and adjusted response data is generated as the output.

[1052] Step 9:

[1053] The device receives the adjusted response data from the server, and the virtual character responds to the user with voice and text. Specifically, the virtual character responds to the user by saying, "I understand. Please take a good rest today." The adjusted response data is input, and the response is displayed as output.

[1054] Step 10:

[1055] The server continuously collects user interaction and emotional data and uses a generative AI model to update the virtual character's personality. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses. The input data includes interaction data and emotional data, and the output is updated personality data.

[1056] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1057] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1058] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1059] [Third embodiment]

[1060] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1061] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1062] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1063] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1064] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1065] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1066] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1067] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1068] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1069] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1070] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1071] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1072] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. This system is realized by linking terminals, servers, and IoT devices.

[1073] System configuration

[1074] 1. User registration and avatar customization

[1075] On device: The user installs the app and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, old).

[1076] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[1077] 2. Avatar personality development

[1078] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[1079] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[1080] 3. Daily life support

[1081] Reminder function:

[1082] User: Instructs the avatar about daily schedules and tasks (such as doctor's appointments and shopping lists).

[1083] Device: Sends the requested information to the server and sets the reminder.

[1084] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[1085] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[1086] Health management function:

[1087] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[1088] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[1089] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1090] 4. Continuous interaction and learning

[1091] Interactive features:

[1092] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[1093] Terminal: Sends the user's questions and conversation content to the server.

[1094] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[1095] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[1096] Learning features:

[1097] Server: Continuously collects user interaction data and trains the AI ​​model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1098] Device: Receive updated personality data and keep your avatar settings up to date.

[1099] Specific examples

[1100] 1. User Registration and Customization:

[1101] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[1102] Server: Stores user information, generates initial avatar data, and sends it to the device.

[1103] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[1104] 2. Reminders and Health Management:

[1105] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[1106] Terminal: Sends a request to the server.

[1107] Server: Set reminders and send confirmation messages to your device.

[1108] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[1109] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[1110] In this way, the system of the present invention not only alleviates the sense of loneliness of the elderly, but also provides support in various aspects of daily life. Through continuous learning and updates, it is possible to provide optimal assistance to users.

[1111] The processing flow will be explained below.

[1112] Step 1:

[1113] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[1114] Step 2:

[1115] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[1116] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[1117] Step 3:

[1118] Device: Presents the user with options for the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[1119] User: Choose your avatar's appearance and voice to complete the customization.

[1120] Step 4:

[1121] Terminal: Sends the information entered and selected by the user to the server.

[1122] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[1123] Step 5:

[1124] Server: Based on the user's personal information and customization information, an artificial intelligence model is used to create an avatar's personality.

[1125] Server: Sends the created personality data to the terminal.

[1126] Step 6:

[1127] Device: Based on the received avatar personality data, the initial avatar settings are performed. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[1128] Step 7:

[1129] User: Enter daily schedules and tasks into the avatar (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?").

[1130] Terminal: Sends input requests to the server.

[1131] Step 8:

[1132] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[1133] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1134] Step 9:

[1135] IoT devices: Collect user health data (heart rate, steps, etc.).

[1136] Server: Periodically receives data collected from IoT devices and analyzes their health status.

[1137] Server: Sends the analysis results to the device and generates appropriate feedback and advice.

[1138] Step 10:

[1139] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[1140] Step 11:

[1141] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[1142] Terminal: Sends the user's query to the server.

[1143] Step 12:

[1144] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[1145] Device: The avatar tells the user, "Today's weather is sunny. The temperature is 25 degrees."

[1146] Step 13:

[1147] Server: Continuously collects user interaction data and updates the avatar's personality using an artificial intelligence model.

[1148] Server: Sends updated personality data to devices, keeping avatar settings up to date.

[1149] Example 1

[1150] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1151] This invention relates to a system for supporting the daily lives of elderly people. Elderly people often experience feelings of loneliness and difficulty managing their health, and therefore support is needed to solve these problems and enrich their daily lives. While existing technologies exist for avatar systems that interact with users, they face challenges in providing optimal support tailored to the user's behavior and preferences through continuous learning and customization based on personal information.

[1152] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1153] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating a personality for the generated avatar using an artificial intelligence model; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT devices; means for the avatar to learn and update its personality through continuous dialogue with the user; means for the avatar to understand the content of the dialogue using a natural language processing algorithm based on the user's personal information and daily dialogue data and generate appropriate responses; and means for the server to analyze data collected from the user and learn the user's behavioral patterns and preferences. This provides comprehensive support for the daily lives of the elderly, alleviating feelings of loneliness, and improving the efficiency of health management.

[1154] "User" refers to an individual or elderly person who uses the system and receives assistance in daily life through interaction with an avatar.

[1155] "Personal Information" refers to information provided by users, including data such as name, birth date, gender, hobbies, and preferences.

[1156] A "customizable avatar" refers to a pseudo-personality whose appearance, voice, etc. can be set based on personal information entered by the user.

[1157] "Artificial intelligence model" refers to the machine learning algorithms and natural language processing technologies used to create the avatar's personality and generate responses.

[1158] "Reminders" refers to a feature that notifies users of their daily schedules and tasks.

[1159] An "IoT device" refers to a sensor device that is connected to the Internet and collects user health data, etc.

[1160] "Health status" refers to a user's physical data, such as heart rate, steps taken, and blood pressure.

[1161] "Feedback" refers to the act of providing advice or notifications to users based on collected data.

[1162] "Continuous dialogue" refers to long-term interaction between the user and the avatar.

[1163] "Natural language processing algorithms" refer to technologies that understand user speech or text input and generate appropriate responses.

[1164] "Behavioral patterns" refer to the user's daily life tendencies and habits.

[1165] "Preferences" refers to a user's hobbies and tendencies.

[1166] This is a system designed to support the daily lives of elderly people. It generates a customizable avatar based on personal information entered by the user, and supports the user's daily life through interaction with this avatar. This system is realized mainly by terminals, servers, and IoT devices.

[1167] First, the user installs and launches the application on their device. This application provides an interface for entering personal information (e.g., name, birthday, gender, hobbies, preferences, etc.). Data such as name, birthday, and hobbies are entered into each input field.

[1168] Next, the user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, old), and this customization information is also sent from the device to the server.

[1169] The server generates a user profile based on the received personal information and customization information. It then uses an artificial intelligence model (generative AI model) to generate a personality for the avatar. This personality data is also sent from the server to the device. Specifically, it uses a natural language processing algorithm to generate dialogue based on the user's interests and behavioral patterns.

[1170] The generated avatar will be displayed on the device and greet the user for the first time. For example, the avatar may say to the user, "Hello, I'm your new lifemate."

[1171] This system has a reminder function, and the user can instruct the avatar about daily schedules and tasks, which are then managed and notified. When the user asks the avatar to "set a reminder to take my medicine at 9:00 tomorrow," the device sends this request to the server, and the reminder is set. At the specified time, the avatar will notify the user, "It's time to take your medicine now."

[1172] It also works with IoT devices as a health management function, collecting user health data (e.g., heart rate, number of steps, blood pressure, etc.). The collected data is sent to a server where it is analyzed. For example, if a user walks 5,000 steps in a day, the server will generate health advice such as "You walked 5,000 steps today. You're very healthy," and notify the user through an avatar on the device.

[1173] Furthermore, this system has the ability to allow the avatar to learn through continuous dialogue and continuously update its personality. When a user asks a question such as "What's in the news today?", the device sends the question to the server, which analyzes the dialogue data, generates appropriate information, and sends it to the device. This allows the avatar to provide information to the user, such as "Here's today's news." This dialogue data is then learned by the server, allowing the avatar to understand the user's preferences and behavioral patterns and provide more appropriate responses.

[1174] For example, if a user inputs the name "Yamada Taro," the hobby "reading," and sets the appearance as "dog" and the voice as "calm female voice," the server will generate a profile for "Yamada Taro" and set the avatar as "a dog who loves reading and has a laid-back personality." Based on this setting, the avatar will greet the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[1175] Examples of prompts include "Set a reminder to take my medicine at 9:00 tomorrow" and "What's the news today?" This allows the present invention to comprehensively support the daily lives of elderly people, alleviating their feelings of loneliness and improving the efficiency of health management.

[1176] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1177] Step 1:

[1178] The user installs the app on their device and launches it. The user enters personal information (such as name, birthday, gender, hobbies, and preferences) through the app's interface. The input data includes "Name: Yamada Taro," "Birthday: January 1, 1945," and "Hobbies: Reading." This input data is sent from the device to the server. The server receives the input data, converts it to JSON format data, and sends it to the server.

[1179] Step 2:

[1180] The server generates a user profile based on the personal information received from the device. The server stores the data in a personal information database and creates a user profile. This user profile includes the user's name, birthday, gender, hobbies, preferences, etc. The output is the initial setting data for the user profile.

[1181] Step 3:

[1182] The user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, elderly). This customization information is sent from the device to the server. Input data includes, for example, "appearance: dog" and "voice: calm female voice." The device sends the customized information to the server.

[1183] Step 4:

[1184] Based on the customization information received by the server, a generative AI model is used to generate initial avatar settings. Specifically, a natural language processing algorithm is used to generate dialogue data based on the user's hobbies and preferences. The output is a customized and personalized avatar settings setup.

[1185] Step 5:

[1186] The server sends the generated avatar personality data to the device. The device receives this personality data and displays the avatar. Based on the initial setup data, the avatar greets the user for the first time. Specifically, the avatar greets the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[1187] Step 6:

[1188] The user instructs the avatar about daily schedules and tasks. For example, the user might instruct the avatar to "set a reminder to take medicine at 9:00 tomorrow." This instruction is sent from the device to the server. The input data includes "a reminder to take medicine at 9:00 tomorrow."

[1189] Step 7:

[1190] The server sets a reminder based on the schedule data it receives. It saves the data in the reminder setting database and notifies the device that the setting is complete. As an output, it generates a message indicating that the reminder setting is complete and sends it to the device.

[1191] Step 8:

[1192] When the set time arrives, the server sends a reminder notification to the device. The device receives this notification, and an avatar notifies the user, "It's time to take your medicine now." Specifically, the avatar notifies the user by voice.

[1193] Step 9:

[1194] IoT devices collect user health data (e.g., heart rate, step count, blood pressure, etc.) and transmit the collected data to a terminal or server in real time. Input data includes heart rate, step count, and blood pressure data.

[1195] Step 10:

[1196] The server analyzes the received health data and evaluates the user's health condition. For example, if the number of steps is low, it generates advice such as "You should walk a little more." This advice data is sent to the device. The generated health advice is included as an output.

[1197] Step 11:

[1198] The device notifies the user through an avatar based on the health advice received from the server. Specifically, the avatar will say to the user, "You walked 5,000 steps today. That's very healthy."

[1199] Step 12:

[1200] The user asks the avatar a question of everyday life, such as "What's in the news today?" This question is sent from the device to the server. The input data includes the question itself.

[1201] Step 13:

[1202] The server generates appropriate information based on the question data it receives. It extracts relevant news information from the dialogue database and sends it to the terminal. As an output, news information is generated and sent to the terminal.

[1203] Step 14:

[1204] Based on the information received from the server, the device uses an avatar to provide news to the user. Specifically, the avatar will say, "Here's today's news."

[1205] Step 15:

[1206] The server continuously collects user interaction data, and the AI ​​model learns from it. The data is stored in an interaction database and the avatar's personality data is updated. The output is the updated personality data.

[1207] Step 16:

[1208] The device keeps the avatar's settings up to date based on the updated personality data it receives from the server. Specifically, the avatar will ask the user, "How did you like the history book you read recently?"

[1209] (Application example 1)

[1210] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1211] When seniors shop in brick-and-mortar stores, they often face challenges in finding the exact product they need and selecting the right product for their health condition. It's also difficult to quickly respond appropriately in emergencies. These challenges can make the shopping experience stressful for seniors, making it difficult for them to receive optimal health care.

[1212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1213] In this invention, the server includes a means for generating a customizable avatar based on personal information entered by the user, a means for creating a personality for the generated avatar using an artificial intelligence model, a means for providing reminders and information in response to the user's daily schedule and questions, a means for managing and providing feedback on the user's health status in cooperation with IoT devices, a means for navigating the user in a store and providing product information in cooperation with smart glasses, a means for suggesting products according to the user's health status, and a means for contacting store staff in an emergency. This allows elderly people to smoothly shop in physical stores and receive appropriate health care.

[1214] A "customizable avatar" is a virtual character that can interact with the user and whose appearance, voice, etc. can be freely set based on personal information entered by the user.

[1215] An "artificial intelligence model" is a technology that uses machine learning and natural language processing to generate appropriate responses and actions in response to user interactions and instructions.

[1216] "Reminder" is a function that notifies the user of scheduled events or tasks at a pre-set time.

[1217] An "IoT device" is a device that has the ability to collect, analyze, and transmit data via the Internet. Examples include health management devices and smart home devices.

[1218] "Smart glasses" are a wearable eyeglass-type device that can display augmented reality and information when worn by the user.

[1219] "Health Status Management" is a system that continuously collects and analyzes users' health data and provides appropriate feedback and advice.

[1220] "In-store navigation" is a function that guides users to the location of the product they are looking for within a physical store.

[1221] "Product Information Provision" is a function that provides users with detailed information about a specified product and related suggestions.

[1222] "Product suggestions based on health status" is a function that recommends products suitable for a user's condition based on their health data.

[1223] "Emergency Contact" is a function that quickly sends alerts to store staff and relevant institutions if there is any change in the user's health condition.

[1224] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. The system also works in conjunction with smart glasses to support shopping in brick-and-mortar stores.

[1225] System configuration

[1226] 1. User registration and avatar customization

[1227] Device: The user installs a dedicated app and enters personal information such as name, birthday, gender, hobbies, and preferences. Next, the user customizes the avatar's appearance (e.g., human or animal) and voice (e.g., male or female). This information is sent to the server.

[1228] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[1229] 2. Avatar personality development

[1230] Server: Based on the user's personal information and input data, a generative AI model is used to create an avatar personality. This model uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[1231] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[1232] 3. Daily life support

[1233] Reminder function:

[1234] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[1235] Device: Sends the requested information to the server and sets the reminder.

[1236] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[1237] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[1238] Health management function:

[1239] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[1240] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[1241] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1242] 4. Shopping support in physical stores

[1243] Shopping Navigation:

[1244] Smart glasses: Navigate users through physical stores and guide them to the location of the product they are looking for. For example, an avatar will guide them, saying, "Milk is in Aisle 3."

[1245] Product information provided:

[1246] Server: Generates detailed product information and related suggestions and sends them to the smart glasses.

[1247] Smart glasses: Avatars present product information to users.

[1248] Product suggestions based on health condition:

[1249] Server: Based on the user's health data, the server recommends suitable products. For example, it generates advice such as "Choose low-salt products."

[1250] Smart glasses: An avatar suggests products based on your health condition.

[1251] 5. Emergency Response

[1252] Smart glasses: Send emergency alerts to store staff if there are any abnormalities in the user's health.

[1253] Server: Records the abnormality in the emergency response database and prompts emergency response.

[1254] Specific examples

[1255] 1. Shopping Navigation:

[1256] User: Asks smart glasses, "Where is the milk?"

[1257] Smart glasses: The avatar navigates by saying "Milk is in Aisle 3."

[1258] 2. Product recommendations based on health conditions:

[1259] User: Asks smart glasses, "What healthy food would you like to eat today?"

[1260] Smart glasses: An avatar suggests, "Choose low-salt products."

[1261] Prompt Sentence Examples

[1262] User: "Where's the milk?"

[1263] Avatar: "Milk is in Aisle 3."

[1264] This system allows elderly people to shop smoothly in brick-and-mortar stores and receive appropriate health care.

[1265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1266] Step 1:

[1267] User registration and avatar customization

[1268] Input: The user enters personal information such as name, birthday, gender, hobbies, and preferences into the device.

[1269] How it works: The user launches the app and enters the required information into a form.

[1270] Data processing: The terminal sends the entered personal information to the server.

[1271] Output: User personal information and customization information is stored on the server.

[1272] Step 2:

[1273] Avatar initial settings

[1274] Input: Your personal and customization information stored on our servers.

[1275] How it works: The server uses this information to generate a user profile and create initial avatar data.

[1276] Data calculation: Generate initial setting data for the avatar using a generative AI model.

[1277] Output: Initialization data is sent to the terminal.

[1278] Step 3:

[1279] Avatar personality development

[1280] Input: Initial setting data for the server-generated avatar.

[1281] How it works: The device configures an avatar based on the data it receives and uses a generative AI model to create the avatar's personality.

[1282] Data calculation: Using a natural language processing algorithm, the avatar's initial setting data is analyzed and personality data is generated.

[1283] Output: The formed personality data is saved on the device.

[1284] Step 4:

[1285] Reminder function

[1286] Input: The user tells the device about their daily schedule and tasks.

[1287] How it works: The user sets a reminder through the app and sends the information to the server.

[1288] Data processing: The server stores the information in the reminder setting database and sends a notification to the device at the specified time.

[1289] Output: The device will notify the user of the reminder via the avatar.

[1290] Step 5:

[1291] Health management function

[1292] Input: User health data collected by IoT devices.

[1293] How it works: The server stores data received from IoT devices and analyzes their health status.

[1294] Data calculation: Generates advice to users based on the results of health data analysis.

[1295] Output: The device provides health advice via an avatar.

[1296] Step 6:

[1297] Shopping support in physical stores - Navigation function

[1298] Input: The user inputs information about the product they are looking for through the smart glasses.

[1299] How it works: The smart glasses send the input information to the server.

[1300] Data calculation: The server references the store database and obtains product location information.

[1301] Output: The smart glasses guide the user to the product location via an avatar.

[1302] Step 7:

[1303] Shopping support in physical stores - Product information provision

[1304] Input: Server-generated product details.

[1305] How it works: The server sends detailed product information and related suggestions to the smart glasses.

[1306] Data processing: Generative AI models select appropriate product information for users.

[1307] Output: The smart glasses present product information to the user via an avatar.

[1308] Step 8:

[1309] Product suggestions based on health status

[1310] Input: User health data and related product information.

[1311] How it works: The server analyzes health data and suggests products suitable for the user.

[1312] Data calculation: Generate optimal product recommendations based on health status data.

[1313] Output: The smart glasses display product suggestions to the user through an avatar.

[1314] Step 9:

[1315] Emergency response

[1316] Input: Anomalous health data detected by IoT devices.

[1317] How it works: The smart glasses send anomaly data to the server.

[1318] Data calculation: The server refers to the emergency response database and generates the necessary response measures.

[1319] Output: An emergency notification is sent to store staff and an avatar informs the user about emergency procedures.

[1320] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1321] This invention is a system that generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized by linking terminals, servers, and IoT devices.

[1322] System configuration

[1323] 1. User registration and avatar customization

[1324] Device: The user installs the application and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[1325] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[1326] 2. Avatar personality development

[1327] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[1328] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[1329] 3. Introducing the Emotion Engine

[1330] On-device: The device is equipped with an emotion engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected.

[1331] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[1332] 4. Daily life support

[1333] Reminder function:

[1334] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[1335] Device: Sends the requested information to the server and sets the reminder.

[1336] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[1337] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1338] Health management function:

[1339] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[1340] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[1341] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1342] 5. Continuous interaction and learning

[1343] Interactive features:

[1344] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[1345] Terminal: Sends the user's questions and conversation content to the server.

[1346] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[1347] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[1348] Learning features:

[1349] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1350] Device: Receive updated personality data and keep your avatar settings up to date.

[1351] Specific examples

[1352] 1. User Registration and Customization:

[1353] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[1354] Server: Stores user information, generates initial avatar data, and sends it to the device.

[1355] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[1356] 2. Reminders and Health Management:

[1357] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[1358] Terminal: Sends a request to the server.

[1359] Server: Set reminders and send confirmation messages to your device.

[1360] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[1361] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[1362] 3. Leveraging Emotion Recognition:

[1363] User: After eating, tell your avatar, "I'm feeling a bit tired today."

[1364] Device: The emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[1365] Server: Based on the emotional data, adjust the content and tone of the avatar's response. "I understand. Please get plenty of rest today," the avatar says in a comforting voice.

[1366] On the device: The avatar provides tailored responses to the user, providing emotionally sensitive support.

[1367] In this way, by combining the emotion engine, the system of the present invention not only alleviates the sense of loneliness felt by the elderly, but also provides emotionally sensitive responses, allowing users to experience richer interactions and receive assistance in their daily lives.

[1368] The processing flow will be explained below.

[1369] Step 1:

[1370] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[1371] Step 2:

[1372] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[1373] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[1374] Step 3:

[1375] On the device: Users are presented with options for avatar appearance (human, animal, etc.) and voice (male, female, young, adult) and can customize their avatar.

[1376] User: Choose your avatar's appearance and voice to complete the customization.

[1377] Step 4:

[1378] Terminal: Sends the information entered and selected by the user to the server.

[1379] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[1380] Step 5:

[1381] Server: Based on the user's personal information and customization information, an AI model is used to create an avatar personality, and the created personality data is sent to the device.

[1382] Step 6:

[1383] Device: Based on the received avatar personality data, the avatar is initialized. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[1384] Step 7:

[1385] User: Enters daily schedules and tasks (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?") and gives instructions to the avatar.

[1386] Terminal: Sends input requests to the server.

[1387] Step 8:

[1388] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[1389] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1390] Step 9:

[1391] IoT devices: Collect user health data (heart rate, steps, etc.).

[1392] Server: Periodically receives data collected from IoT devices and analyzes health status. Sends the analysis results to the device. For example, if the number of steps is low, the avatar will advise the user, "You should walk a little more."

[1393] Step 10:

[1394] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[1395] Step 11:

[1396] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[1397] Terminal: Sends the user's query to the server.

[1398] Step 12:

[1399] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[1400] Device: The avatar explains to the user, "Today's weather is sunny. The temperature is 25 degrees."

[1401] Step 13:

[1402] Device: The emotion engine analyzes the user's voice and facial expressions. For example, if a user says, "I'm a little tired today," the voice tone and facial expression data are collected.

[1403] Step 14:

[1404] Server: Adjusts the avatar's response content and tone based on the emotional data analyzed by the emotion engine. "I understand. Please get plenty of rest today," the avatar says comfortingly.

[1405] Step 15:

[1406] Device: The avatar provides tailored responses to the user, providing appropriate feedback based on emotional data and providing support that is sensitive to the user's emotions.

[1407] Step 16:

[1408] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an artificial intelligence model.

[1409] Device: Receives the latest personality data and updates avatar settings.

[1410] Example 2

[1411] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1412] There is a need for an effective system that can support the daily lives of elderly people while alleviating their feelings of loneliness. Conventional support systems are limited to one-way information provision or simple reminder functions, and do not adequately address the emotions and individual needs of users. This has led to issues such as limited user satisfaction and utilization effectiveness.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1414] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating the personality of the generated avatar using an artificial intelligence model; means for analyzing the user's voice and facial expressions to acquire emotional data; means for adjusting the avatar's response content and tone based on the emotional data; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT sensors; and means for the avatar to learn and update its personality through continuous dialogue with the user. This makes it possible to respond to the user's emotions and individual needs and provide comprehensive support for the elderly in their daily lives.

[1415] "User" refers to the individual who uses the system, who is often a senior citizen.

[1416] "Personal information" refers to information that identifies a user, such as the user's name, birthday, gender, hobbies, and preferences.

[1417] A "customizable avatar" is a virtual character whose appearance and voice can be changed based on information entered by the user.

[1418] "Artificial intelligence model" refers to the algorithms and machine learning models used to generate avatar personalities and responses based on user input data.

[1419] "Emotional data" is data that indicates the emotional state of a user, obtained by analyzing the user's voice and facial expressions.

[1420] An "IoT sensor" is a device that is connected to the Internet and collects data on a user's health and environment.

[1421] "Health Guidance" means advice or suggestions to improve a user's health.

[1422] "Reminder" is a feature that notifies users of their schedules and tasks on time.

[1423] "Dialogue" refers to verbal exchanges between a user and an avatar.

[1424] "Continuous interaction" refers to interaction between a user and an avatar that occurs regularly over an extended period of time.

[1425] "Personality updating" refers to the process of improving an avatar's responses and behavioral patterns based on interaction data and emotional data with the user.

[1426] MODE FOR CARRYING OUT THE INVENTION

[1427] This system generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized through collaboration between terminals, servers, and IoT devices.

[1428] System Configuration

[1429] 1. User registration and avatar customization

[1430] Device: The user installs the application and enters the necessary personal information (such as name, birthday, gender, and hobbies). The user then customizes the avatar's appearance and voice. Specific hardware examples include smartphones and tablets.

[1431] Server: Receives the entered personal information and customization information and generates a user profile. This is done using a database server and an AI server. Furthermore, based on this information, it generates initial setting data for the avatar and sends it to the device.

[1432] Examples:

[1433] The user enters their name "Taro Tanaka," their date of birth "March 15, 1950," their hobby "travel," and sets their avatar's appearance to "cat" and voice to "young man's voice."

[1434] 2. Avatar personality development

[1435] Server: Based on the user's personal information and input data, an artificial intelligence model (e.g., GPT-4) is used to create an avatar personality. Specific software examples include a machine learning platform and a natural language processing engine. The created personality data is sent to the device.

[1436] Device: Initial settings are made based on the received avatar personality data. For example, the avatar may say, "Hello, I'm your new lifemate."

[1437] Examples:

[1438] The avatar greets, "Hello, Tanaka-san. How was your day?"

[1439] 3. Introducing the Emotion Engine

[1440] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions. Examples of specifications include a camera-equipped device and a voice recognition microphone.

[1441] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[1442] Examples:

[1443] The user says, "I'm feeling a bit tired today." The emotion engine interprets this and the avatar responds, "That must be tough. Why don't you take a break?"

[1444] 4. Daily life support

[1445] Reminder function:

[1446] User: Instructs the avatar on daily schedules and tasks.

[1447] Server: Stores the information in the reminder setting database and sends a notification to the device at the specified time.

[1448] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1449] Health management function:

[1450] IoT devices: Collect user health data (e.g., heart rate, steps, blood pressure, etc.). Specific examples of such devices include fitness bands and smartwatches.

[1451] Server: Receives and analyzes the collected data. For example, it generates advice such as "If your step count is low, you should walk a little more."

[1452] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1453] Examples:

[1454] If a user requests, "Set a reminder to take my medicine at 9:00 tomorrow," the server will set this reminder and the device will notify them at the specified time.

[1455] 5. Continuous interaction and learning

[1456] Interactive features:

[1457] Users: Ask the avatar everyday topics and questions.

[1458] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[1459] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[1460] Learning features:

[1461] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1462] Device: Receive updated personality data and keep your avatar settings up to date.

[1463] Example prompt sentence:

[1464] "What news is there today?"

[1465] "Tell me about the weather lately."

[1466] "When's your next doctor's appointment?"

[1467] In this way, this system generates and operates customizable avatars based on personal information and emotional data entered by the user, providing comprehensive support for the user's daily life.

[1468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1469] Step 1:

[1470] User Registration and Customization

[1471] Input: Users install applications on their devices and enter personal information such as name, birthday, gender, hobbies, etc. They also customize the appearance and voice of their avatar.

[1472] Operation: The device sends the entered personal information and customization information to the server. Specifically, the user fills in the information in the input form and presses the send button.

[1473] Output: The server generates a user profile based on the received data and creates initial avatar data. This initial data is then sent to the device.

[1474] Step 2:

[1475] Avatar personality development

[1476] Input: The server receives the user's personal information and customization data and inputs it into an artificial intelligence model (e.g., GPT-4).

[1477] How it works: The server uses an artificial intelligence model to generate the avatar's personality, passing the incoming data to a natural language processing engine and processing the output from the model.

[1478] Output: The generated avatar's personality data is sent to the device. The device configures the avatar based on the received data and displays it on the interface.

[1479] Step 3:

[1480] Completing the initial setup and first greetings

[1481] Input: The device performs initial settings based on the avatar's personality data received from the server.

[1482] How it works: The device applies the avatar's appearance, voice, and personality data to display the avatar for the first time. The avatar then greets the user for the first time.

[1483] Output: The avatar will start a customized interaction with the user, such as "Hello, I'm your new lifemate."

[1484] Step 4:

[1485] Emotion engine analysis and response adjustment

[1486] Input: The device collects the user's voice and facial expressions through the camera and microphone.

[1487] How it works: The emotion engine analyzes the collected data, extracts emotion data, and sends this data to the server.

[1488] Output: The server adjusts the avatar's response content and tone based on the emotional data and sends the data to the device. The avatar on the device then provides the adjusted response to the user.

[1489] Step 5:

[1490] Reminder settings and notifications

[1491] Input: The user gives the avatar instructions for daily schedules and tasks, such as "Set a reminder to take my medicine tomorrow at 9 AM."

[1492] How it works: The device sends instructions from the user to the server, which stores the information in the reminder settings database. When the specified time arrives, the server sends the notification data to the device.

[1493] Output: The device receives notification data from the server, and the avatar notifies the user, such as "It's time to take your medicine now."

[1494] Step 6:

[1495] Health management features for data collection and feedback

[1496] Input: IoT devices collect user health data (e.g., heart rate, steps, blood pressure, etc.).

[1497] How it works: The IoT device sends the collected data to a server, which analyzes the data and generates advice such as "You should walk a little more" if the number of steps is low.

[1498] Output: The analysis results are sent to the device, and the avatar tells the user, "You walked 5,000 steps today. You're very healthy."

[1499] Step 7:

[1500] Continuous interaction and learning

[1501] Input: The user asks the avatar everyday topics and questions, such as "What's in the news today?"

[1502] How it works: The device sends the user's question data to the server, which then generates appropriate information based on the received data and sends it to the device. Furthermore, the server continuously collects conversation data and emotional data with the user, and updates the avatar's personality using an artificial intelligence model.

[1503] Output: The device receives information from the server, the avatar tells the user "Here's today's news" and provides the appropriate information, and keeps the avatar's settings up to date.

[1504] As described above, the program of the system of the present invention provides a function to support the daily lives of elderly people and alleviate their feelings of loneliness through a series of processing steps.

[1505] (Application example 2)

[1506] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1507] To support the daily lives of elderly people while alleviating their feelings of loneliness, simple reminder and health management functions are insufficient. Support tailored to individual needs through continuous interaction with the elderly and flexible response to emotional changes is necessary. However, current systems lack the ability to read the user's unique emotions and respond appropriately based on them, and therefore are unable to provide sufficient psychological satisfaction and security to the elderly.

[1508] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1509] In this invention, the server includes means for generating a customizable virtual character based on personal information entered by the user, means for forming the personality of the generated virtual character using an artificial intelligence model, and means for adjusting the content and tone of the virtual character's responses based on emotional data from an emotion analysis engine. This enables personalized responses that are sensitive to the emotions of the elderly, providing them with a greater sense of satisfaction and security.

[1510] "user"

[1511] refers to individuals who use the system, often specifically elderly people.

[1512] "personal information"

[1513] This refers to data entered by the user to identify the user, such as name, birthday, gender, hobbies, and preferences.

[1514] "Virtual Character"

[1515] is a digital character that is customized based on a user's personal information, with visual and audio elements including appearance and voice.

[1516] "Artificial Intelligence Model"

[1517] refers to algorithms and learning models that use user input data to shape the personality of a virtual character.

[1518] Emotion analysis engine

[1519] This refers to a system that analyzes a user's voice, facial expressions, etc. to detect their emotional state.

[1520] Reminders

[1521] This refers to a function that notifies users of scheduled events and tasks in a timely manner.

[1522] "IoT devices"

[1523] refers to a physical device that is connected to the Internet and is capable of sending and receiving sensor data, including wearable devices for health management.

[1524] "Health status"

[1525] refers to biometric data such as a user's heart rate, blood pressure, number of steps, etc. These data are used to monitor the user's health.

[1526] "Response content"

[1527] refers to the responses that a virtual character provides based on the user's questions and emotional state, including information and advice.

[1528] "tone"

[1529] This refers to the tone of voice and speaking style of a virtual character, which is adjusted by an emotion analysis engine, depending on the user's emotional state.

[1530] This invention provides a system that supports the daily lives of elderly people and alleviates their feelings of loneliness by generating a customizable virtual character based on the user's personal information and combining it with an emotion analysis engine. This system is composed of a terminal, a server, and an IoT device. The details of this system are described below.

[1531] System configuration

[1532] 1. User registration and virtual character customization

[1533] The terminal provides an interface where the user installs the application and inputs necessary personal information (such as name, birthday, gender, hobbies, and preferences). The user can then customize the virtual character's appearance (human, animal, etc.) and voice (male, female, young, adult). The server receives the input personal information and customization information and generates a user profile. Based on this information, the server then generates initial setting data for the virtual character and sends it to the terminal.

[1534] 2. Personality development of virtual characters

[1535] The server uses a generative AI model to create a virtual character's personality based on the user's personal information and input data. This uses natural language processing and machine learning algorithms. The created personality data is then sent to the device. The device then performs initial setup based on the received virtual character's personality data and greets the user for the first time.

[1536] 3. Introduction of a sentiment analysis engine

[1537] The device is equipped with an emotion analysis engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected. The server receives the emotional data analyzed by the emotion analysis engine and adjusts the content and tone of the virtual character's responses.

[1538] 4. Daily life support

[1539] The device can instruct the virtual character about the user's daily schedule and tasks (such as doctor's appointments and shopping lists). It also provides reminder setting and health management functions. The server receives the information specified by the user, stores it in a reminder setting database, and sends notifications to the device at the specified time. It also analyzes health data (heart rate, number of steps, blood pressure, etc.) collected from IoT devices and generates health advice.

[1540] 5. Continuous interaction and learning

[1541] The device has a dialogue function that allows users to ask the virtual character everyday topics and questions. The server generates appropriate information based on the dialogue data received and sends it to the device. It also continuously collects dialogue data and emotional data and updates the virtual character's personality using a generative AI model. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1542] Specific examples

[1543] User Registration and Customization

[1544] The user launches the app on the device, inputs their name "General Taro," their date of birth "January 1, 1945," and their hobby "reading," selects an "animal" as the virtual character's appearance, and sets the voice to a "calm female voice." The server saves the user information, generates initial setting data for the virtual character, and sends it to the device. The virtual character greets the device by saying, "Hello, Taro. I'm your new friend."

[1545] Utilizing Emotion Recognition

[1546] When a user says, "I feel a little tired today," the device uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data. The server then adjusts the content and tone of the virtual character's response based on the emotional data, and the virtual character then comforts the user by saying, "I understand. Please take it easy today."

[1547] Example prompts to input to a generative AI model:

[1548] Username: General Taro

[1549] User input: Set a reminder to take my medicine tomorrow at 9am

[1550] Emotion engine: Analyzes user input and extracts emotional data (e.g., "I'm tired")

[1551] Virtual character response: Sets a reminder and responds to the user (e.g., "I've set a reminder to take my medicine tomorrow at 9 o'clock.")

[1552] In this way, the system of the present invention can respond to the user's emotions and individual needs, and can support the elderly in their daily lives while alleviating their sense of loneliness.

[1553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1554] Step 1:

[1555] The user inputs personal information (such as name, birthday, gender, hobbies, and preferences) through the device. The device acquires this information and sends it to the server. The personal information is the input data, and the data sent to the server is the output data.

[1556] Step 2:

[1557] The server receives the personal information sent by the user and generates a user profile. The server stores the personal information in a database and generates initial setting data for the virtual character. This initial setting data includes customization information such as the virtual character's appearance and voice. The input data is personal information, and the output is the initial setting data, which is then sent to the terminal.

[1558] Step 3:

[1559] The device displays the virtual character and greets the user based on the initial setting data of the virtual character received from the server. Specifically, it plays a voice and text such as "Hello, Taro. I'm your new friend." The initial setting data is the input data, and the greeting is displayed as the output.

[1560] Step 4:

[1561] The user inputs daily plans and questions to the virtual character. For example, they can give instructions such as "Set a reminder to take my medicine at 9:00 tomorrow." The device acquires the user's input and sends it to the server. The input data includes the user's instructions, and the output data is sent to the server.

[1562] Step 5:

[1563] The server receives instructions sent by the user, analyzes them, and sets reminders. Specifically, it saves the information in a reminder setting database and prepares to send a reminder notification to the device at the specified time. The input data is the user's instructions, and the output is reminder setting data.

[1564] Step 6:

[1565] The device will notify the user of the reminder at the specified time based on the reminder setting data received from the server. Specifically, it will display a voice and text message saying, "It's time to take your medicine now." The input data is the reminder setting data, and the output is a reminder notification.

[1566] Step 7:

[1567] To analyze a user's emotions, the device analyzes the user's voice and facial expressions using an emotion analysis engine. The emotion analysis engine receives the user's voice and facial expression data and detects their emotional state (for example, "tired" or "energetic"). The input data is the user's voice and facial expression, and the output is emotion data.

[1568] Step 8:

[1569] The server adjusts the content and tone of the virtual character's responses based on the emotional data. For example, if a user says, "I'm a little tired today," the server generates an adjusted response, such as "I understand. Please take it easy today," and sends it to the device. Emotional data is the input data, and adjusted response data is generated as the output.

[1570] Step 9:

[1571] The device receives the adjusted response data from the server, and the virtual character responds to the user with voice and text. Specifically, the virtual character responds to the user by saying, "I understand. Please take a good rest today." The adjusted response data is input, and the response is displayed as output.

[1572] Step 10:

[1573] The server continuously collects user interaction and emotional data and uses a generative AI model to update the virtual character's personality. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses. The input data includes interaction data and emotional data, and the output is updated personality data.

[1574] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1575] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1576] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1577] [Fourth embodiment]

[1578] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1579] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1580] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1581] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1582] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1583] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1584] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1585] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1586] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1587] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1588] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1589] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1590] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1591] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. This system is realized by linking terminals, servers, and IoT devices.

[1592] System configuration

[1593] 1. User registration and avatar customization

[1594] On device: The user installs the app and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, old).

[1595] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[1596] 2. Avatar personality development

[1597] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[1598] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[1599] 3. Daily life support

[1600] Reminder function:

[1601] User: Instructs the avatar about daily schedules and tasks (such as doctor's appointments and shopping lists).

[1602] Device: Sends the requested information to the server and sets the reminder.

[1603] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[1604] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[1605] Health management function:

[1606] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[1607] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[1608] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1609] 4. Continuous interaction and learning

[1610] Interactive features:

[1611] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[1612] Terminal: Sends the user's questions and conversation content to the server.

[1613] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[1614] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[1615] Learning features:

[1616] Server: Continuously collects user interaction data and trains the AI ​​model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1617] Device: Receive updated personality data and keep your avatar settings up to date.

[1618] Specific examples

[1619] 1. User Registration and Customization:

[1620] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[1621] Server: Stores user information, generates initial avatar data, and sends it to the device.

[1622] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[1623] 2. Reminders and Health Management:

[1624] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[1625] Terminal: Sends a request to the server.

[1626] Server: Set reminders and send confirmation messages to your device.

[1627] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[1628] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[1629] In this way, the system of the present invention not only alleviates the sense of loneliness of the elderly, but also provides support in various aspects of daily life. Through continuous learning and updates, it is possible to provide optimal assistance to users.

[1630] The processing flow will be explained below.

[1631] Step 1:

[1632] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[1633] Step 2:

[1634] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[1635] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[1636] Step 3:

[1637] Device: Presents the user with options for the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[1638] User: Choose your avatar's appearance and voice to complete the customization.

[1639] Step 4:

[1640] Terminal: Sends the information entered and selected by the user to the server.

[1641] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[1642] Step 5:

[1643] Server: Based on the user's personal information and customization information, an artificial intelligence model is used to create an avatar's personality.

[1644] Server: Sends the created personality data to the terminal.

[1645] Step 6:

[1646] Device: Based on the received avatar personality data, the initial avatar settings are performed. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[1647] Step 7:

[1648] User: Enter daily schedules and tasks into the avatar (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?").

[1649] Terminal: Sends input requests to the server.

[1650] Step 8:

[1651] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[1652] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1653] Step 9:

[1654] IoT devices: Collect user health data (heart rate, steps, etc.).

[1655] Server: Periodically receives data collected from IoT devices and analyzes their health status.

[1656] Server: Sends the analysis results to the device and generates appropriate feedback and advice.

[1657] Step 10:

[1658] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[1659] Step 11:

[1660] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[1661] Terminal: Sends the user's query to the server.

[1662] Step 12:

[1663] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[1664] Device: The avatar tells the user, "Today's weather is sunny. The temperature is 25 degrees."

[1665] Step 13:

[1666] Server: Continuously collects user interaction data and updates the avatar's personality using an artificial intelligence model.

[1667] Server: Sends updated personality data to devices, keeping avatar settings up to date.

[1668] Example 1

[1669] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1670] This invention relates to a system for supporting the daily lives of elderly people. Elderly people often experience feelings of loneliness and difficulty managing their health, and therefore support is needed to solve these problems and enrich their daily lives. While existing technologies exist for avatar systems that interact with users, they face challenges in providing optimal support tailored to the user's behavior and preferences through continuous learning and customization based on personal information.

[1671] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1672] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating a personality for the generated avatar using an artificial intelligence model; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT devices; means for the avatar to learn and update its personality through continuous dialogue with the user; means for the avatar to understand the content of the dialogue using a natural language processing algorithm based on the user's personal information and daily dialogue data and generate appropriate responses; and means for the server to analyze data collected from the user and learn the user's behavioral patterns and preferences. This provides comprehensive support for the daily lives of the elderly, alleviating feelings of loneliness, and improving the efficiency of health management.

[1673] "User" refers to an individual or elderly person who uses the system and receives assistance in daily life through interaction with an avatar.

[1674] "Personal Information" refers to information provided by users, including data such as name, birth date, gender, hobbies, and preferences.

[1675] A "customizable avatar" refers to a pseudo-personality whose appearance, voice, etc. can be set based on personal information entered by the user.

[1676] "Artificial intelligence model" refers to the machine learning algorithms and natural language processing technologies used to create the avatar's personality and generate responses.

[1677] "Reminders" refers to a feature that notifies users of their daily schedules and tasks.

[1678] An "IoT device" refers to a sensor device that is connected to the Internet and collects user health data, etc.

[1679] "Health status" refers to a user's physical data, such as heart rate, steps taken, and blood pressure.

[1680] "Feedback" refers to the act of providing advice or notifications to users based on collected data.

[1681] "Continuous dialogue" refers to long-term interaction between the user and the avatar.

[1682] "Natural language processing algorithms" refer to technologies that understand user speech or text input and generate appropriate responses.

[1683] "Behavioral patterns" refer to the user's daily life tendencies and habits.

[1684] "Preferences" refers to a user's hobbies and tendencies.

[1685] This is a system designed to support the daily lives of elderly people. It generates a customizable avatar based on personal information entered by the user, and supports the user's daily life through interaction with this avatar. This system is realized mainly by terminals, servers, and IoT devices.

[1686] First, the user installs and launches the application on their device. This application provides an interface for entering personal information (e.g., name, birthday, gender, hobbies, preferences, etc.). Data such as name, birthday, and hobbies are entered into each input field.

[1687] Next, the user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, old), and this customization information is also sent from the device to the server.

[1688] The server generates a user profile based on the received personal information and customization information. It then uses an artificial intelligence model (generative AI model) to generate a personality for the avatar. This personality data is also sent from the server to the device. Specifically, it uses a natural language processing algorithm to generate dialogue based on the user's interests and behavioral patterns.

[1689] The generated avatar will be displayed on the device and greet the user for the first time. For example, the avatar may say to the user, "Hello, I'm your new lifemate."

[1690] This system has a reminder function, and the user can instruct the avatar about daily schedules and tasks, which are then managed and notified. When the user asks the avatar to "set a reminder to take my medicine at 9:00 tomorrow," the device sends this request to the server, and the reminder is set. At the specified time, the avatar will notify the user, "It's time to take your medicine now."

[1691] It also works with IoT devices as a health management function, collecting user health data (e.g., heart rate, number of steps, blood pressure, etc.). The collected data is sent to a server where it is analyzed. For example, if a user walks 5,000 steps in a day, the server will generate health advice such as "You walked 5,000 steps today. You're very healthy," and notify the user through an avatar on the device.

[1692] Furthermore, this system has the ability to allow the avatar to learn through continuous dialogue and continuously update its personality. When a user asks a question such as "What's in the news today?", the device sends the question to the server, which analyzes the dialogue data, generates appropriate information, and sends it to the device. This allows the avatar to provide information to the user, such as "Here's today's news." This dialogue data is then learned by the server, allowing the avatar to understand the user's preferences and behavioral patterns and provide more appropriate responses.

[1693] For example, if a user inputs the name "Yamada Taro," the hobby "reading," and sets the appearance as "dog" and the voice as "calm female voice," the server will generate a profile for "Yamada Taro" and set the avatar as "a dog who loves reading and has a laid-back personality." Based on this setting, the avatar will greet the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[1694] Examples of prompts include "Set a reminder to take my medicine at 9:00 tomorrow" and "What's the news today?" This allows the present invention to comprehensively support the daily lives of elderly people, alleviating their feelings of loneliness and improving the efficiency of health management.

[1695] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1696] Step 1:

[1697] The user installs the app on their device and launches it. The user enters personal information (such as name, birthday, gender, hobbies, and preferences) through the app's interface. The input data includes "Name: Yamada Taro," "Birthday: January 1, 1945," and "Hobbies: Reading." This input data is sent from the device to the server. The server receives the input data, converts it to JSON format data, and sends it to the server.

[1698] Step 2:

[1699] The server generates a user profile based on the personal information received from the device. The server stores the data in a personal information database and creates a user profile. This user profile includes the user's name, birthday, gender, hobbies, preferences, etc. The output is the initial setting data for the user profile.

[1700] Step 3:

[1701] The user customizes the avatar's appearance (e.g., human, animal) and voice (e.g., male, female, young, elderly). This customization information is sent from the device to the server. Input data includes, for example, "appearance: dog" and "voice: calm female voice." The device sends the customized information to the server.

[1702] Step 4:

[1703] Based on the customization information received by the server, a generative AI model is used to generate initial avatar settings. Specifically, a natural language processing algorithm is used to generate dialogue data based on the user's hobbies and preferences. The output is a customized and personalized avatar settings setup.

[1704] Step 5:

[1705] The server sends the generated avatar personality data to the device. The device receives this personality data and displays the avatar. Based on the initial setup data, the avatar greets the user for the first time. Specifically, the avatar greets the user by saying, "Hello, Yamada-san. I'm your new dog friend."

[1706] Step 6:

[1707] The user instructs the avatar about daily schedules and tasks. For example, the user might instruct the avatar to "set a reminder to take medicine at 9:00 tomorrow." This instruction is sent from the device to the server. The input data includes "a reminder to take medicine at 9:00 tomorrow."

[1708] Step 7:

[1709] The server sets a reminder based on the schedule data it receives. It saves the data in the reminder setting database and notifies the device that the setting is complete. As an output, it generates a message indicating that the reminder setting is complete and sends it to the device.

[1710] Step 8:

[1711] When the set time arrives, the server sends a reminder notification to the device. The device receives this notification, and an avatar notifies the user, "It's time to take your medicine now." Specifically, the avatar notifies the user by voice.

[1712] Step 9:

[1713] IoT devices collect user health data (e.g., heart rate, step count, blood pressure, etc.) and transmit the collected data to a terminal or server in real time. Input data includes heart rate, step count, and blood pressure data.

[1714] Step 10:

[1715] The server analyzes the received health data and evaluates the user's health condition. For example, if the number of steps is low, it generates advice such as "You should walk a little more." This advice data is sent to the device. The generated health advice is included as an output.

[1716] Step 11:

[1717] The device notifies the user through an avatar based on the health advice received from the server. Specifically, the avatar will say to the user, "You walked 5,000 steps today. That's very healthy."

[1718] Step 12:

[1719] The user asks the avatar a question of everyday life, such as "What's in the news today?" This question is sent from the device to the server. The input data includes the question itself.

[1720] Step 13:

[1721] The server generates appropriate information based on the question data it receives. It extracts relevant news information from the dialogue database and sends it to the terminal. As an output, news information is generated and sent to the terminal.

[1722] Step 14:

[1723] Based on the information received from the server, the device uses an avatar to provide news to the user. Specifically, the avatar will say, "Here's today's news."

[1724] Step 15:

[1725] The server continuously collects user interaction data, and the AI ​​model learns from it. The data is stored in an interaction database and the avatar's personality data is updated. The output is the updated personality data.

[1726] Step 16:

[1727] The device keeps the avatar's settings up to date based on the updated personality data it receives from the server. Specifically, the avatar will ask the user, "How did you like the history book you read recently?"

[1728] (Application example 1)

[1729] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1730] When seniors shop in brick-and-mortar stores, they often face challenges in finding the exact product they need and selecting the right product for their health condition. It's also difficult to quickly respond appropriately in emergencies. These challenges can make the shopping experience stressful for seniors, making it difficult for them to receive optimal health care.

[1731] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1732] In this invention, the server includes a means for generating a customizable avatar based on personal information entered by the user, a means for creating a personality for the generated avatar using an artificial intelligence model, a means for providing reminders and information in response to the user's daily schedule and questions, a means for managing and providing feedback on the user's health status in cooperation with IoT devices, a means for navigating the user in a store and providing product information in cooperation with smart glasses, a means for suggesting products according to the user's health status, and a means for contacting store staff in an emergency. This allows elderly people to smoothly shop in physical stores and receive appropriate health care.

[1733] A "customizable avatar" is a virtual character that can interact with the user and whose appearance, voice, etc. can be freely set based on personal information entered by the user.

[1734] An "artificial intelligence model" is a technology that uses machine learning and natural language processing to generate appropriate responses and actions in response to user interactions and instructions.

[1735] "Reminder" is a function that notifies the user of scheduled events or tasks at a pre-set time.

[1736] An "IoT device" is a device that has the ability to collect, analyze, and transmit data via the Internet. Examples include health management devices and smart home devices.

[1737] "Smart glasses" are a wearable eyeglass-type device that can display augmented reality and information when worn by the user.

[1738] "Health Status Management" is a system that continuously collects and analyzes users' health data and provides appropriate feedback and advice.

[1739] "In-store navigation" is a function that guides users to the location of the product they are looking for within a physical store.

[1740] "Product Information Provision" is a function that provides users with detailed information about a specified product and related suggestions.

[1741] "Product suggestions based on health status" is a function that recommends products suitable for a user's condition based on their health data.

[1742] "Emergency Contact" is a function that quickly sends alerts to store staff and relevant institutions if there is any change in the user's health condition.

[1743] This invention is a system that generates customizable avatars based on personal information entered by users and supports the daily lives of elderly people through interactions with the avatars. The system also works in conjunction with smart glasses to support shopping in brick-and-mortar stores.

[1744] System configuration

[1745] 1. User registration and avatar customization

[1746] Device: The user installs a dedicated app and enters personal information such as name, birthday, gender, hobbies, and preferences. Next, the user customizes the avatar's appearance (e.g., human or animal) and voice (e.g., male or female). This information is sent to the server.

[1747] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[1748] 2. Avatar personality development

[1749] Server: Based on the user's personal information and input data, a generative AI model is used to create an avatar personality. This model uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[1750] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[1751] 3. Daily life support

[1752] Reminder function:

[1753] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[1754] Device: Sends the requested information to the server and sets the reminder.

[1755] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[1756] Device: The avatar informs the user, "You have a doctor's appointment at 2 PM."

[1757] Health management function:

[1758] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[1759] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[1760] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1761] 4. Shopping support in physical stores

[1762] Shopping Navigation:

[1763] Smart glasses: Navigate users through physical stores and guide them to the location of the product they are looking for. For example, an avatar will guide them, saying, "Milk is in Aisle 3."

[1764] Product information provided:

[1765] Server: Generates detailed product information and related suggestions and sends them to the smart glasses.

[1766] Smart glasses: Avatars present product information to users.

[1767] Product suggestions based on health condition:

[1768] Server: Based on the user's health data, the server recommends suitable products. For example, it generates advice such as "Choose low-salt products."

[1769] Smart glasses: An avatar suggests products based on your health condition.

[1770] 5. Emergency Response

[1771] Smart glasses: Send emergency alerts to store staff if there are any abnormalities in the user's health.

[1772] Server: Records the abnormality in the emergency response database and prompts emergency response.

[1773] Specific examples

[1774] 1. Shopping Navigation:

[1775] User: Asks smart glasses, "Where is the milk?"

[1776] Smart glasses: The avatar navigates by saying "Milk is in Aisle 3."

[1777] 2. Product recommendations based on health conditions:

[1778] User: Asks smart glasses, "What healthy food would you like to eat today?"

[1779] Smart glasses: An avatar suggests, "Choose low-salt products."

[1780] Prompt Sentence Examples

[1781] User: "Where's the milk?"

[1782] Avatar: "Milk is in Aisle 3."

[1783] This system allows elderly people to shop smoothly in brick-and-mortar stores and receive appropriate health care.

[1784] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1785] Step 1:

[1786] User registration and avatar customization

[1787] Input: The user enters personal information such as name, birthday, gender, hobbies, and preferences into the device.

[1788] How it works: The user launches the app and enters the required information into a form.

[1789] Data processing: The terminal sends the entered personal information to the server.

[1790] Output: User personal information and customization information is stored on the server.

[1791] Step 2:

[1792] Avatar initial settings

[1793] Input: Your personal and customization information stored on our servers.

[1794] How it works: The server uses this information to generate a user profile and create initial avatar data.

[1795] Data calculation: Generate initial setting data for the avatar using a generative AI model.

[1796] Output: Initialization data is sent to the terminal.

[1797] Step 3:

[1798] Avatar personality development

[1799] Input: Initial setting data for the server-generated avatar.

[1800] How it works: The device configures an avatar based on the data it receives and uses a generative AI model to create the avatar's personality.

[1801] Data calculation: Using a natural language processing algorithm, the avatar's initial setting data is analyzed and personality data is generated.

[1802] Output: The formed personality data is saved on the device.

[1803] Step 4:

[1804] Reminder function

[1805] Input: The user tells the device about their daily schedule and tasks.

[1806] How it works: The user sets a reminder through the app and sends the information to the server.

[1807] Data processing: The server stores the information in the reminder setting database and sends a notification to the device at the specified time.

[1808] Output: The device will notify the user of the reminder via the avatar.

[1809] Step 5:

[1810] Health management function

[1811] Input: User health data collected by IoT devices.

[1812] How it works: The server stores data received from IoT devices and analyzes their health status.

[1813] Data calculation: Generates advice to users based on the results of health data analysis.

[1814] Output: The device provides health advice via an avatar.

[1815] Step 6:

[1816] Shopping support in physical stores - Navigation function

[1817] Input: The user inputs information about the product they are looking for through the smart glasses.

[1818] How it works: The smart glasses send the input information to the server.

[1819] Data calculation: The server references the store database and obtains product location information.

[1820] Output: The smart glasses guide the user to the product location via an avatar.

[1821] Step 7:

[1822] Shopping support in physical stores - Product information provision

[1823] Input: Server-generated product details.

[1824] How it works: The server sends detailed product information and related suggestions to the smart glasses.

[1825] Data processing: Generative AI models select appropriate product information for users.

[1826] Output: The smart glasses present product information to the user via an avatar.

[1827] Step 8:

[1828] Product suggestions based on health status

[1829] Input: User health data and related product information.

[1830] How it works: The server analyzes health data and suggests products suitable for the user.

[1831] Data calculation: Generate optimal product recommendations based on health status data.

[1832] Output: The smart glasses display product suggestions to the user through an avatar.

[1833] Step 9:

[1834] Emergency response

[1835] Input: Anomalous health data detected by IoT devices.

[1836] How it works: The smart glasses send anomaly data to the server.

[1837] Data calculation: The server refers to the emergency response database and generates the necessary response measures.

[1838] Output: An emergency notification is sent to store staff and an avatar informs the user about emergency procedures.

[1839] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1840] This invention is a system that generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized by linking terminals, servers, and IoT devices.

[1841] System configuration

[1842] 1. User registration and avatar customization

[1843] Device: The user installs the application and enters the required personal information (name, birthday, gender, hobbies, preferences, etc.). The user then customizes the avatar's appearance (human, animal, etc.) and voice (male, female, young, adult).

[1844] Server: Receives the entered personal information and customization information, generates a user profile, and based on this information, generates initial avatar data and sends it to the device.

[1845] 2. Avatar personality development

[1846] Server: Based on the user's personal information and input data, an artificial intelligence model creates an avatar personality. This uses natural language processing and machine learning algorithms. The created personality data is sent to the device.

[1847] Device: Based on the received avatar personality data, the device performs initial setup and greets the user. For example, the avatar might say, "Hello, I'm your new lifemate."

[1848] 3. Introducing the Emotion Engine

[1849] On-device: The device is equipped with an emotion engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected.

[1850] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[1851] 4. Daily life support

[1852] Reminder function:

[1853] User: Instructs the avatar on daily schedules and tasks (such as doctor's appointments or shopping lists).

[1854] Device: Sends the requested information to the server and sets the reminder.

[1855] Server: Saves the reminder setting in the database and sends a notification to the device when the specified time arrives.

[1856] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1857] Health management function:

[1858] IoT devices: Collect user health data (heart rate, steps, blood pressure, etc.).

[1859] Server: Receives and analyzes the collected data. For example, if the number of steps is low, it generates advice such as "You should walk a little more."

[1860] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1861] 5. Continuous interaction and learning

[1862] Interactive features:

[1863] Users: Ask the avatar everyday topics and questions, such as "What's the news today?" or "What's the weather like these days?"

[1864] Terminal: Sends the user's questions and conversation content to the server.

[1865] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[1866] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[1867] Learning features:

[1868] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1869] Device: Receive updated personality data and keep your avatar settings up to date.

[1870] Specific examples

[1871] 1. User Registration and Customization:

[1872] Device: The user launches the app and enters their name "Yamada Taro," their date of birth "January 1, 1945," and their hobby "reading." They then select a "dog" as the avatar's appearance and a "calm female voice."

[1873] Server: Stores user information, generates initial avatar data, and sends it to the device.

[1874] Device: The avatar greets, "Hello, Yamada-san. I'm your new dog friend."

[1875] 2. Reminders and Health Management:

[1876] User: Asks avatar to "set a reminder to take my medicine tomorrow at 9am."

[1877] Terminal: Sends a request to the server.

[1878] Server: Set reminders and send confirmation messages to your device.

[1879] Device: The avatar tells the user, "I've set a reminder to take your medicine tomorrow at 9am."

[1880] At 9:00 the next day, a reminder notification is sent from the server and an avatar on the device notifies the user, "It's time to take your medicine now."

[1881] 3. Leveraging Emotion Recognition:

[1882] User: After eating, tell your avatar, "I'm feeling a bit tired today."

[1883] Device: The emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[1884] Server: Based on the emotional data, adjust the content and tone of the avatar's response. "I understand. Please get plenty of rest today," the avatar says in a comforting voice.

[1885] On the device: The avatar provides tailored responses to the user, providing emotionally sensitive support.

[1886] In this way, by combining the emotion engine, the system of the present invention not only alleviates the sense of loneliness felt by the elderly, but also provides emotionally sensitive responses, allowing users to experience richer interactions and receive assistance in their daily lives.

[1887] The processing flow will be explained below.

[1888] Step 1:

[1889] Device: The user installs the app and launches it for the first time. A welcome screen appears, prompting the user to press the "Register" button.

[1890] Step 2:

[1891] User: Enter personal information such as name, birthday, hobbies, preferences, etc.

[1892] Device: Temporarily saves the entered information and proceeds to the next customization screen.

[1893] Step 3:

[1894] On the device: Users are presented with options for avatar appearance (human, animal, etc.) and voice (male, female, young, adult) and can customize their avatar.

[1895] User: Choose your avatar's appearance and voice to complete the customization.

[1896] Step 4:

[1897] Terminal: Sends the information entered and selected by the user to the server.

[1898] Server: Generates a user profile based on the received information and uses this data to create initial avatar data.

[1899] Step 5:

[1900] Server: Based on the user's personal information and customization information, an AI model is used to create an avatar personality, and the created personality data is sent to the device.

[1901] Step 6:

[1902] Device: Based on the received avatar personality data, the avatar is initialized. An initial greeting is displayed to the user. The avatar greets the user by saying, "Hello, I'm your new lifemate."

[1903] Step 7:

[1904] User: Enters daily schedules and tasks (e.g., "Do you have a doctor's appointment tomorrow at 2 PM?") and gives instructions to the avatar.

[1905] Terminal: Sends input requests to the server.

[1906] Step 8:

[1907] Server: Updates the reminder setting database based on the user's input information. Sends a notification to the device at the specified time.

[1908] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1909] Step 9:

[1910] IoT devices: Collect user health data (heart rate, steps, etc.).

[1911] Server: Periodically receives data collected from IoT devices and analyzes health status. Sends the analysis results to the device. For example, if the number of steps is low, the avatar will advise the user, "You should walk a little more."

[1912] Step 10:

[1913] Device: The avatar provides advice to the user based on their health data, for example, "You walked 5,000 steps today. You're very healthy."

[1914] Step 11:

[1915] Users: Ask the avatar everyday topics and questions (e.g., "What's the weather like today?", "What's the latest news?").

[1916] Terminal: Sends the user's query to the server.

[1917] Step 12:

[1918] Server: Receives queries from users, searches for and generates relevant information, and sends the generated information to the terminal.

[1919] Device: The avatar explains to the user, "Today's weather is sunny. The temperature is 25 degrees."

[1920] Step 13:

[1921] Device: The emotion engine analyzes the user's voice and facial expressions. For example, if a user says, "I'm a little tired today," the voice tone and facial expression data are collected.

[1922] Step 14:

[1923] Server: Adjusts the avatar's response content and tone based on the emotional data analyzed by the emotion engine. "I understand. Please get plenty of rest today," the avatar says comfortingly.

[1924] Step 15:

[1925] Device: The avatar provides tailored responses to the user, providing appropriate feedback based on emotional data and providing support that is sensitive to the user's emotions.

[1926] Step 16:

[1927] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an artificial intelligence model.

[1928] Device: Receives the latest personality data and updates avatar settings.

[1929] Example 2

[1930] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1931] There is a need for an effective system that can support the daily lives of elderly people while alleviating their feelings of loneliness. Conventional support systems are limited to one-way information provision or simple reminder functions, and do not adequately address the emotions and individual needs of users. This has led to issues such as limited user satisfaction and utilization effectiveness.

[1932] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1933] In this invention, the server includes: means for generating a customizable avatar based on personal information entered by the user; means for creating the personality of the generated avatar using an artificial intelligence model; means for analyzing the user's voice and facial expressions to acquire emotional data; means for adjusting the avatar's response content and tone based on the emotional data; means for providing reminders and information in response to the user's daily schedule and questions; means for managing and providing feedback on the user's health status in cooperation with IoT sensors; and means for the avatar to learn and update its personality through continuous dialogue with the user. This makes it possible to respond to the user's emotions and individual needs and provide comprehensive support for the elderly in their daily lives.

[1934] "User" refers to the individual who uses the system, who is often a senior citizen.

[1935] "Personal information" refers to information that identifies a user, such as the user's name, birthday, gender, hobbies, and preferences.

[1936] A "customizable avatar" is a virtual character whose appearance and voice can be changed based on information entered by the user.

[1937] "Artificial intelligence model" refers to the algorithms and machine learning models used to generate avatar personalities and responses based on user input data.

[1938] "Emotional data" is data that indicates the emotional state of a user, obtained by analyzing the user's voice and facial expressions.

[1939] An "IoT sensor" is a device that is connected to the Internet and collects data on a user's health and environment.

[1940] "Health Guidance" means advice or suggestions to improve a user's health.

[1941] "Reminder" is a feature that notifies users of their schedules and tasks on time.

[1942] "Dialogue" refers to verbal exchanges between a user and an avatar.

[1943] "Continuous interaction" refers to interaction between a user and an avatar that occurs regularly over an extended period of time.

[1944] "Personality updating" refers to the process of improving an avatar's responses and behavioral patterns based on interaction data and emotional data with the user.

[1945] MODE FOR CARRYING OUT THE INVENTION

[1946] This system generates customizable avatars based on personal information entered by users and combines them with an emotion engine that recognizes the user's emotions to support the daily lives of elderly people while alleviating their feelings of loneliness. This system is realized through collaboration between terminals, servers, and IoT devices.

[1947] System Configuration

[1948] 1. User registration and avatar customization

[1949] Device: The user installs the application and enters the necessary personal information (such as name, birthday, gender, and hobbies). The user then customizes the avatar's appearance and voice. Specific hardware examples include smartphones and tablets.

[1950] Server: Receives the entered personal information and customization information and generates a user profile. This is done using a database server and an AI server. Furthermore, based on this information, it generates initial setting data for the avatar and sends it to the device.

[1951] Examples:

[1952] The user enters their name "Taro Tanaka," their date of birth "March 15, 1950," their hobby "travel," and sets their avatar's appearance to "cat" and voice to "young man's voice."

[1953] 2. Avatar personality development

[1954] Server: Based on the user's personal information and input data, an artificial intelligence model (e.g., GPT-4) is used to create an avatar personality. Specific software examples include a machine learning platform and a natural language processing engine. The created personality data is sent to the device.

[1955] Device: Initial settings are made based on the received avatar personality data. For example, the avatar may say, "Hello, I'm your new lifemate."

[1956] Examples:

[1957] The avatar greets, "Hello, Tanaka-san. How was your day?"

[1958] 3. Introducing the Emotion Engine

[1959] Device: Equipped with an emotion engine that analyzes the user's voice and facial expressions. Examples of specifications include a camera-equipped device and a voice recognition microphone.

[1960] Server: Receives emotional data analyzed by the emotion engine and adjusts the avatar's response content and tone.

[1961] Examples:

[1962] The user says, "I'm feeling a bit tired today." The emotion engine interprets this and the avatar responds, "That must be tough. Why don't you take a break?"

[1963] 4. Daily life support

[1964] Reminder function:

[1965] User: Instructs the avatar on daily schedules and tasks.

[1966] Server: Stores the information in the reminder setting database and sends a notification to the device at the specified time.

[1967] Terminal: An avatar informs the user, "You have a doctor's appointment tomorrow at 2 PM."

[1968] Health management function:

[1969] IoT devices: Collect user health data (e.g., heart rate, steps, blood pressure, etc.). Specific examples of such devices include fitness bands and smartwatches.

[1970] Server: Receives and analyzes the collected data. For example, it generates advice such as "If your step count is low, you should walk a little more."

[1971] Device: The avatar provides health advice based on the analysis results, for example, telling the user, "You walked 5,000 steps today. You're very healthy."

[1972] Examples:

[1973] If a user requests, "Set a reminder to take my medicine at 9:00 tomorrow," the server will set this reminder and the device will notify them at the specified time.

[1974] 5. Continuous interaction and learning

[1975] Interactive features:

[1976] Users: Ask the avatar everyday topics and questions.

[1977] Server: Based on the received dialogue data, generates appropriate information and sends it to the terminal.

[1978] Terminal: The avatar will tell the user appropriate information, such as "Here's today's news."

[1979] Learning features:

[1980] Server: Continuously collects user interaction and emotional data and updates the avatar's personality using an AI model, allowing the avatar to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[1981] Device: Receive updated personality data and keep your avatar settings up to date.

[1982] Example prompt sentence:

[1983] "What news is there today?"

[1984] "Tell me about the weather lately."

[1985] "When's your next doctor's appointment?"

[1986] In this way, this system generates and operates customizable avatars based on personal information and emotional data entered by the user, providing comprehensive support for the user's daily life.

[1987] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1988] Step 1:

[1989] User Registration and Customization

[1990] Input: Users install applications on their devices and enter personal information such as name, birthday, gender, hobbies, etc. They also customize the appearance and voice of their avatar.

[1991] Operation: The device sends the entered personal information and customization information to the server. Specifically, the user fills in the information in the input form and presses the send button.

[1992] Output: The server generates a user profile based on the received data and creates initial avatar data. This initial data is then sent to the device.

[1993] Step 2:

[1994] Avatar personality development

[1995] Input: The server receives the user's personal information and customization data and inputs it into an artificial intelligence model (e.g., GPT-4).

[1996] How it works: The server uses an artificial intelligence model to generate the avatar's personality, passing the incoming data to a natural language processing engine and processing the output from the model.

[1997] Output: The generated avatar's personality data is sent to the device. The device configures the avatar based on the received data and displays it on the interface.

[1998] Step 3:

[1999] Completing the initial setup and first greetings

[2000] Input: The device performs initial settings based on the avatar's personality data received from the server.

[2001] How it works: The device applies the avatar's appearance, voice, and personality data to display the avatar for the first time. The avatar then greets the user for the first time.

[2002] Output: The avatar will start a customized interaction with the user, such as "Hello, I'm your new lifemate."

[2003] Step 4:

[2004] Emotion engine analysis and response adjustment

[2005] Input: The device collects the user's voice and facial expressions through the camera and microphone.

[2006] How it works: The emotion engine analyzes the collected data, extracts emotion data, and sends this data to the server.

[2007] Output: The server adjusts the avatar's response content and tone based on the emotional data and sends the data to the device. The avatar on the device then provides the adjusted response to the user.

[2008] Step 5:

[2009] Reminder settings and notifications

[2010] Input: The user gives the avatar instructions for daily schedules and tasks, such as "Set a reminder to take my medicine tomorrow at 9 AM."

[2011] How it works: The device sends instructions from the user to the server, which stores the information in the reminder settings database. When the specified time arrives, the server sends the notification data to the device.

[2012] Output: The device receives notification data from the server, and the avatar notifies the user, such as "It's time to take your medicine now."

[2013] Step 6:

[2014] Health management features for data collection and feedback

[2015] Input: IoT devices collect user health data (e.g., heart rate, steps, blood pressure, etc.).

[2016] How it works: The IoT device sends the collected data to a server, which analyzes the data and generates advice such as "You should walk a little more" if the number of steps is low.

[2017] Output: The analysis results are sent to the device, and the avatar tells the user, "You walked 5,000 steps today. You're very healthy."

[2018] Step 7:

[2019] Continuous interaction and learning

[2020] Input: The user asks the avatar everyday topics and questions, such as "What's in the news today?"

[2021] How it works: The device sends the user's question data to the server, which then generates appropriate information based on the received data and sends it to the device. Furthermore, the server continuously collects conversation data and emotional data with the user, and updates the avatar's personality using an artificial intelligence model.

[2022] Output: The device receives information from the server, the avatar tells the user "Here's today's news" and provides the appropriate information, and keeps the avatar's settings up to date.

[2023] As described above, the program of the system of the present invention provides a function to support the daily lives of elderly people and alleviate their feelings of loneliness through a series of processing steps.

[2024] (Application example 2)

[2025] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2026] To support the daily lives of elderly people while alleviating their feelings of loneliness, simple reminder and health management functions are insufficient. Support tailored to individual needs through continuous interaction with the elderly and flexible response to emotional changes is necessary. However, current systems lack the ability to read the user's unique emotions and respond appropriately based on them, and therefore are unable to provide sufficient psychological satisfaction and security to the elderly.

[2027] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2028] In this invention, the server includes means for generating a customizable virtual character based on personal information entered by the user, means for forming the personality of the generated virtual character using an artificial intelligence model, and means for adjusting the content and tone of the virtual character's responses based on emotional data from an emotion analysis engine. This enables personalized responses that are sensitive to the emotions of the elderly, providing them with a greater sense of satisfaction and security.

[2029] "user"

[2030] refers to individuals who use the system, often specifically elderly people.

[2031] "personal information"

[2032] This refers to data entered by the user to identify the user, such as name, birthday, gender, hobbies, and preferences.

[2033] "Virtual Character"

[2034] is a digital character that is customized based on a user's personal information, with visual and audio elements including appearance and voice.

[2035] "Artificial Intelligence Model"

[2036] refers to algorithms and learning models that use user input data to shape the personality of a virtual character.

[2037] Emotion analysis engine

[2038] This refers to a system that analyzes a user's voice, facial expressions, etc. to detect their emotional state.

[2039] Reminders

[2040] This refers to a function that notifies users of scheduled events and tasks in a timely manner.

[2041] "IoT devices"

[2042] refers to a physical device that is connected to the Internet and is capable of sending and receiving sensor data, including wearable devices for health management.

[2043] "Health status"

[2044] refers to biometric data such as a user's heart rate, blood pressure, number of steps, etc. These data are used to monitor the user's health.

[2045] "Response content"

[2046] refers to the responses that a virtual character provides based on the user's questions and emotional state, including information and advice.

[2047] "tone"

[2048] This refers to the tone of voice and speaking style of a virtual character, which is adjusted by an emotion analysis engine, depending on the user's emotional state.

[2049] This invention provides a system that supports the daily lives of elderly people and alleviates their feelings of loneliness by generating a customizable virtual character based on the user's personal information and combining it with an emotion analysis engine. This system is composed of a terminal, a server, and an IoT device. The details of this system are described below.

[2050] System configuration

[2051] 1. User registration and virtual character customization

[2052] The terminal provides an interface where the user installs the application and inputs necessary personal information (such as name, birthday, gender, hobbies, and preferences). The user can then customize the virtual character's appearance (human, animal, etc.) and voice (male, female, young, adult). The server receives the input personal information and customization information and generates a user profile. Based on this information, the server then generates initial setting data for the virtual character and sends it to the terminal.

[2053] 2. Personality development of virtual characters

[2054] The server uses a generative AI model to create a virtual character's personality based on the user's personal information and input data. This uses natural language processing and machine learning algorithms. The created personality data is then sent to the device. The device then performs initial setup based on the received virtual character's personality data and greets the user for the first time.

[2055] 3. Introduction of a sentiment analysis engine

[2056] The device is equipped with an emotion analysis engine that analyzes the user's voice and facial expressions. If the user is using a device with a camera, facial expressions are also detected. The server receives the emotional data analyzed by the emotion analysis engine and adjusts the content and tone of the virtual character's responses.

[2057] 4. Daily life support

[2058] The device can instruct the virtual character about the user's daily schedule and tasks (such as doctor's appointments and shopping lists). It also provides reminder setting and health management functions. The server receives the information specified by the user, stores it in a reminder setting database, and sends notifications to the device at the specified time. It also analyzes health data (heart rate, number of steps, blood pressure, etc.) collected from IoT devices and generates health advice.

[2059] 5. Continuous interaction and learning

[2060] The device has a dialogue function that allows users to ask the virtual character everyday topics and questions. The server generates appropriate information based on the dialogue data received and sends it to the device. It also continuously collects dialogue data and emotional data and updates the virtual character's personality using a generative AI model. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses.

[2061] Specific examples

[2062] User Registration and Customization

[2063] The user launches the app on the device, inputs their name "General Taro," their date of birth "January 1, 1945," and their hobby "reading," selects an "animal" as the virtual character's appearance, and sets the voice to a "calm female voice." The server saves the user information, generates initial setting data for the virtual character, and sends it to the device. The virtual character greets the device by saying, "Hello, Taro. I'm your new friend."

[2064] Utilizing Emotion Recognition

[2065] When a user says, "I feel a little tired today," the device uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data. The server then adjusts the content and tone of the virtual character's response based on the emotional data, and the virtual character then comforts the user by saying, "I understand. Please take it easy today."

[2066] Example prompts to input to a generative AI model:

[2067] Username: General Taro

[2068] User input: Set a reminder to take my medicine tomorrow at 9am

[2069] Emotion engine: Analyzes user input and extracts emotional data (e.g., "I'm tired")

[2070] Virtual character response: Sets a reminder and responds to the user (e.g., "I've set a reminder to take my medicine tomorrow at 9 o'clock.")

[2071] In this way, the system of the present invention can respond to the user's emotions and individual needs, and can support the elderly in their daily lives while alleviating their sense of loneliness.

[2072] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2073] Step 1:

[2074] The user inputs personal information (such as name, birthday, gender, hobbies, and preferences) through the device. The device acquires this information and sends it to the server. The personal information is the input data, and the data sent to the server is the output data.

[2075] Step 2:

[2076] The server receives the personal information sent by the user and generates a user profile. The server stores the personal information in a database and generates initial setting data for the virtual character. This initial setting data includes customization information such as the virtual character's appearance and voice. The input data is personal information, and the output is the initial setting data, which is then sent to the terminal.

[2077] Step 3:

[2078] The device displays the virtual character and greets the user based on the initial setting data of the virtual character received from the server. Specifically, it plays a voice and text such as "Hello, Taro. I'm your new friend." The initial setting data is the input data, and the greeting is displayed as the output.

[2079] Step 4:

[2080] The user inputs daily plans and questions to the virtual character. For example, they can give instructions such as "Set a reminder to take my medicine at 9:00 tomorrow." The device acquires the user's input and sends it to the server. The input data includes the user's instructions, and the output data is sent to the server.

[2081] Step 5:

[2082] The server receives instructions sent by the user, analyzes them, and sets reminders. Specifically, it saves the information in a reminder setting database and prepares to send a reminder notification to the device at the specified time. The input data is the user's instructions, and the output is reminder setting data.

[2083] Step 6:

[2084] The device will notify the user of the reminder at the specified time based on the reminder setting data received from the server. Specifically, it will display a voice and text message saying, "It's time to take your medicine now." The input data is the reminder setting data, and the output is a reminder notification.

[2085] Step 7:

[2086] To analyze a user's emotions, the device analyzes the user's voice and facial expressions using an emotion analysis engine. The emotion analysis engine receives the user's voice and facial expression data and detects their emotional state (for example, "tired" or "energetic"). The input data is the user's voice and facial expression, and the output is emotion data.

[2087] Step 8:

[2088] The server adjusts the content and tone of the virtual character's responses based on the emotional data. For example, if a user says, "I'm a little tired today," the server generates an adjusted response, such as "I understand. Please take it easy today," and sends it to the device. Emotional data is the input data, and adjusted response data is generated as the output.

[2089] Step 9:

[2090] The device receives the adjusted response data from the server, and the virtual character responds to the user with voice and text. Specifically, the virtual character responds to the user by saying, "I understand. Please take a good rest today." The adjusted response data is input, and the response is displayed as output.

[2091] Step 10:

[2092] The server continuously collects user interaction and emotional data and uses a generative AI model to update the virtual character's personality. This allows the virtual character to better understand the user's preferences and behavioral patterns and provide appropriate responses. The input data includes interaction data and emotional data, and the output is updated personality data.

[2093] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2094] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2095] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2096] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2097] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric...

Claims

1. means for generating a customizable avatar based on personal information entered by the user; A means for forming the personality of the generated avatar using an artificial intelligence model; A way to provide reminders and information based on users' daily schedules and questions, and A means of linking with IoT devices to manage and provide feedback on the user's health status, A way for the avatar to learn and update its personality through continuous interaction with the user, A system including:

2. 10. The system of claim 1, further comprising means for providing an interface for a user to select customizations for the appearance and voice of the avatar.

3. The system of claim 1 , further comprising means for storing received IoT device data for analyzing a user's health status and providing health advice based thereon.

Images