System

Abstract

To provide a system capable of achieving support for elderly people living alone and early detection of dementia.SOLUTION: Provided is a system with an AI partner that can be customized according to a user's preferences and interests, the system comprising: means for estimating an emotional state judging from the user's tone of voice, manner of speaking, used words, etc. with help of an emotion engine; means for generating prompt sentences to instruct generation of appropriate contents of conversation according to a current emotional state of the user on the basis of the estimated emotional state of the user; means for generating contents of conversation with the user using the generated prompt sentences and the AI partner; means for replying the generated contents of conversation to the user with voice or text; means for finding out an early symptom of dementia through conversation with the user by analyzing changes, etc. in conversation patterns and memory in a short period of time; and means for transmitting an alert to his / her family or a medical institution if the user shows a symptom that may be dementia.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need to reduce loneliness among elderly people living alone and to detect early symptoms of dementia, but there is no effective system to solve these issues. [Means for solving the problem]

[0005] This invention provides an AI companion that can be customized to suit the user's preferences and interests. This AI companion reduces the user's sense of loneliness through daily conversations and simultaneously detects early symptoms of dementia. It also analyzes conversation patterns and short-term changes in memory, and sends alerts to family members and medical institutions if it detects any worrisome symptoms. This will enable support for elderly people living alone and early detection of dementia. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention provides a system for supporting elderly people living alone. This system includes an AI companion that can be customized to suit the user's preferences and interests. The AI ​​companion engages in daily conversations with the user, helping to reduce the user's sense of loneliness through these conversations. Furthermore, the AI ​​companion has the ability to detect early symptoms of dementia through dialogue with the user. Specifically, the AI ​​companion analyzes conversation patterns and short-term changes in memory, and if any worrisome symptoms are detected, it sends an alert to family members or medical institutions.

[0029] "Example 2"

[0030] In another embodiment of the present invention, the AI ​​buddy can adjust the conversation content according to the user's daily life situation and health condition. For example, if the user has a cold, the AI ​​buddy can ask about the symptoms of the cold and advise the user to get adequate rest.

[0031] "Example 3"

[0032] In a further embodiment of the present invention, the system further includes a function for the AI ​​buddy to estimate the user's emotional state and adjust the conversation content according to the estimated emotional state. For example, if the AI ​​buddy estimates that the user is feeling depressed, the AI ​​buddy may offer words of encouragement or start a conversation about the user's favorite topics.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The AI ​​companion is customized based on the user's preferences and interests. For example, if the user is interested in music or movies, the AI ​​companion will provide conversations about those topics.

[0036] Step 2: The AI ​​companion will have daily conversations with the user, which are designed to reduce the user's sense of loneliness.

[0037] Step 3: AI Aibou will detect early signs of dementia through dialogue with the user, specifically by analyzing conversation patterns and short-term changes in memory.

[0038] Step 4: If any worrisome symptoms are observed, your AI companion will send an alert to your family or a medical institution.

[0039] "Example 2"

[0040] Step 1: The AI ​​companion will adjust the conversation content according to the user's daily life situation and health condition. For example, if the user has a cold, the AI ​​companion will ask about the symptoms of the cold and advise them to get adequate rest.

[0041] "Example 3"

[0042] Step 1: The AI ​​companion estimates the user's emotional state, which is inferred from the user's tone of voice, speaking style, and words used.

[0043] Step 2: The AI ​​buddy adapts the conversation based on the estimated emotional state. For example, if the AI ​​buddy estimates that the user is feeling down, it might offer words of encouragement or bring up topics the user likes.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] For elderly people living alone, it is important to reduce feelings of loneliness and detect early symptoms of dementia early. However, effective methods to achieve this are lacking. There is also a need for systems that can provide conversations tailored to the user's preferences and interests, and adjust the content of conversations according to their health status. Furthermore, there is a need for technology that can accurately analyze users' comments and generate appropriate responses. A system that can solve these issues is needed.

[0047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0048] In this invention, the server includes a means for converting a user's speech into text data using speech recognition software, a means for inputting prompts into a generative AI model to generate a response, and a means for displaying the generated response to the user. This reduces the user's sense of loneliness and enables early detection of early symptoms of dementia. It also enables the provision of conversations tailored to the user's preferences and interests, and the content of the conversation to be adjusted according to the user's health condition.

[0049] The "Artificial Intelligence Partner" is a system that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[0050] "Voice recognition software" is a technology that converts a user's speech from voice data into text data.

[0051] A "generative AI model" is an artificial intelligence technology that generates appropriate responses based on input prompts.

[0052] A "prompt sentence" is text data input into a generative AI model and is an instruction sentence for generating a response.

[0053] "Text Data" means the written information of a user's speech converted by voice recognition software.

[0054] An "alert" is a warning message sent to notify family members or medical institutions when an abnormality is detected.

[0055] "Conversation data" is a record of the conversation between the user and the AI ​​companion.

[0056] An "anomaly" is an abnormal condition, such as early symptoms of dementia, detected based on changes in a user's speech patterns or memory.

[0057] The present invention is a system for supporting elderly people living alone, including an AI companion that can be customized to suit the user's preferences and interests. The system engages in daily conversations with the user, helping to reduce the user's sense of loneliness. The AI ​​companion also has the ability to detect early symptoms of dementia through dialogue with the user.

[0058] Hardware and software used

[0059] Hardware:

[0060] Devices such as smartphones and tablets

[0061] server

[0062] software:

[0063] Voice recognition software (e.g., Google® Speech-to-Text)

[0064] Natural language processing technology (e.g., NLTK and spaCy)

[0065] Generative AI models (e.g., OpenAI's GPT-4)

[0066] Data processing and calculation

[0067] Processing user utterances:

[0068] When a user initiates a conversation with their AI partner through the device, the device uses voice recognition software to convert the user's speech into text, which is then sent to a server.

[0069] Analyzing text data:

[0070] The server analyzes the received text data using natural language processing technology, and as a result of the analysis, it understands the intent and content of the user's comments.

[0071] Response generation:

[0072] Based on the analysis results, the server inputs a prompt sentence into the generative AI model, which then generates an appropriate response based on this prompt sentence.

[0073] View response:

[0074] The server generates a response and sends it to the terminal, which displays the response to the user.

[0075] Accumulation of conversation data and monitoring for anomalies:

[0076] The server stores conversation data with the user in a database and monitors changes in the user's conversation patterns and memory. If an abnormality is detected, the server sends an alert to family members or medical institutions.

[0077] Specific examples

[0078] For example, if a user says, "I can't remember what I had for dinner today, what was for dinner last night?", the following happens:

[0079] 1. User: "I can't remember what I had for dinner today. What was for dinner last night?"

[0080] 2. Terminal: Uses voice recognition software to convert speech into text data and send it to the server.

[0081] 3. Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[0082] 4. Generative AI model: Generates the response, "You had curry rice for dinner last night. What are you planning to eat today?"

[0083] 5. Server: Sends the generated response to the terminal.

[0084] 6. Terminal: displays the response to the user.

[0085] 7. Server: Stores conversation data and monitors for abnormalities. If an abnormality is detected, an alert is sent.

[0086] Example prompt sentence:

[0087] User: "I can't remember what I had for dinner today. What was for dinner last night?"

[0088] AI Buddy: "You had curry rice for dinner last night. What are you planning to eat today?"

[0089] In this way, the system of the present invention can reduce the user's sense of loneliness and detect early symptoms of dementia. It can also provide conversations tailored to the user's preferences and interests, and adjust the content of conversations according to the user's health condition.

[0090] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0091] Step 1:

[0092] The user initiates a conversation with the AI ​​partner through the terminal.

[0093] Input: User's spoken utterance

[0094] What happens: A user says, "I can't remember what I had for dinner today. What was for dinner last night?"

[0095] Output: Audio data

[0096] Step 2:

[0097] The device converts the user's speech into text data using voice recognition software.

[0098] Input: Audio data

[0099] What happens: Your device uses voice recognition software (e.g., Google Speech-to-Text) to convert voice data into text.

[0100] Output: Text data

[0101] Step 3:

[0102] The terminal transmits the converted text data to the server.

[0103] Input: Text data

[0104] Specific operation: The terminal sends text data to the server.

[0105] Output: Text data sent to the server

[0106] Step 4:

[0107] The text data received by the server is analyzed using natural language processing technology.

[0108] Input: Text data

[0109] Specific operation: The server uses natural language processing technology (e.g., NLTK or spaCy) to analyze the text data and understand the intent and content of the user's statements.

[0110] Output: Analysis results

[0111] Step 5:

[0112] Based on the analysis results, the server inputs a prompt sentence into the generative AI model and generates a response.

[0113] Input: Analysis results

[0114] How it works: The server creates a prompt based on the analysis results and inputs it into a generative AI model (e.g., OpenAI's GPT-4), which then generates an appropriate response.

[0115] Output: The generated response

[0116] Step 6:

[0117] The server generates a response and sends it to the terminal.

[0118] Input: Generated response

[0119] Specific operation: The server sends the generated response to the terminal.

[0120] Output: Response sent to the terminal

[0121] Step 7:

[0122] The terminal displays the response to the user.

[0123] Input: Response sent to terminal

[0124] Specific operation: The device displays the response to the user, who then sees the AI ​​partner's response on the device screen.

[0125] Output: The response displayed to the user

[0126] Step 8:

[0127] The server stores conversation data and monitors for abnormalities.

[0128] Input: Conversation data

[0129] Specific operation: The server stores conversation data with the user in a database and monitors changes in the user's conversation patterns and memories.

[0130] Output: Accumulated conversation data and monitoring results

[0131] Step 9:

[0132] If the server detects an abnormality, it will send an alert.

[0133] Input: Monitoring results

[0134] Specific operation: If the server detects an abnormality, it will send an alert to family members or medical institutions via email or SMS.

[0135] Output: Alerts sent

[0136] (Application example 1)

[0137] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0138] For elderly people living alone, reducing feelings of loneliness and early detection of early symptoms of dementia are important issues. However, systems to solve these issues are limited, and there is a lack of support for elderly people to enjoy shopping with peace of mind, especially in physical stores. Furthermore, while services for elderly people in physical stores need to be tailored to individual users' preferences and health conditions, current systems are not adequately able to do so.

[0139] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0140] In this invention, the server includes: a means for including an AI buddy that can be customized to suit the user's preferences and interests; a means for the AI ​​buddy to engage in entertaining daily conversations; a means for detecting early symptoms of dementia through dialogue with the user; a means for analyzing conversation patterns and short-term changes in memory; a means for sending an alert to family or medical institutions if any worrisome symptoms are observed; a means for providing the AI ​​buddy function to elderly customers through robots or smart displays installed in physical stores; and a means for an AI concierge to understand the customer's preferences and interests through conversation when the elderly customer visits the store and suggest appropriate products and services. This allows elderly people to enjoy shopping in physical stores with peace of mind, reduces feelings of loneliness, and enables early detection of early symptoms of dementia.

[0141] "User" refers to an individual who uses the System.

[0142] "Preferences" refer to the things that a user particularly likes or is interested in.

[0143] "Interests" refers to the things or areas that a user is interested in.

[0144] "Customizable" means that settings and features can be changed to suit the user's individual needs and preferences.

[0145] "AI Partner" refers to a part of the system that uses artificial intelligence to interact with and assist users.

[0146] "Daily conversations" refer to the daily interactions between the user and their AI companion.

[0147] "Early symptoms of dementia" refers to symptoms such as a decline in memory and judgment that are seen in the early stages of dementia.

[0148] "Conversation patterns" refer to the format and content of repeated dialogue between the user and their AI companion.

[0149] "Short-term changes in memory" refers to how a user's memory changes over a short period of time.

[0150] An "alert" is a warning or notification that is issued when a specific condition is met.

[0151] "Family" refers to the User's immediate family or close relatives.

[0152] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0153] "Brick and mortar store" refers to a commercial establishment located in a physical location.

[0154] A "robot" refers to a mechanical device that is programmed to perform specific tasks.

[0155] A "smart display" refers to a display device that has internet connectivity and displays information.

[0156] The term "elderly" generally refers to people aged 65 and over.

[0157] "Customers" refer to consumers who visit physical stores.

[0158] "AI concierge" refers to a part of a system that uses artificial intelligence to provide guidance and assistance to customers.

[0159] "Goods" means items sold in physical stores.

[0160] "Services" refers to various assistance and benefits provided in physical stores.

[0161] This invention is a system that includes an AI companion that can be customized to suit the user's preferences and interests, and provides the AI ​​companion function to elderly customers in physical stores. Specific embodiments of this system are described below.

[0162] System configuration

[0163] The system consists of the following main components:

[0164] 1. User device: A device such as a smartphone or tablet.

[0165] 2. Robots or smart displays in brick-and-mortar stores: devices to interact with elderly customers.

[0166] 3. Server: The back-end system for running generative AI models and processing data.

[0167] Program processing

[0168] The server performs the following process.

[0169] 1. Customization of user preferences and interests: When users first access the system, they enter information about their preferences and interests, which is used to customize their AI companion.

[0170] 2. Engaging in everyday conversations: When users visit a physical store, robots and smart displays will engage in everyday conversations with them, using generative AI models.

[0171] 3. Early Dementia Symptom Detection: Analyzes short-term changes in speech patterns and memory to detect early signs of dementia. If specific keywords or phrases are detected, an alert will be sent.

[0172] 4. Sending alerts: If early symptoms of dementia are detected, an alert will be sent to family members or medical institutions.

[0173] Hardware and software used

[0174] Hardware: smartphones, tablets, robots, smart displays

[0175] Software: Generative AI models (e.g., OpenAI APIs)

[0176] Specific examples

[0177] For example, when an elderly customer visits a physical store, the following conversation takes place:

[0178] Example conversation

[0179] User: "Hi, I've been forgetting things a lot lately."

[0180] AI Buddy: "Hello, Tanaka-san. Can you tell me about recent events?"

[0181] During this conversation, the AI ​​companion analyzes the user's memory and conversation patterns and sends out alerts as needed.

[0182] Prompt Sentence Examples

[0183] User: Hi, I've been forgetting things a lot lately.

[0184] AI Companion:

[0185] By inputting this prompt into a generative AI model, an appropriate response can be obtained.

[0186] As described above, this system allows elderly people to enjoy shopping in physical stores with peace of mind, reduces feelings of loneliness, and makes it possible to detect early symptoms of dementia early.

[0187] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0188] Step 1:

[0189] When a user first accesses the system, they enter information about their preferences and interests using their user terminal.

[0190] Input: Information about your preferences and interests (e.g., favorite foods, hobbies)

[0191] Output: Customized user profile

[0192] Specific operation: The user terminal sends the input information to the server, and the server generates a user profile based on the information.

[0193] Step 2:

[0194] When a user visits a physical store, the store's robot or smart display will recognize the user and begin a casual conversation.

[0195] Input: User visit information, user profile

[0196] Output: Generated conversation

[0197] How it works: The robot or smart display detects a user's visit and queries the server for the user's profile. The server then uses the generative AI model to generate appropriate conversation content and sends it to the robot or smart display.

[0198] Step 3:

[0199] The server analyzes conversation patterns and short-term changes in memory to detect early symptoms of dementia through conversations with the user.

[0200] Input: User conversation

[0201] Output: Detection results of early symptoms of dementia

[0202] How it works: The server uses a generative AI model to analyze the conversation and detect specific keywords and phrases, which can then determine whether early symptoms of dementia are present.

[0203] Step 4:

[0204] If early symptoms of dementia are detected, the server will send an alert to family members and medical institutions.

[0205] Input: Detection results of early symptoms of dementia

[0206] Output: Alert notification

[0207] Specific operation: The server generates an alert based on the detection results and sends a notification to pre-registered family and medical contacts.

[0208] Step 5:

[0209] Robots or smart displays in physical stores suggest appropriate products and services based on users' preferences and interests.

[0210] Input: User profile, generated conversation

[0211] Output: Product or service suggestions

[0212] Specific operation: The robot or smart display suggests products and services suitable for the user based on the user profile and conversation content received from the server.

[0213] Through these steps, this system will enable elderly people to enjoy shopping in physical stores with peace of mind, reduce feelings of loneliness, and enable early detection of early symptoms of dementia.

[0214] Example 2

[0215] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0216] While existing AI companion systems can be customized to suit a user's preferences and interests, they are inadequate in adjusting the content of conversations to suit the user's daily life situation and health condition. Furthermore, they lack the ability to assess a user's health condition in real time and provide appropriate advice, limiting their support for health management. Furthermore, they lack the technical means to accurately analyze user input and generate appropriate conversation content. To address these issues, a system capable of more advanced analysis and response generation is needed.

[0217] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0218] In this invention, the server includes means for receiving user input and converting the voice into text using voice recognition software, means for analyzing the text data using a generative AI model to determine the user's health condition, and means for generating conversation content according to the user's situation using the generative AI model, thereby making it possible to provide appropriate conversation content according to the user's daily life situation and health condition.

[0219] "AI Buddy" is an interactive AI system that provides various information through dialogue with the user and can be customized to suit the user's preferences and interests.

[0220] "Speech recognition software" means software that converts a user's voice input into text data.

[0221] A "generative AI model" is an artificial intelligence model that analyzes input text data and generates appropriate responses and conversation content.

[0222] "Text data" refers to a user's voice input converted by voice recognition software or text information entered directly by a user.

[0223] "Health Status" refers to the state of a User's physical and mental health, including specific symptoms such as colds or fatigue.

[0224] "Conversation content" refers to the information and advice provided by the AI ​​partner during the dialogue with the user.

[0225] "Analysis" is the process by which the generative AI model determines the user's situation and health status based on the input text data.

[0226] "Response generation" is the process by which a generative AI model creates appropriate conversation content based on the analysis results.

[0227] "Speech synthesis software" is software that converts generated text data into speech and responds to the user in speech.

[0228] The present invention relates to an AI companion system that adjusts the content of conversations depending on the user's daily life situation and health condition. Specific embodiments of this system will be described below.

[0229] Hardware and software used

[0230] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[0231] Software: Generative AI models (e.g., GPT-4), speech recognition software (e.g., Google Speech-to-Text API), speech synthesis software (e.g., Google Text-to-Speech API)

[0232] System Overview

[0233] The system receives user input, converts speech to text using speech recognition software, analyzes the text data using a generative AI model to determine the user's health status, and then uses the generative AI model to generate conversational content tailored to the user's situation, returning the generated conversational content to the user in voice or text.

[0234] Specific examples

[0235] Consider the case where a user speaks into a smartphone microphone, "I'm feeling a bit under the weather today." This voice input is converted into text data using speech recognition software. Next, the server inputs this text data into a generative AI model, which recognizes the keyword "feeling under the weather" and determines that the user has a cold. The server then inputs a prompt sentence into the generative AI model: "If the user has a cold, ask about their cold symptoms and advise them to get adequate rest." The model then generates conversational content such as "How are your cold symptoms?" and "Please get plenty of rest today." Finally, the generated text data is converted into speech using speech synthesis software, and a voice response is provided to the user, or the same content is displayed in text form.

[0236] Prompt Sentence Examples

[0237] "If the user has a cold, the AI ​​companion should generate conversational content that asks about cold symptoms and advises them to get adequate rest."

[0238] In this way, the AI ​​companion can provide appropriate conversation content based on the user's health condition.

[0239] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0240] Step 1:

[0241] Users use their smartphone or PC to talk to their AI companion via voice or text.

[0242] Input: User voice or text input

[0243] What happens: The user speaks into the smartphone microphone and says, "I'm feeling a bit under the weather today," or types the same thing using text input.

[0244] Output: User's voice or text data

[0245] Step 2:

[0246] The server uses speech recognition software to convert the user's speech into text.

[0247] Input: User's voice data

[0248] What happens: The server receives the voice data and uses speech recognition software (e.g., Google Speech-to-Text API) to convert it into text data such as "I'm feeling a bit under the weather today."

[0249] Output: Text data

[0250] Step 3:

[0251] The server uses the generated AI model to analyze the text data and determine the user's health status.

[0252] Input: Text data

[0253] Specific operation: The server inputs text data into the generative AI model, and the model recognizes the keyword "feeling like I have a cold" and determines that the user has a cold.

[0254] Output: Information about the user's health status (e.g., they have a cold)

[0255] Step 4:

[0256] The server uses a generative AI model to generate conversation content that is appropriate for the user's situation.

[0257] Input: User's health information

[0258] Specific operation: The server inputs the prompt sentence "If the user has a cold, ask about their cold symptoms and advise them to get adequate rest" into the generative AI model, and the model generates conversational content such as "What are your cold symptoms?" and "Please get plenty of rest today."

[0259] Output: Generated conversation content (text data)

[0260] Step 5:

[0261] The device responds to the user with the generated conversation content in voice or text.

[0262] Input: Generated conversation content (text data)

[0263] Specific operation: The server sends the generated text data to the device, which then uses speech synthesis software (e.g., Google Text-to-Speech API) to convert it into speech, asking the user, "How are your cold symptoms?" or "Please get plenty of rest today," or displaying the same information in text.

[0264] Output: A voice or text response to the user

[0265] (Application example 2)

[0266] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0267] In modern society, there is a demand for providing appropriate advice tailored to users' daily lives and health conditions. However, conventional systems have difficulty adjusting the content of conversations based on the user's health condition and daily life situation, and there is a lack of means to effectively utilize generative AI models to provide appropriate advice. Therefore, an effective system is needed to improve users' health management and quality of life.

[0268] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0269] In this invention, the server includes a means for including an AI companion that can be customized to suit the user's preferences and interests, a means for adjusting the content of the conversation depending on the user's daily life situation and health condition, and a means for providing appropriate advice using a generative AI model based on the user's health condition, thereby enabling the provision of appropriate advice according to the user's health condition and daily life situation, thereby improving the user's health management and quality of life.

[0270] "User" refers to an individual who uses the System.

[0271] "Preferences" refer to the things that a user particularly likes or is interested in.

[0272] "Interests" refers to the things or areas that a user is interested in.

[0273] "Customizable" means that settings and features can be changed according to the user's preferences and interests.

[0274] "AI Partner" refers to a system that uses artificial intelligence to interact with users and provide support.

[0275] "Daily life" refers to the activities and lifestyle habits of the user on a daily basis.

[0276] "Health Status" refers to the state of your physical and mental health.

[0277] "Conversation content" refers to the content of the dialogue that the AI ​​partner has with the user.

[0278] A "generative AI model" refers to a model that uses artificial intelligence to generate text and data.

[0279] "Advice" means any advice or suggestion provided to a User.

[0280] "Dementia" refers to a disease characterized by a decline in memory and cognitive function.

[0281] "Early symptoms" refer to symptoms seen in the early stages of the onset of a disease.

[0282] A "pattern" refers to a series of actions or phenomena that have a particular regularity or tendency.

[0283] "Short-term changes in memory" refers to fluctuations or changes in memory over a short period of time.

[0284] An "alert" is a notification or warning intended to draw attention.

[0285] "Family" refers to people related to you by blood or marriage.

[0286] "Medical institution" refers to a facility or organization that provides medical services.

[0287] "Emotional state" refers to the user's emotional or mood state.

[0288] A system for implementing this invention includes an AI companion that can be customized to a user's preferences and interests, adapts conversations based on the user's daily life situation and health status, and provides appropriate advice using a generative AI model based on the user's health status.

[0289] System Configuration

[0290] The system uses the following hardware and software:

[0291] Hardware: Smartphones, servers

[0292] Software: Python, OpenAI API

[0293] Program processing explanation

[0294] The server provides an interface for acquiring the user's health status. The user inputs their current health status using their smartphone. For example, they can input their status such as "cold," "headache," or "feeling well."

[0295] The server generates a prompt based on the user's health status. This prompt is then input into a generative AI model to generate appropriate advice. The generative AI model used is OpenAI's "text-davinci-003."

[0296] The generated advice is sent from the server to the user's smartphone and displayed, allowing the user to receive appropriate advice based on their own health condition.

[0297] Specific examples

[0298] For example, if the user types "cold," the generated prompt text will be:

[0299] If the user has a cold, provide appropriate advice.

[0300] If you feed this prompt into a generative AI model, it might generate responses like these:

[0301] If you have a cold, it's important to get plenty of rest, drink plenty of fluids, eat foods with vitamin C, and keep warm. Consult your doctor if necessary.

[0302] In this way, appropriate advice can be provided according to the user's health condition.

[0303] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0304] Step 1:

[0305] The user inputs their current health status using their smartphone.

[0306] Input: User's health condition (e.g., "cold," "headache," "feeling well")

[0307] Output: Health status data

[0308] Specific actions: The user opens the application on their smartphone, enters their current health status in the text box, and presses the send button.

[0309] Step 2:

[0310] A server receives health status data sent by a user.

[0311] Input: Health status data

[0312] Output: Acknowledgement message

[0313] Specific operation: Health status data sent from the smartphone reaches the server, which receives and stores the data.

[0314] Step 3:

[0315] The server generates a prompt sentence based on the health status data.

[0316] Input: Health status data

[0317] Output: prompt statement

[0318] Specific operation: The server analyzes the received health status data and generates a prompt sentence, for example, "If the user has a cold, please provide appropriate advice."

[0319] Step 4:

[0320] The prompt sentence generated by the server is input into the generative AI model.

[0321] Input: prompt statement

[0322] Output: Input data to a generative AI model

[0323] Specific operation: The server sends the generated prompt text to OpenAI's API and inputs it into the generative AI model (e.g., text-davinci-003).

[0324] Step 5:

[0325] A generative AI model generates appropriate advice based on the prompt.

[0326] Input: prompt statement

[0327] Output: Advice text

[0328] Specific operation: The generative AI model analyzes the prompt sentence and generates advice, for example, "If you have a cold, it is important to get plenty of rest and drink plenty of fluids."

[0329] Step 6:

[0330] The server receives the generated advice text and transmits it to the smartphone.

[0331] Input: Advice text

[0332] Output: Send confirmation message

[0333] Specific operation: The server sends the advice text received from the generative AI model to the smartphone and confirms that the transmission was successful.

[0334] Step 7:

[0335] The user receives and displays the advice text on the smartphone.

[0336] Input: Advice text

[0337] Output: Advice displayed

[0338] Specific operation: The smartphone displays the advice text received from the server on the screen, and the user confirms it.

[0339] Example 3

[0340] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] Conventional AI companion systems have had difficulty accurately estimating a user's emotional state and providing conversation content appropriate to that state. Furthermore, they lacked the ability to adjust conversation content based on the user's health condition and daily life situation, making them ineffective in providing psychological support and managing the user's health. Furthermore, their conversation analysis capabilities to detect early symptoms of dementia and their ability to send alerts to family members and medical institutions were limited. To address these issues, an AI companion system with more advanced emotion analysis and conversation generation capabilities is needed.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0343] In this invention, the server includes means for receiving input data sent from a user, means for analyzing the received input data and estimating an emotional state, and means for sending a prompt sentence to the generative AI model based on the estimated emotional state. This enables the server to generate appropriate conversation content according to the user's emotional state, thereby enabling effective psychological support and health management for the user.

[0344] "Artificial Intelligence Partner" is an artificial intelligence system that provides various functions through dialogue with the user.

[0345] "Input data" refers to information such as text messages and voice data that a user sends through a terminal.

[0346] "Emotional state" refers to the psychological state of the user, and includes emotions such as joy, sadness, anger, and fatigue.

[0347] A "generative artificial intelligence model" is an artificial intelligence algorithm for generating appropriate conversation content based on a given prompt sentence.

[0348] A "prompt" is an instruction sent to a generative AI model based on an emotional state, and serves as the basis for generating appropriate conversation content.

[0349] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[0350] "Sentiment analysis" is the process of analyzing input data to infer a user's emotional state.

[0351] An "alert" is a warning message that notifies family members or medical institutions if any worrisome symptoms are observed.

[0352] "Conversation content" refers to the content of the dialogue that the AI ​​partner provides to the user.

[0353] The present invention relates to an AI companion system that estimates the emotional state of a user and adjusts the content of a conversation in accordance with that state. Specific embodiments of this system will be described below.

[0354] First, a user inputs text messages or voice data using a terminal. For example, consider the case where a user inputs a text message such as "I'm very tired today." This input data is sent from the terminal to a server.

[0355] The server receives input data sent by the user. The received data is temporarily stored in a database within the server. The server then analyzes the input data using emotion analysis software, such as Google Cloud Natural Language API or IBM Watson® Tone Analyzer. This analysis estimates the user's emotional state (e.g., joy, sadness, anger, fatigue, etc.).

[0356] Based on the estimated emotional state, the server sends a prompt to the generative AI model. OpenAI's GPT-4 and other models are used as generative AI models. The prompt includes the user's emotional state and an appropriate response accordingly. For example, a prompt such as "The user feels tired. Please offer some words of encouragement" is sent to the generative AI model.

[0357] The generative AI model generates appropriate conversation content based on this prompt. For example, it generates a response such as, "Today was really tough. Let's take a break and refresh yourself!" The generated conversation content is sent from the server to the user's device. The user can receive the response from their AI partner through their device.

[0358] For example, if a user inputs "I feel very tired today," the server receives this message and uses the Google Cloud Natural Language API to estimate the user's emotional state, such as "fatigue" or "depression." The server then sends a prompt to the generative AI model: "The user feels tired. Please offer some words of encouragement." The generative AI model generates a response: "Today was really hard. Take a break and refresh yourself!" The server then sends this response to the user's device. The user can view this message on their device screen and receive words of encouragement.

[0359] In this way, by providing appropriate conversation content according to the emotional state of the user, it becomes possible to effectively provide psychological support and health management for the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0360] Step 1:

[0361] The user sends input data through the terminal.

[0362] A user inputs text messages or voice data using a device such as a smartphone or PC. For example, the user inputs a text message such as "I'm very tired today" and presses the send button. This input data is sent from the device to the server.

[0363] Step 2:

[0364] The server receives input data from the user.

[0365] The server receives text messages and voice data sent by the user from the device. The received data is temporarily stored in a database on the server. Input data includes the user's text message, "I'm very tired today."

[0366] Step 3:

[0367] The server uses emotion analysis software to analyze the input data and estimate the emotional state.

[0368] The server passes the received input data to emotion analysis software (e.g., Google Cloud Natural Language API or IBM Watson Tone Analyzer). The emotion analysis software analyzes the text message and estimates the user's emotional state (e.g., fatigue or depression). The analysis results in an emotional state such as "fatigue" or "depression" being output.

[0369] Step 4:

[0370] The server sends a prompt to the generative AI model.

[0371] The server sends a prompt to a generative AI model (e.g., OpenAI's GPT-4) based on the estimated emotional state. The prompt includes the user's emotional state and an appropriate response accordingly. For example, the server generates a prompt such as "The user feels tired. Please offer some words of encouragement." and sends it to the generative AI model.

[0372] Step 5:

[0373] The generative AI model generates appropriate conversation content based on the prompt text.

[0374] The generative AI model generates appropriate conversational content based on the received prompt. For example, it generates a response such as, "Today was really tough. Let's take a break and refresh yourself!" The generated conversational content is output.

[0375] Step 6:

[0376] The server transmits the generated conversation content to the user's terminal.

[0377] The server sends the conversational content received from the generative AI model to the user's device. The data is displayed on the user's device. The output includes a message such as, "Today was really hard. Take a break and refresh yourself!"

[0378] Step 7:

[0379] The user receives a response from the AI ​​partner through the terminal.

[0380] The user receives a response from the AI ​​buddy via the device. For example, the user can see a message on their smartphone screen saying, "Today was really tough. Take a break and refresh yourself!" and receive words of encouragement.

[0381] (Application example 3)

[0382] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Conventional food delivery systems make suggestions without considering the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Furthermore, they are unable to provide appropriate meal suggestions when the user is feeling stressed or depressed, and lack functionality to support the user's health and mental state.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for including an AI companion that can be customized to suit the user's preferences and interests, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, etc., means for sending an alert to family members or medical institutions if any worrisome symptoms are observed, and means for estimating the user's emotional state and making food delivery suggestions based on that emotional state. This enables personalized food delivery suggestions based on the user's emotional state, increasing the user's psychological satisfaction and supporting their health and mental state.

[0385] "User" refers to an individual who uses the System.

[0386] "Preferences and interests" refers to the user's particular interests and preferences.

[0387] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[0388] "AI Partner" refers to a system that uses artificial intelligence to interact with users and provide various types of support.

[0389] "A means of making everyday conversations enjoyable" refers to a function that provides enjoyment through everyday interactions with users.

[0390] "Early symptoms of dementia" refers to early signs of cognitive decline.

[0391] "Conversational patterns" refer to the tendency to use words and progress in conversations with users.

[0392] "Short-term changes in memory" refers to how a user's memory changes over a short period of time.

[0393] "Means of sending alerts" refers to the ability to send notifications when certain conditions occur.

[0394] "Emotional state" refers to a user's current psychological state or mood.

[0395] "Food delivery suggestion" means suggesting food and drink delivery services to a User.

[0396] A system for carrying out this invention includes an AI companion that can be customized to suit a user's preferences and interests. The AI ​​companion includes means for engaging in daily conversations with the user, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for issuing alerts to family members or medical institutions if worrisome symptoms are observed, and means for estimating the user's emotional state and suggesting food delivery options based on that emotional state.

[0397] System Program

[0398] This system includes the following programs:

[0399] 1. Loading the emotion estimation model: The server uses OpenAI's emotion analysis model to estimate the emotional state from the user's utterances.

[0400] 2. Emotional state estimation: The server receives the user's input and estimates the emotional state.

[0401] 3. Food Suggestion: The server makes appropriate food suggestions based on the estimated emotional state.

[0402] Hardware and software used

[0403] Hardware: A device, such as a smartphone or tablet, that receives user input.

[0404] Software: Uses OpenAI's sentiment analysis model for sentiment estimation and a customized algorithm for food recommendations.

[0405] Data processing and calculation

[0406] Data processing: User comments are obtained as text data and input into the sentiment analysis model.

[0407] Data computation: The sentiment analysis model estimates the emotional state of the user from their statements, and the food suggestion algorithm generates appropriate suggestions based on the results.

[0408] Specific examples

[0409] For example, if a user types, "I'm very tired today," the emotion estimation model will infer that they are "feeling stressed." Based on this result, the system will suggest relaxing herbal tea and salad.

[0410] Example prompt sentence:

[0411] User says: I'm so tired today

[0412] Estimate your emotional state.

[0413] In this way, personalized food delivery suggestions based on the user's emotional state become possible, increasing the user's psychological satisfaction and supporting their health and mental state.

[0414] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0415] Step 1:

[0416] The user inputs a message using a terminal, which is then saved as text data on the terminal.

[0417] Step 2:

[0418] The terminal transmits the input text data to the server, which receives the text data and prepares it for input into the emotion estimation model.

[0419] Step 3:

[0420] The server uses OpenAI's sentiment analysis model to estimate the user's emotional state from the received text data. Specifically, the server inputs the text data into the sentiment analysis model and obtains the emotional state (e.g., stress, depression, joy, etc.) as the output.

[0421] Step 4:

[0422] The server runs a food recommendation algorithm based on the estimated emotional state, generating appropriate food delivery recommendations depending on the emotional state. For example, if the emotional state is "stressed," it will suggest relaxing herbal tea and salad.

[0423] Step 5:

[0424] The server sends the generated food suggestions to the terminal, which displays the suggestions to the user, who can review the suggested food delivery and place an order if necessary.

[0425] Step 6:

[0426] If the user decides to order the suggested food delivery, the terminal sends the order information to the server, which receives the order information and forwards the order to the food delivery service.

[0427] Step 7:

[0428] The food delivery service receives the order and delivers the meal to the user. After the delivery is complete, the server sends a notification to the user that the delivery is complete.

[0429] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0430] "Example 1"

[0431] In one embodiment of the present invention, an AI companion is provided that includes an emotion engine that recognizes a user's emotions. The emotion engine infers the user's emotional state from the user's tone of voice, speaking style, and words used. For example, when a user is angry, their voice tone often becomes higher and their speech rate increases. Based on this information, the emotion engine infers that the user is angry.

[0432] "Example 2"

[0433] Furthermore, AI Buddy adjusts the content of the conversation depending on the user's emotional state estimated by the emotion engine. For example, if the user is estimated to be angry, AI Buddy will suggest topics to calm the user. If the user is estimated to be happy, AI Buddy will suggest topics to share that joy.

[0434] "Example 3"

[0435] As described above, the present invention makes it possible to accurately grasp the user's emotional state and provide appropriate dialogue accordingly. This allows the user to enjoy more natural dialogue and reduce feelings of loneliness. Furthermore, early symptoms of dementia can be detected early through changes in the user's emotional state.

[0436] The processing flow of each embodiment will be described below.

[0437] "Example 1"

[0438] Step 1: Through conversations with the user, the AI ​​companion collects information about the user's tone of voice, speaking style, and words used.

[0439] Step 2: Send the collected information to the emotion engine.

[0440] Step 3: The emotion engine estimates the user's emotional state based on the transmitted information.

[0441] "Example 2"

[0442] Step 1: The AI ​​partner adjusts the content of the conversation based on the estimated emotional state received from the emotion engine.

[0443] Step 2: For example, if the user is presumed to be angry, the AI ​​companion will provide a topic to calm the user.

[0444] Step 3: If the AI ​​companion determines that the user is happy, it will provide topics that share that joy.

[0445] "Example 3"

[0446] Step 1: The AI ​​partner understands the user's emotional state through dialogue with them. Step 2: The AI ​​partner detects early symptoms of dementia through changes in the user's emotional state.

[0447] Step 3: If early symptoms of dementia are detected, the AI ​​partner will send an alert to family members or medical institutions.

[0448] Example 1

[0449] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0450] For elderly people living alone, reducing feelings of loneliness and early detection of early symptoms of dementia are important issues. Conventional systems are unable to fully analyze the user's emotional state and conversation patterns, making it difficult to provide appropriate feedback or warnings. Furthermore, there is a need for a system that can effectively collect and analyze the user's voice data and adjust the content of conversations according to their daily life situation and health condition.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0452] In this invention, the server includes means for collecting the user's voice and transmitting the voice data to the server, means for converting the voice data into text, and means for analyzing the text data with an emotion recognition engine, which enables detailed analysis of the user's emotional state and speech patterns and provides appropriate feedback and warnings.

[0453] The "Artificial Intelligence Partner" is a system that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[0454] "A means of making everyday conversations enjoyable" is a function that provides conversations that are tailored to the user's interests and concerns through natural dialogue with the user.

[0455] The "means for detecting early symptoms of dementia" is a function that detects early symptoms of dementia by analyzing conversation patterns and short-term changes in memory through dialogue with the user.

[0456] "Means for analyzing conversation patterns and short-term changes in memory" refers to a function that analyzes changes in conversation content and memory by comparing them with the user's past conversation data.

[0457] "Means for sending warnings to family members and medical institutions" is a function that sends warnings to family members and medical institutions when early symptoms of dementia are detected.

[0458] "Means for collecting voice and transmitting voice data to a server" refers to a function for collecting the user's voice with a microphone and transmitting it to a server via the Internet.

[0459] "Means for converting voice data into text" refers to a function that converts collected voice data into text data using a natural language processing engine.

[0460] "Means for analyzing text data using an emotion recognition engine" refers to a function that inputs text data into an emotion recognition engine and estimates the user's emotional state.

[0461] "Means for analyzing conversation patterns and detecting early symptoms of dementia" is a function that analyzes a user's conversation patterns based on text data and detects early symptoms of dementia.

[0462] The "means for providing feedback to the user" is a function that provides appropriate feedback to the user by voice based on the analysis results.

[0463] The present invention is a system that provides an AI companion to assist elderly people living alone. The system includes an AI companion that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[0464] Hardware and software used

[0465] Hardware: Devices with microphones, speakers, and internet connectivity (e.g. smartphones, tablets, smart speakers)

[0466] Software: Natural language processing engines (e.g., Google Cloud Natural Language API), emotion recognition engines (e.g., IBM Watson Tone Analyzer), data analysis tools (e.g., Python's Pandas library)

[0467] Data processing and calculation

[0468] 1. Collecting user conversations

[0469] The terminal collects the user's voice through a microphone.

[0470] The collected voice data is sent to the server.

[0471] 2. Analysis of audio data

[0472] The server converts the voice data into text using a natural language processing engine.

[0473] The converted text data is input into an emotion recognition engine to estimate the user's emotional state.

[0474] 3. Analysis of conversation patterns

[0475] The server stores the text data in a database and compares it with past conversation data.

[0476] Data analysis tools are used to analyze short-term changes in conversation patterns and memory.

[0477] 4. Detecting early symptoms of dementia

[0478] Based on the analysis results, the server determines whether early symptoms of dementia are present.

[0479] If any worrisome symptoms are detected, an alert will be sent to family members or medical institutions.

[0480] 5. User Feedback

[0481] The device provides feedback to the user through the speaker.

[0482] For example, if the AI ​​partner estimates that the user is angry, it will make suggestions such as, "Maybe you should try to calm down a bit?"

[0483] Specific examples

[0484] User conversation: "I went for a walk today, but I got lost."

[0485] System Action:

[0486] The device collects the audio and sends it to the server.

[0487] The server converts the speech into text and analyzes it using an emotion recognition engine.

[0488] The emotion recognition engine infers that the user is feeling anxious.

[0489] The server compares the data with past conversation data and analyzes the possibility of early symptoms of dementia.

[0490] If necessary, alert family members or medical institutions.

[0491] The device will provide feedback to the user saying, "Are you OK? Let's check the map together next time."

[0492] Prompt Sentence Examples

[0493] "Analyze the emotions users express when talking about recent events."

[0494] "It analyzes the user's speech patterns and determines whether they are exhibiting early signs of dementia."

[0495] The above is an embodiment of the present invention.

[0496] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0497] Step 1:

[0498] The user provides voice input.

[0499] The user speaks everyday conversations or questions into the device. For example, "I went for a walk today, but I got lost." The input is the user's voice data, and the output is the voice data collected by the device.

[0500] Step 2:

[0501] The device collects the audio data.

[0502] The device uses a built-in microphone to collect the user's voice. The collected voice data is temporarily stored in the device. The input is the user's voice, and the output is the voice data stored in the device.

[0503] Step 3:

[0504] The terminal transmits the voice data to the server.

[0505] The device sends the collected voice data to a server via the Internet. The data is encrypted before transmission. The input is the voice data stored on the device, and the output is the voice data sent to the server.

[0506] Step 4:

[0507] The server converts the audio data into text.

[0508] The server uses a natural language processing engine to convert the voice data into text, for example, "I went for a walk today, but I got lost." The input is the voice data sent to the server, and the output is the converted text data.

[0509] Step 5:

[0510] The server analyzes the text data using an emotion recognition engine.

[0511] The server uses an emotion recognition engine to infer the user's emotional state from the text data. For example, it infers that the user is feeling anxious. The input is the converted text data, and the output is the inferred emotional state.

[0512] Step 6:

[0513] The server analyzes the conversation patterns.

[0514] The server stores the text data in a database and compares it with past conversation data. Data analysis tools are used to analyze short-term changes in conversation patterns and memory. The input is the estimated emotional state and past conversation data, and the output is the analysis results.

[0515] Step 7:

[0516] The server detects early symptoms of dementia.

[0517] The server determines whether early symptoms of dementia are present based on the analysis results. For example, if the user has recently shown a pattern of frequently getting lost, it determines that this may be an early symptom of dementia. The input is the analysis results, and the output is the detection result of early symptoms of dementia.

[0518] Step 8:

[0519] The server issues an alert.

[0520] If the server detects early symptoms of dementia, it sends an alert to family members or medical institutions. The alert is sent via email or SMS. The input is the detection result of early symptoms of dementia, and the output is the alert sent.

[0521] Step 9:

[0522] The terminal provides feedback to the user.

[0523] The device provides feedback to the user through the speaker, such as "Are you OK? Let's check the map together next time." The input is the feedback from the server, and the output is the feedback provided to the user.

[0524] (Application example 1)

[0525] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0526] For elderly people living alone, it is important to reduce feelings of loneliness and detect early symptoms of dementia early. However, to solve these issues, a system is needed that can grasp the elderly's condition through everyday conversation and respond quickly if an abnormality is detected. It is also necessary to accurately recognize the user's emotional state and respond accordingly. Conventional systems have found it difficult to comprehensively provide these functions.

[0527] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0528] In this invention, the server includes means for including an AI companion that can be customized to suit the user's preferences and interests, means for engaging in daily conversations, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for sending alerts to family members and medical institutions if worrisome symptoms are observed, means for recognizing the user's voice and analyzing emotions, and means for sending alerts if an abnormality is detected. This makes it possible to reduce loneliness among the elderly, detect early symptoms of dementia, and respond quickly.

[0529] "User" refers to an individual who uses the System.

[0530] "AI companion customizable to your preferences and interests" refers to artificial intelligence that can adjust its functions and dialogue content based on the user's individual preferences and interests.

[0531] "A means of making everyday conversations enjoyable" refers to a function that allows users to have daily conversations and provide fun and satisfaction.

[0532] "Means for detecting early symptoms of dementia" refers to a function that detects early signs of dementia through dialogue with the user.

[0533] "Means for analyzing conversation patterns and short-term changes in memory" refers to a function that analyzes the content of a user's conversations and fluctuations in memory and detects abnormalities.

[0534] "Means of sending alerts to family members and medical institutions" refers to the function of sending notifications to the user's family members and medical institutions when an abnormality is detected.

[0535] "Means for recognizing voice and analyzing emotions" refers to the function of analyzing the user's voice and inferring their emotional state.

[0536] "Means for sending alerts when an abnormality is detected" refers to the function of sending an alert to designated contacts when the system detects an abnormality.

[0537] A system for implementing this invention includes an AI companion that can be customized to suit a user's preferences and interests. The system is realized as an application installed on a device such as a smartphone or smart glasses.

[0538] Hardware and software used

[0539] Hardware: Smartphones, smart glasses

[0540] Software: Python, TENSORFLOW (registered trademark), NLP library (e.g., spaCy), speech recognition library (e.g., Google Speech-to-Text)

[0541] Data processing and calculation

[0542] Acquiring audio data

[0543] The device's microphone is used to capture the user's voice. Voice data is collected in real time and converted into text data through a speech recognition library.

[0544] Voice Recognition

[0545] A speech recognition library (e.g., Google Speech-to-Text API) is used to convert the captured speech data into text, which is then used for subsequent sentiment analysis and early dementia symptom detection.

[0546] emotion recognition

[0547] The converted text data is analyzed using an NLP library (e.g., spaCy) and an emotion engine (TensorFlow model) is used to estimate the user's emotional state. The emotion engine estimates emotions from the user's tone of voice, speaking style, and the words they use.

[0548] Detecting early symptoms of dementia

[0549] It analyzes short-term changes in conversation patterns and memory to detect anomalies, by analyzing a user's conversation history and monitoring the frequency and changes of specific keywords and phrases.

[0550] Sending alerts

[0551] If an abnormality is detected, the system will send an alert to family members or medical institutions via email or SMS.

[0552] Specific examples

[0553] For example, if a user forgets recent events, the system operates as follows:

[0554] 1. A user asks, "What was for dinner last night?"

[0555] 2. The system recognizes the speech and converts it into text.

[0556] 3. Analyze the converted text and determine that the user has forgotten recent events.

[0557] 4. The emotion engine infers emotions from the user's tone of voice and speaking style, and sends an alert if an anomaly is detected.

[0558] Prompt Sentence Examples

[0559] "If users forget what's happened recently, send them an alert."

[0560] In this way, the invention can reduce the sense of loneliness felt by the elderly and enable early detection and prompt response to early symptoms of dementia.

[0561] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0562] Step 1:

[0563] The device captures the user's voice.

[0564] Input: User's voice

[0565] Output: Audio data

[0566] What it does: Uses the device's microphone to collect the user's voice in real time.

[0567] Step 2:

[0568] The device converts the voice data into text.

[0569] Input: Audio data

[0570] Output: Text data

[0571] Specific operation: Uses a speech recognition library (e.g., Google Speech-to-Text API) to convert the acquired voice data into text.

[0572] Step 3:

[0573] The server analyzes the text data and estimates the emotion.

[0574] Input: Text data

[0575] Output: Emotional state

[0576] Specific behavior: Analyze text data using an NLP library (e.g., spaCy) and infer emotional states using an emotion engine (TensorFlow model).

[0577] Step 4:

[0578] The server analyzes conversation patterns and short-term changes in memory.

[0579] Input: Text data

[0580] Output: Analysis results

[0581] What it does: Analyzes a user's conversation history and monitors the frequency and changes of certain keywords and phrases.

[0582] Step 5:

[0583] The server detects an abnormality.

[0584] Input: Analysis results

[0585] Output: Anomaly detection results

[0586] Specific operation: Based on the analysis results, detect early symptoms of dementia and emotional abnormalities.

[0587] Step 6:

[0588] The server sends an alert if an abnormality is detected.

[0589] Input: Anomaly detection result

[0590] Output: Alert notification

[0591] Specific operation: If an abnormality is detected, an alert will be sent to family members or medical institutions via email or SMS.

[0592] Example 2

[0593] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0594] Conventional AI companion systems can be customized to suit the user's preferences and interests and have the ability to engage in enjoyable daily conversations, but they do not adequately adjust the content of the conversation to reflect the user's health or emotional state. While they offer dialogues to detect early symptoms of dementia and alerts when worrisome symptoms are detected, they lack the ability to generate conversation content that takes into account the user's real-time health and emotional state. This creates the challenge of insufficient health management and psychological support for users.

[0595] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0596] In this invention, the server includes means for collecting user health condition data, means for collecting user emotional state data, and means for analyzing the collected data and generating conversation content using a generative artificial intelligence model, thereby making it possible to provide appropriate conversation content according to the user's health condition and emotional state.

[0597] "Artificial Intelligence Partner" is a system equipped with artificial intelligence that provides various functions through dialogue with the user.

[0598] "Daily life status" refers to information about a user's daily activities and lifestyle habits.

[0599] "Health Status" refers to data relating to a user's physical condition or health.

[0600] "Emotional state" refers to information about a user's psychological state or emotions.

[0601] "Means for adjusting conversation content" refers to methods and technologies for generating and providing appropriate conversation content according to the user's situation or state.

[0602] "Means for collecting health data" refers to the methods and technologies used to obtain data about a user's health.

[0603] "Means for collecting emotional state data" refers to methods or techniques for obtaining data about a user's emotions.

[0604] A "generative artificial intelligence model" refers to an artificial intelligence model that generates appropriate responses and conversation content based on user input and the situation.

[0605] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0606] The present invention relates to an AI companion system that adjusts the content of conversations depending on the user's daily life situation, health condition, and emotional state. Specific embodiments of this system will be described below.

[0607] Hardware and software used

[0608] Hardware:

[0609] High-performance servers (e.g., servers equipped with NVIDIA GPUs)

[0610] Wearable devices (e.g., devices that measure heart rate or temperature)

[0611] Device (e.g. smartphone, tablet)

[0612] software:

[0613] Generative AI models (e.g., OpenAI's GPT-4)

[0614] Emotion engines (e.g., Affectiva's emotion recognition API)

[0615] Text-to-speech engine

[0616] System Operation

[0617] Collecting user health data

[0618] The device collects health status data from the wearable device worn by the user. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time and transmits it to the device via Bluetooth or Wi-Fi.

[0619] Examples:

[0620] The terminal acquires heart rate data from the wearable device.

[0621] The terminal acquires body temperature data from the body temperature sensor.

[0622] The terminal acquires step count data from the activity meter.

[0623] Collecting user emotional state data

[0624] The device uses a camera and microphone to estimate the user's emotional state from their facial expressions and voice, and this data is sent to the emotion engine for analysis.

[0625] Examples:

[0626] The device uses a camera to capture the user's facial expressions.

[0627] The terminal uses a microphone to record the user's voice.

[0628] The terminal transmits the captured facial expression data to the emotion engine.

[0629] The device transmits the recorded voice data to the emotion engine.

[0630] Analyzing data and generating conversation content

[0631] The server analyzes the collected health and emotional state data. Based on the analysis results, it uses a generative AI model to generate appropriate conversation content. It inputs prompt sentences into the generative AI model to obtain the conversation content.

[0632] Examples:

[0633] The server analyzes the health status data to determine whether the user has a cold.

[0634] The server analyzes the emotional state data to determine whether the user is angry.

[0635] The server inputs the prompt sentence to the generative AI model: "Please tell me how the AI ​​companion should proceed with the conversation if the user has a cold."

[0636] The server stores the conversation content obtained from the generative AI model.

[0637] Providing conversation content

[0638] The device provides the user with the conversation content obtained from the server. Specifically, the AI ​​partner uses voice synthesis technology to converse with the user.

[0639] Examples:

[0640] The terminal inputs the conversation content acquired from the server into a voice synthesis engine.

[0641] The terminal uses a speech synthesis engine to convert the conversation into speech.

[0642] The device's AI partner will converse with the user through the speaker.

[0643] Prompt Sentence Examples

[0644] "How should the AI ​​companion proceed with the conversation if the user has a cold?"

[0645] "If the user is angry, tell us how the AI ​​companion should proceed with the conversation."

[0646] In this way, a system is realized in which the server, terminal, and user work together to provide appropriate conversation content according to the user's health condition and emotional state.

[0647] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0648] Step 1:

[0649] Collecting user health data

[0650] The device collects health status data from the wearable device worn by the user. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time and transmits it to the device via Bluetooth or Wi-Fi.

[0651] Input: Heart rate, body temperature, and activity data from wearable devices

[0652] Output: Health status data sent to the device

[0653] Specific behavior:

[0654] The terminal acquires heart rate data from the wearable device.

[0655] The terminal acquires body temperature data from the body temperature sensor.

[0656] The terminal acquires step count data from the activity meter.

[0657] Step 2:

[0658] Collecting user emotional state data

[0659] The device uses a camera and microphone to estimate the user's emotional state from their facial expressions and voice, and this data is sent to the emotion engine for analysis.

[0660] Input: facial expression data captured by a camera, audio data recorded by a microphone

[0661] Output: Emotional state data sent to the emotion engine

[0662] Specific behavior:

[0663] The device uses a camera to capture the user's facial expressions.

[0664] The terminal uses a microphone to record the user's voice.

[0665] The terminal transmits the captured facial expression data to the emotion engine.

[0666] The device transmits the recorded voice data to the emotion engine.

[0667] Step 3:

[0668] Analyzing data and generating conversation content

[0669] The server analyzes the collected health and emotional state data. Based on the analysis results, it uses a generative AI model to generate appropriate conversation content. It inputs prompt sentences into the generative AI model to obtain the conversation content.

[0670] Input: Health status data, emotional status data, prompt text

[0671] Output: Generated conversation

[0672] Specific behavior:

[0673] The server analyzes the health status data to determine whether the user has a cold.

[0674] The server analyzes the emotional state data to determine whether the user is angry.

[0675] The server inputs the prompt sentence to the generative AI model: "Please tell me how the AI ​​companion should proceed with the conversation if the user has a cold."

[0676] The server stores the conversation content obtained from the generative AI model.

[0677] Step 4:

[0678] Providing conversation content

[0679] The device provides the user with the conversation content obtained from the server. Specifically, the AI ​​partner uses voice synthesis technology to converse with the user.

[0680] Input: Conversation content obtained from the server

[0681] Output: The audio conversation that is presented to the user

[0682] Specific behavior:

[0683] The terminal inputs the conversation content acquired from the server into a voice synthesis engine.

[0684] The terminal uses a speech synthesis engine to convert the conversation into speech.

[0685] The device's AI partner will converse with the user through the speaker.

[0686] (Application example 2)

[0687] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0688] Conventional AI partner systems can be customized to suit the user's preferences and interests, and are equipped with functions to facilitate enjoyable daily conversations and detect early symptoms of dementia. However, when it comes to customer service in physical stores, they lack the functionality to analyze customers' facial expressions and physical condition in real time and provide optimal customer service to each individual. This poses a challenge in that personalized customer service that would improve customer satisfaction has not been realized.

[0689] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0690] In this invention, the server includes a means for including an AI companion that can be customized to suit the user's preferences and interests, a means for adjusting the content of conversation depending on the user's daily life situation and health condition, a means for estimating the user's emotional state and adjusting the content of conversation depending on that emotional state, and a means for analyzing the facial expressions and physical condition of customers in real time and providing optimal customer service to each individual customer, thereby enabling personalized customer service to improve customer satisfaction in physical stores.

[0691] "User" refers to an individual or customer who uses the System.

[0692] "Preferences" refer to the things that a user particularly likes or is interested in.

[0693] "Interests" refers to the things or areas that a user is interested in.

[0694] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[0695] "AI Partner" is an interactive system equipped with artificial intelligence that provides various types of support through conversations with users.

[0696] "Everyday life situations" refer to events and conditions that users experience in their daily lives.

[0697] "Health Status" refers to the state of a User's physical and mental health.

[0698] "Conversation content" refers to the words and topics exchanged between the AI ​​partner and the user.

[0699] "Emotional state" refers to a user's current feelings or mood.

[0700] "Facial Expressions" refers to the user's facial expressions and movements.

[0701] "Physical condition" refers to the user's physical health and condition.

[0702] "Real-time" refers to processing or reaction occurring immediately, without delay.

[0703] "Analysis" refers to examining and understanding data and information in detail.

[0704] "Customer service" refers to the service and response provided by store staff to customers.

[0705] "Personalized customer service" refers to providing services that are individually optimized to suit the needs and circumstances of each customer.

[0706] The system for implementing this invention includes an AI partner that can be customized to suit the user's preferences and interests. The AI ​​partner adjusts the content of conversations based on the user's daily life situation and health condition, and further estimates the user's emotional state and provides conversation content appropriate to that emotional state. In addition, in physical stores, the AI ​​partner has the ability to analyze customers' facial expressions and physical condition in real time to provide optimal customer service to each individual.

[0707] Hardware and software used

[0708] Hardware: smart glasses, camera

[0709] Software: OpenCV (face detection), EmotionRecognizer (emotion recognition), HealthStatusRecognizer (health status recognition), ConversationEngine (conversation content generation)

[0710] System processing overview

[0711] 1. Video capture from camera: Using a camera to capture video in real time, the camera built into the smart glasses captures the faces of customers.

[0712] 2. Emotion and health status recognition: Detect faces from captured video and recognize emotions using EmotionRecognizer. Also, recognize health status using HealthStatusRecognizer.

[0713] 3. Conversation content generation: Based on the recognized emotions and health status, the Conversation Engine generates appropriate conversation content.

[0714] 4. Display on smart glasses: The generated conversation content is displayed on smart glasses, allowing staff to provide personalized service to customers.

[0715] Specific examples

[0716] For example, if the customer is determined to be tired through emotion and health status recognition, the smart glasses will display a message saying, "Please suggest products that will help them relax." If the customer is determined to be depressed, the smart glasses will display a message saying, "Please offer some encouraging conversation."

[0717] Prompt Sentence Examples

[0718] Below are some example prompts to input to a generative AI model:

[0719] Analyze the user's facial expressions and health status to generate appropriate conversations. For example, if the user is tired, suggest products that will help them relax, and if they are feeling down, provide encouraging conversations.

[0720] In this way, a system can be realized that allows for more personalized customer service in physical stores.

[0721] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0722] Step 1:

[0723] Capture video from the camera

[0724] The server captures video in real time using the camera built into the smart glasses. The input is video data from the camera, and the output is the captured video frames, which are used in subsequent processing steps.

[0725] Step 2:

[0726] Face detection

[0727] The server processes the captured video frames using OpenCV to detect faces. The input is the video frame, and the output is the position information of the detected face. Specifically, the server extracts the facial contours and feature points.

[0728] Step 3:

[0729] emotion recognition

[0730] The server uses EmotionRecognizer to recognize emotions based on the detected face location information. The input is face location information and video frames, and the output is the user's emotional state (e.g., joy, anger, sadness, etc.). Specifically, emotions are estimated from facial expressions.

[0731] Step 4:

[0732] Health Status Awareness

[0733] The server recognizes the health status using HealthStatusRecognizer based on the location information of the detected face. The input is the location information of the face and the video frame, and the output is the user's health status (e.g., fatigue, vitality, etc.). Specifically, the health status is estimated from factors such as facial color and eye condition.

[0734] Step 5:

[0735] Conversation generation

[0736] The server uses the ConversationEngine to generate appropriate conversation content based on the recognized emotional state and health status. The input is the emotional state and health status, and the output is the generated conversation content. Specifically, a prompt sentence is input into the generative AI model to generate appropriate conversation content.

[0737] Step 6:

[0738] Displayed on smart glasses

[0739] The server displays the generated conversation content on the smart glasses. The input is the generated conversation content, and the output is text or messages displayed on the smart glasses' display. Specifically, instructions are displayed for staff to provide personalized customer service.

[0740] Example 3

[0741] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0742] In modern society, the number of people experiencing loneliness and mental stress is increasing. It is also difficult to detect the early symptoms of dementia, which often delays appropriate measures. Furthermore, while there is a need for appropriate dialogue based on the user's emotional state, current systems have difficulty achieving this. To solve these issues, a system is needed that can accurately grasp the user's emotional state and provide appropriate dialogue accordingly.

[0743] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0744] In this invention, the server includes a means for converting the user's voice data into text, a means for inputting the text data into an emotion analysis model to estimate the emotional state, and a means for inputting a prompt sentence according to the emotional state into a generative AI model, thereby making it possible to accurately grasp the user's emotional state and provide appropriate dialogue accordingly.

[0745] "Artificial Intelligence Partner" is an artificial intelligence system that understands the user's emotional state and health status through dialogue with them and provides appropriate conversation content.

[0746] "Voice data" refers to data that digitally represents the voice signals generated when a user speaks to a terminal.

[0747] The "means for converting to text" refers to a means that uses voice recognition technology to convert voice data into text information.

[0748] A "sentiment analysis model" is a model that includes machine learning algorithms for analyzing text data and inferring a user's emotional state.

[0749] A "prompt" is an instruction sentence that is input into a generative AI model to generate an appropriate response based on the user's emotional state.

[0750] A "generative artificial intelligence model" is an artificial intelligence model for generating natural dialogue based on an input prompt sentence.

[0751] The "means for generating conversation content" is a means for generating appropriate conversation content according to the emotional state of the user using an emotion analysis model and a generative artificial intelligence model.

[0752] "Means of communicating by voice" refers to a means that uses voice synthesis technology to convert the generated text data into voice and communicate it to the user.

[0753] "Means for sending alerts" refers to the means for sending notifications to family members or medical institutions if any abnormalities are detected in the user's health or emotional state.

[0754] The present invention is a system including an AI companion that estimates the emotional state of a user and adjusts the content of a conversation depending on the emotional state. Specific embodiments of this system are described below.

[0755] First, when a user speaks into the device, the device captures the voice data. The device needs a microphone and a network connection so that the voice data can be transmitted to the server.

[0756] The server uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the voice data into text, which is a written representation of what you said.

[0757] The server then uses a sentiment analysis model (e.g., a BERT-based sentiment analysis model) to estimate the user's emotional state from the text data. The sentiment analysis model includes a machine learning algorithm to analyze the text data and estimate the user's emotional state.

[0758] After the emotional state is estimated, the server inputs a prompt sentence into a generative AI model (e.g., OpenAI's GPT-3 (registered trademark)). The prompt sentence is an instruction sentence for generating an appropriate response according to the user's emotional state. For example, if the user says, "I'm very tired today," the prompt sentence would be, "If the user is feeling tired, how would you offer words of encouragement?"

[0759] The generative AI model generates natural-sounding dialogue based on the input prompt, such as "You've worked really hard today. Let's take a break and refresh yourself."

[0760] The generated conversation content is sent from the server to the device, which then uses speech synthesis software (e.g., Google Cloud Text-to-Speech) to convert the generated text data into speech and convey it to the user, allowing the user to enjoy natural conversations with their AI companion.

[0761] As a concrete example, consider the case where a user says to a device, "I'm very tired today." This voice data is sent from the device to a server and converted into text by speech recognition software. A sentiment analysis model analyzes this text and infers that the user is tired. The server inputs the prompt sentence "If the user feels tired, what words of encouragement would you give them?" into the generative AI model. The generative AI model generates a response such as "You worked really hard today. Take a break and refresh yourself," and the server sends it to the device. The device then converts the text into speech and conveys it to the user.

[0762] This system enables users to reduce their sense of loneliness through natural conversation with their AI companion and to detect early symptoms of dementia through changes in their emotional state. The flow of the identification process in Example 3 will be explained with reference to Figure 21.

[0763] Step 1:

[0764] The user speaks into the terminal.

[0765] For example, the user might say to the terminal, "I'm very tired today." This voice data becomes the input for proceeding to the next processing step.

[0766] Step 2:

[0767] The terminal transmits the voice data to the server.

[0768] The device captures the user's voice data and sends it to the server, using the device's microphone and network connection. The input is the user's voice data, and the output is digital voice data sent to the server.

[0769] Step 3:

[0770] The server converts the audio data into text.

[0771] The server uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the voice data into text. The input is digital voice data, and the output is text data. For example, the generated text might say, "I'm very tired today."

[0772] Step 4:

[0773] The server inputs the text data into a sentiment analysis model to estimate the emotional state.

[0774] The server uses a BERT-based sentiment analysis model to estimate the user's emotional state from text data. The input is text data, and the output is the estimated emotional state. In this case, the estimated emotional state is "tired."

[0775] Step 5:

[0776] The server inputs prompt sentences according to the emotional state into the generative AI model.

[0777] The server generates a prompt sentence based on the emotional state. For example, the prompt sentence "If the user feels tired, how would you offer words of encouragement?" is input to the generative AI model. The input is the estimated emotional state, and the output is the prompt sentence.

[0778] Step 6:

[0779] The server receives the response from the generative AI model and generates appropriate conversation content.

[0780] The server receives responses from a generative AI model (e.g., OpenAI's GPT-3) and generates appropriate conversational content. The input is a prompt, and the output is the generated conversational content. For example, a response might be, "You've worked really hard today. Let's take a break and refresh yourself."

[0781] Step 7:

[0782] The server transmits the generated conversation content to the terminal.

[0783] The server sends the generated conversation content to the terminal. The input is the generated conversation content, and the output is text data sent to the terminal.

[0784] Step 8:

[0785] The terminal conveys the contents of the conversation to the user by voice.

[0786] The device converts the received text data into speech using speech synthesis software (e.g., Google Cloud Text-to-Speech) and conveys it to the user. The input is text data, and the output is audio data. The user can hear a voice message saying, "You've worked really hard today. Let's take a break and refresh yourself."

[0787] (Application example 3)

[0788] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0789] In modern society, understanding a user's emotional state in real time and providing appropriate dialogue based on that state is important for reducing feelings of loneliness and maintaining mental health. However, conventional systems have had difficulty accurately estimating a user's emotional state and adjusting the content of conversations accordingly. Furthermore, even in brick-and-mortar stores, it has been difficult to provide customer service that reflects the emotional state of customers, making it difficult to improve customer satisfaction.

[0790] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a means including an AI companion that can be customized to suit the user's preferences and interests, a means for detecting early symptoms of dementia through dialogue with the user, a means for analyzing conversation patterns and short-term changes in memory, a means for issuing an alert to family members or medical institutions if any worrisome symptoms are observed, and a means for grasping the user's emotional state in real time in a physical store and generating a customer service response accordingly. This makes it possible to accurately grasp the user's emotional state and provide appropriate dialogue and customer service accordingly.

[0791] "User" refers to an individual who uses the System.

[0792] "Preferences and interests" refers to the user's particular interests and preferences.

[0793] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[0794] "AI Aibou" is an interactive system equipped with artificial intelligence, and refers to a device or software that communicates with users.

[0795] "Daily conversations" refers to the daily interactions between the user and their AI companion.

[0796] "Early symptoms of dementia" refers to early signs of cognitive decline.

[0797] "Conversational patterns" refer to the repetition of words and phrases that a user uses on a daily basis.

[0798] "Short-term memory changes" refers to how a user's memory fluctuates over a short period of time.

[0799] "Alert" refers to a function that notifies you when a specific situation occurs.

[0800] "Family or Healthcare Provider" refers to the User's next of kin or organizations that provide health care services.

[0801] "Daily life situations" refers to the user's daily activities and living environment.

[0802] "Health Status" refers to the state of a User's physical and mental health.

[0803] An "emotional state" refers to the emotion a user is feeling at a particular point in time.

[0804] "Real-time" refers to near-instant processing or response.

[0805] "Customer service response" refers to the responses and assistance provided by staff to customers in a physical store.

[0806] A system for carrying out this invention includes an AI companion that can be customized to suit a user's preferences and interests. The AI ​​companion includes means for entertaining daily conversations, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for sending alerts to family members or medical institutions if worrisome symptoms are observed, and means for grasping the user's emotional state in real time in a physical store and generating corresponding customer service responses.

[0807] Hardware and software used

[0808] Hardware

[0809] Smart Glasses: Smart glasses with a camera are used to capture the faces of customers.

[0810] Computer: Used to process data and run emotion recognition models.

[0811] software

[0812] OpenCV: An image processing library used for face detection and preprocessing.

[0813] Keras: A deep learning framework for running emotion recognition models.

[0814] TextBlob: Used for text generation and natural language processing.

[0815] Data processing and calculation

[0816] Image capture and preprocessing

[0817] The smart glasses camera captures the customer's face, and facial detection is performed using OpenCV. The detected facial image is converted to grayscale and resized to an appropriate size for input into the emotion recognition model.

[0818] emotion recognition

[0819] A facial image is input to a pre-trained emotion recognition model using Keras to estimate the user's emotional state, and an appropriate customer response is generated based on the estimated emotional state.

[0820] Response Generation

[0821] TextBlob is used to generate a text response based on the estimated emotional state, which is then displayed on the smart glasses display, allowing staff to serve the customer accordingly.

[0822] Specific examples

[0823] For example, if a customer is feeling stressed, the emotion recognition model will infer "stress" and generate a response using TextBlob: "Sir, we would like to introduce some products that will help you relax." This response will be displayed on the smart glasses display, and staff will assist the customer accordingly.

[0824] Prompt Sentence Examples

[0825] Develop an application that applies the AI ​​Aibou system, which estimates the user's emotional state and adjusts the conversation content accordingly, to customer service in a brick-and-mortar store. Using smart glasses, the system will grasp the emotional state of customers in real time and help them serve them accordingly. Please explain the specific program code and its processing.

[0826] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0827] Step 1:

[0828] The camera in the smart glasses captures the customer's face. The input is the camera image, and the output is the captured facial image. The smart glasses acquire the image in real time and generate image data for face detection.

[0829] Step 2:

[0830] The server performs face detection using OpenCV. The input is the face image captured in step 1, and the output is the area (coordinate information) of the detected face. The face detection algorithm identifies the face in the image and outputs its position as coordinates.

[0831] Step 3:

[0832] The server converts the detected face image to grayscale and resizes it to an appropriate size for input to the emotion recognition model. The input is the face region obtained in step 2, and the output is the preprocessed face image. An image processing algorithm standardizes the face image and converts it into a format suitable for model input.

[0833] Step 4:

[0834] The server inputs the preprocessed facial images into a pre-trained emotion recognition model using Keras to estimate the user's emotional state. The input is the preprocessed facial image from step 3, and the output is the estimated emotional state (e.g., happiness, sadness, anger, etc.). The emotion recognition model analyzes the image data and outputs emotion labels.

[0835] Step 5:

[0836] The server uses TextBlob to generate a text response according to the estimated emotional state. The input is the emotional state estimated in step 4, and the output is the generated text response. A natural language processing algorithm generates an appropriate response based on the emotional state.

[0837] Step 6:

[0838] The server displays the generated text response on the display of the smart glasses. The input is the text response generated in step 5, and the output is the response sentence displayed on the display of the smart glasses. The smart glasses receive the text data and display it on the display.

[0839] Step 7:

[0840] The user (staff member) serves the customer according to the response displayed on the smart glasses display. The input is the text response displayed in step 6, and the output is the user's customer service behavior. The staff member refers to the displayed response and provides appropriate service.

[0841] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0842] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0843] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0844] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0845] [Second embodiment]

[0846] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0847] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0848] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0849] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0850] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0851] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0852] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0853] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0854] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0855] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0856] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0857] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0858] "Example 1"

[0859] One embodiment of the present invention provides a system for supporting elderly people living alone. This system includes an AI companion that can be customized to suit the user's preferences and interests. The AI ​​companion engages in daily conversations with the user, helping to reduce the user's sense of loneliness through these conversations. Furthermore, the AI ​​companion has the ability to detect early symptoms of dementia through dialogue with the user. Specifically, the AI ​​companion analyzes conversation patterns and short-term changes in memory, and if any worrisome symptoms are detected, it sends an alert to family members or medical institutions.

[0860] "Example 2"

[0861] In another embodiment of the present invention, the AI ​​buddy can adjust the conversation content according to the user's daily life situation and health condition. For example, if the user has a cold, the AI ​​buddy can ask about the symptoms of the cold and advise the user to get adequate rest.

[0862] "Example 3"

[0863] As a further embodiment of the present invention, a system is provided that further includes a function in which the AI companion estimates the user's emotional state and adjusts the conversation content according to the emotional state. For example, when the user is estimated to be depressed, the AI companion may offer words of encouragement or bring up topics that the user likes.

[0864] The processing flow of each exemplary embodiment will be described below.

[0865] "Exemplary Embodiment 1"

[0866] Step 1: The AI companion is customized based on the user's preferences and interests. For example, if the user is interested in music or movies, the AI companion offers conversations about those topics.

[0867] Step 2: The AI companion has daily conversations with the user. These conversations are for reducing the user's sense of loneliness.

[0868] Step 3: The AI companion detects early symptoms of dementia through interactions with the user. Specifically, it analyzes changes in conversation patterns and short-term memory.

[0869] Step 4: If concerning symptoms are detected, the AI companion sends alerts to the family or medical institutions.

[0870] "Exemplary Embodiment 2"

[0871] Step 1: The AI companion adjusts the conversation content according to the user's daily life situation and health condition. For example, if the user has a cold, the AI companion asks about the cold symptoms and gives advice on taking appropriate rest.

[0872] "Exemplary Embodiment 3"

[0873] Step 1: The AI companion estimates the user's emotional state. This is estimated from the user's voice tone, way of speaking, words used, etc.

[0874] Step 2: The AI ​​buddy adapts the conversation based on the estimated emotional state. For example, if the AI ​​buddy estimates that the user is feeling down, it might offer words of encouragement or bring up topics the user likes.

[0875] Example 1

[0876] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0877] For elderly people living alone, it is important to reduce feelings of loneliness and detect early symptoms of dementia early. However, effective methods to achieve this are lacking. There is also a need for systems that can provide conversations tailored to the user's preferences and interests, and adjust the content of conversations according to their health status. Furthermore, there is a need for technology that can accurately analyze users' comments and generate appropriate responses. A system that can solve these issues is needed.

[0878] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0879] In this invention, the server includes a means for converting a user's speech into text data using speech recognition software, a means for inputting prompts into a generative AI model to generate a response, and a means for displaying the generated response to the user. This reduces the user's sense of loneliness and enables early detection of early symptoms of dementia. It also enables the provision of conversations tailored to the user's preferences and interests, and the content of the conversation to be adjusted according to the user's health condition.

[0880] The "Artificial Intelligence Partner" is a system that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[0881] "Voice recognition software" is a technology that converts a user's speech from voice data into text data.

[0882] A "generative AI model" is an artificial intelligence technology that generates appropriate responses based on input prompts.

[0883] A "prompt sentence" is text data input into a generative AI model and is an instruction sentence for generating a response.

[0884] "Text Data" means the written information of a user's speech converted by voice recognition software.

[0885] An "alert" is a warning message sent to notify family members or medical institutions when an abnormality is detected.

[0886] "Conversation data" is a record of the conversation between the user and the AI ​​companion.

[0887] An "anomaly" is an abnormal condition, such as early symptoms of dementia, detected based on changes in a user's speech patterns or memory.

[0888] The present invention is a system for supporting elderly people living alone, including an AI companion that can be customized to suit the user's preferences and interests. The system engages in daily conversations with the user, helping to reduce the user's sense of loneliness. The AI ​​companion also has the ability to detect early symptoms of dementia through dialogue with the user.

[0889] Hardware and software used

[0890] Hardware:

[0891] Devices such as smartphones and tablets

[0892] server

[0893] software:

[0894] Voice recognition software (e.g., Google Speech-to-Text)

[0895] Natural language processing technology (e.g., NLTK and spaCy)

[0896] Generative AI models (e.g., OpenAI's GPT-4)

[0897] Data processing and calculation

[0898] Processing user utterances:

[0899] When a user initiates a conversation with their AI partner through the device, the device uses voice recognition software to convert the user's speech into text data, which is then sent to a server.

[0900] Analyzing text data:

[0901] The server analyzes the received text data using natural language processing technology, and as a result of the analysis, it understands the intent and content of the user's comments.

[0902] Response generation:

[0903] Based on the analysis results, the server inputs a prompt sentence into the generative AI model, which then generates an appropriate response based on this prompt sentence.

[0904] View response:

[0905] The server generates a response and sends it to the terminal, which displays the response to the user.

[0906] Accumulation of conversation data and monitoring for anomalies:

[0907] The server stores conversation data with the user in a database and monitors changes in the user's conversation patterns and memory. If an abnormality is detected, the server sends an alert to family members or medical institutions.

[0908] Specific examples

[0909] For example, if a user says, "I can't remember what I had for dinner today, what was for dinner last night?", the following happens:

[0910] 1. User: "I can't remember what I had for dinner today. What was for dinner last night?"

[0911] 2. Terminal: Uses voice recognition software to convert speech into text data and send it to the server.

[0912] 3. Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[0913] 4. Generative AI model: Generates the response, "You had curry rice for dinner last night. What are you planning to eat today?"

[0914] 5. Server: Sends the generated response to the terminal.

[0915] 6. Terminal: displays the response to the user.

[0916] 7. Server: Stores conversation data and monitors for abnormalities. If an abnormality is detected, an alert is sent.

[0917] Example prompt sentence:

[0918] User: "I can't remember what I had for dinner today. What was for dinner last night?"

[0919] AI Buddy: "You had curry rice for dinner last night. What are you planning to eat today?"

[0920] In this way, the system of the present invention can reduce the user's sense of loneliness and detect early symptoms of dementia. It can also provide conversations tailored to the user's preferences and interests, and adjust the content of conversations according to the user's health condition.

[0921] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0922] Step 1:

[0923] The user initiates a conversation with the AI ​​partner through the terminal.

[0924] Input: User's spoken utterance

[0925] What happens: A user says, "I can't remember what I had for dinner today. What was for dinner last night?"

[0926] Output: Audio data

[0927] Step 2:

[0928] The device converts the user's speech into text data using voice recognition software.

[0929] Input: Audio data

[0930] What happens: Your device uses voice recognition software (e.g., Google Speech-to-Text) to convert voice data into text.

[0931] Output: Text data

[0932] Step 3:

[0933] The terminal transmits the converted text data to the server.

[0934] Input: Text data

[0935] Specific operation: The terminal sends text data to the server.

[0936] Output: Text data sent to the server

[0937] Step 4:

[0938] The text data received by the server is analyzed using natural language processing technology.

[0939] Input: Text data

[0940] Specific operation: The server uses natural language processing technology (e.g., NLTK or spaCy) to analyze the text data and understand the intent and content of the user's statements.

[0941] Output: Analysis results

[0942] Step 5:

[0943] Based on the analysis results, the server inputs a prompt sentence into the generative AI model and generates a response.

[0944] Input: Analysis results

[0945] How it works: The server creates a prompt based on the analysis results and inputs it into a generative AI model (e.g., OpenAI's GPT-4), which then generates an appropriate response.

[0946] Output: The generated response

[0947] Step 6:

[0948] The server generates a response and sends it to the terminal.

[0949] Input: Generated response

[0950] Specific operation: The server sends the generated response to the terminal.

[0951] Output: Response sent to the terminal

[0952] Step 7:

[0953] The terminal displays the response to the user.

[0954] Input: Response sent to terminal

[0955] Specific operation: The device displays the response to the user, who then sees the AI ​​partner's response on the device screen.

[0956] Output: The response displayed to the user

[0957] Step 8:

[0958] The server stores conversation data and monitors for abnormalities.

[0959] Input: Conversation data

[0960] Specific operation: The server stores conversation data with the user in a database and monitors changes in the user's conversation patterns and memories.

[0961] Output: Accumulated conversation data and monitoring results

[0962] Step 9:

[0963] If the server detects an abnormality, it will send an alert.

[0964] Input: Monitoring results

[0965] Specific operation: If the server detects an abnormality, it will send an alert to family members or medical institutions via email or SMS.

[0966] Output: Alerts sent

[0967] (Application example 1)

[0968] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0969] For elderly people living alone, reducing feelings of loneliness and early detection of early symptoms of dementia are important issues. However, systems to solve these issues are limited, and there is a lack of support for elderly people to enjoy shopping with peace of mind, especially in physical stores. Furthermore, while services for elderly people in physical stores need to be tailored to individual users' preferences and health conditions, current systems are not adequately able to do so.

[0970] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0971] In this invention, the server includes: a means for including an AI buddy that can be customized to suit the user's preferences and interests; a means for the AI ​​buddy to engage in entertaining daily conversations; a means for detecting early symptoms of dementia through dialogue with the user; a means for analyzing conversation patterns and short-term changes in memory; a means for sending an alert to family or medical institutions if any worrisome symptoms are observed; a means for providing the AI ​​buddy function to elderly customers through robots or smart displays installed in physical stores; and a means for an AI concierge to understand the customer's preferences and interests through conversation when the elderly customer visits the store and suggest appropriate products and services. This allows elderly people to enjoy shopping in physical stores with peace of mind, reduces feelings of loneliness, and enables early detection of early symptoms of dementia.

[0972] "User" refers to an individual who uses the System.

[0973] "Preferences" refer to the things that a user particularly likes or is interested in.

[0974] "Interests" refers to the things or areas that a user is interested in.

[0975] "Customizable" means that settings and features can be changed to suit the user's individual needs and preferences.

[0976] "AI Partner" refers to a part of the system that uses artificial intelligence to interact with and assist users.

[0977] "Daily conversations" refer to the daily interactions between the user and their AI companion.

[0978] "Early symptoms of dementia" refers to symptoms such as a decline in memory and judgment that are seen in the early stages of dementia.

[0979] "Conversation patterns" refer to the format and content of repeated dialogue between the user and their AI companion.

[0980] "Short-term changes in memory" refers to how a user's memory changes over a short period of time.

[0981] An "alert" is a warning or notification that is issued when a specific condition is met.

[0982] "Family" refers to the User's immediate family or close relatives.

[0983] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0984] "Brick and mortar store" refers to a commercial establishment located in a physical location.

[0985] A "robot" refers to a mechanical device that is programmed to perform specific tasks.

[0986] A "smart display" refers to a display device that has internet connectivity and displays information.

[0987] The term "elderly" generally refers to people aged 65 and over.

[0988] "Customers" refer to consumers who visit physical stores.

[0989] "AI concierge" refers to a part of a system that uses artificial intelligence to provide guidance and assistance to customers.

[0990] "Goods" means items sold in physical stores.

[0991] "Services" refers to various assistance and benefits provided in physical stores.

[0992] This invention is a system that includes an AI companion that can be customized to suit the user's preferences and interests, and provides the AI ​​companion function to elderly customers in physical stores. Specific embodiments of this system are described below.

[0993] System configuration

[0994] The system consists of the following main components:

[0995] 1. User device: A device such as a smartphone or tablet.

[0996] 2. Robots or smart displays in brick-and-mortar stores: devices to interact with elderly customers.

[0997] 3. Server: The back-end system for running generative AI models and processing data.

[0998] Program processing

[0999] The server performs the following process.

[1000] 1. Customization of user preferences and interests: When users first access the system, they enter information about their preferences and interests, which is used to customize their AI companion.

[1001] 2. Engaging in everyday conversations: When users visit a physical store, robots and smart displays will engage in everyday conversations with them, using generative AI models.

[1002] 3. Early Dementia Symptom Detection: Analyzes short-term changes in speech patterns and memory to detect early signs of dementia. If specific keywords or phrases are detected, an alert will be sent.

[1003] 4. Sending alerts: If early symptoms of dementia are detected, an alert will be sent to family members or medical institutions.

[1004] Hardware and software used

[1005] Hardware: smartphones, tablets, robots, smart displays

[1006] Software: Generative AI models (e.g., OpenAI APIs)

[1007] Specific examples

[1008] For example, when an elderly customer visits a physical store, the following conversation takes place:

[1009] Example conversation

[1010] User: "Hi, I've been forgetting things a lot lately."

[1011] AI Buddy: "Hello, Tanaka-san. Can you tell me about recent events?"

[1012] During this conversation, the AI ​​companion analyzes the user's memory and conversation patterns and sends out alerts as needed.

[1013] Prompt Sentence Examples

[1014] User: Hi, I've been forgetting things a lot lately.

[1015] AI Companion:

[1016] By inputting this prompt into a generative AI model, an appropriate response can be obtained.

[1017] As described above, this system allows elderly people to enjoy shopping in physical stores with peace of mind, reduces feelings of loneliness, and makes it possible to detect early symptoms of dementia early.

[1018] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1019] Step 1:

[1020] When a user first accesses the system, they enter information about their preferences and interests using their user terminal.

[1021] Input: Information about your preferences and interests (e.g., favorite foods, hobbies)

[1022] Output: Customized user profile

[1023] Specific operation: The user terminal sends the input information to the server, and the server generates a user profile based on the information.

[1024] Step 2:

[1025] When a user visits a physical store, the store's robot or smart display will recognize the user and begin a casual conversation.

[1026] Input: User visit information, user profile

[1027] Output: Generated conversation

[1028] How it works: The robot or smart display detects a user's visit and queries the server for the user's profile. The server then uses the generative AI model to generate appropriate conversation content and sends it to the robot or smart display.

[1029] Step 3:

[1030] The server analyzes conversation patterns and short-term changes in memory to detect early symptoms of dementia through conversations with the user.

[1031] Input: User conversation

[1032] Output: Detection results of early symptoms of dementia

[1033] How it works: The server uses a generative AI model to analyze the conversation and detect specific keywords and phrases, which can then determine whether early symptoms of dementia are present.

[1034] Step 4:

[1035] If early symptoms of dementia are detected, the server will send an alert to family members and medical institutions.

[1036] Input: Detection results of early symptoms of dementia

[1037] Output: Alert notification

[1038] Specific operation: The server generates an alert based on the detection results and sends a notification to pre-registered family and medical contacts.

[1039] Step 5:

[1040] Robots or smart displays in physical stores suggest appropriate products and services based on users' preferences and interests.

[1041] Input: User profile, generated conversation

[1042] Output: Product or service suggestions

[1043] Specific operation: The robot or smart display suggests products and services suitable for the user based on the user profile and conversation content received from the server.

[1044] Through these steps, this system will enable elderly people to enjoy shopping in physical stores with peace of mind, reduce feelings of loneliness, and enable early detection of early symptoms of dementia.

[1045] Example 2

[1046] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1047] While existing AI companion systems can be customized to suit a user's preferences and interests, they are inadequate in adjusting the content of conversations to suit the user's daily life situation and health condition. Furthermore, they lack the ability to assess a user's health condition in real time and provide appropriate advice, limiting their support for health management. Furthermore, they lack the technical means to accurately analyze user input and generate appropriate conversation content. To address these issues, a system capable of more advanced analysis and response generation is needed.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1049] In this invention, the server includes means for receiving user input and converting the voice into text using voice recognition software, means for analyzing the text data using a generative AI model to determine the user's health condition, and means for generating conversation content according to the user's situation using the generative AI model, thereby making it possible to provide appropriate conversation content according to the user's daily life situation and health condition.

[1050] "AI Buddy" is an interactive AI system that provides various information through dialogue with the user and can be customized to suit the user's preferences and interests.

[1051] "Speech recognition software" means software that converts a user's voice input into text data.

[1052] A "generative AI model" is an artificial intelligence model that analyzes input text data and generates appropriate responses and conversation content.

[1053] "Text data" refers to a user's voice input converted by voice recognition software or text information entered directly by a user.

[1054] "Health Status" refers to the state of a User's physical and mental health, including specific symptoms such as colds or fatigue.

[1055] "Conversation content" refers to the information and advice provided by the AI ​​partner during the dialogue with the user.

[1056] "Analysis" is the process by which the generative AI model determines the user's situation and health status based on the input text data.

[1057] "Response generation" is the process by which a generative AI model creates appropriate conversation content based on the analysis results.

[1058] "Speech synthesis software" is software that converts generated text data into speech and responds to the user in speech.

[1059] The present invention relates to an AI companion system that adjusts the content of conversations depending on the user's daily life situation and health condition. Specific embodiments of this system will be described below.

[1060] Hardware and software used

[1061] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1062] Software: Generative AI models (e.g., GPT-4), speech recognition software (e.g., Google Speech-to-Text API), speech synthesis software (e.g., Google Text-to-Speech API)

[1063] System Overview

[1064] The system receives user input, converts speech to text using speech recognition software, analyzes the text data using a generative AI model to determine the user's health status, and then uses the generative AI model to generate conversational content tailored to the user's situation, returning the generated conversational content to the user in voice or text.

[1065] Specific examples

[1066] Consider the case where a user speaks into a smartphone microphone, "I'm feeling a bit under the weather today." This voice input is converted into text data using speech recognition software. Next, the server inputs this text data into a generative AI model, which recognizes the keyword "feeling under the weather" and determines that the user has a cold. The server then inputs a prompt sentence into the generative AI model: "If the user has a cold, ask about their cold symptoms and advise them to get adequate rest." The model then generates conversational content such as "How are your cold symptoms?" and "Please get plenty of rest today." Finally, the generated text data is converted into speech using speech synthesis software, and a voice response is provided to the user, or the same content is displayed in text form.

[1067] Prompt Sentence Examples

[1068] "If the user has a cold, the AI ​​companion should generate conversational content that asks about cold symptoms and advises them to get adequate rest."

[1069] In this way, the AI ​​companion can provide appropriate conversation content based on the user's health condition.

[1070] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1071] Step 1:

[1072] Users use their smartphone or PC to talk to their AI companion via voice or text.

[1073] Input: User voice or text input

[1074] What happens: The user speaks into the smartphone microphone and says, "I'm feeling a bit under the weather today," or types the same thing using text input.

[1075] Output: User's voice or text data

[1076] Step 2:

[1077] The server uses speech recognition software to convert the user's speech into text.

[1078] Input: User's voice data

[1079] What happens: The server receives the voice data and uses speech recognition software (e.g., Google Speech-to-Text API) to convert it into text data such as "I'm feeling a bit under the weather today."

[1080] Output: Text data

[1081] Step 3:

[1082] The server uses the generated AI model to analyze the text data and determine the user's health status.

[1083] Input: Text data

[1084] Specific operation: The server inputs text data into the generative AI model, and the model recognizes the keyword "feeling like I have a cold" and determines that the user has a cold.

[1085] Output: Information about the user's health status (e.g., they have a cold)

[1086] Step 4:

[1087] The server uses a generative AI model to generate conversation content that is appropriate for the user's situation.

[1088] Input: User's health information

[1089] Specific operation: The server inputs the prompt sentence "If the user has a cold, ask about their cold symptoms and advise them to get adequate rest" into the generative AI model, and the model generates conversational content such as "What are your cold symptoms?" and "Please get plenty of rest today."

[1090] Output: Generated conversation content (text data)

[1091] Step 5:

[1092] The device responds to the user with the generated conversation content in voice or text.

[1093] Input: Generated conversation content (text data)

[1094] Specific operation: The server sends the generated text data to the device, which then uses speech synthesis software (e.g., Google Text-to-Speech API) to convert it into speech, asking the user, "How are your cold symptoms?" or "Please get plenty of rest today," or displaying the same information in text.

[1095] Output: A voice or text response to the user

[1096] (Application example 2)

[1097] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1098] In modern society, there is a demand for providing appropriate advice tailored to users' daily lives and health conditions. However, conventional systems have difficulty adjusting the content of conversations based on the user's health condition and daily life situation, and there is a lack of means to effectively utilize generative AI models to provide appropriate advice. Therefore, an effective system is needed to improve users' health management and quality of life.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1100] In this invention, the server includes a means for including an AI companion that can be customized to suit the user's preferences and interests, a means for adjusting the content of the conversation depending on the user's daily life situation and health condition, and a means for providing appropriate advice using a generative AI model based on the user's health condition, thereby enabling the provision of appropriate advice according to the user's health condition and daily life situation, thereby improving the user's health management and quality of life.

[1101] "User" refers to an individual who uses the System.

[1102] "Preferences" refer to the things that a user particularly likes or is interested in.

[1103] "Interests" refers to the things or areas that a user is interested in.

[1104] "Customizable" means that settings and features can be changed according to the user's preferences and interests.

[1105] "AI Partner" refers to a system that uses artificial intelligence to interact with users and provide support.

[1106] "Daily life" refers to the activities and lifestyle habits of the user on a daily basis.

[1107] "Health Status" refers to the state of your physical and mental health.

[1108] "Conversation content" refers to the content of the dialogue that the AI ​​partner has with the user.

[1109] A "generative AI model" refers to a model that uses artificial intelligence to generate text and data.

[1110] "Advice" means any advice or suggestion provided to a User.

[1111] "Dementia" refers to a disease characterized by a decline in memory and cognitive function.

[1112] "Early symptoms" refer to symptoms seen in the early stages of the onset of a disease.

[1113] A "pattern" refers to a series of actions or phenomena that have a particular regularity or tendency.

[1114] "Short-term changes in memory" refers to fluctuations or changes in memory over a short period of time.

[1115] An "alert" is a notification or warning intended to draw attention.

[1116] "Family" refers to people related to you by blood or marriage.

[1117] "Medical institution" refers to a facility or organization that provides medical services.

[1118] "Emotional state" refers to the user's emotional or mood state.

[1119] A system for implementing this invention includes an AI companion that can be customized to a user's preferences and interests, adapts conversations based on the user's daily life situation and health status, and provides appropriate advice using a generative AI model based on the user's health status.

[1120] System Configuration

[1121] The system uses the following hardware and software:

[1122] Hardware: Smartphones, servers

[1123] Software: Python, OpenAI API

[1124] Program processing explanation

[1125] The server provides an interface for acquiring the user's health status. The user inputs their current health status using their smartphone. For example, they can input their status such as "cold," "headache," or "feeling well."

[1126] The server generates a prompt based on the user's health status. This prompt is then input into a generative AI model to generate appropriate advice. The generative AI model used is OpenAI's "text-davinci-003."

[1127] The generated advice is sent from the server to the user's smartphone and displayed, allowing the user to receive appropriate advice based on their own health condition.

[1128] Specific examples

[1129] For example, if the user types "cold," the generated prompt text will be:

[1130] If the user has a cold, provide appropriate advice.

[1131] If you feed this prompt into a generative AI model, it might generate responses like these:

[1132] If you have a cold, it's important to get plenty of rest, drink plenty of fluids, eat foods with vitamin C, and keep warm. Consult your doctor if necessary.

[1133] In this way, appropriate advice can be provided according to the user's health condition.

[1134] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1135] Step 1:

[1136] The user inputs their current health status using their smartphone.

[1137] Input: User's health condition (e.g., "cold," "headache," "feeling well")

[1138] Output: Health status data

[1139] Specific actions: The user opens the application on their smartphone, enters their current health status in the text box, and presses the send button.

[1140] Step 2:

[1141] A server receives health status data sent by a user.

[1142] Input: Health status data

[1143] Output: Acknowledgement message

[1144] Specific operation: Health status data sent from the smartphone reaches the server, which receives and stores the data.

[1145] Step 3:

[1146] The server generates a prompt sentence based on the health status data.

[1147] Input: Health status data

[1148] Output: prompt statement

[1149] Specific operation: The server analyzes the received health status data and generates a prompt sentence, for example, "If the user has a cold, please provide appropriate advice."

[1150] Step 4:

[1151] The prompt sentence generated by the server is input into the generative AI model.

[1152] Input: prompt statement

[1153] Output: Input data to a generative AI model

[1154] Specific operation: The server sends the generated prompt text to OpenAI's API and inputs it into the generative AI model (e.g., text-davinci-003).

[1155] Step 5:

[1156] A generative AI model generates appropriate advice based on the prompt.

[1157] Input: prompt statement

[1158] Output: Advice text

[1159] Specific operation: The generative AI model analyzes the prompt sentence and generates advice, for example, "If you have a cold, it is important to get plenty of rest and drink plenty of fluids."

[1160] Step 6:

[1161] The server receives the generated advice text and transmits it to the smartphone.

[1162] Input: Advice text

[1163] Output: Send confirmation message

[1164] Specific operation: The server sends the advice text received from the generative AI model to the smartphone and confirms that the transmission was successful.

[1165] Step 7:

[1166] The user receives and displays the advice text on the smartphone.

[1167] Input: Advice text

[1168] Output: Advice displayed

[1169] Specific operation: The smartphone displays the advice text received from the server on the screen, and the user confirms it.

[1170] Example 3

[1171] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1172] Conventional AI companion systems have had difficulty accurately estimating a user's emotional state and providing conversation content appropriate to that state. Furthermore, they lacked the ability to adjust conversation content based on the user's health condition and daily life situation, making them ineffective in providing psychological support and managing the user's health. Furthermore, their conversation analysis capabilities to detect early symptoms of dementia and their ability to send alerts to family members and medical institutions were limited. To address these issues, an AI companion system with more advanced emotion analysis and conversation generation capabilities is needed.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1174] In this invention, the server includes means for receiving input data sent from a user, means for analyzing the received input data and estimating an emotional state, and means for sending a prompt sentence to the generative AI model based on the estimated emotional state. This enables the server to generate appropriate conversation content according to the user's emotional state, thereby enabling effective psychological support and health management for the user.

[1175] "Artificial Intelligence Partner" is an artificial intelligence system that provides various functions through dialogue with the user.

[1176] "Input data" refers to information such as text messages and voice data that a user sends through a terminal.

[1177] "Emotional state" refers to the psychological state of the user, and includes emotions such as joy, sadness, anger, and fatigue.

[1178] A "generative artificial intelligence model" is an artificial intelligence algorithm for generating appropriate conversation content based on a given prompt sentence.

[1179] A "prompt" is an instruction sent to a generative AI model based on an emotional state, and serves as the basis for generating appropriate conversation content.

[1180] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[1181] "Sentiment analysis" is the process of analyzing input data to infer a user's emotional state.

[1182] An "alert" is a warning message that notifies family members or medical institutions if any worrisome symptoms are observed.

[1183] "Conversation content" refers to the content of the dialogue that the AI ​​partner provides to the user.

[1184] The present invention relates to an AI companion system that estimates the emotional state of a user and adjusts the content of a conversation in accordance with that state. Specific embodiments of this system will be described below.

[1185] First, a user inputs text messages or voice data using a terminal. For example, consider the case where a user inputs a text message such as "I'm very tired today." This input data is sent from the terminal to a server.

[1186] The server receives input data sent by the user. The received data is temporarily stored in a database on the server. The server then analyzes the input data using emotion analysis software, such as Google Cloud Natural Language API or IBM Watson Tone Analyzer. This analysis estimates the user's emotional state (e.g., joy, sadness, anger, fatigue, etc.).

[1187] Based on the estimated emotional state, the server sends a prompt to the generative AI model. OpenAI's GPT-4 and other models are used as generative AI models. The prompt includes the user's emotional state and an appropriate response accordingly. For example, a prompt such as "The user feels tired. Please offer some words of encouragement" is sent to the generative AI model.

[1188] The generative AI model generates appropriate conversation content based on this prompt. For example, it generates a response such as, "Today was really tough. Let's take a break and refresh yourself!" The generated conversation content is sent from the server to the user's device. The user can receive the response from their AI partner through their device.

[1189] For example, if a user inputs "I feel very tired today," the server receives this message and uses the Google Cloud Natural Language API to estimate the user's emotional state, such as "fatigue" or "depression." The server then sends a prompt to the generative AI model: "The user feels tired. Please offer some words of encouragement." The generative AI model generates a response: "Today was really hard. Take a break and refresh yourself!" The server then sends this response to the user's device. The user can view this message on their device screen and receive words of encouragement.

[1190] In this way, by providing appropriate conversation content according to the emotional state of the user, it becomes possible to effectively provide psychological support and health management for the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1191] Step 1:

[1192] The user sends input data through the terminal.

[1193] A user inputs text messages or voice data using a device such as a smartphone or PC. For example, the user inputs a text message such as "I'm very tired today" and presses the send button. This input data is sent from the device to the server.

[1194] Step 2:

[1195] The server receives input data from the user.

[1196] The server receives text messages and voice data sent by the user from the device. The received data is temporarily stored in a database on the server. Input data includes the user's text message, "I'm very tired today."

[1197] Step 3:

[1198] The server uses emotion analysis software to analyze the input data and estimate the emotional state.

[1199] The server passes the received input data to emotion analysis software (e.g., Google Cloud Natural Language API or IBM Watson Tone Analyzer). The emotion analysis software analyzes the text message and estimates the user's emotional state (e.g., fatigue or depression). The analysis results in an emotional state such as "fatigue" or "depression" being output.

[1200] Step 4:

[1201] The server sends a prompt to the generative AI model.

[1202] The server sends a prompt to a generative AI model (e.g., OpenAI's GPT-4) based on the estimated emotional state. The prompt includes the user's emotional state and an appropriate response accordingly. For example, the server generates a prompt such as "The user feels tired. Please offer some words of encouragement." and sends it to the generative AI model.

[1203] Step 5:

[1204] The generative AI model generates appropriate conversation content based on the prompt text.

[1205] The generative AI model generates appropriate conversational content based on the received prompt. For example, it generates a response such as, "Today was really tough. Let's take a break and refresh yourself!" The generated conversational content is output.

[1206] Step 6:

[1207] The server transmits the generated conversation content to the user's terminal.

[1208] The server sends the conversational content received from the generative AI model to the user's device. The data is displayed on the user's device. The output includes a message such as, "Today was really hard. Take a break and refresh yourself!"

[1209] Step 7:

[1210] The user receives a response from the AI ​​partner through the terminal.

[1211] The user receives a response from the AI ​​buddy via the device. For example, the user can see a message on their smartphone screen saying, "Today was really tough. Take a break and refresh yourself!" and receive words of encouragement.

[1212] (Application example 3)

[1213] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1214] Conventional food delivery systems make suggestions without considering the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Furthermore, they are unable to provide appropriate meal suggestions when the user is feeling stressed or depressed, and lack functionality to support the user's health and mental state.

[1215] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for including an AI companion that can be customized to suit the user's preferences and interests, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, etc., means for sending an alert to family members or medical institutions if any worrisome symptoms are observed, and means for estimating the user's emotional state and making food delivery suggestions based on that emotional state. This enables personalized food delivery suggestions based on the user's emotional state, increasing the user's psychological satisfaction and supporting their health and mental state.

[1216] "User" refers to an individual who uses the System.

[1217] "Preferences and interests" refers to the user's particular interests and preferences.

[1218] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[1219] "AI Partner" refers to a system that uses artificial intelligence to interact with users and provide various types of support.

[1220] "A means of making everyday conversations enjoyable" refers to a function that provides enjoyment through everyday interactions with users.

[1221] "Early symptoms of dementia" refers to early signs of cognitive decline.

[1222] "Conversational patterns" refer to the tendency to use words and progress in conversations with users.

[1223] "Short-term changes in memory" refers to how a user's memory changes over a short period of time.

[1224] "Means of sending alerts" refers to the ability to send notifications when certain conditions occur.

[1225] "Emotional state" refers to a user's current psychological state or mood.

[1226] "Food delivery suggestion" means suggesting food and drink delivery services to a User.

[1227] A system for carrying out this invention includes an AI companion that can be customized to suit a user's preferences and interests. The AI ​​companion includes means for engaging in daily conversations with the user, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for issuing alerts to family members or medical institutions if worrisome symptoms are observed, and means for estimating the user's emotional state and suggesting food delivery options based on that emotional state.

[1228] System Program

[1229] This system includes the following programs:

[1230] 1. Loading the emotion estimation model: The server uses OpenAI's emotion analysis model to estimate the emotional state from the user's utterances.

[1231] 2. Emotional state estimation: The server receives the user's input and estimates the emotional state.

[1232] 3. Food Suggestion: The server makes appropriate food suggestions based on the estimated emotional state.

[1233] Hardware and software used

[1234] Hardware: A device, such as a smartphone or tablet, that receives user input.

[1235] Software: Uses OpenAI's sentiment analysis model for sentiment estimation and a customized algorithm for food recommendations.

[1236] Data processing and calculation

[1237] Data processing: User comments are obtained as text data and input into the sentiment analysis model.

[1238] Data computation: The sentiment analysis model estimates the emotional state of the user from their statements, and the food suggestion algorithm generates appropriate suggestions based on the results.

[1239] Specific examples

[1240] For example, if a user types, "I'm very tired today," the emotion estimation model will infer that they are "feeling stressed." Based on this result, the system will suggest relaxing herbal tea and salad.

[1241] Example prompt sentence:

[1242] User says: I'm so tired today

[1243] Estimate your emotional state.

[1244] In this way, personalized food delivery suggestions based on the user's emotional state become possible, increasing the user's psychological satisfaction and supporting their health and mental state.

[1245] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1246] Step 1:

[1247] The user inputs a message using a terminal, which is then saved as text data on the terminal.

[1248] Step 2:

[1249] The terminal transmits the input text data to the server, which receives the text data and prepares it for input into the emotion estimation model.

[1250] Step 3:

[1251] The server uses OpenAI's sentiment analysis model to estimate the user's emotional state from the received text data. Specifically, the server inputs the text data into the sentiment analysis model and obtains the emotional state (e.g., stress, depression, joy, etc.) as the output.

[1252] Step 4:

[1253] The server runs a food recommendation algorithm based on the estimated emotional state, generating appropriate food delivery recommendations depending on the emotional state. For example, if the emotional state is "stressed," it will suggest relaxing herbal tea and salad.

[1254] Step 5:

[1255] The server sends the generated food suggestions to the terminal, which displays the suggestions to the user, who can review the suggested food delivery and place an order if necessary.

[1256] Step 6:

[1257] If the user decides to order the suggested food delivery, the terminal sends the order information to the server, which receives the order information and forwards the order to the food delivery service.

[1258] Step 7:

[1259] The food delivery service receives the order and delivers the meal to the user. After the delivery is complete, the server sends a notification to the user that the delivery is complete.

[1260] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1261] "Example 1"

[1262] In one embodiment of the present invention, an AI companion is provided that includes an emotion engine that recognizes a user's emotions. The emotion engine infers the user's emotional state from the user's tone of voice, speaking style, and words used. For example, when a user is angry, their voice tone often becomes higher and their speech rate increases. Based on this information, the emotion engine infers that the user is angry.

[1263] "Example 2"

[1264] Furthermore, AI Buddy adjusts the content of the conversation depending on the user's emotional state estimated by the emotion engine. For example, if the user is estimated to be angry, AI Buddy will suggest topics to calm the user. If the user is estimated to be happy, AI Buddy will suggest topics to share that joy.

[1265] "Example 3"

[1266] As described above, the present invention makes it possible to accurately grasp the user's emotional state and provide appropriate dialogue accordingly. This allows the user to enjoy more natural dialogue and reduce feelings of loneliness. Furthermore, early symptoms of dementia can be detected early through changes in the user's emotional state.

[1267] The processing flow of each embodiment will be described below.

[1268] "Example 1"

[1269] Step 1: Through conversations with the user, the AI ​​companion collects information about the user's tone of voice, speaking style, and words used.

[1270] Step 2: Send the collected information to the emotion engine.

[1271] Step 3: The emotion engine estimates the user's emotional state based on the transmitted information.

[1272] "Example 2"

[1273] Step 1: The AI ​​partner adjusts the content of the conversation based on the estimated emotional state received from the emotion engine.

[1274] Step 2: For example, if the user is presumed to be angry, the AI ​​companion will provide a topic to calm the user.

[1275] Step 3: If the AI ​​companion determines that the user is happy, it will provide topics that share that joy.

[1276] "Example 3"

[1277] Step 1: The AI ​​partner understands the user's emotional state through dialogue with them. Step 2: The AI ​​partner detects early symptoms of dementia through changes in the user's emotional state.

[1278] Step 3: If early symptoms of dementia are detected, the AI ​​partner will send an alert to family members or medical institutions.

[1279] Example 1

[1280] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1281] For elderly people living alone, reducing feelings of loneliness and early detection of early symptoms of dementia are important issues. Conventional systems are unable to fully analyze the user's emotional state and conversation patterns, making it difficult to provide appropriate feedback or warnings. Furthermore, there is a need for a system that can effectively collect and analyze the user's voice data and adjust the content of conversations according to their daily life situation and health condition.

[1282] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1283] In this invention, the server includes means for collecting the user's voice and transmitting the voice data to the server, means for converting the voice data into text, and means for analyzing the text data with an emotion recognition engine, which enables detailed analysis of the user's emotional state and speech patterns and provides appropriate feedback and warnings.

[1284] The "Artificial Intelligence Partner" is a system that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[1285] "A means of making everyday conversations enjoyable" is a function that provides conversations that are tailored to the user's interests and concerns through natural dialogue with the user.

[1286] The "means for detecting early symptoms of dementia" is a function that detects early symptoms of dementia by analyzing conversation patterns and short-term changes in memory through dialogue with the user.

[1287] "Means for analyzing conversation patterns and short-term changes in memory" refers to a function that analyzes changes in conversation content and memory by comparing them with the user's past conversation data.

[1288] "Means for sending warnings to family members and medical institutions" is a function that sends warnings to family members and medical institutions when early symptoms of dementia are detected.

[1289] "Means for collecting voice and transmitting voice data to a server" refers to a function for collecting the user's voice with a microphone and transmitting it to a server via the Internet.

[1290] "Means for converting voice data into text" refers to a function that converts collected voice data into text data using a natural language processing engine.

[1291] "Means for analyzing text data using an emotion recognition engine" refers to a function that inputs text data into an emotion recognition engine and estimates the user's emotional state.

[1292] "Means for analyzing conversation patterns and detecting early symptoms of dementia" is a function that analyzes a user's conversation patterns based on text data and detects early symptoms of dementia.

[1293] The "means for providing feedback to the user" is a function that provides appropriate feedback to the user by voice based on the analysis results.

[1294] The present invention is a system that provides an AI companion to assist elderly people living alone. The system includes an AI companion that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[1295] Hardware and software used

[1296] Hardware: Devices with microphones, speakers, and internet connectivity (e.g. smartphones, tablets, smart speakers)

[1297] Software: Natural language processing engines (e.g., Google Cloud Natural Language API), emotion recognition engines (e.g., IBM Watson Tone Analyzer), data analysis tools (e.g., Python's Pandas library)

[1298] Data processing and calculation

[1299] 1. Collecting user conversations

[1300] The terminal collects the user's voice through a microphone.

[1301] The collected voice data is sent to the server.

[1302] 2. Analysis of audio data

[1303] The server converts the voice data into text using a natural language processing engine.

[1304] The converted text data is input into an emotion recognition engine to estimate the user's emotional state.

[1305] 3. Analysis of conversation patterns

[1306] The server stores the text data in a database and compares it with past conversation data.

[1307] Data analysis tools are used to analyze short-term changes in conversation patterns and memory.

[1308] 4. Detecting early symptoms of dementia

[1309] Based on the analysis results, the server determines whether early symptoms of dementia are present.

[1310] If any worrisome symptoms are detected, an alert will be sent to family members or medical institutions.

[1311] 5. User Feedback

[1312] The device provides feedback to the user through the speaker.

[1313] For example, if the AI ​​partner estimates that the user is angry, it will make suggestions such as, "Maybe you should try to calm down a bit?"

[1314] Specific examples

[1315] User conversation: "I went for a walk today, but I got lost."

[1316] System Action:

[1317] The device collects the audio and sends it to the server.

[1318] The server converts the speech into text and analyzes it using an emotion recognition engine.

[1319] The emotion recognition engine infers that the user is feeling anxious.

[1320] The server compares the data with past conversation data and analyzes the possibility of early symptoms of dementia.

[1321] If necessary, alert family members or medical institutions.

[1322] The device will provide feedback to the user saying, "Are you OK? Let's check the map together next time."

[1323] Prompt Sentence Examples

[1324] "Analyze the emotions users express when talking about recent events."

[1325] "It analyzes the user's speech patterns and determines whether they are exhibiting early signs of dementia."

[1326] The above is an embodiment of the present invention.

[1327] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1328] Step 1:

[1329] The user provides voice input.

[1330] The user speaks everyday conversations or questions into the device. For example, "I went for a walk today, but I got lost." The input is the user's voice data, and the output is the voice data collected by the device.

[1331] Step 2:

[1332] The device collects the audio data.

[1333] The device uses a built-in microphone to collect the user's voice. The collected voice data is temporarily stored in the device. The input is the user's voice, and the output is the voice data stored in the device.

[1334] Step 3:

[1335] The terminal transmits the voice data to the server.

[1336] The device sends the collected voice data to a server via the Internet. The data is encrypted before transmission. The input is the voice data stored on the device, and the output is the voice data sent to the server.

[1337] Step 4:

[1338] The server converts the audio data into text.

[1339] The server uses a natural language processing engine to convert the voice data into text, for example, "I went for a walk today, but I got lost." The input is the voice data sent to the server, and the output is the converted text data.

[1340] Step 5:

[1341] The server analyzes the text data using an emotion recognition engine.

[1342] The server uses an emotion recognition engine to infer the user's emotional state from the text data. For example, it infers that the user is feeling anxious. The input is the converted text data, and the output is the inferred emotional state.

[1343] Step 6:

[1344] The server analyzes the conversation patterns.

[1345] The server stores the text data in a database and compares it with past conversation data. Data analysis tools are used to analyze short-term changes in conversation patterns and memory. The input is the estimated emotional state and past conversation data, and the output is the analysis results.

[1346] Step 7:

[1347] The server detects early symptoms of dementia.

[1348] The server determines whether early symptoms of dementia are present based on the analysis results. For example, if the user has recently shown a pattern of frequently getting lost, it determines that this may be an early symptom of dementia. The input is the analysis results, and the output is the detection result of early symptoms of dementia.

[1349] Step 8:

[1350] The server issues an alert.

[1351] If the server detects early symptoms of dementia, it sends an alert to family members or medical institutions. The alert is sent via email or SMS. The input is the detection result of early symptoms of dementia, and the output is the alert sent.

[1352] Step 9:

[1353] The terminal provides feedback to the user.

[1354] The device provides feedback to the user through the speaker, such as "Are you OK? Let's check the map together next time." The input is the feedback from the server, and the output is the feedback provided to the user.

[1355] (Application example 1)

[1356] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1357] For elderly people living alone, it is important to reduce feelings of loneliness and detect early symptoms of dementia early. However, to solve these issues, a system is needed that can grasp the elderly's condition through everyday conversation and respond quickly if an abnormality is detected. It is also necessary to accurately recognize the user's emotional state and respond accordingly. Conventional systems have found it difficult to provide these functions comprehensively.

[1358] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1359] In this invention, the server includes means for including an AI companion that can be customized to suit the user's preferences and interests, means for engaging in daily conversations, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for sending alerts to family members and medical institutions if worrisome symptoms are observed, means for recognizing the user's voice and analyzing emotions, and means for sending alerts if an abnormality is detected. This makes it possible to reduce loneliness among the elderly, detect early symptoms of dementia, and respond quickly.

[1360] "User" refers to an individual who uses the System.

[1361] "AI companion customizable to your preferences and interests" refers to artificial intelligence that can adjust its functions and dialogue content based on the user's individual preferences and interests.

[1362] "A means of making everyday conversations enjoyable" refers to a function that allows users to have daily conversations and provide fun and satisfaction.

[1363] "Means for detecting early symptoms of dementia" refers to a function that detects early signs of dementia through dialogue with the user.

[1364] "Means for analyzing conversation patterns and short-term changes in memory" refers to a function that analyzes the content of a user's conversations and fluctuations in memory and detects abnormalities.

[1365] "Means of sending alerts to family members and medical institutions" refers to the function of sending notifications to the user's family members and medical institutions when an abnormality is detected.

[1366] "Means for recognizing voice and analyzing emotions" refers to the function of analyzing the user's voice and inferring their emotional state.

[1367] "Means for sending alerts when an abnormality is detected" refers to the function of sending an alert to designated contacts when the system detects an abnormality.

[1368] A system for implementing this invention includes an AI companion that can be customized to suit a user's preferences and interests. The system is realized as an application installed on a device such as a smartphone or smart glasses.

[1369] Hardware and software used

[1370] Hardware: Smartphones, smart glasses

[1371] Software: Python, TensorFlow, NLP libraries (e.g., spaCy), speech recognition libraries (e.g., Google Speech-to-Text)

[1372] Data processing and calculation

[1373] Acquiring audio data

[1374] The device's microphone is used to capture the user's voice. Voice data is collected in real time and converted into text data through a speech recognition library.

[1375] Voice Recognition

[1376] A speech recognition library (e.g., Google Speech-to-Text API) is used to convert the captured voice data into text, which is then used for subsequent sentiment analysis and early dementia symptom detection.

[1377] emotion recognition

[1378] The converted text data is analyzed using an NLP library (e.g., spaCy) and an emotion engine (TensorFlow model) is used to estimate the user's emotional state. The emotion engine estimates emotions from the user's tone of voice, speaking style, and the words they use.

[1379] Detecting early symptoms of dementia

[1380] It analyzes short-term changes in conversation patterns and memory to detect anomalies, by analyzing a user's conversation history and monitoring the frequency and changes of specific keywords and phrases.

[1381] Sending alerts

[1382] If an abnormality is detected, the system will send an alert to family members or medical institutions via email or SMS.

[1383] Specific examples

[1384] For example, if a user forgets recent events, the system operates as follows:

[1385] 1. A user asks, "What was for dinner last night?"

[1386] 2. The system recognizes the speech and converts it into text.

[1387] 3. Analyze the converted text and determine that the user has forgotten recent events.

[1388] 4. The emotion engine infers emotions from the user's tone of voice and speaking style, and sends an alert if an anomaly is detected.

[1389] Prompt Sentence Examples

[1390] "If users forget what's happened recently, send them an alert."

[1391] In this way, the invention can reduce the sense of loneliness felt by the elderly and enable early detection and prompt response to early symptoms of dementia.

[1392] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1393] Step 1:

[1394] The device captures the user's voice.

[1395] Input: User's voice

[1396] Output: Audio data

[1397] What it does: Uses the device's microphone to collect the user's voice in real time.

[1398] Step 2:

[1399] The device converts the voice data into text.

[1400] Input: Audio data

[1401] Output: Text data

[1402] Specific operation: Uses a speech recognition library (e.g., Google Speech-to-Text API) to convert the acquired voice data into text.

[1403] Step 3:

[1404] The server analyzes the text data and estimates the emotion.

[1405] Input: Text data

[1406] Output: Emotional state

[1407] Specific behavior: Analyze text data using an NLP library (e.g., spaCy) and infer emotional states using an emotion engine (TensorFlow model).

[1408] Step 4:

[1409] The server analyzes conversation patterns and short-term changes in memory.

[1410] Input: Text data

[1411] Output: Analysis results

[1412] What it does: Analyzes a user's conversation history and monitors the frequency and changes of certain keywords and phrases.

[1413] Step 5:

[1414] The server detects an abnormality.

[1415] Input: Analysis results

[1416] Output: Anomaly detection results

[1417] Specific operation: Based on the analysis results, detect early symptoms of dementia and emotional abnormalities.

[1418] Step 6:

[1419] The server sends an alert if an abnormality is detected.

[1420] Input: Anomaly detection result

[1421] Output: Alert notification

[1422] Specific operation: If an abnormality is detected, an alert will be sent to family members or medical institutions via email or SMS.

[1423] Example 2

[1424] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1425] Conventional AI companion systems can be customized to suit the user's preferences and interests and have the ability to engage in enjoyable daily conversations, but they do not adequately adjust the content of the conversation to reflect the user's health or emotional state. While they offer dialogues to detect early symptoms of dementia and alerts when worrisome symptoms are detected, they lack the ability to generate conversation content that takes into account the user's real-time health and emotional state. This creates the challenge of insufficient health management and psychological support for users.

[1426] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1427] In this invention, the server includes means for collecting user health condition data, means for collecting user emotional state data, and means for analyzing the collected data and generating conversation content using a generative artificial intelligence model, thereby making it possible to provide appropriate conversation content according to the user's health condition and emotional state.

[1428] "Artificial Intelligence Partner" is a system equipped with artificial intelligence that provides various functions through dialogue with the user.

[1429] "Daily life status" refers to information about a user's daily activities and lifestyle habits.

[1430] "Health Status" refers to data relating to a user's physical condition or health.

[1431] "Emotional state" refers to information about a user's psychological state or emotions.

[1432] "Means for adjusting conversation content" refers to methods and technologies for generating and providing appropriate conversation content according to the user's situation or state.

[1433] "Means for collecting health data" refers to the methods and technologies used to obtain data about a user's health.

[1434] "Means for collecting emotional state data" refers to methods or techniques for obtaining data about a user's emotions.

[1435] A "generative artificial intelligence model" refers to an artificial intelligence model that generates appropriate responses and conversation content based on user input and the situation.

[1436] A "prompt" refers to an instruction or question that is input into a generative AI model.

[1437] The present invention relates to an AI companion system that adjusts the content of conversations depending on the user's daily life situation, health condition, and emotional state. Specific embodiments of this system will be described below.

[1438] Hardware and software used

[1439] Hardware:

[1440] High-performance servers (e.g., servers equipped with NVIDIA GPUs)

[1441] Wearable devices (e.g., devices that measure heart rate or temperature)

[1442] Device (e.g. smartphone, tablet)

[1443] software:

[1444] Generative AI models (e.g., OpenAI's GPT-4)

[1445] Emotion engines (e.g., Affectiva's emotion recognition API)

[1446] Text-to-speech engine

[1447] System Operation

[1448] Collecting user health data

[1449] The device collects health status data from the wearable device worn by the user. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time and transmits it to the device via Bluetooth or Wi-Fi.

[1450] Examples:

[1451] The terminal acquires heart rate data from the wearable device.

[1452] The terminal acquires body temperature data from the body temperature sensor.

[1453] The terminal acquires step count data from the activity meter.

[1454] Collecting user emotional state data

[1455] The device uses a camera and microphone to estimate the user's emotional state from their facial expressions and voice, and this data is sent to the emotion engine for analysis.

[1456] Examples:

[1457] The device uses a camera to capture the user's facial expressions.

[1458] The terminal uses a microphone to record the user's voice.

[1459] The terminal transmits the captured facial expression data to the emotion engine.

[1460] The device transmits the recorded voice data to the emotion engine.

[1461] Analyzing data and generating conversation content

[1462] The server analyzes the collected health and emotional state data. Based on the analysis results, it uses a generative AI model to generate appropriate conversation content. It inputs prompt sentences into the generative AI model to obtain the conversation content.

[1463] Examples:

[1464] The server analyzes the health status data to determine whether the user has a cold.

[1465] The server analyzes the emotional state data to determine whether the user is angry.

[1466] The server inputs the prompt sentence to the generative AI model: "Please tell me how the AI ​​companion should proceed with the conversation if the user has a cold."

[1467] The server stores the conversation content obtained from the generative AI model.

[1468] Providing conversation content

[1469] The device provides the user with the conversation content obtained from the server. Specifically, the AI ​​partner uses voice synthesis technology to converse with the user.

[1470] Examples:

[1471] The terminal inputs the conversation content acquired from the server into a voice synthesis engine.

[1472] The terminal uses a speech synthesis engine to convert the conversation into speech.

[1473] The device's AI partner will converse with the user through the speaker.

[1474] Prompt Sentence Examples

[1475] "How should the AI ​​companion proceed with the conversation if the user has a cold?"

[1476] "If the user is angry, tell us how the AI ​​companion should proceed with the conversation."

[1477] In this way, a system is realized in which the server, terminal, and user work together to provide appropriate conversation content according to the user's health condition and emotional state.

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1:

[1480] Collecting user health data

[1481] The device collects health status data from the wearable device worn by the user. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time and transmits it to the device via Bluetooth or Wi-Fi.

[1482] Input: Heart rate, body temperature, and activity data from wearable devices

[1483] Output: Health status data sent to the device

[1484] Specific behavior:

[1485] The terminal acquires heart rate data from the wearable device.

[1486] The terminal acquires body temperature data from the body temperature sensor.

[1487] The terminal acquires step count data from the activity meter.

[1488] Step 2:

[1489] Collecting user emotional state data

[1490] The device uses a camera and microphone to estimate the user's emotional state from their facial expressions and voice, and this data is sent to the emotion engine for analysis.

[1491] Input: facial expression data captured by a camera, audio data recorded by a microphone

[1492] Output: Emotional state data sent to the emotion engine

[1493] Specific behavior:

[1494] The device uses a camera to capture the user's facial expressions.

[1495] The terminal uses a microphone to record the user's voice.

[1496] The terminal transmits the captured facial expression data to the emotion engine.

[1497] The device transmits the recorded voice data to the emotion engine.

[1498] Step 3:

[1499] Analyzing data and generating conversation content

[1500] The server analyzes the collected health and emotional state data. Based on the analysis results, it uses a generative AI model to generate appropriate conversation content. It inputs prompt sentences into the generative AI model to obtain the conversation content.

[1501] Input: Health status data, emotional status data, prompt text

[1502] Output: Generated conversation

[1503] Specific behavior:

[1504] The server analyzes the health status data to determine whether the user has a cold.

[1505] The server analyzes the emotional state data to determine whether the user is angry.

[1506] The server inputs the prompt sentence to the generative AI model: "Please tell me how the AI ​​companion should proceed with the conversation if the user has a cold."

[1507] The server stores the conversation content obtained from the generative AI model.

[1508] Step 4:

[1509] Providing conversation content

[1510] The device provides the user with the conversation content obtained from the server. Specifically, the AI ​​partner uses voice synthesis technology to converse with the user.

[1511] Input: Conversation content obtained from the server

[1512] Output: The audio conversation that is presented to the user

[1513] Specific behavior:

[1514] The terminal inputs the conversation content acquired from the server into a voice synthesis engine.

[1515] The terminal uses a speech synthesis engine to convert the conversation into speech.

[1516] The device's AI partner will converse with the user through the speaker.

[1517] (Application example 2)

[1518] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1519] Conventional AI partner systems can be customized to suit the user's preferences and interests, and are equipped with functions to facilitate enjoyable daily conversations and detect early symptoms of dementia. However, when it comes to customer service in physical stores, they lack the functionality to analyze customers' facial expressions and physical condition in real time and provide optimal customer service to each individual. This poses a challenge in that personalized customer service that would improve customer satisfaction has not been realized.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1521] In this invention, the server includes a means for including an AI companion that can be customized to suit the user's preferences and interests, a means for adjusting the content of conversation depending on the user's daily life situation and health condition, a means for estimating the user's emotional state and adjusting the content of conversation depending on that emotional state, and a means for analyzing the facial expressions and physical condition of customers in real time and providing optimal customer service to each individual customer, thereby enabling personalized customer service to improve customer satisfaction in physical stores.

[1522] "User" refers to an individual or customer who uses the System.

[1523] "Preferences" refer to the things that a user particularly likes or is interested in.

[1524] "Interests" refers to the things or areas that a user is interested in.

[1525] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[1526] "AI Partner" is an interactive system equipped with artificial intelligence that provides various types of support through conversations with users.

[1527] "Everyday life situations" refer to events and conditions that users experience in their daily lives.

[1528] "Health Status" refers to the state of a User's physical and mental health.

[1529] "Conversation content" refers to the words and topics exchanged between the AI ​​partner and the user.

[1530] "Emotional state" refers to a user's current feelings or mood.

[1531] "Facial Expressions" refers to the user's facial expressions and movements.

[1532] "Physical condition" refers to the user's physical health and condition.

[1533] "Real-time" refers to processing or reaction occurring immediately, without delay.

[1534] "Analysis" refers to examining and understanding data and information in detail.

[1535] "Customer service" refers to the service and response provided by store staff to customers.

[1536] "Personalized customer service" refers to providing services that are individually optimized to suit the needs and circumstances of each customer.

[1537] The system for implementing this invention includes an AI partner that can be customized to suit the user's preferences and interests. The AI ​​partner adjusts the content of conversations based on the user's daily life situation and health condition, and further estimates the user's emotional state and provides conversation content appropriate to that emotional state. In addition, in physical stores, the AI ​​partner has the ability to analyze customers' facial expressions and physical condition in real time to provide optimal customer service to each individual.

[1538] Hardware and software used

[1539] Hardware: smart glasses, camera

[1540] Software: OpenCV (face detection), EmotionRecognizer (emotion recognition), HealthStatusRecognizer (health status recognition), ConversationEngine (conversation content generation)

[1541] System processing overview

[1542] 1. Video capture from camera: Using a camera to capture video in real time, the camera built into the smart glasses captures the faces of customers.

[1543] 2. Emotion and health status recognition: Detect faces from captured video and recognize emotions using EmotionRecognizer. Also, recognize health status using HealthStatusRecognizer.

[1544] 3. Conversation content generation: Based on the recognized emotions and health status, the Conversation Engine generates appropriate conversation content.

[1545] 4. Display on smart glasses: The generated conversation content is displayed on smart glasses, allowing staff to provide personalized service to customers.

[1546] Specific examples

[1547] For example, if the customer is determined to be tired through emotion and health status recognition, the smart glasses will display a message saying, "Please suggest products that will help them relax." If the customer is determined to be depressed, the smart glasses will display a message saying, "Please offer some encouraging conversation."

[1548] Prompt Sentence Examples

[1549] Below are some example prompts to input to a generative AI model:

[1550] Analyze the user's facial expressions and health status to generate appropriate conversations. For example, if the user is tired, suggest products that will help them relax, and if they are feeling down, provide encouraging conversations.

[1551] In this way, a system can be realized that allows for more personalized customer service in physical stores.

[1552] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1553] Step 1:

[1554] Capture video from the camera

[1555] The server captures video in real time using the camera built into the smart glasses. The input is video data from the camera, and the output is the captured video frames, which are used in subsequent processing steps.

[1556] Step 2:

[1557] Face detection

[1558] The server processes the captured video frames using OpenCV to detect faces. The input is the video frame, and the output is the position information of the detected face. Specifically, the server extracts the facial contours and feature points.

[1559] Step 3:

[1560] emotion recognition

[1561] The server uses EmotionRecognizer to recognize emotions based on the detected face location information. The input is face location information and video frames, and the output is the user's emotional state (e.g., joy, anger, sadness, etc.). Specifically, emotions are estimated from facial expressions.

[1562] Step 4:

[1563] Health Status Awareness

[1564] The server recognizes the health status using HealthStatusRecognizer based on the location information of the detected face. The input is the location information of the face and the video frame, and the output is the user's health status (e.g., fatigue, vitality, etc.). Specifically, the health status is estimated from factors such as facial color and eye condition.

[1565] Step 5:

[1566] Conversation generation

[1567] The server uses the ConversationEngine to generate appropriate conversation content based on the recognized emotional state and health status. The input is the emotional state and health status, and the output is the generated conversation content. Specifically, a prompt sentence is input into the generative AI model to generate appropriate conversation content.

[1568] Step 6:

[1569] Displayed on smart glasses

[1570] The server displays the generated conversation content on the smart glasses. The input is the generated conversation content, and the output is text or messages displayed on the smart glasses' display. Specifically, instructions are displayed for staff to provide personalized customer service.

[1571] Example 3

[1572] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1573] In modern society, the number of people experiencing loneliness and mental stress is increasing. It is also difficult to detect the early symptoms of dementia, which often delays appropriate measures. Furthermore, while there is a need for appropriate dialogue based on the user's emotional state, current systems have difficulty achieving this. To solve these issues, a system is needed that can accurately grasp the user's emotional state and provide appropriate dialogue accordingly.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1575] In this invention, the server includes a means for converting the user's voice data into text, a means for inputting the text data into an emotion analysis model to estimate the emotional state, and a means for inputting a prompt sentence according to the emotional state into a generative AI model, thereby making it possible to accurately grasp the user's emotional state and provide appropriate dialogue accordingly.

[1576] "Artificial Intelligence Partner" is an artificial intelligence system that understands the user's emotional state and health condition through dialogue with them and provides appropriate conversation content.

[1577] "Voice data" refers to data that digitally represents the voice signals generated when a user speaks to a terminal.

[1578] The "means for converting to text" refers to a means that uses voice recognition technology to convert voice data into text information.

[1579] A "sentiment analysis model" is a model that includes machine learning algorithms for analyzing text data and inferring a user's emotional state.

[1580] A "prompt" is an instruction sentence that is input into a generative AI model to generate an appropriate response based on the user's emotional state.

[1581] A "generative artificial intelligence model" is an artificial intelligence model for generating natural dialogue based on an input prompt sentence.

[1582] The "means for generating conversation content" is a means for generating appropriate conversation content according to the emotional state of the user using an emotion analysis model and a generative artificial intelligence model.

[1583] "Means of communicating by voice" refers to a means that uses voice synthesis technology to convert the generated text data into voice and communicate it to the user.

[1584] "Means for sending alerts" refers to the means for sending notifications to family members or medical institutions if any abnormalities are detected in the user's health or emotional state.

[1585] The present invention is a system including an AI companion that estimates the emotional state of a user and adjusts the content of a conversation depending on the emotional state. Specific embodiments of this system are described below.

[1586] First, when a user speaks into the device, the device captures the voice data. The device needs a microphone and a network connection so that the voice data can be transmitted to the server.

[1587] The server uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the voice data into text, which is a written representation of what you said.

[1588] The server then uses a sentiment analysis model (e.g., a BERT-based sentiment analysis model) to estimate the user's emotional state from the text data. The sentiment analysis model includes a machine learning algorithm to analyze the text data and estimate the user's emotional state.

[1589] After the emotional state is estimated, the server inputs a prompt into the generative AI model (e.g., OpenAI's GPT-3). The prompt is an instruction to generate an appropriate response according to the user's emotional state. For example, if the user says, "I'm very tired today," the prompt would be, "If the user is feeling tired, how would you encourage them?"

[1590] The generative AI model generates natural-sounding dialogue based on the input prompt, such as "You've worked really hard today. Let's take a break and refresh yourself."

[1591] The generated conversation content is sent from the server to the device, which then uses speech synthesis software (e.g., Google Cloud Text-to-Speech) to convert the generated text data into speech and convey it to the user, allowing the user to enjoy natural conversations with their AI companion.

[1592] As a concrete example, consider the case where a user says to a device, "I'm very tired today." This voice data is sent from the device to a server and converted into text by speech recognition software. A sentiment analysis model analyzes this text and infers that the user is tired. The server inputs the prompt sentence "If the user feels tired, what words of encouragement would you give them?" into the generative AI model. The generative AI model generates a response such as "You worked really hard today. Take a break and refresh yourself," and the server sends it to the device. The device then converts the text into speech and conveys it to the user.

[1593] This system enables users to reduce their sense of loneliness through natural conversation with their AI companion and to detect early symptoms of dementia through changes in their emotional state. The flow of the identification process in Example 3 will be explained with reference to Figure 21.

[1594] Step 1:

[1595] The user speaks into the terminal.

[1596] For example, the user might say to the terminal, "I'm very tired today." This voice data becomes the input for proceeding to the next processing step.

[1597] Step 2:

[1598] The terminal transmits the voice data to the server.

[1599] The device captures the user's voice data and sends it to the server, using the device's microphone and network connection. The input is the user's voice data, and the output is digital voice data sent to the server.

[1600] Step 3:

[1601] The server converts the audio data into text.

[1602] The server uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the voice data into text. The input is digital voice data, and the output is text data. For example, the generated text might say, "I'm very tired today."

[1603] Step 4:

[1604] The server inputs the text data into a sentiment analysis model to estimate the emotional state.

[1605] The server uses a BERT-based sentiment analysis model to estimate the user's emotional state from text data. The input is text data, and the output is the estimated emotional state. In this case, the estimated emotional state is "tired."

[1606] Step 5:

[1607] The server inputs prompt sentences according to the emotional state into the generative AI model.

[1608] The server generates a prompt sentence based on the emotional state. For example, the prompt sentence "If the user feels tired, how would you offer words of encouragement?" is input to the generative AI model. The input is the estimated emotional state, and the output is the prompt sentence.

[1609] Step 6:

[1610] The server receives the response from the generative AI model and generates appropriate conversation content.

[1611] The server receives responses from a generative AI model (e.g., OpenAI's GPT-3) and generates appropriate conversational content. The input is a prompt, and the output is the generated conversational content. For example, a response might be, "You've worked really hard today. Let's take a break and refresh yourself."

[1612] Step 7:

[1613] The server transmits the generated conversation content to the terminal.

[1614] The server sends the generated conversation content to the terminal. The input is the generated conversation content, and the output is text data sent to the terminal.

[1615] Step 8:

[1616] The terminal conveys the contents of the conversation to the user by voice.

[1617] The device converts the received text data into speech using speech synthesis software (e.g., Google Cloud Text-to-Speech) and conveys it to the user. The input is text data, and the output is audio data. The user can hear a voice message saying, "You've worked really hard today. Let's take a break and refresh yourself."

[1618] (Application example 3)

[1619] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1620] In modern society, understanding a user's emotional state in real time and providing appropriate dialogue based on that state is important for reducing feelings of loneliness and maintaining mental health. However, conventional systems have had difficulty accurately estimating a user's emotional state and adjusting the content of conversations accordingly. Furthermore, even in brick-and-mortar stores, it has been difficult to provide customer service that reflects the emotional state of customers, making it difficult to improve customer satisfaction.

[1621] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a means including an AI companion that can be customized to suit the user's preferences and interests, a means for detecting early symptoms of dementia through dialogue with the user, a means for analyzing conversation patterns and short-term changes in memory, a means for issuing an alert to family members or medical institutions if any worrisome symptoms are observed, and a means for grasping the user's emotional state in real time in a physical store and generating a customer service response accordingly. This makes it possible to accurately grasp the user's emotional state and provide appropriate dialogue and customer service accordingly.

[1622] "User" refers to an individual who uses the System.

[1623] "Preferences and interests" refers to the user's particular interests and preferences.

[1624] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[1625] "AI Aibou" is an interactive system equipped with artificial intelligence, and refers to a device or software that communicates with users.

[1626] "Daily conversations" refers to the daily interactions between the user and their AI companion.

[1627] "Early symptoms of dementia" refers to early signs of cognitive decline.

[1628] "Conversational patterns" refer to the repetition of words and phrases that a user uses on a daily basis.

[1629] "Short-term memory changes" refers to how a user's memory fluctuates over a short period of time.

[1630] "Alert" refers to a function that notifies you when a specific situation occurs.

[1631] "Family or Healthcare Provider" refers to the User's next of kin or organizations that provide health care services.

[1632] "Daily life situations" refers to the user's daily activities and living environment.

[1633] "Health Status" refers to the state of a User's physical and mental health.

[1634] An "emotional state" refers to the emotion a user is feeling at a particular point in time.

[1635] "Real-time" refers to near-instant processing or response.

[1636] "Customer service response" refers to the responses and assistance provided by staff to customers in a physical store.

[1637] A system for carrying out this invention includes an AI companion that can be customized to suit a user's preferences and interests. The AI ​​companion includes means for entertaining daily conversations, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for sending alerts to family members or medical institutions if worrisome symptoms are observed, and means for grasping the user's emotional state in real time in a physical store and generating corresponding customer service responses.

[1638] Hardware and software used

[1639] Hardware

[1640] Smart Glasses: Smart glasses with a camera are used to capture the faces of customers.

[1641] Computer: Used to process data and run emotion recognition models.

[1642] software

[1643] OpenCV: An image processing library used for face detection and preprocessing.

[1644] Keras: A deep learning framework for running emotion recognition models.

[1645] TextBlob: Used for text generation and natural language processing.

[1646] Data processing and calculation

[1647] Image capture and preprocessing

[1648] The smart glasses camera captures the customer's face, and facial detection is performed using OpenCV. The detected facial image is converted to grayscale and resized to an appropriate size for input into the emotion recognition model.

[1649] emotion recognition

[1650] A facial image is input to a pre-trained emotion recognition model using Keras to estimate the user's emotional state, and an appropriate customer response is generated based on the estimated emotional state.

[1651] Response Generation

[1652] TextBlob is used to generate a text response based on the estimated emotional state, which is then displayed on the smart glasses display, allowing staff to serve the customer accordingly.

[1653] Specific examples

[1654] For example, if a customer is feeling stressed, the emotion recognition model will infer "stress" and generate a response using TextBlob: "Sir, we would like to introduce some products that will help you relax." This response will be displayed on the smart glasses display, and staff will assist the customer accordingly.

[1655] Prompt Sentence Examples

[1656] Develop an application that applies the AI ​​Aibou system, which estimates the user's emotional state and adjusts the conversation content accordingly, to customer service in a brick-and-mortar store. Using smart glasses, the system will grasp the emotional state of customers in real time and help them serve them accordingly. Please explain the specific program code and its processing.

[1657] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1658] Step 1:

[1659] The camera in the smart glasses captures the customer's face. The input is the camera image, and the output is the captured facial image. The smart glasses acquire the image in real time and generate image data for face detection.

[1660] Step 2:

[1661] The server performs face detection using OpenCV. The input is the face image captured in step 1, and the output is the area (coordinate information) of the detected face. The face detection algorithm identifies the face in the image and outputs its position as coordinates.

[1662] Step 3:

[1663] The server converts the detected face image to grayscale and resizes it to an appropriate size for input to the emotion recognition model. The input is the face region obtained in step 2, and the output is the preprocessed face image. An image processing algorithm standardizes the face image and converts it into a format suitable for model input.

[1664] Step 4:

[1665] The server inputs the preprocessed facial images into a pre-trained emotion recognition model using Keras to estimate the user's emotional state. The input is the preprocessed facial image from step 3, and the output is the estimated emotional state (e.g., happiness, sadness, anger, etc.). The emotion recognition model analyzes the image data and outputs emotion labels.

[1666] Step 5:

[1667] The server uses TextBlob to generate a text response according to the estimated emotional state. The input is the emotional state estimated in step 4, and the output is the generated text response. A natural language processing algorithm generates an appropriate response based on the emotional state.

[1668] Step 6:

[1669] The server displays the generated text response on the display of the smart glasses. The input is the text response generated in step 5, and the output is the response sentence displayed on the display of the smart glasses. The smart glasses receive the text data and display it on the display.

[1670] Step 7:

[1671] The user (staff member) serves the customer according to the response displayed on the smart glasses display. The input is the text response displayed in step 6, and the output is the user's customer service behavior. The staff member refers to the displayed response and provides appropriate service.

[1672] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1673] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1674] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1675] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1676] [Third embodiment]

[1677] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1678] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1679] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1680] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1681] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1682] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1683] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1684] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1685] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1686] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1687] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1688] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1689] "Example 1"

[1690] One embodiment of the present invention provides a system for supporting elderly people living alone. This system includes an AI companion that can be customized to suit the user's preferences and interests. The AI ​​companion engages in daily conversations with the user, helping to reduce the user's sense of loneliness through these conversations. Furthermore, the AI ​​companion has the ability to detect early symptoms of dementia through dialogue with the user. Specifically, the AI ​​companion analyzes conversation patterns and short-term changes in memory, and if any worrisome symptoms are detected, it sends an alert to family members or medical institutions.

[1691] "Example 2"

[1692] In another embodiment of the present invention, the AI ​​buddy can adjust the conversation content according to the user's daily life situation and health condition. For example, if the user has a cold, the AI ​​buddy can ask about the symptoms of the cold and advise the user to get adequate rest.

[1693] "Example 3"

[1694] In a further embodiment of the present invention, the system further includes a function for the AI ​​buddy to estimate the user's emotional state and adjust the conversation content according to the estimated emotional state. For example, if the AI ​​buddy estimates that the user is feeling depressed, the AI ​​buddy may offer words of encouragement or start a conversation about the user's favorite topics.

[1695] The processing flow of each embodiment will be described below.

[1696] "Example 1"

[1697] Step 1: The AI ​​companion is customized based on the user's preferences and interests. For example, if the user is interested in music or movies, the AI ​​companion will provide conversations about those topics.

[1698] Step 2: The AI ​​companion will have daily conversations with the user, which are designed to reduce the user's sense of loneliness.

[1699] Step 3: AI Aibou will detect early signs of dementia through dialogue with the user, specifically by analyzing conversation patterns and short-term changes in memory.

[1700] Step 4: If any worrisome symptoms are observed, your AI companion will send an alert to your family or a medical institution.

[1701] "Example 2"

[1702] Step 1: The AI ​​companion will adjust the conversation content according to the user's daily life situation and health condition. For example, if the user has a cold, the AI ​​companion will ask about the symptoms of the cold and advise them to get adequate rest.

[1703] "Example 3"

[1704] Step 1: The AI ​​companion estimates the user's emotional state, which is inferred from the user's tone of voice, speaking style, and words used.

[1705] Step 2: The AI ​​buddy adapts the conversation based on the estimated emotional state. For example, if the AI ​​buddy estimates that the user is feeling down, it might offer words of encouragement or bring up topics the user likes.

[1706] Example 1

[1707] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1708] For elderly people living alone, it is important to reduce feelings of loneliness and detect early symptoms of dementia early. However, effective methods to achieve this are lacking. There is also a need for systems that can provide conversations tailored to the user's preferences and interests, and adjust the content of conversations according to their health status. Furthermore, there is a need for technology that can accurately analyze users' comments and generate appropriate responses. A system that can solve these issues is needed.

[1709] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1710] In this invention, the server includes a means for converting a user's speech into text data using speech recognition software, a means for inputting prompts into a generative AI model to generate a response, and a means for displaying the generated response to the user. This reduces the user's sense of loneliness and enables early detection of early symptoms of dementia. It also enables the provision of conversations tailored to the user's preferences and interests, and the content of the conversation to be adjusted according to the user's health condition.

[1711] The "Artificial Intelligence Partner" is a system that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[1712] "Voice recognition software" is a technology that converts a user's speech from voice data into text data.

[1713] A "generative AI model" is an artificial intelligence technology that generates appropriate responses based on input prompts.

[1714] A "prompt sentence" is text data input into a generative AI model and is an instruction sentence for generating a response.

[1715] "Text Data" means the written information of a user's speech converted by voice recognition software.

[1716] An "alert" is a warning message sent to notify family members or medical institutions when an abnormality is detected.

[1717] "Conversation data" is a record of the conversation between the user and the AI ​​companion.

[1718] An "anomaly" is an abnormal condition, such as early symptoms of dementia, detected based on changes in a user's speech patterns or memory.

[1719] The present invention is a system for supporting elderly people living alone, including an AI companion that can be customized to suit the user's preferences and interests. The system engages in daily conversations with the user, helping to reduce the user's sense of loneliness. The AI ​​companion also has the ability to detect early symptoms of dementia through dialogue with the user.

[1720] Hardware and software used

[1721] Hardware:

[1722] Devices such as smartphones and tablets

[1723] server

[1724] software:

[1725] Voice recognition software (e.g., Google Speech-to-Text)

[1726] Natural language processing technology (e.g., NLTK and spaCy)

[1727] Generative AI models (e.g., OpenAI's GPT-4)

[1728] Data processing and calculation

[1729] Processing user utterances:

[1730] When a user initiates a conversation with their AI partner through the device, the device uses voice recognition software to convert the user's speech into text, which is then sent to a server.

[1731] Analyzing text data:

[1732] The server analyzes the received text data using natural language processing technology, and as a result of the analysis, it understands the intent and content of the user's comments.

[1733] Response generation:

[1734] Based on the analysis results, the server inputs a prompt sentence into the generative AI model, which then generates an appropriate response based on this prompt sentence.

[1735] View response:

[1736] The server generates a response and sends it to the terminal, which displays the response to the user.

[1737] Accumulation of conversation data and monitoring for anomalies:

[1738] The server stores conversation data with the user in a database and monitors changes in the user's conversation patterns and memory. If an abnormality is detected, the server sends an alert to family members or medical institutions.

[1739] Specific examples

[1740] For example, if a user says, "I can't remember what I had for dinner today, what was for dinner last night?", the following happens:

[1741] 1. User: "I can't remember what I had for dinner today. What was for dinner last night?"

[1742] 2. Terminal: Uses voice recognition software to convert speech into text data and send it to the server.

[1743] 3. Server: Analyzes the text data and inputs prompt sentences into the generative AI model.

[1744] 4. Generative AI model: Generates the response, "You had curry rice for dinner last night. What are you planning to eat today?"

[1745] 5. Server: Sends the generated response to the terminal.

[1746] 6. Terminal: displays the response to the user.

[1747] 7. Server: Stores conversation data and monitors for abnormalities. If an abnormality is detected, an alert is sent.

[1748] Example prompt sentence:

[1749] User: "I can't remember what I had for dinner today. What was for dinner last night?"

[1750] AI Buddy: "You had curry rice for dinner last night. What are you planning to eat today?"

[1751] In this way, the system of the present invention can reduce the user's sense of loneliness and detect early symptoms of dementia. It can also provide conversations tailored to the user's preferences and interests, and adjust the content of conversations according to the user's health condition.

[1752] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1753] Step 1:

[1754] The user initiates a conversation with the AI ​​partner through the terminal.

[1755] Input: User's spoken utterance

[1756] What happens: A user says, "I can't remember what I had for dinner today. What was for dinner last night?"

[1757] Output: Audio data

[1758] Step 2:

[1759] The device converts the user's speech into text data using voice recognition software.

[1760] Input: Audio data

[1761] What happens: Your device uses voice recognition software (e.g., Google Speech-to-Text) to convert voice data into text.

[1762] Output: Text data

[1763] Step 3:

[1764] The terminal transmits the converted text data to the server.

[1765] Input: Text data

[1766] Specific operation: The terminal sends text data to the server.

[1767] Output: Text data sent to the server

[1768] Step 4:

[1769] The text data received by the server is analyzed using natural language processing technology.

[1770] Input: Text data

[1771] Specific operation: The server uses natural language processing technology (e.g., NLTK or spaCy) to analyze the text data and understand the intent and content of the user's statements.

[1772] Output: Analysis results

[1773] Step 5:

[1774] Based on the analysis results, the server inputs a prompt sentence into the generative AI model and generates a response.

[1775] Input: Analysis results

[1776] How it works: The server creates a prompt based on the analysis results and inputs it into a generative AI model (e.g., OpenAI's GPT-4), which then generates an appropriate response.

[1777] Output: The generated response

[1778] Step 6:

[1779] The server generates a response and sends it to the terminal.

[1780] Input: Generated response

[1781] Specific operation: The server sends the generated response to the terminal.

[1782] Output: Response sent to the terminal

[1783] Step 7:

[1784] The terminal displays the response to the user.

[1785] Input: Response sent to terminal

[1786] Specific operation: The device displays the response to the user, who then sees the AI ​​partner's response on the device screen.

[1787] Output: The response displayed to the user

[1788] Step 8:

[1789] The server stores conversation data and monitors for abnormalities.

[1790] Input: Conversation data

[1791] Specific operation: The server stores conversation data with the user in a database and monitors changes in the user's conversation patterns and memories.

[1792] Output: Accumulated conversation data and monitoring results

[1793] Step 9:

[1794] If the server detects an abnormality, it will send an alert.

[1795] Input: Monitoring results

[1796] Specific operation: If the server detects an abnormality, it will send an alert to family members or medical institutions via email or SMS.

[1797] Output: Alerts sent

[1798] (Application example 1)

[1799] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1800] For elderly people living alone, reducing feelings of loneliness and early detection of early symptoms of dementia are important issues. However, systems to solve these issues are limited, and there is a lack of support for elderly people to enjoy shopping with peace of mind, especially in physical stores. Furthermore, while services for elderly people in physical stores need to be tailored to individual users' preferences and health conditions, current systems are not adequately able to do so.

[1801] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1802] In this invention, the server includes: a means for including an AI buddy that can be customized to suit the user's preferences and interests; a means for the AI ​​buddy to engage in entertaining daily conversations; a means for detecting early symptoms of dementia through dialogue with the user; a means for analyzing conversation patterns and short-term changes in memory; a means for sending an alert to family or medical institutions if any worrisome symptoms are observed; a means for providing the AI ​​buddy function to elderly customers through robots or smart displays installed in physical stores; and a means for an AI concierge to understand the customer's preferences and interests through conversation when the elderly customer visits the store and suggest appropriate products and services. This allows elderly people to enjoy shopping in physical stores with peace of mind, reduces feelings of loneliness, and enables early detection of early symptoms of dementia.

[1803] "User" refers to an individual who uses the System.

[1804] "Preferences" refer to the things that a user particularly likes or is interested in.

[1805] "Interests" refers to the things or areas that a user is interested in.

[1806] "Customizable" means that settings and features can be changed to suit the user's individual needs and preferences.

[1807] "AI Partner" refers to a part of the system that uses artificial intelligence to interact with and assist users.

[1808] "Daily conversations" refer to the daily interactions between the user and their AI companion.

[1809] "Early symptoms of dementia" refers to symptoms such as a decline in memory and judgment that are seen in the early stages of dementia.

[1810] "Conversation patterns" refer to the format and content of repeated dialogue between the user and their AI companion.

[1811] "Short-term changes in memory" refers to how a user's memory changes over a short period of time.

[1812] An "alert" is a warning or notification that is issued when a specific condition is met.

[1813] "Family" refers to the User's immediate family or close relatives.

[1814] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1815] "Brick and mortar store" refers to a commercial establishment located in a physical location.

[1816] A "robot" refers to a mechanical device that is programmed to perform specific tasks.

[1817] A "smart display" refers to a display device that has internet connectivity and displays information.

[1818] The term "elderly" generally refers to people aged 65 and over.

[1819] "Customers" refer to consumers who visit physical stores.

[1820] "AI concierge" refers to a part of a system that uses artificial intelligence to provide guidance and assistance to customers.

[1821] "Goods" means items sold in physical stores.

[1822] "Services" refers to various assistance and benefits provided in physical stores.

[1823] This invention is a system that includes an AI companion that can be customized to suit the user's preferences and interests, and provides the AI ​​companion function to elderly customers in physical stores. Specific embodiments of this system are described below.

[1824] System configuration

[1825] The system consists of the following main components:

[1826] 1. User device: A device such as a smartphone or tablet.

[1827] 2. Robots or smart displays in brick-and-mortar stores: devices to interact with elderly customers.

[1828] 3. Server: The back-end system for running generative AI models and processing data.

[1829] Program processing

[1830] The server performs the following process.

[1831] 1. Customization of user preferences and interests: When users first access the system, they enter information about their preferences and interests, which is used to customize their AI companion.

[1832] 2. Engaging in everyday conversations: When users visit a physical store, robots and smart displays will engage in everyday conversations with them, using generative AI models.

[1833] 3. Early Dementia Symptom Detection: Analyzes short-term changes in speech patterns and memory to detect early signs of dementia. If specific keywords or phrases are detected, an alert will be sent.

[1834] 4. Sending alerts: If early symptoms of dementia are detected, an alert will be sent to family members or medical institutions.

[1835] Hardware and software used

[1836] Hardware: smartphones, tablets, robots, smart displays

[1837] Software: Generative AI models (e.g., OpenAI APIs)

[1838] Specific examples

[1839] For example, when an elderly customer visits a physical store, the following conversation takes place:

[1840] Example conversation

[1841] User: "Hi, I've been forgetting things a lot lately."

[1842] AI Buddy: "Hello, Tanaka-san. Can you tell me about recent events?"

[1843] During this conversation, the AI ​​companion analyzes the user's memory and conversation patterns and sends out alerts as needed.

[1844] Prompt Sentence Examples

[1845] User: Hi, I've been forgetting things a lot lately.

[1846] AI Companion:

[1847] By inputting this prompt into a generative AI model, an appropriate response can be obtained.

[1848] As described above, this system allows elderly people to enjoy shopping in physical stores with peace of mind, reduces feelings of loneliness, and makes it possible to detect early symptoms of dementia early.

[1849] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1850] Step 1:

[1851] When a user first accesses the system, they enter information about their preferences and interests using their user terminal.

[1852] Input: Information about your preferences and interests (e.g., favorite foods, hobbies)

[1853] Output: Customized user profile

[1854] Specific operation: The user terminal sends the input information to the server, and the server generates a user profile based on the information.

[1855] Step 2:

[1856] When a user visits a physical store, the store's robot or smart display will recognize the user and begin a casual conversation.

[1857] Input: User visit information, user profile

[1858] Output: Generated conversation

[1859] How it works: The robot or smart display detects a user's visit and queries the server for the user's profile. The server then uses the generative AI model to generate appropriate conversation content and sends it to the robot or smart display.

[1860] Step 3:

[1861] The server analyzes conversation patterns and short-term changes in memory to detect early symptoms of dementia through conversations with the user.

[1862] Input: User conversation

[1863] Output: Detection results of early symptoms of dementia

[1864] How it works: The server uses a generative AI model to analyze the conversation and detect specific keywords and phrases, which can then determine whether early symptoms of dementia are present.

[1865] Step 4:

[1866] If early symptoms of dementia are detected, the server will send an alert to family members and medical institutions.

[1867] Input: Detection results of early symptoms of dementia

[1868] Output: Alert notification

[1869] Specific operation: The server generates an alert based on the detection results and sends a notification to pre-registered family and medical contacts.

[1870] Step 5:

[1871] Robots or smart displays in physical stores suggest appropriate products and services based on users' preferences and interests.

[1872] Input: User profile, generated conversation

[1873] Output: Product or service suggestions

[1874] Specific operation: The robot or smart display suggests products and services suitable for the user based on the user profile and conversation content received from the server.

[1875] Through these steps, this system will enable elderly people to enjoy shopping in physical stores with peace of mind, reduce feelings of loneliness, and enable early detection of early symptoms of dementia.

[1876] Example 2

[1877] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1878] While existing AI companion systems can be customized to suit a user's preferences and interests, they are inadequate in adjusting the content of conversations to suit the user's daily life situation and health condition. Furthermore, they lack the ability to assess a user's health condition in real time and provide appropriate advice, limiting their support for health management. Furthermore, they lack the technical means to accurately analyze user input and generate appropriate conversation content. To address these issues, a system capable of more advanced analysis and response generation is needed.

[1879] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1880] In this invention, the server includes means for receiving user input and converting the voice into text using voice recognition software, means for analyzing the text data using a generative AI model to determine the user's health condition, and means for generating conversation content according to the user's situation using the generative AI model, thereby making it possible to provide appropriate conversation content according to the user's daily life situation and health condition.

[1881] "AI Buddy" is an interactive AI system that provides various information through dialogue with the user and can be customized to suit the user's preferences and interests.

[1882] "Speech recognition software" means software that converts a user's voice input into text data.

[1883] A "generative AI model" is an artificial intelligence model that analyzes input text data and generates appropriate responses and conversation content.

[1884] "Text data" refers to a user's voice input converted by voice recognition software or text information entered directly by a user.

[1885] "Health Status" refers to the state of a User's physical and mental health, including specific symptoms such as colds or fatigue.

[1886] "Conversation content" refers to the information and advice provided by the AI ​​partner during the dialogue with the user.

[1887] "Analysis" is the process by which the generative AI model determines the user's situation and health status based on the input text data.

[1888] "Response generation" is the process by which a generative AI model creates appropriate conversation content based on the analysis results.

[1889] "Speech synthesis software" is software that converts generated text data into speech and responds to the user in speech.

[1890] The present invention relates to an AI companion system that adjusts the content of conversations depending on the user's daily life situation and health condition. Specific embodiments of this system will be described below.

[1891] Hardware and software used

[1892] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1893] Software: Generative AI models (e.g., GPT-4), speech recognition software (e.g., Google Speech-to-Text API), speech synthesis software (e.g., Google Text-to-Speech API)

[1894] System Overview

[1895] The system receives user input, converts speech to text using speech recognition software, analyzes the text data using a generative AI model to determine the user's health status, and then uses the generative AI model to generate conversational content tailored to the user's situation, returning the generated conversational content to the user in voice or text.

[1896] Specific examples

[1897] Consider the case where a user speaks into a smartphone microphone, "I'm feeling a bit under the weather today." This voice input is converted into text data using speech recognition software. Next, the server inputs this text data into a generative AI model, which recognizes the keyword "feeling under the weather" and determines that the user has a cold. The server then inputs a prompt sentence into the generative AI model: "If the user has a cold, ask about their cold symptoms and advise them to get adequate rest." The model then generates conversational content such as "How are your cold symptoms?" and "Please get plenty of rest today." Finally, the generated text data is converted into speech using speech synthesis software, and a voice response is provided to the user, or the same content is displayed in text form.

[1898] Prompt Sentence Examples

[1899] "If the user has a cold, the AI ​​companion should generate conversational content that asks about cold symptoms and advises them to get adequate rest."

[1900] In this way, the AI ​​companion can provide appropriate conversation content based on the user's health condition.

[1901] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1902] Step 1:

[1903] Users use their smartphone or PC to talk to their AI companion via voice or text.

[1904] Input: User voice or text input

[1905] What happens: The user speaks into the smartphone microphone and says, "I'm feeling a bit under the weather today," or types the same thing using text input.

[1906] Output: User's voice or text data

[1907] Step 2:

[1908] The server uses speech recognition software to convert the user's speech into text.

[1909] Input: User's voice data

[1910] What happens: The server receives the voice data and uses speech recognition software (e.g., Google Speech-to-Text API) to convert it into text data such as "I'm feeling a bit under the weather today."

[1911] Output: Text data

[1912] Step 3:

[1913] The server uses the generated AI model to analyze the text data and determine the user's health status.

[1914] Input: Text data

[1915] Specific operation: The server inputs text data into the generative AI model, and the model recognizes the keyword "feeling like I have a cold" and determines that the user has a cold.

[1916] Output: Information about the user's health status (e.g., they have a cold)

[1917] Step 4:

[1918] The server uses a generative AI model to generate conversation content that is appropriate for the user's situation.

[1919] Input: User's health information

[1920] Specific operation: The server inputs the prompt sentence "If the user has a cold, ask about their cold symptoms and advise them to get adequate rest" into the generative AI model, and the model generates conversational content such as "What are your cold symptoms?" and "Please get plenty of rest today."

[1921] Output: Generated conversation content (text data)

[1922] Step 5:

[1923] The device responds to the user with the generated conversation content in voice or text.

[1924] Input: Generated conversation content (text data)

[1925] Specific operation: The server sends the generated text data to the device, which then uses speech synthesis software (e.g., Google Text-to-Speech API) to convert it into speech, asking the user, "How are your cold symptoms?" or "Please get plenty of rest today," or displaying the same information in text.

[1926] Output: A voice or text response to the user

[1927] (Application example 2)

[1928] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1929] In modern society, there is a demand for providing appropriate advice tailored to users' daily lives and health conditions. However, conventional systems have difficulty adjusting the content of conversations based on the user's health condition and daily life situation, and there is a lack of means to effectively utilize generative AI models to provide appropriate advice. Therefore, an effective system is needed to improve users' health management and quality of life.

[1930] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1931] In this invention, the server includes a means for including an AI companion that can be customized to suit the user's preferences and interests, a means for adjusting the content of the conversation depending on the user's daily life situation and health condition, and a means for providing appropriate advice using a generative AI model based on the user's health condition, thereby enabling the provision of appropriate advice according to the user's health condition and daily life situation, thereby improving the user's health management and quality of life.

[1932] "User" refers to an individual who uses the System.

[1933] "Preferences" refer to the things that a user particularly likes or is interested in.

[1934] "Interests" refers to the things or areas that a user is interested in.

[1935] "Customizable" means that settings and features can be changed according to the user's preferences and interests.

[1936] "AI Partner" refers to a system that uses artificial intelligence to interact with users and provide support.

[1937] "Daily life" refers to the activities and lifestyle habits of the user on a daily basis.

[1938] "Health Status" refers to the state of your physical and mental health.

[1939] "Conversation content" refers to the content of the dialogue that the AI ​​partner has with the user.

[1940] A "generative AI model" refers to a model that uses artificial intelligence to generate text and data.

[1941] "Advice" means any advice or suggestion provided to a User.

[1942] "Dementia" refers to a disease characterized by a decline in memory and cognitive function.

[1943] "Early symptoms" refer to symptoms seen in the early stages of the onset of a disease.

[1944] A "pattern" refers to a series of actions or phenomena that have a particular regularity or tendency.

[1945] "Short-term changes in memory" refers to fluctuations or changes in memory over a short period of time.

[1946] An "alert" is a notification or warning intended to draw attention.

[1947] "Family" refers to people related to you by blood or marriage.

[1948] "Medical institution" refers to a facility or organization that provides medical services.

[1949] "Emotional state" refers to the user's emotional or mood state.

[1950] A system for implementing this invention includes an AI companion that can be customized to a user's preferences and interests, adapts conversations based on the user's daily life situation and health status, and provides appropriate advice using a generative AI model based on the user's health status.

[1951] System Configuration

[1952] The system uses the following hardware and software:

[1953] Hardware: Smartphones, servers

[1954] Software: Python, OpenAI API

[1955] Program processing explanation

[1956] The server provides an interface for acquiring the user's health status. The user inputs their current health status using their smartphone. For example, they can input their current status such as "cold," "headache," or "feeling well."

[1957] The server generates a prompt based on the user's health status. This prompt is then input into a generative AI model to generate appropriate advice. The generative AI model used is OpenAI's "text-davinci-003."

[1958] The generated advice is sent from the server to the user's smartphone and displayed, allowing the user to receive appropriate advice based on their own health condition.

[1959] Specific examples

[1960] For example, if the user types "cold," the generated prompt text will be:

[1961] If the user has a cold, provide appropriate advice.

[1962] If you feed this prompt into a generative AI model, it might generate responses like these:

[1963] If you have a cold, it's important to get plenty of rest, drink plenty of fluids, eat foods with vitamin C, and keep warm. Consult your doctor if necessary.

[1964] In this way, appropriate advice can be provided according to the user's health condition.

[1965] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1966] Step 1:

[1967] The user inputs their current health status using their smartphone.

[1968] Input: User's health condition (e.g., "cold," "headache," "feeling well")

[1969] Output: Health status data

[1970] Specific actions: The user opens the application on their smartphone, enters their current health status in the text box, and presses the send button.

[1971] Step 2:

[1972] A server receives health status data sent by a user.

[1973] Input: Health status data

[1974] Output: Acknowledgement message

[1975] Specific operation: Health status data sent from the smartphone reaches the server, which receives and stores the data.

[1976] Step 3:

[1977] The server generates a prompt sentence based on the health status data.

[1978] Input: Health status data

[1979] Output: prompt statement

[1980] Specific operation: The server analyzes the received health status data and generates a prompt sentence, for example, "If the user has a cold, please provide appropriate advice."

[1981] Step 4:

[1982] The prompt sentence generated by the server is input into the generative AI model.

[1983] Input: prompt statement

[1984] Output: Input data to a generative AI model

[1985] Specific operation: The server sends the generated prompt text to OpenAI's API and inputs it into the generative AI model (e.g., text-davinci-003).

[1986] Step 5:

[1987] A generative AI model generates appropriate advice based on the prompt.

[1988] Input: prompt statement

[1989] Output: Advice text

[1990] Specific operation: The generative AI model analyzes the prompt sentence and generates advice, for example, "If you have a cold, it is important to get plenty of rest and drink plenty of fluids."

[1991] Step 6:

[1992] The server receives the generated advice text and transmits it to the smartphone.

[1993] Input: Advice text

[1994] Output: Send confirmation message

[1995] Specific operation: The server sends the advice text received from the generative AI model to the smartphone and confirms that the transmission was successful.

[1996] Step 7:

[1997] The user receives and displays the advice text on the smartphone.

[1998] Input: Advice text

[1999] Output: Advice displayed

[2000] Specific operation: The smartphone displays the advice text received from the server on the screen, and the user confirms it.

[2001] Example 3

[2002] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2003] Conventional AI companion systems have had difficulty accurately estimating a user's emotional state and providing conversation content appropriate to that state. Furthermore, they lacked the ability to adjust conversation content based on the user's health condition and daily life situation, making them ineffective in providing psychological support and health management. Furthermore, their conversation analysis capabilities to detect early symptoms of dementia and their ability to send alerts to family members and medical institutions were limited. To address these issues, an AI companion system with more advanced emotion analysis and conversation generation capabilities is needed.

[2004] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2005] In this invention, the server includes means for receiving input data sent from a user, means for analyzing the received input data and estimating an emotional state, and means for sending a prompt sentence to the generative AI model based on the estimated emotional state, thereby generating appropriate conversation content according to the user's emotional state and enabling effective psychological support and health management for the user.

[2006] "Artificial Intelligence Partner" is an artificial intelligence system that provides various functions through dialogue with the user.

[2007] "Input data" refers to information such as text messages and voice data that a user sends through a terminal.

[2008] "Emotional state" refers to the psychological state of the user, and includes emotions such as joy, sadness, anger, and fatigue.

[2009] A "generative artificial intelligence model" is an artificial intelligence algorithm for generating appropriate conversation content based on a given prompt sentence.

[2010] A "prompt" is an instruction sent to a generative AI model based on an emotional state, and serves as the basis for generating appropriate conversation content.

[2011] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[2012] "Sentiment analysis" is the process of analyzing input data to infer a user's emotional state.

[2013] An "alert" is a warning message that notifies family members or medical institutions if any worrisome symptoms are observed.

[2014] "Conversation content" refers to the content of the dialogue that the AI ​​partner provides to the user.

[2015] The present invention relates to an AI companion system that estimates the emotional state of a user and adjusts the content of a conversation in accordance with that state. Specific embodiments of this system will be described below.

[2016] First, a user inputs text messages or voice data using a terminal. For example, consider the case where a user inputs a text message such as "I'm very tired today." This input data is sent from the terminal to a server.

[2017] The server receives input data sent by the user. The received data is temporarily stored in a database on the server. The server then analyzes the input data using emotion analysis software, such as Google Cloud Natural Language API or IBM Watson Tone Analyzer. This analysis estimates the user's emotional state (e.g., joy, sadness, anger, fatigue, etc.).

[2018] Based on the estimated emotional state, the server sends a prompt to the generative AI model. OpenAI's GPT-4 and other models are used as generative AI models. The prompt includes the user's emotional state and an appropriate response accordingly. For example, a prompt such as "The user feels tired. Please offer some words of encouragement" is sent to the generative AI model.

[2019] The generative AI model generates appropriate conversation content based on this prompt. For example, it generates a response such as, "Today was really tough. Let's take a break and refresh yourself!" The generated conversation content is sent from the server to the user's device. The user can receive the response from their AI partner through their device.

[2020] For example, if a user inputs "I feel very tired today," the server receives this message and uses the Google Cloud Natural Language API to estimate the user's emotional state, such as "fatigue" or "depression." The server then sends a prompt to the generative AI model: "The user feels tired. Please offer some words of encouragement." The generative AI model generates a response: "Today was really hard. Take a break and refresh yourself!" The server then sends this response to the user's device. The user can view this message on their device screen and receive words of encouragement.

[2021] In this way, by providing appropriate conversation content according to the emotional state of the user, it becomes possible to effectively provide psychological support and health management for the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[2022] Step 1:

[2023] The user sends input data through the terminal.

[2024] A user inputs text messages or voice data using a device such as a smartphone or PC. For example, the user inputs a text message such as "I'm very tired today" and presses the send button. This input data is sent from the device to the server.

[2025] Step 2:

[2026] The server receives input data from the user.

[2027] The server receives text messages and voice data sent by the user from the device. The received data is temporarily stored in a database on the server. Input data includes the user's text message, "I'm very tired today."

[2028] Step 3:

[2029] The server uses emotion analysis software to analyze the input data and estimate the emotional state.

[2030] The server passes the received input data to emotion analysis software (e.g., Google Cloud Natural Language API or IBM Watson Tone Analyzer). The emotion analysis software analyzes the text message and estimates the user's emotional state (e.g., fatigue or depression). The analysis results in an emotional state such as "fatigue" or "depression" being output.

[2031] Step 4:

[2032] The server sends a prompt to the generative AI model.

[2033] The server sends a prompt to a generative AI model (e.g., OpenAI's GPT-4) based on the estimated emotional state. The prompt includes the user's emotional state and an appropriate response accordingly. For example, the server generates a prompt such as "The user feels tired. Please offer some words of encouragement." and sends it to the generative AI model.

[2034] Step 5:

[2035] The generative AI model generates appropriate conversation content based on the prompt text.

[2036] The generative AI model generates appropriate conversational content based on the received prompt. For example, it generates a response such as, "Today was really tough. Let's take a break and refresh yourself!" The generated conversational content is output.

[2037] Step 6:

[2038] The server transmits the generated conversation content to the user's terminal.

[2039] The server sends the conversational content received from the generative AI model to the user's device. The data is displayed on the user's device. The output includes a message such as, "Today was really hard. Take a break and refresh yourself!"

[2040] Step 7:

[2041] The user receives a response from the AI ​​partner through the terminal.

[2042] The user receives a response from the AI ​​buddy via the device. For example, the user can see a message on their smartphone screen saying, "Today was really tough. Take a break and refresh yourself!" and receive words of encouragement.

[2043] (Application example 3)

[2044] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2045] Conventional food delivery systems make suggestions without considering the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Furthermore, they are unable to provide appropriate meal suggestions when the user is feeling stressed or depressed, and lack functionality to support the user's health and mental state.

[2046] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for including an AI companion that can be customized to suit the user's preferences and interests, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, etc., means for sending an alert to family members or medical institutions if any worrisome symptoms are observed, and means for estimating the user's emotional state and making food delivery suggestions based on that emotional state. This enables personalized food delivery suggestions based on the user's emotional state, increasing the user's psychological satisfaction and supporting their health and mental state.

[2047] "User" refers to an individual who uses the System.

[2048] "Preferences and interests" refers to the user's particular interests and preferences.

[2049] "Customizable" means that settings and features can be modified to suit a user's specific needs and preferences.

[2050] "AI Partner" refers to a system that uses artificial intelligence to interact with users and provide various types of support.

[2051] "A means of making everyday conversations enjoyable" refers to a function that provides enjoyment through everyday interactions with users.

[2052] "Early symptoms of dementia" refers to early signs of cognitive decline.

[2053] "Conversational patterns" refer to the tendency to use words and progress in conversations with users.

[2054] "Short-term changes in memory" refers to how a user's memory changes over a short period of time.

[2055] "Means of sending alerts" refers to the ability to send notifications when certain conditions occur.

[2056] "Emotional state" refers to a user's current psychological state or mood.

[2057] "Food delivery suggestion" means suggesting food and drink delivery services to a User.

[2058] A system for carrying out this invention includes an AI companion that can be customized to suit a user's preferences and interests. The AI ​​companion includes means for engaging in daily conversations with the user, means for detecting early symptoms of dementia through dialogue with the user, means for analyzing conversation patterns and short-term changes in memory, means for sending alerts to family members or medical institutions if worrisome symptoms are observed, and means for estimating the user's emotional state and suggesting food delivery options based on that emotional state.

[2059] System Program

[2060] This system includes the following programs:

[2061] 1. Loading the emotion estimation model: The server uses OpenAI's emotion analysis model to estimate the emotional state from the user's utterances.

[2062] 2. Emotional state estimation: The server receives the user's input and estimates the emotional state.

[2063] 3. Food Suggestion: The server makes appropriate food suggestions based on the estimated emotional state.

[2064] Hardware and software used

[2065] Hardware: A device, such as a smartphone or tablet, that receives user input.

[2066] Software: Uses OpenAI's sentiment analysis model for sentiment estimation and a customized algorithm for food recommendations.

[2067] Data processing and calculation

[2068] Data processing: User comments are obtained as text data and input into the sentiment analysis model.

[2069] Data computation: The sentiment analysis model estimates the emotional state of the user from their statements, and the food suggestion algorithm generates appropriate suggestions based on the results.

[2070] Specific examples

[2071] For example, if a user types, "I'm very tired today," the emotion estimation model will infer that they are "feeling stressed." Based on this result, the system will suggest relaxing herbal tea and salad.

[2072] Example prompt sentence:

[2073] User says: I'm so tired today

[2074] Estimate your emotional state.

[2075] In this way, personalized food delivery suggestions based on the user's emotional state become possible, increasing the user's psychological satisfaction and supporting their health and mental state.

[2076] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[2077] Step 1:

[2078] The user inputs a message using a terminal, which is then saved as text data on the terminal.

[2079] Step 2:

[2080] The terminal transmits the input text data to the server, which receives the text data and prepares it for input into the emotion estimation model.

[2081] Step 3:

[2082] The server uses OpenAI's sentiment analysis model to estimate the user's emotional state from the received text data. Specifically, the server inputs the text data into the sentiment analysis model and obtains the emotional state (e.g., stress, depression, joy, etc.) as the output.

[2083] Step 4:

[2084] The server runs a food recommendation algorithm based on the estimated emotional state, generating appropriate food delivery recommendations depending on the emotional state. For example, if the emotional state is "stressed," it will suggest relaxing herbal tea and salad.

[2085] Step 5:

[2086] The server sends the generated food suggestions to the terminal, which displays the suggestions to the user, who can review the suggested food delivery and place an order if necessary.

[2087] Step 6:

[2088] If the user decides to order the suggested food delivery, the terminal sends the order information to the server, which receives the order information and forwards the order to the food delivery service.

[2089] Step 7:

[2090] The food delivery service receives the order and delivers the meal to the user. After the delivery is complete, the server sends a notification to the user that the delivery is complete.

[2091] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2092] "Example 1"

[2093] In one embodiment of the present invention, an AI companion is provided that includes an emotion engine that recognizes a user's emotions. The emotion engine infers the user's emotional state from the user's tone of voice, speaking style, and words used. For example, when a user is angry, their voice tone often becomes higher and their speech rate increases. Based on this information, the emotion engine infers that the user is angry.

[2094] "Example 2"

[2095] Furthermore, AI Buddy adjusts the content of the conversation depending on the user's emotional state estimated by the emotion engine. For example, if the user is estimated to be angry, AI Buddy will suggest topics to calm the user. If the user is estimated to be happy, AI Buddy will suggest topics to share that joy.

[2096] "Example 3"

[2097] As described above, the present invention makes it possible to accurately grasp the user's emotional state and provide appropriate dialogue accordingly. This allows the user to enjoy more natural dialogue and reduce feelings of loneliness. Furthermore, early symptoms of dementia can be detected early through changes in the user's emotional state.

[2098] The processing flow of each embodiment will be described below.

[2099] "Example 1"

[2100] Step 1: Through conversations with the user, the AI ​​companion collects information about the user's tone of voice, speaking style, and words used.

[2101] Step 2: Send the collected information to the emotion engine.

[2102] Step 3: The emotion engine estimates the user's emotional state based on the transmitted information.

[2103] "Example 2"

[2104] Step 1: The AI ​​partner adjusts the content of the conversation based on the estimated emotional state received from the emotion engine.

[2105] Step 2: For example, if the user is presumed to be angry, the AI ​​companion will provide a topic to calm the user.

[2106] Step 3: If the AI ​​companion determines that the user is happy, it will provide topics that share that joy.

[2107] "Example 3"

[2108] Step 1: The AI ​​partner understands the user's emotional state through dialogue with them. Step 2: The AI ​​partner detects early symptoms of dementia through changes in the user's emotional state.

[2109] Step 3: If early symptoms of dementia are detected, the AI ​​partner will send an alert to family members or medical institutions.

[2110] Example 1

[2111] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2112] For elderly people living alone, reducing feelings of loneliness and early detection of early symptoms of dementia are important issues. Conventional systems are unable to fully analyze the user's emotional state and conversation patterns, making it difficult to provide appropriate feedback or warnings. Furthermore, there is a need for a system that can effectively collect and analyze the user's voice data and adjust the content of conversations according to their daily life situation and health condition.

[2113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2114] In this invention, the server includes means for collecting the user's voice and transmitting the voice data to the server, means for converting the voice data into text, and means for analyzing the text data with an emotion recognition engine, which enables detailed analysis of the user's emotional state and speech patterns and provides appropriate feedback and warnings.

[2115] The "Artificial Intelligence Partner" is a system that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[2116] "A means of making everyday conversations enjoyable" is a function that provides conversations that are tailored to the user's interests and concerns through natural dialogue with the user.

[2117] The "means for detecting early symptoms of dementia" is a function that detects early symptoms of dementia by analyzing conversation patterns and short-term changes in memory through dialogue with the user.

[2118] "Means for analyzing conversation patterns and short-term changes in memory" refers to a function that analyzes changes in conversation content and memory by comparing them with the user's past conversation data.

[2119] "Means for sending warnings to family members and medical institutions" is a function that sends warnings to family members and medical institutions when early symptoms of dementia are detected.

[2120] "Means for collecting voice and transmitting voice data to a server" refers to a function for collecting the user's voice with a microphone and transmitting it to a server via the Internet.

[2121] "Means for converting voice data into text" refers to a function that converts collected voice data into text data using a natural language processing engine.

[2122] "Means for analyzing text data using an emotion recognition engine" refers to a function that inputs text data into an emotion recognition engine and estimates the user's emotional state.

[2123] "Means for analyzing conversation patterns and detecting early symptoms of dementia" is a function that analyzes a user's conversation patterns based on text data and detects early symptoms of dementia.

[2124] The "means for providing feedback to the user" is a function that provides appropriate feedback to the user by voice based on the analysis results.

[2125] The present invention is a system that provides an AI companion to assist elderly people living alone. The system includes an AI companion that can be customized to suit the user's preferences and interests, and has the ability to reduce the user's sense of loneliness through daily conversations and detect early symptoms of dementia.

[2126] Hardware and software used

[2127] Hardware: Devices with microphones, speakers, and internet connectivity (e.g. smartphones, tablets, smart speakers)

[2128] Software: Natural language processing engines (e.g., Google Cloud Natural Language API), emotion recognition engines (e.g., IBM Watson Tone Analyzer), data analysis tools (e.g., Python's Pandas library)

[2129] Data processing and calculation

[2130] 1. Collecting user conversations

[2131] The terminal collects the user's voice through a microphone.

[2132] The collected voice data is sent to the server.

[2133] 2. Analysis of audio data

[2134] The server converts the voice data into text using a natural language processing engine.

[2135] The converted text data is input into an emotion recognition engine to estimate the user's emotional state.

[2136] 3. Analysis of conversation patterns

[2137] The server stores the text data in a database and compares it with past conversation data.

[2138] Data analysis tools are used to analyze short-term changes in conversation patterns and memory.

[2139] 4. Detecting early symptoms of dementia

[2140] Based on the analysis results, the server determines whether early symptoms of dementia are present.

[2141] If any worrisome symptoms are detected, an alert will be sent to family members or medical institutions.

[2142] 5. User Feedback

[2143] The device provides feedback to the user through the speaker.

[2144] For example, if the AI ​​partner estimates that the user is angry, it will make suggestions such as, "Maybe you should try to calm down a bit?"

[2145] Specific examples

[2146] User conversation: "I went for a walk today, but I got lost."

[2147] System Action:

[2148] The device collects the audio and sends it to the server.

[2149] The server converts the speech into text and analyzes it using an emotion recognition engine.

[2150] The emotion recognition engine infers that the user is feeling anxious.

[2151] The server compares the data with past conversation data and analyzes the possibility of early symptoms of dementia.

[2152] If necessary, alert family members or medical institutions.

[2153] The device will provide feedback to the user saying, "Are you OK? Let's check the map together next time."

[2154] Prompt Sentence Examples

[2155] "Analyze the emotions users express when talking about recent events."

[2156] "It analyzes the user's speech patterns and determines whether they are exhibiting early signs of dementia."

[2157] The above is an embodiment of the present invention.

[2158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2159] Step 1:

[2160] The user provides voice input. 【2...

Claims

1. A system including an AI companion that can be customized to suit a user's preferences and interests, A means for inferring the user's emotional state from the tone of voice, speaking style, words used, etc., using an emotion engine; a means for generating a prompt sentence for instructing the generation of appropriate conversation content according to the user's current emotional state based on the estimated emotional state of the user; A means for generating a conversation content with a user using the generated prompt sentence and the AI ​​companion; A means for responding to the generated conversation content by voice or text to the user; By analyzing conversation patterns and short-term changes in memory, the company hopes to discover early signs of dementia through dialogue with users, A means to send alerts to family members or medical institutions if the user shows symptoms that may be dementia, A system including:

2. The system of claim 1 , wherein the AI ​​buddy further comprises means for adjusting the content of the conversation depending on the user's daily living situation and health condition.

3. The system of claim 1 , wherein the AI ​​buddy includes means for adjusting conversational content in response to the emotional state estimated by the emotion engine.

Images