System

Abstract

To increase conversation opportunities for the elderly, enable early detection of dementia, and provide support for their lives.SOLUTION: A system includes: means for imitating a conversation with a human using natural language processing technology; means for generating a conversation including questions for determining the progression of dementia; means for recognizing a user's voice and converting it into text data; means for analyzing the user's emotional state using an emotion engine; means for generating an appropriate reply using a generative AI model based on a prompt sentence using reply contents of the generated conversation and the analyzed emotional state; means for analyzing the conversation and determining the progression situations of dementia; and means for communicating results of the determination to the user's relatives or a medical institution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Due to the increase in the elderly population and the trend toward nuclear families, an increasing number of elderly people are living alone. As a result, opportunities for conversation decrease, raising the risk of dementia. In addition, existing interactive robots have a mechanical and unnatural way of speaking, which is different from the feeling of talking to a human, resulting in a lack of continued use. [Means for solving the problem]

[0005] This invention uses natural language processing technology to mimic human conversations and generate conversations that include questions to assess the progression of dementia. The replies in the generated conversation are analyzed to assess the progression of dementia. The assessment results are shared with relatives, medical institutions, etc. Furthermore, it supports users in using services such as e-commerce sites and arranging taxis, understands the user's lifestyle based on the content of the conversation and behavioral patterns, and provides appropriate support as needed. This increases conversation opportunities for the elderly, realizes early detection of dementia, and supports daily life. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] As an example embodiment of the present invention, an interactive robot is provided. This robot uses natural language processing technology to imitate conversations with humans. Specifically, a natural language generation model such as a generative AI (generative pretrained transformer) is used to generate natural conversations with humans. The robot also generates conversations that include questions for determining the progression of dementia. For example, the robot asks questions such as "What day is it today?" and "What time is it now?" and analyzes the replies. This analysis determines the progression of dementia. The results of the determination are shared with relatives, medical institutions, etc.

[0029] "Example 2"

[0030] Furthermore, as an example of the present invention, a function is provided to assist users in using services such as e-commerce sites and arranging taxis. Specifically, the robot purchases products on e-commerce sites and arranges taxis based on the user's instructions. This makes it possible to support the daily lives of elderly people.

[0031] "Example 3"

[0032] In addition, as an example of the present invention, a function is provided that grasps the user's living situation from the content of the user's conversations and behavioral patterns, etc., and provides appropriate support as needed. Specifically, the function grasps the user's dietary intake and changes in physical condition from the content of the user's conversations, and connects with relatives, medical institutions, etc. as needed. This enables health management for the elderly.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The conversational robot starts and starts a conversation with the user.

[0036] Step 2: The robot uses natural language processing technology to understand what the user is saying and generate an appropriate response.

[0037] Step 3: The robot generates a conversation containing questions to determine the progression of dementia and asks the user questions.

[0038] Step 4: The robot receives the user's response and analyzes it.

[0039] Step 5: Based on the analysis results, the progression of dementia is determined and the results are shared with relatives, medical institutions, etc.

[0040] "Example 2"

[0041] Step 1: The user instructs the robot to use a service such as purchasing a product on an e-commerce site or arranging a taxi.

[0042] Step 2: The robot understands the user's instructions and follows them to make purchases on an e-commerce site or order a taxi.

[0043] "Example 3"

[0044] Step 1: The robot analyzes the user's conversation content, behavioral patterns, etc. to understand their living situation.

[0045] Step 2: Based on the analysis results, the robot will provide appropriate support as needed. For example, it will understand the patient's dietary intake and changes in physical condition, and contact relatives or medical institutions as needed.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] In modern society, early detection of dementia and understanding its progression are important issues. However, conventional methods require diagnosis at specialized medical institutions, and there is a lack of easy ways to monitor the progression of dementia in daily life. Furthermore, systems for understanding users' living conditions and providing appropriate support are not well developed. This creates the risk of overlooking the progression of dementia and the problem of not being able to provide appropriate support.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes means for generating a program for an interactive robot using a generative artificial intelligence model, means for executing the generated program and starting a conversation with a user, means for generating questions for determining the progression of dementia, means for asking the generated questions to the user, means for receiving the user's responses and analyzing them using natural language processing technology, and means for transmitting the analysis results to the server and communicating with relatives and medical institutions. This makes it possible to easily monitor the progression of dementia in everyday life and provide appropriate support.

[0051] A "generative artificial intelligence model" is an artificial intelligence technology that is trained in advance with large amounts of data and has the ability to generate natural language.

[0052] An "interactive robot" is a robot that uses natural language processing technology to imitate conversations with humans, providing information and asking questions through dialogue.

[0053] The "means for generating a program" is a method for using a generative artificial intelligence model to create scripts and codes for execution by an interactive robot.

[0054] The "means for executing the program" refers to a method for loading the generated program into the interactive robot and for actually running it.

[0055] The "means for generating questions" is a method for using a generative artificial intelligence model to create questions necessary to determine the progression of dementia.

[0056] The "means of asking a question to a user" refers to a method by which the interactive robot presents the generated question to the user by voice or text.

[0057] The "means for analyzing responses" is a method for analyzing responses from users using natural language processing technology to understand their meaning and intent.

[0058] The "means for transmitting the analysis results" is a method for transmitting the analysis results to a server and further communicating them to relatives or medical institutions.

[0059] "Means of linking with relatives and medical institutions" refers to methods for communicating information about the progression of dementia to appropriate parties.

[0060] "Means to support the use of e-commerce sites" refers to methods that provide users with the support they need when shopping online.

[0061] "Transportation arrangement support" refers to a method that provides the user with the support they need to arrange a taxi or other means of transportation.

[0062] "Means for understanding the living situation" refers to a method for analyzing the content of a user's conversations and behavioral patterns to understand the situation of daily life.

[0063] "Means for providing appropriate support" refers to methods for providing necessary assistance and services according to the user's living situation.

[0064] This invention is a system that uses a generative artificial intelligence model to generate a program for an interactive robot, determines the progression of dementia through natural conversation with the user, and communicates the results to relatives and medical institutions. Specific embodiments of this system are described below.

[0065] Program Generation

[0066] Subject: Server

[0067] The server generates a program for the conversational robot using a generative artificial intelligence model (e.g., GPT-3 (registered trademark)). The server uses a natural language processing library (e.g., spaCy or NLTK) to design the program so that the robot can have a natural conversation. Specifically, the server inputs the following prompt sentence into the generative artificial intelligence model:

[0068] Example prompt sentence:

[0069] "Generate a scenario in which the robot asks the user questions to determine the progression of dementia. Example questions include, 'What day is it today?' and 'What time is it now?'"

[0070] Running the program

[0071] Subject: Robot

[0072] The robot receives and executes the program provided by the server. As an initial setting, the robot prepares to start a conversation with the user. Specifically, the robot greets the user with a greeting such as "Hello, how are you doing today?" and creates a flow of conversation.

[0073] Question Generation

[0074] Subject: Robot

[0075] The robot uses a generative artificial intelligence model to generate questions to assess the progression of dementia, such as "What day is it today?" or "What time is it now?" This allows it to collect information to assess the user's cognitive function.

[0076] User interaction

[0077] Subject: Robot

[0078] The robot then asks the user the generated question. For example, the robot asks, "What day is it today?" and the user replies, "Today is Monday." In this way, the robot collects the necessary information through dialogue with the user.

[0079] Analysis of responses

[0080] Subject: Robot

[0081] The robot receives the user's responses and analyzes them using natural language processing technology. Specifically, it analyzes the content of the responses and calculates indicators to determine the progression of dementia. For example, it evaluates whether the user's responses are accurate and how long it takes to respond.

[0082] Sending the results

[0083] Subject: Robot

[0084] The robot sends the analysis results to a server, which then shares the results with relatives and medical institutions. For example, the server may notify relatives that the user's dementia is at a mild stage. This allows the relevant parties to take appropriate action.

[0085] Specific examples

[0086] Hardware and software used

[0087] Hardware: Interactive robot, server

[0088] Software: Generative AI models (GPT-3, etc.), natural language processing libraries (spaCy, NLTK)

[0089] This system makes it possible to easily monitor the progression of dementia in everyday life and provide appropriate support.

[0090] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0091] Step 1: Generate the program

[0092] The server generates a program for the interactive robot using a generative artificial intelligence model (such as GPT-3). The server inputs a prompt statement into the generative artificial intelligence model and generates a script for the interactive robot to execute. Specifically, the server inputs the prompt statement, "Please generate a scenario in which the robot asks the user questions to determine the progression of dementia." The input is the prompt statement, and the output is the script for the interactive robot.

[0093] Step 2: Run the program

[0094] The robot receives and executes a program provided by the server. As an initial setting, the robot prepares to start a conversation with the user. Specifically, the robot greets the user with a greeting such as "Hello, how are you doing today?" and creates a flow of conversation. The input is the program provided by the server, and the output is the initial conversation with the user.

[0095] Step 3: Generate questions

[0096] The robot uses a generative AI model to generate questions to determine the progression of dementia. For example, it generates questions such as "What day is it today?" or "What time is it now?" The input is the generative AI model and user information, and the output is the generated question.

[0097] Step 4: User interaction

[0098] The robot asks the user the generated question. For example, the robot asks "What day is it today?" and the user replies "Today is Monday." The input is the generated question, and the output is the user's response.

[0099] Step 5: Analyze the responses

[0100] The robot receives the user's response and analyzes it using natural language processing technology. Specifically, it analyzes the content of the response and calculates indicators to determine the progression of dementia. For example, it evaluates whether the user's response is accurate and the time it takes to respond. The input is the user's response, and the output is the analysis result.

[0101] Step 6: Sending the results

[0102] The robot sends the analysis results to a server, which then shares the results with relatives and medical institutions. For example, the server may notify relatives that the user's dementia is at a mild stage. The input is the analysis results, and the output is a notification to relatives and medical institutions.

[0103] In this way, the interactive robot can assess the progression of dementia through natural conversation and provide the necessary information to those involved.

[0104] (Application example 1)

[0105] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] There is a need to detect the progression of dementia in the elderly early and take appropriate measures. However, it is difficult to continuously monitor the progression of dementia in the elderly in their daily lives, and there is a lack of means to respond quickly when abnormalities occur. Therefore, an effective system is needed to ensure the safety of the elderly and to detect the progression of dementia early.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0108] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for sending a notification when an abnormality is detected, and means for being installed on a smartphone. This makes it possible to continuously monitor the progression of dementia in elderly people and to respond quickly when an abnormality occurs.

[0109] "Natural language processing technology" is a technology that enables computers to understand, generate, and respond to human language.

[0110] "Questions to determine the progression of dementia" are questions asked to elderly people to assess the progression of dementia.

[0111] The "means for analyzing the content of replies to the generated conversation" is a means for analyzing the content of replies to the generated conversation and determining the progression of dementia.

[0112] "Means of communicating the results of the assessment to relatives, medical institutions, etc." refers to means of notifying relatives and medical institutions of the results of the assessment of the progression of dementia.

[0113] The "means for sending a notification when an abnormality is detected" is a means for sending a notification to relatives or medical institutions when an abnormality is detected in the progression of dementia.

[0114] "Means for being installed on a smartphone" refers to means for being installed as an application on a smartphone.

[0115] "Means for understanding a user's living situation from the content of their conversations and patterns of behavior, etc." refers to means for analyzing the content of a user's conversations and patterns of behavior to understand their living situation.

[0116] A "security conversational assistant" is a conversational assistant that monitors the user's safety and provides appropriate support when necessary.

[0117] A system for implementing this invention is configured as an application installed on a smartphone. The system uses natural language processing technology to mimic human conversation and generate conversations including questions for assessing the progression of dementia. The system analyzes the responses in the generated conversation to determine the progression of dementia. The results of the assessment are shared with relatives, medical institutions, etc., and a notification is sent if an abnormality is detected.

[0118] Hardware and software used

[0119] Hardware: Smartphone

[0120] Software: OpenAI (registered trademark) API, Python, smtplib (email sending library)

[0121] Data processing and calculation

[0122] The server processes the data in the following steps:

[0123] 1. Prepare a list of questions to ask the elderly, such as "What day is it today?" and "What time is it now?"

[0124] 2. Conversation generation: Using OpenAI's API, natural responses are generated to questions. A generative AI model (Generative Pretrained Transformer) is used to generate responses based on the prompt sentence.

[0125] 3. Analysis of response content: The generated response content is analyzed to determine the progression of dementia. For example, responses such as "I don't know" or "I forgot" are considered abnormal.

[0126] 4. Notification: If an abnormality is detected, a notification is sent to relatives or medical institutions by email. The notification is sent to the specified email address using smtplib.

[0127] Specific examples

[0128] Example questions and answers

[0129] Question: "What day is it today?"

[0130] Response: "Today is Monday."

[0131] Analysis result: normal

[0132] Prompt Sentence Examples

[0133] Question: What day is it today?

[0134] A: Today is Monday.

[0135] This system makes it possible to continuously monitor the progression of dementia in elderly people and respond quickly when abnormalities occur. It also analyzes the user's conversation content and behavioral patterns to understand their living situation and provide appropriate support as needed. As a security dialogue assistant, it monitors the user's safety and provides necessary support.

[0136] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0137] Step 1:

[0138] The server sets a question list. The question list contains questions to be asked to the elderly. For example, questions such as "What day is it today?" and "What time is it now?" are set. The input is the question list, and the output is the set question list. The server saves these questions for use in subsequent processing.

[0139] Step 2:

[0140] The server selects a question from a configured question list and generates a conversation using OpenAI's API. The input is the question selected from the question list, and the output is the generated response. The server uses a generative AI model (Generative Pretrained Transformer) to generate natural responses based on the prompt sentence. Specifically, the server sends an API request and receives a response.

[0141] Step 3:

[0142] The server analyzes the generated response content. The input is the generated response, and the output is the analysis result. The server analyzes the response content and determines the progression of dementia. For example, if the response includes "I don't know" or "I forgot," it is determined to be abnormal. Specifically, the server performs text analysis of the response content and determines whether or not there is an abnormality.

[0143] Step 4:

[0144] The server shares the analysis results with relatives, medical institutions, etc. The input is the analysis results, and the output is a notification message. If the server detects an abnormality, it sends a notification to relatives or medical institutions by email. Specifically, the server uses smtplib to send the notification to the configured email address.

[0145] Step 5:

[0146] The device operates as an application installed on a smartphone. Input is instructions from the server, and output is dialogue with the user. The device presents questions received from the server to the user and sends the user's responses to the server. Specifically, the device displays questions through a user interface and accepts voice and text input.

[0147] Step 6:

[0148] The user responds to questions presented by the terminal. The input is the question from the terminal, and the output is the user's response. The user responds by voice or text, and the response is sent to the server via the terminal. Specifically, the user inputs the response using the terminal's microphone or keyboard.

[0149] Example 2

[0150] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0151] In order to support the lives of elderly people and dementia patients, it is important to reduce the effort required to use various daily services and provide appropriate support. However, previous systems lacked the functionality to automatically perform specific operations based on user instructions, which placed a heavy burden on users. In addition, there were insufficient means to determine the progression of dementia and provide appropriate support.

[0152] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of responses to the generated conversation and determining the progression of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for receiving and analyzing user instructions, means for inputting prompt sentences into the generation AI model and generating specific operation procedures, means for transmitting the generated procedures to a robot and performing operations through the API of an e-commerce site or a taxi dispatch service, and means for notifying the user of the operation results. This makes it possible to efficiently support the lives of elderly people and dementia patients, reduce the burden on users, and provide appropriate support.

[0153] "Natural language processing technology" is a technology that uses computers to understand, interpret, and generate human language.

[0154] "Means for imitating conversation" refers to means for reproducing dialogue with humans using natural language processing technology.

[0155] "Questions to determine dementia progression" are questions designed to assess the progression of dementia.

[0156] The "means for analyzing the content of replies to the generated conversation" is a means for analyzing replies from users and evaluating their content.

[0157] The "means for determining the progression of dementia" is a means for evaluating the degree of progression of dementia based on the responses in the generated conversation.

[0158] "Means of communicating the results of the assessment to relatives, medical institutions, etc." refers to means of notifying relatives and medical institutions of the results of the assessment of the progression of dementia.

[0159] The "means for receiving and analyzing user instructions" refers to a means for receiving instructions from a user and analyzing the contents of those instructions.

[0160] A "generative AI model" is a model that uses artificial intelligence to generate responses or steps for a specific task.

[0161] A "prompt sentence" is an instruction sentence to be input into a generative AI model.

[0162] The "means for generating specific operating procedures" refers to a means for inputting a prompt sentence into a generative AI model and generating specific operating procedures.

[0163] A "robot" is a machine that automatically performs operations based on programmed instructions.

[0164] An "EC site" is a website that sells products and services over the Internet.

[0165] A "taxi dispatch service" is an online service for arranging a taxi.

[0166] "API" stands for Application Program Interface, an interface for exchanging functions and data between different software programs.

[0167] The "means for notifying the user of the operation result" is a means for notifying the user of the result of the operation performed by the robot.

[0168] MODE FOR CARRYING OUT THE INVENTION

[0169] The present invention is a system for supporting the daily lives of users, particularly elderly people and people with dementia. A specific embodiment of this system will be described below.

[0170] System configuration

[0171] This system consists of a server, user devices (PCs, smartphones, tablets), and a robot. The server uses natural language processing technology to analyze user instructions and generates specific operating procedures using a generative AI model. The robot performs operations based on the instructions received from the server through the API of an e-commerce site or taxi dispatch service.

[0172] Hardware and software used

[0173] Hardware: Servers, user devices (PCs, smartphones, tablets), robots

[0174] Software: Natural language processing technology, generative AI models (e.g., GPT-4 (registered trademark)), APIs for e-commerce sites, APIs for taxi dispatch services

[0175] Program processing

[0176] When the server receives instructions from the user, it analyzes them using natural language processing technology. Based on the analysis results, it inputs prompt statements into the generative AI model to generate specific operating procedures. The generated procedures are sent to the robot, which then performs the operations via the API of the e-commerce site or taxi dispatch service. The results of the operations are notified to the user via the server.

[0177] Specific examples

[0178] Example 1: Purchasing a product on an e-commerce site

[0179] User Instruction: "I want you to buy this product"

[0180] Server analysis: Extract product name and quantity

[0181] Prompt for generative AI model: "What are the steps to purchase a product on an e-commerce site? The product name is 'XYZ' and the quantity is 1."

[0182] Robot operation: Calls the API of the e-commerce site, searches for products, adds them to the cart, and completes the purchase process

[0183] Notify the user: "Your purchase of product 'XYZ' has been completed. Delivery is expected in 3 days."

[0184] Example 2: Arranging a taxi

[0185] User instruction: "Please arrange a taxi for me."

[0186] Server analysis: Extracting origin and destination

[0187] Prompt for generative AI model: "What are the steps to order a taxi? From point A to point B."

[0188] Robot operation: Call the API of a taxi dispatch service, specify the departure and destination points, and arrange a taxi.

[0189] Tell the user: "Your taxi has been booked and is expected to arrive in 10 minutes."

[0190] In this way, the server, user terminal, and robot work together to perform specific operations based on the user's instructions, making it a system that supports the user's daily life. This system can efficiently support the daily lives of elderly people and dementia patients, reduce the burden on the user, and provide appropriate support.

[0191] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0192] Step 1:

[0193] The user inputs instructions into the system using a terminal (PC, smartphone, tablet). For example, the user inputs "Please arrange a taxi." This instruction is sent to the server.

[0194] Input: User instructions (e.g., "Please arrange a taxi for me")

[0195] Output: User instructions sent to the server

[0196] Step 2:

[0197] The server analyzes the instructions received from the user, using natural language processing technology to understand the intent of the instructions and extract the necessary information (e.g., departure and destination points).

[0198] Input: User instructions

[0199] Data processing: Analyze instructions using natural language processing technology and extract necessary information

[0200] Output: Analysis results (e.g., origin and destination)

[0201] Step 3:

[0202] The server inputs a prompt sentence into the generative AI model based on the analysis results. For example, it generates a prompt sentence such as, "Please tell me the steps to order a taxi. The starting point is 'Point A' and the destination is 'Point B'."

[0203] Input: Analysis results

[0204] Data processing: Input prompts into the generative AI model to generate specific operating procedures

[0205] Output: Generated operating instructions

[0206] Step 4:

[0207] The server sends the generated operation instructions to the robot, which then follows the instructions received from the server and performs operations via the API of the e-commerce site or taxi dispatch service.

[0208] Input: Generated operating instructions

[0209] Data processing: Send instructions to the robot and execute operations through API

[0210] Output: Operation result

[0211] Step 5:

[0212] The robot calls the API of an e-commerce site or a taxi dispatch service to perform the specified operation, for example, calling the API of a taxi dispatch service to order a taxi based on the departure point and destination.

[0213] Input: Instructions sent to the robot

[0214] Specific behavior: Call the API and perform the specified operation

[0215] Output: Response from the API (e.g., taxi booking completed)

[0216] Step 6:

[0217] The server receives the operation result from the robot and notifies the user. For example, it sends a message to the user's terminal saying, "Taxi arrangements have been completed. Estimated arrival time is in 10 minutes."

[0218] Input: Operation result from the robot

[0219] Data processing: Converts the operation result into a message that notifies the user

[0220] Output: A notification message to the user

[0221] In this way, the server, terminal, and robot work together to perform specific operations based on the user's instructions, creating a system that supports the user's daily life.

[0222] (Application example 2)

[0223] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] In order to provide support for the elderly and dementia patients, there is a need for a means to easily access the services they need in their daily lives. However, these users often have difficulty with technical operations, especially when using services such as purchasing products on e-commerce sites or ordering a taxi. It is also important to properly assess the progression of dementia and provide the necessary support, but current systems have difficulty in comprehensively resolving these issues.

[0225] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the responses to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for searching for products on an e-commerce site based on the user's voice instructions and completing the purchase procedure, means for learning the user's preferences and past purchase history and recommending optimal products, and means for arranging a taxi based on the user's voice instructions. This enables elderly people and dementia patients to easily use services they need in their daily lives, and makes it possible to appropriately determine the progression of dementia and provide necessary support.

[0226] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0227] "Means for mimicking conversation" refers to technologies and algorithms that mimic human interactions and generate natural conversations.

[0228] "Dementia progression questions" are specific questions designed to assess the progression of dementia.

[0229] "Means for analyzing the content of responses generated from conversations" refers to technology for analyzing the user's responses and understanding their meaning and intent from their content.

[0230] "Means for determining the progression of dementia" refers to algorithms or systems for assessing the progression of dementia based on the user's responses.

[0231] "Means of communicating the assessment results to relatives, medical institutions, etc." refers to a means of communication for notifying relatives and medical institutions of the assessment results of the progression of dementia.

[0232] "Means for searching for products on an e-commerce site and completing the purchase procedure based on the user's voice instructions" refers to a system that receives the user's voice instructions, searches for products on an e-commerce site, and automatically completes the purchase procedure.

[0233] "A means of learning a user's preferences and past purchase history and recommending the most suitable products" is an algorithm that recommends the most suitable products to a user based on the user's past behavioral data.

[0234] The "means for dispatching a taxi based on a user's voice instructions" is a system for automatically dispatching a taxi based on a user's voice instructions.

[0235] To implement the present invention, the following system configuration and program are required.

[0236] System Configuration

[0237] The system is configured using the following hardware and software.

[0238] Hardware: Smartphone, microphone

[0239] Software: Python, Google® Cloud Speech-to-Text API, Selenium

[0240] Program processing explanation

[0241] Voice Recognition

[0242] When a user speaks into the smartphone's microphone, the smartphone converts the voice into text using the Google Cloud Speech-to-Text API. This speech recognition technology allows the smartphone to accurately understand the user's commands.

[0243] Product search and purchase on e-commerce sites

[0244] Based on the text data obtained through speech recognition, the smartphone uses Selenium to access e-commerce sites and search for the specified product. The most suitable product is selected from the search results and the purchase process is automatically carried out. This makes it easy for even elderly or technology-inexperienced users to purchase products.

[0245] Product recommendation

[0246] It uses a generative AI model to learn users' past purchase history and preferences. This model analyzes user behavior data and recommends the best products, allowing users to easily find the products that suit them.

[0247] Taxi arrangements

[0248] When a user wants to order a taxi, they can access the taxi dispatch service based on their voice commands and call a taxi to the specified location. This function is also implemented using Selenium.

[0249] Specific examples

[0250] For example, if a user says, "Buy milk," the smartphone converts this voice command into text, searches for "milk" on an e-commerce site, and completes the purchase. Or, if a user says, "Order a taxi," the smartphone accesses a taxi dispatch service and calls a taxi to the specified location.

[0251] Prompt Sentence Examples

[0252] "Buy some milk."

[0253] "Arrange a taxi."

[0254] In this way, a system can be realized that uses voice instructions to support the lives of elderly people and users who are unfamiliar with technology.

[0255] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0256] Step 1:

[0257] The user issues a voice command into the smartphone's microphone. The input is the user's voice, and the output is audio data, which is captured by the smartphone's microphone.

[0258] Step 2:

[0259] The device sends the captured voice data to the Google Cloud Speech-to-Text API for conversion to text data. The input is voice data, and the output is text data. In this conversion process, the voice data is analyzed and the corresponding text is generated.

[0260] Step 3:

[0261] The device analyzes the converted text data and understands the user's instructions. The input is the text data, and the output is the analysis result. Specifically, it checks whether the text data contains keywords such as "purchase" or "taxi."

[0262] Step 4:

[0263] If the user's instruction is related to "purchase," the device uses Selenium to access the e-commerce site and search for the specified product. The input is the analysis result and product name, and the output is a search result page. Specifically, Selenium is used to operate the browser, enter the product name in the search box, and click the search button.

[0264] Step 5:

[0265] The terminal selects the most suitable product from the search results and carries out the purchase procedure. The input is the search result page, and the output is confirmation of purchase completion. Specifically, the user clicks on the most suitable product from the search result page and presses the purchase button to complete the purchase procedure.

[0266] Step 6:

[0267] If the user's instruction is related to "taxi," the device will use Selenium to access a taxi dispatch service and call a taxi to the specified location. The input is the analysis result and the specified location, and the output is confirmation of the taxi dispatch. Specifically, the device will access the taxi dispatch service page, enter the specified location, and click the button to dispatch a taxi.

[0268] Step 7:

[0269] The device uses a generative AI model to learn the user's past purchase history and preferences and recommend the most suitable products. The input is the user's past behavioral data, and the output is the recommended products. Specifically, the generative AI model analyzes the user's past purchase history and recommends the most suitable products to the user.

[0270] Step 8:

[0271] The server generates a conversation including questions to assess the progression of dementia and presents it to the user. The input is the user's past conversation data, and the output is the generated conversation. Specifically, natural language processing technology is used to generate questions to assess the progression of the user's dementia.

[0272] Step 9:

[0273] The server analyzes the generated conversation responses and determines the progression of dementia. The input is the user's responses, and the output is the evaluation result of the dementia progression. Specifically, the server analyzes the user's responses and applies an algorithm to evaluate the dementia progression.

[0274] Step 10:

[0275] The server then shares the results of the assessment with relatives, medical institutions, etc. The input is the assessment result of the dementia progression, and the output is a notification message. Specifically, a notification message is sent to relatives and medical institutions based on the assessment result.

[0276] Example 3

[0277] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0278] It is important to understand the living conditions of elderly people and dementia patients and provide appropriate support, but conventional methods have had difficulty understanding the situation in real time or detecting abnormalities. In addition, communication with relatives and medical institutions had to be done manually, making it difficult to respond quickly.

[0279] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0280] In this invention, the server includes means for imitating human conversation using natural language processing technology, means for converting voice data into text data, means for analyzing the text data and extracting keywords for understanding the living situation, means for evaluating the living situation based on the extracted keywords, means for sharing the evaluation results with relatives, medical institutions, etc., and means for providing appropriate support as needed. This makes it possible to understand the user's living situation in real time and to promptly notify relatives and medical institutions if an abnormality is detected.

[0281] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0282] "Voice data" means data that is a digital recording of a user's speech.

[0283] "Text data" refers to data obtained by converting voice data into character information.

[0284] "Keywords" are words or phrases extracted from text data that are important for understanding a person's living situation.

[0285] "Lifestyle" is information that indicates the user's daily activities and health status.

[0286] "Evaluation" is the process of analyzing the user's living situation based on the extracted keywords and detecting abnormalities or specific patterns.

[0287] "Family" refers to the User's family or close relatives.

[0288] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0289] "Support" refers to the assistance and advice provided to users based on their life situation.

[0290] "Collaboration" is the process of notifying relatives and medical institutions of the evaluation results and sharing information.

[0291] The present invention is a system for understanding a user's living situation and providing appropriate support as needed. A specific embodiment of this system will be described below.

[0292] First, a user engages in everyday conversation using a smartphone or other device with a microphone. The device collects this conversation data in real time and sends it to a server. The server then converts the received voice data into text data using the Google Speech-to-Text API. This API provides highly accurate speech recognition and accurately transcribes the user's speech into text.

[0293] The server then analyzes the text data using spaCy, a natural language processing library. Specifically, it performs morphological analysis to extract keywords related to dietary intake and changes in physical condition. For example, keywords such as "eating," "physical condition," and "healthy" are included.

[0294] The server evaluates the user's lifestyle based on the extracted keywords. It uses a machine learning model based on Tensorflow® to compare the data with past data and detect abnormal patterns. For example, if the keyword "eat" does not appear repeatedly, it determines that there is something abnormal with the user's dietary intake.

[0295] If an abnormality is detected, the server will send a notification to relatives and medical institutions. The device will display an interface to provide appropriate support to the user. For example, if the user says, "I haven't had much of an appetite recently," the device will send that information to the server, which will then send a notification to relatives.

[0296] As a concrete example, consider the case where a user says, "I didn't eat anything yesterday." This utterance is collected through the smartphone's microphone and converted into text using the Google Speech-to-Text API. The text is then analyzed using spaCy, and the keyword "didn't eat" is extracted. Based on this information, the server determines that there is something abnormal with the user's dietary intake and sends a notification to their relatives.

[0297] Example prompts to input to a generative AI model:

[0298] User says: "I didn't eat anything yesterday."

[0299] Prompt: "The user said they didn't eat anything yesterday. Use this information to assess whether their dietary intake is abnormal and notify family members or healthcare providers as needed."

[0300] In this way, the system supports the user's health management. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0301] Step 1:

[0302] Users use smartphones or other devices with microphones to carry out everyday conversations. The devices collect this conversation data in real time and send it to a server. The input is the user's voice data, and the output is the voice data sent to the server.

[0303] Step 2:

[0304] The server converts the received voice data into text data using the Google Speech-to-Text API. The input is voice data and the output is text data. Specifically, the voice recognition engine analyzes the voice data and generates the corresponding text.

[0305] Step 3:

[0306] The server analyzes the text data using spaCy, a natural language processing library. The input is text data, and the output is extracted keywords. Specifically, it performs morphological analysis and extracts keywords related to dietary intake and changes in physical condition.

[0307] Step 4:

[0308] The server evaluates the user's living situation based on the extracted keywords. The input is the extracted keywords, and the output is the evaluation result of the living situation. Specifically, it uses a machine learning model using TensorFlow to compare past data and detect abnormal patterns.

[0309] Step 5:

[0310] If an abnormality is detected, the server sends a notification to relatives or a medical institution. The input is the evaluation result of the living situation, and the output is a notification message. Specifically, the notification system sends a message to relatives or a medical institution based on the evaluation result.

[0311] Step 6:

[0312] The terminal displays an interface to provide appropriate support to the user. The input is a notification message from the server, and the output is a support message that is displayed to the user. Specifically, the terminal displays a message on the screen saying, "There has been something unusual with your recent diet. Your relatives have been notified."

[0313] In this way, the system supports the user in managing their health.

[0314] (Application example 3)

[0315] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0316] Health management for the elderly is an important issue, and early detection of changes in health status, particularly the progression of dementia and changes in daily life, is required. However, current systems have difficulty analyzing the content of users' conversations and behavioral patterns in detail and providing appropriate support. Furthermore, there is a lack of means to quickly notify relatives and medical institutions. This makes it difficult to effectively manage the health status of the elderly.

[0317] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0318] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for analyzing the content of the user's conversation and behavioral patterns and monitoring the health condition, means for recording information about the health condition in a log, and means for sending notifications to relatives and medical institutions as necessary. This makes it possible to grasp the health condition of elderly people in detail and provide appropriate support.

[0319] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0320] "Conversational imitation methods" refers to technologies and algorithms that simulate human interaction.

[0321] "Questions to determine the progression of dementia" are questions designed to assess the cognitive function of the user.

[0322] "Means for analyzing the content of responses generated from conversations" refers to technology for analyzing the user's responses and understanding their meaning and intent from their content.

[0323] The "means for determining the progression of dementia" is a technology for assessing the progression of dementia based on the user's responses.

[0324] "Means of linking with relatives, medical institutions, etc." refers to technologies and methods for notifying relatives and medical institutions of the user's health condition and the progression of dementia.

[0325] "Means for analyzing user conversation content and behavioral patterns" refers to technology that analyzes users' everyday conversations and behavioral data and understands those patterns.

[0326] "Means for monitoring health status" refers to technology for continuously monitoring a user's health status and detecting abnormalities.

[0327] "Means for logging health information" refers to technology that records data about a user's health status for later reference.

[0328] "Means for sending notifications to relatives and medical institutions as necessary" refers to technology for sending notifications to relatives and medical institutions when an abnormality in the user's health condition is detected.

[0329] A system for carrying out the present invention includes a program for analyzing the content of a user's conversations and behavioral patterns and monitoring the health condition of the user. A specific embodiment of this system will be described below.

[0330] System configuration

[0331] The system includes a smartphone, a server, and a network infrastructure for sending notifications to relatives and medical institutions. The smartphone is installed with an application that uses voice recognition software and natural language processing technology. The server runs a program that analyzes the user's health status and sends notifications as needed.

[0332] Hardware and software used

[0333] Hardware: Smartphone microphone, server

[0334] Software: speech_recognition library, requests library, natural language processing technology

[0335] Data processing and calculation

[0336] 1. Acquiring voice data: Capture the user's voice in real time using the smartphone microphone.

[0337] 2. Speech Recognition: Use the speech_recognition library to convert the captured speech data into text.

[0338] 3. Text analysis: Using natural language processing technology, the converted text is analyzed to understand changes in dietary intake and physical condition.

[0339] 4. Health status recording: Send the analysis results to the server and record the health status information in a log.

[0340] 5. Send notifications: Call APIs to notify relatives and medical institutions, if necessary.

[0341] Specific examples

[0342] For example, if a user says, "I don't have an appetite today," the smartphone's microphone picks up the speech and the speech_recognition library converts it into text. Natural language processing technology then analyzes the text and extracts information about the meal. The server logs this information and notifies relatives or medical institutions as needed.

[0343] Example prompt sentence:

[0344] "If a user says, 'I don't have an appetite today,' the app will analyze that and send a notification to their relatives."

[0345] In this way, the system can gain a detailed understanding of the health status of elderly people and provide appropriate support.

[0346] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0347] Step 1:

[0348] The smartphone's microphone picks up the user's voice.

[0349] Input: User's voice

[0350] Output: Audio data

[0351] What it does: When a user speaks into their smartphone, the microphone captures their voice.

[0352] Step 2:

[0353] The smartphone's voice recognition software converts the voice data into text.

[0354] Input: Audio data

[0355] Output: Text data

[0356] What it does: It uses the speech_recognition library to analyze audio data and generate corresponding text.

[0357] Step 3:

[0358] The smartphone's natural language processing technology analyzes the text data to understand changes in dietary intake and physical condition.

[0359] Input: Text data

[0360] Output: Health status information

[0361] Specific operation: Using natural language processing technology, keywords related to diet and physical condition are extracted from text data and health status is evaluated.

[0362] Step 4:

[0363] The smartphone sends information about the health status to a server.

[0364] Input: Health information

[0365] Output: Data sent to the server

[0366] Specific operation: The smartphone sends data to a server via the Internet.

[0367] Step 5:

[0368] Logs health information received by the server.

[0369] Input: Data sent to the server

[0370] Output: Data recorded in the log file

[0371] Specific operation: The server analyzes the data received and records it in a log file along with the date and time.

[0372] Step 6:

[0373] The server will notify relatives and medical institutions as needed.

[0374] Input: Data recorded in the log file

[0375] Output: Notification sent to relatives and medical facilities

[0376] What it does: The server monitors the log file and sends notifications using the requests library if an anomaly is detected.

[0377] In this way, the system can gain a detailed understanding of the user's health status and provide appropriate support.

[0378] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0379] "Example 1"

[0380] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system imitates a conversation with a user and generates a conversation including questions for determining the progression of dementia. The replies in the generated conversation are analyzed to determine the progression of dementia. The determination result is shared with relatives, medical institutions, etc. Furthermore, this system includes an emotion engine that recognizes the user's emotions. This emotion engine analyzes emotions from the user's tone of voice, facial expressions, choice of words, etc. For example, if the user is feeling lonely, this information can be shared with relatives, medical institutions, etc., to provide appropriate support.

[0381] "Example 2"

[0382] Another embodiment of the present invention provides a system incorporating an emotion engine. This system supports users in using services such as e-commerce sites and taxi booking. The system also includes an emotion engine that recognizes the user's emotions. This emotion engine analyzes the user's emotions from the user's tone of voice, facial expressions, choice of words, etc. For example, if the user is feeling stressed, this information can be shared with relatives, medical institutions, etc., to provide appropriate support.

[0383] "Example 3"

[0384] As yet another embodiment of the present invention, a system incorporating an emotion engine is provided. This system understands the user's living situation from the content of their conversations, behavioral patterns, etc., and provides appropriate support as needed. Furthermore, this system includes an emotion engine that recognizes the user's emotions. This emotion engine analyzes the user's emotions from the user's tone of voice, facial expressions, choice of words, etc. For example, if the user is feeling happy, this information can be shared with relatives, medical institutions, etc., so that appropriate support can be provided.

[0385] The processing flow of each embodiment will be described below.

[0386] "Example 1"

[0387] Step 1: Generate a conversation with the user, including questions to determine the progression of dementia.

[0388] Step 2: Analyze the responses generated from the conversation to determine the progression of dementia.

[0389] Step 3: Communicate the results of the decision to relatives, medical institutions, etc.

[0390] Step 4: Recognize the user's emotions Using an emotion engine, analyze the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0391] Step 5: Share the analysis results with relatives, medical institutions, etc., and provide appropriate support.

[0392] "Example 2"

[0393] Step 1: Support users in using e-commerce sites and services such as arranging taxis.

[0394] Step 2: Recognize the user's emotions Using an emotion engine, analyze the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0395] Step 3: Share the analysis results with relatives, medical institutions, etc., and provide appropriate support.

[0396] "Example 3"

[0397] Step 1: Understand the user's living situation based on their conversations and behavioral patterns, and provide appropriate support as needed.

[0398] Step 2: Recognize the user's emotions Using an emotion engine, analyze the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0399] Step 3: Share the analysis results with relatives, medical institutions, etc., and provide appropriate support.

[0400] Example 1

[0401] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0402] Conventional interactive robot systems have difficulty imitating natural conversations with users, and have had issues with being unable to adequately analyze questions and responses to determine the progression of dementia. Furthermore, there was a lack of means to properly analyze the user's emotions and share the results with relatives and medical institutions. This made it difficult to properly understand the user's health and emotional state and provide the necessary support.

[0403] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0404] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for recognizing the user's voice and converting it into text data, means for generating appropriate replies using a generative AI model, means for analyzing the user's emotions using an emotion engine, and means for sharing the analysis results with relatives, medical institutions, etc. This makes it possible to determine the progression of dementia through natural conversation with the user, appropriately analyze the user's emotional state, and provide necessary support.

[0405] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0406] "Questions for determining the progression of dementia" are questions designed to assess the cognitive function of the user.

[0407] A "generative AI model" is an artificial intelligence model that has been trained in advance with large amounts of data and is designed to generate natural language.

[0408] An "emotion engine" is a technology for analyzing emotions from a user's tone of voice, facial expressions, choice of words, etc.

[0409] "Means of communicating with relatives, medical institutions, etc." refers to means for notifying relatives and medical institutions of the results of analysis and judgment.

[0410] "Speech recognition" is a technology that converts a user's voice into text data.

[0411] "Text data" is character information converted using voice recognition technology.

[0412] The "means for analyzing the content of the reply" is a technique for analyzing the user's reply and evaluating its content.

[0413] The "means for generating an appropriate response" is a technique for generating a natural response to a user's input.

[0414] The "analysis results" are the results of analyzing the user's responses and emotional state.

[0415] "Means for providing support" refers to means for providing appropriate support and services according to the user's condition.

[0416] This invention uses an interactive robot system to realize natural conversation with the user and determine the progression of dementia. The system is composed of a combination of natural language processing technology, a generative AI model, and an emotion engine.

[0417] The server uses natural language processing technology to mimic conversations with users. Specifically, it uses a generative AI model (e.g., a Generative Pretrained Transformer) to generate appropriate responses to user input. This generative AI model has learned a large amount of data in advance and is capable of natural language generation.

[0418] The device uses speech recognition technology to recognize the user's voice and convert it into text data. For example, the Google Speech-to-Text API can be used as the speech recognition software. The converted text data is sent to a server and analyzed by a generative AI model.

[0419] The server generates questions to assess the progression of dementia, such as "What day is it today?" and "What time is it now?" These questions are designed to assess the user's cognitive function.

[0420] The device converts the generated question into speech using speech synthesis software (e.g., Amazon Polly) and outputs it to the user through the speaker. When the user responds to the question, the device again uses speech recognition technology to convert the response into text data and sends it to the server.

[0421] The server analyzes the responses and determines the progression of the user's dementia. For example, it checks whether the user correctly answered the day of the week. It also uses an emotion engine (for example, Microsoft® Azure® emotion recognition API) to analyze the user's emotions from their tone of voice and facial expressions. The analysis results are shared with relatives and medical institutions.

[0422] As a specific example, when a user says "Hello" to an interactive robot, the device captures the audio and converts it into text data using the Google Speech-to-Text API. The server inputs the text data into GPT-3 and generates a response such as "Hello, how was your day?". The server then generates a question to determine the progression of dementia, such as "What day of the week is it today?". The device uses Amazon Polly to convert this question into speech and outputs it to the user through the speaker.

[0423] If the user responds with "Today is Monday," the device again uses the Google Speech-to-Text API to convert the response into text data. The server then analyzes the text data and verifies whether the user correctly answered the day of the week. It also uses Microsoft Azure's emotion recognition API to analyze the user's emotions from their tone of voice and facial expressions. For example, if the user sounds lonely, it will analyze their emotions. Finally, the server communicates the results of its judgment and emotion analysis to relatives and medical institutions via email or a dedicated app.

[0424] Example prompt sentence:

[0425] User: "What day is it today?"

[0426] Robot: "Today is Monday."

[0427] Server: "Analyze the user's responses and determine the progression of dementia."

[0428] In this way, the interactive robot can assess the progression of dementia through natural conversation with the user, properly analyze the user's emotional state, and provide the necessary support.

[0429] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0430] Step 1:

[0431] The user speaks to the interactive robot.

[0432] Specifically, the user speaks to the interactive robot saying "hello." The input is the user's voice, and the output is voice data.

[0433] Step 2:

[0434] The device recognizes the user's voice and converts it into text data.

[0435] Specifically, the device captures the user's voice through a microphone and converts the voice into text data using speech recognition software (e.g., Google Speech-to-Text API). The input is voice data and the output is text data.

[0436] Step 3:

[0437] The server uses a generative AI model to generate an appropriate response.

[0438] Specifically, the server inputs the received text data into a generative AI model (e.g., GPT-3) to generate an appropriate response. The input is text data, and the output is the generated response text data.

[0439] Step 4:

[0440] The server generates questions to determine the progression of dementia.

[0441] Specifically, the server generates questions to determine the progression of dementia. For example, it generates a question such as, "What day is it today?" The input is the user's basic information and past dialogue history, and the output is the question text data.

[0442] Step 5:

[0443] The terminal outputs the generated question to the user by voice.

[0444] Specifically, the device converts text data into speech using speech synthesis software (e.g., Amazon Polly) and outputs it to the user through a speaker. The input is the text data of the question, and the output is speech data.

[0445] Step 6:

[0446] The user responds to the question.

[0447] As a specific operation, the user responds, "Today is Monday." The input is the user's voice, and the output is voice data.

[0448] Step 7:

[0449] The terminal converts the user's response into text data.

[0450] Specifically, the terminal again uses voice recognition software to convert the user's response into text data, with the input being voice data and the output being text data.

[0451] Step 8:

[0452] The server analyzes the responses and determines the progression of dementia.

[0453] Specifically, the server analyzes the response and checks whether the user correctly answered the day of the week. The input is the text data of the response, and the output is the judgment result regarding the progression of dementia.

[0454] Step 9:

[0455] The server analyzes the user's emotions using an emotion engine.

[0456] Specifically, the server uses an emotion engine (for example, Microsoft Azure's emotion recognition API) to analyze the user's emotions from their tone of voice and facial expressions. The input is the user's voice data and video data, and the output is the emotion analysis results.

[0457] Step 10:

[0458] The server will share the judgment results and emotion analysis results with relatives and medical institutions.

[0459] Specifically, the server communicates the judgment results and emotion analysis results to relatives and medical institutions via email or a dedicated app. The input is the judgment results and emotion analysis results, and the output is notification data.

[0460] (Application example 1)

[0461] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0462] In modern society, it is important to detect the progression of dementia in the elderly early and provide appropriate support. However, there are limited means to determine the progression of dementia, and there is no system that can simultaneously analyze user emotions and security risks, making it difficult to provide comprehensive support.

[0463] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for analyzing the user's emotions, and means for determining security risks based on the emotion analysis results and sharing the results with relatives and security services. This makes it possible to determine the progression of dementia while simultaneously analyzing the user's emotions and security risks, thereby providing comprehensive support.

[0464] "Natural language processing technology" is a technology that enables computers to understand, generate, and respond to human language.

[0465] "Questions to assess dementia progression" are questions designed to assess a user's cognitive function, such as "What day is it today?" and "What time is it now?"

[0466] The "means for analyzing the content of responses generated from conversations" refers to technology that analyzes the responses provided by users and evaluates the progression of dementia from their content.

[0467] "Means of connecting with relatives, medical institutions, etc." refers to technology for notifying the user's relatives and medical institutions of the results of the system's judgment.

[0468] "Means for analyzing user emotions" refers to technology for analyzing emotions from the user's tone of voice, facial expressions, choice of words, etc.

[0469] "Means for determining security risks based on the results of emotion analysis" is a technology for evaluating security risks such as fear and anxiety felt by users based on the results of emotion analysis.

[0470] "Means for linking with relatives and security services" refers to technology that notifies the user's relatives and security services when a security risk is detected.

[0471] A system for implementing this invention uses natural language processing technology to mimic human conversations and generate conversations that include questions for assessing the progression of dementia. The system analyzes the responses in the generated conversations to determine the progression of dementia. The system also analyzes the user's emotions, determines security risks based on the emotion analysis results, and connects with relatives and security services.

[0472] Hardware and software used

[0473] Hardware: Smartphones, robots

[0474] Software: OpenAI API, IBM Watson(R) Tone Analyzer API

[0475] Data processing and calculation

[0476] natural language generation

[0477] The server uses OpenAI's GPT-3 model to generate a conversation with the user, using prompts such as:

[0478] Example prompt sentence:

[0479] Hello, how was your day today?

[0480] Based on this prompt, the server receives responses from the user and generates natural conversation.

[0481] Emotion analysis

[0482] The server uses IBM Watson Tone Analyzer to analyze the emotions of the generated conversation, based on the user's tone of voice, facial expressions, and word choice, and produces results such as the following:

[0483] Example of sentiment analysis results:

[0484] We have detected that you are feeling anxious.

[0485] Determining security risks

[0486] The server evaluates the user's fear, anxiety, and other security risks based on the results of emotion analysis, and if a risk is detected, it notifies the user's relatives and security services.

[0487] Specific examples

[0488] 1. Talk to your users:

[0489] Server: "Hello, how was your day today?"

[0490] User: "I was a little tired today, but nothing too bad."

[0491] 2. Emotion analysis:

[0492] The server sends the user's response to the IBM Watson Tone Analyzer to analyze the emotion.

[0493] Result: "We have detected that you are feeling anxious."

[0494] 3. Determining security risks:

[0495] The server determines security risks based on the results of emotion analysis and, if necessary, connects with relatives or security services.

[0496] In this way, the system can simultaneously analyze the user's emotions and security risks while determining the progression of dementia, and provide comprehensive support.

[0497] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0498] Step 1:

[0499] The server generates a prompt to start a conversation with the user. Specifically, it uses OpenAI's GPT-3 model to generate a prompt such as "Hello, how was your day today?". This prompt is then presented to the user.

[0500] Input: None

[0501] Output: Prompt statement (e.g. "Hello, how was your day today?")

[0502] Step 2:

[0503] The user responds to prompts provided by the server, and the responses are sent to the server in text format.

[0504] Input: prompt statement

[0505] Output: User response (e.g., "I was a little tired today, but nothing too bad.")

[0506] Step 3:

[0507] The server receives the user's responses and generates a conversation using natural language processing technology, specifically OpenAI's GPT-3 model, to generate appropriate responses to the user's responses.

[0508] Input: User response

[0509] Output: The generated conversation (e.g., "That's tough. Is there anything I can help you with?")

[0510] Step 4:

[0511] The server analyzes the generated conversation responses to determine the progression of dementia. Specifically, it analyzes the user's responses and evaluates whether there are any signs of dementia progression.

[0512] Input: Generated conversation

[0513] Output: Assessment result of dementia progression (e.g., "No progression of dementia is observed.")

[0514] Step 5:

[0515] The server sends the user's response to the IBM Watson Tone Analyzer, which analyzes their emotions based on their tone of voice, facial expressions, and choice of words.

[0516] Input: User response

[0517] Output: Sentiment analysis results (e.g., "We have detected that the user is feeling anxious.")

[0518] Step 6:

[0519] The server determines security risks based on the results of emotion analysis. Specifically, it evaluates security risks such as fear and anxiety felt by the user based on the results of emotion analysis.

[0520] Input: Sentiment analysis results

[0521] Output: Security risk assessment result (e.g. "Security risk detected.")

[0522] Step 7:

[0523] If a security risk is detected, the server notifies the user's relatives and security services of the information by generating a notification message and sending it to them.

[0524] Input: Security risk assessment results

[0525] Output: Notification message (e.g. "A security risk has been detected due to user concerns.")

[0526] In this way, the system can simultaneously analyze the user's emotions and security risks while determining the progression of dementia, and provide comprehensive support.

[0527] Example 2

[0528] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0529] Elderly people and people with dementia face difficulties in their daily lives, making it difficult to use e-commerce sites or order taxis. It is also necessary to properly understand the stress and anxiety these users feel and provide the necessary support. It is also important to determine the progression of dementia early and take appropriate measures.

[0530] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for receiving instructions from the user and supporting the use of services, means for analyzing the user's emotions using an emotion engine, and means for providing appropriate support based on the analysis results. This enables elderly people and dementia patients to live their daily lives smoothly, and makes it possible to appropriately understand the user's emotional state and provide necessary support.

[0531] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0532] "Questions to determine dementia progression" are questions designed to assess the progression of dementia.

[0533] The "means for analyzing the content of responses to the generated conversation" refers to a means for analyzing responses provided by users and evaluating their content.

[0534] The "means for determining the progression of dementia" is a means for evaluating the degree of progression of dementia based on the responses in the generated conversation.

[0535] "Means of coordinating with relatives, medical institutions, etc." refers to the means of notifying relatives and medical institutions of the results of the judgment and encouraging them to take necessary measures.

[0536] "Means for receiving user instructions and supporting service usage" refers to means for receiving instructions from a user and providing a service based on those instructions.

[0537] The "emotion engine" is an engine that analyzes emotions from the user's tone of voice, facial expressions, choice of words, etc.

[0538] "Means for analyzing user emotions" refers to means for assessing the emotional state of a user using an emotion engine.

[0539] "Means for providing appropriate support" refers to means for providing necessary assistance and services to users based on the analysis results.

[0540] MODE FOR CARRYING OUT THE INVENTION

[0541] This invention is a system that supports users in using services such as e-commerce sites and taxi bookings, and further analyzes the user's emotions using an emotion engine to provide appropriate support. Specific embodiments of this system are described below.

[0542] Hardware and software used

[0543] Hardware: Servers, devices (smartphones, tablets, PCs), robots

[0544] Software: Emotion engine, e-commerce API, taxi booking API, generative AI model

[0545] System Overview

[0546] The user uses the device to input instructions, such as purchasing a product on an e-commerce site or ordering a taxi. The device then sends these instructions to the server, which then executes them using the appropriate API. The server also uses an emotion engine to analyze the user's emotions and contacts relatives or medical institutions as needed.

[0547] Specific examples

[0548] 1. Receive user instructions

[0549] The user uses their smartphone to say, "Buy this item," and the device uses voice recognition software to convert the command into text.

[0550] 2. Send instructions to the server

[0551] The terminal sends a text instruction to the server, "Purchase this product." The server receives this instruction and starts processing the product purchase using the EC site API.

[0552] 3. Purchasing products on an e-commerce site

[0553] The server calls the EC site API, adds the specified product to the cart, then completes the purchase and notifies the user of the purchase completion.

[0554] 4. Arranging a taxi

[0555] When a user requests a taxi, the server calls the taxi dispatch API and dispatches a taxi to the user's current location. The server then notifies the user of the estimated arrival time of the taxi.

[0556] 5. Emotion analysis using an emotion engine

[0557] If a user says, "I've been feeling stressed lately," the server inputs the voice data into the emotion engine and analyzes the user's stress level. The analysis result is, "The user is feeling high stress."

[0558] 6. Providing appropriate support

[0559] Based on the analysis results, the server notifies the user's relatives that the user is experiencing high levels of stress, and if necessary, contacts a medical institution and arranges for appropriate support.

[0560] Prompt Sentence Examples

[0561] "Please enter instructions for users to purchase products on your e-commerce site."

[0562] "Users can enter instructions to order a taxi."

[0563] "Analyze the user's emotions and provide instructions to provide appropriate support."

[0564] The above is a specific embodiment for carrying out the present invention. This system enables elderly people and dementia patients to smoothly lead their daily lives, and it is possible to properly grasp the user's emotional state and provide necessary support.

[0565] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0566] Step 1:

[0567] A user uses a terminal to input instructions for purchasing a product on an e-commerce site or ordering a taxi. Specifically, the user uses a smartphone to give a voice command such as "purchase this product." The terminal uses voice recognition software to convert this command into text. The input is voice data, and the output is text data.

[0568] Step 2:

[0569] The terminal sends a text instruction to the server. Specifically, the terminal sends the text data "Purchase this product" to the server. The input is text data, and the output is data sent to the server.

[0570] Step 3:

[0571] The server uses the EC site API to purchase the product specified by the user. Specifically, the server calls the API, adds the specified product to the cart, and completes the purchase. The input is text data, and the output is a notification that the purchase is complete.

[0572] Step 4:

[0573] When a user issues a taxi request, the server uses the taxi dispatch API to dispatch a taxi to the user's specified location. Specifically, the server calls the API to dispatch a taxi to the user's current location or a specified location. The input is text data, and the output is a notification of the estimated arrival time of the taxi.

[0574] Step 5:

[0575] The server uses an emotion engine to analyze emotions based on the user's tone of voice, facial expressions, and choice of words. Specifically, the server inputs voice data and image data into the emotion engine to analyze the user's emotional state. The input is voice data and image data, and the output is the emotion analysis results.

[0576] Step 6:

[0577] The server provides appropriate support based on the analysis results. Specifically, the server communicates the analysis results to relatives and medical institutions and arranges appropriate support for the user. The input is the emotion analysis results, and the output is notifications to relatives and medical institutions.

[0578] (Application example 2)

[0579] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0580] Elderly people and people with dementia often have difficulty using services such as e-commerce sites, arranging taxis, and ordering food delivery in their daily lives. Furthermore, when these users experience stress or anxiety, they lack the means to properly recognize their emotions and notify their relatives or medical institutions. This makes it difficult for them to receive appropriate support.

[0581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0582] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing replies to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for ordering food delivery based on the user's voice instructions, means for analyzing the user's tone of voice and facial expressions to recognize emotions, and means for notifying relatives or medical institutions if the user is feeling stressed or anxious. This not only enables elderly people and dementia patients to easily use services necessary for their daily lives, but also makes it possible to appropriately recognize changes in emotions and quickly provide necessary support.

[0583] "Natural language processing technology" is a technology that allows computers to process and understand human language.

[0584] "Conversational imitation methods" refers to technologies and algorithms that simulate human interaction.

[0585] "Questions to assess dementia progression" are questions designed to assess the symptoms and progression of dementia.

[0586] The "means for analyzing the content of responses" refers to a technology for analyzing the user's responses and extracting specific information from the content.

[0587] "Means of communicating with relatives, medical institutions, etc." refers to technologies and methods for notifying relatives and medical institutions of analysis results and important information.

[0588] "Means for placing food delivery orders based on voice instructions" refers to technology that recognizes a user's voice commands and places food delivery orders based on them.

[0589] "Means for recognizing emotions by analyzing tone of voice and facial expressions" refers to technology that analyzes the tone of a user's voice and facial expressions to determine their emotional state.

[0590] "Means for notifying relatives or medical institutions when a user is feeling stressed or anxious" refers to technology that notifies relatives or medical institutions of the information when it is determined that the user is feeling stressed or anxious.

[0591] A system for carrying out this invention has the following configuration: The system includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the replies in the generated conversation and determining the progression of dementia, means for communicating the determination results to relatives, medical institutions, etc., means for ordering food delivery based on the user's voice instructions, means for analyzing the user's tone of voice and facial expressions and recognizing emotions, and means for notifying relatives or medical institutions if the user is feeling stressed or anxious.

[0592] Hardware and software used

[0593] Hardware: smartphone, microphone, camera

[0594] Software: Natural language processing libraries (e.g., speech_recognition), emotion recognition libraries (e.g., emotion_recognition), HTTP request libraries (e.g., requests), email sending libraries (e.g., smtplib)

[0595] Data processing and calculation

[0596] 1. Speech recognition: Use the smartphone's microphone to capture the user's voice and convert it into text using the speech_recognition library.

[0597] 2. Emotion analysis: The acquired voice data is analyzed using the emotion_recognition library to determine the user's emotional state.

[0598] 3. Food delivery order: Based on voice instructions, an order is sent to the food delivery API using an HTTP request.

[0599] 4. Notification: If the user is feeling stressed or anxious, the smtplib library is used to send notification emails to relatives or medical institutions.

[0600] Specific examples

[0601] When a user says "I want to order pizza," the smartphone's microphone picks up the audio and converts it into text using the speech_recognition library. Next, the emotion_recognition library is used to analyze the user's emotions from the audio data. If the user is feeling stressed or anxious, that information is sent via email to their relatives or a medical institution. Finally, an HTTP request is used to send the order to a food delivery API, and the results are displayed to the user.

[0602] Prompt Sentence Examples

[0603] If a user says "I want to order a pizza," convert that speech to text and send the order to a food delivery API. Also, analyze the user's tone of voice to determine their emotions and send an email notification to their relatives if they are feeling stressed or anxious.

[0604] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0605] Step 1:

[0606] The user issues a voice command into the smartphone's microphone.

[0607] Input: User's voice commands

[0608] Output: Audio data

[0609] Specific operation: The user says, "I want to order a pizza."

[0610] Step 2:

[0611] The device receives the voice data and converts it to text using the speech_recognition library.

[0612] Input: Audio data

[0613] Output: Text data

[0614] What it does: Your smartphone records the voice data and uses the speech_recognition library to convert it into text: "I'd like to order a pizza."

[0615] Step 3:

[0616] The device inputs the text data into the emotion_recognition library to analyze the user's emotions.

[0617] Input: Text data

[0618] Output: Emotion data (e.g., stress, anxiety, joy, etc.)

[0619] Specific behavior: Analyze text data and determine whether the user is experiencing stress.

[0620] Step 4:

[0621] If the device detects that the user is feeling stressed or anxious based on emotional data, it uses the smtplib library to send a notification email to relatives or medical institutions.

[0622] Input: Emotion data

[0623] Output: Notification email

[0624] Specific operation: If it is determined that the user is feeling stressed, an email stating "The user is feeling stressed" is sent to the user's relatives.

[0625] Step 5:

[0626] The device uses the text data to send an order to the food delivery API using an HTTP request.

[0627] Input: Text data

[0628] Output: Order result data

[0629] Specific operation: Sends text data to the food delivery API and makes a request to "order pizza."

[0630] Step 6:

[0631] The terminal displays the order result data from the food delivery API to the user.

[0632] Input: Order result data

[0633] Output: User notification

[0634] Specific behavior: Receives a response from the food delivery API and notifies the user whether the order was successful or not.

[0635] Example 3

[0636] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0637] There is a need to understand the living conditions and health status of elderly people and dementia patients in real time and provide appropriate support. However, conventional systems have the problem of being unable to fully analyze the user's emotions and behavioral patterns, making it difficult to provide appropriate support. In addition, there is a lack of means to quickly notify relatives or medical institutions of changes in the user's physical condition or emotions, which can delay response in emergencies.

[0638] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of responses to the generated conversation and determining the progression state of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for analyzing the content of the user's conversation and behavioral patterns and understanding the living situation, means for analyzing the user's emotions from the user's tone of voice and facial expressions, and means for contacting relatives or medical institutions as necessary. This makes it possible to understand the living situation and health condition of elderly people and dementia patients in real time and provide appropriate support.

[0639] "Natural language processing technology" is a technology that uses computers to understand, interpret, and generate human language.

[0640] "Means to mimic conversation" refers to technology that reproduces a dialogue with the user in a natural way.

[0641] "Questions to determine the progression of dementia" are questions designed to assess the cognitive function of the user.

[0642] The "means for analyzing the content of responses generated from conversations" refers to technology that analyzes the user's responses and evaluates the progression of dementia from their content.

[0643] "Means for determining the progression of dementia" refers to technology that evaluates the progression of dementia based on the user's responses and behavior.

[0644] "Means of connecting with relatives, medical institutions, etc." refers to technology for notifying relatives and medical institutions of the user's health condition and living situation.

[0645] "Means for analyzing user conversation content and behavioral patterns" refers to technology that analyzes users' statements and behavior to understand their living situation.

[0646] "Means for understanding living conditions" refers to technology that analyzes the user's daily life situation and provides the necessary support.

[0647] "Means for analyzing emotions from a user's tone of voice and facial expression" refers to a technology that analyzes a user's tone of voice and facial expression and evaluates their emotions.

[0648] "Means of contacting relatives or medical institutions as needed" refers to technology that allows for quick contact with relatives or medical institutions depending on the user's condition.

[0649] This invention is a system that analyzes the content of a user's conversations and behavioral patterns, grasps their living situation, and provides appropriate support as needed. Furthermore, by combining it with an emotion engine, it can recognize the user's emotions and provide more accurate support.

[0650] Hardware and software used

[0651] Hardware: microphones, cameras, servers

[0652] Software: speech recognition software (e.g., Google Speech-to-Text), sentiment analysis engines (e.g., Affectiva), database management systems (e.g., MySQL®)

[0653] Data processing and calculation flow

[0654] 1. Audio data collection:

[0655] The device collects the user's conversation in real time through a microphone.

[0656] The collected voice data is immediately sent to the server.

[0657] 2. Analysis of audio data:

[0658] The server uses voice recognition software to convert the voice data into text data.

[0659] The converted text data is stored in a database.

[0660] 3. Behavioral Pattern Analysis:

[0661] The device uses a camera to capture the user's actions.

[0662] The captured video data is transmitted to a server.

[0663] The server uses video analysis software to extract user behavior patterns.

[0664] 4. Emotion Analysis:

[0665] The server uses an emotion analysis engine to analyze the user's emotions from their tone of voice and facial expressions.

[0666] The analysis results are stored in a database.

[0667] 5. Understanding your living situation:

[0668] The server integrates the voice data, behavioral patterns, and emotional data collected to understand the user's living situation.

[0669] For example, it detects changes in food intake and physical condition.

[0670] 6. Providing Support:

[0671] The server will contact relatives and medical institutions as needed.

[0672] For example, if a user complains of feeling unwell, a medical institution will be notified.

[0673] Specific examples

[0674] Example 1: When a user says, "I have a headache today"

[0675] The server uses voice recognition software to generate text data such as "I have a headache today."

[0676] An emotion analysis engine analyzes the level of pain from the user's tone of voice.

[0677] The server uses this information to contact medical institutions and provide appropriate support.

[0678] Example 2: When a user smiles and says, "Today was fun"

[0679] The server uses voice recognition software to generate text data such as "Today was fun."

[0680] The emotion analysis engine analyzes the user's smile to determine their joy.

[0681] The server then communicates this information to relatives, sharing the user's positive feelings.

[0682] Prompt Sentence Examples

[0683] Prompt 1: "Please explain how you would respond if a user complained of feeling unwell."

[0684] Prompt 2: "If the user experiences joy, explain how they would communicate this information to their next of kin."

[0685] This system makes it possible to grasp the living conditions and health conditions of elderly people and dementia patients in real time and provide appropriate support. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0686] Step 1:

[0687] The device collects the user's conversation in real time through a microphone. The input is the user's voice, and the output is digital voice data. Specifically, when a user says, "I don't remember what I ate today," the device captures this voice and sends it to the server as digital voice data.

[0688] Step 2:

[0689] The server uses speech recognition software to convert the voice data into text data. The input is digital voice data, and the output is text data. Specifically, the server converts the voice data "I can't remember what I ate today" into text, and stores the converted text "I can't remember what I ate today" in a database.

[0690] Step 3:

[0691] The device uses a camera to capture the user's actions. The input is the user's actions, and the output is video data. Specifically, the camera captures the user walking around the room and sends the video data to the server.

[0692] Step 4:

[0693] The server uses video analysis software to extract the user's behavioral patterns. The input is video data, and the output is behavioral pattern data. Specifically, the server analyzes the video data and detects that the user frequently visits the kitchen.

[0694] Step 5:

[0695] The server uses an emotion analysis engine to analyze emotions from the user's tone of voice and facial expression. The input is audio and video data, and the output is emotional data. In concrete terms, when a user says with a smile, "Today was fun," the server analyzes the tone of voice and facial expression and determines that the user is feeling happy.

[0696] Step 6:

[0697] The server integrates collected voice data, behavioral patterns, and emotional data to understand the user's living situation. The input is text data, behavioral pattern data, and emotional data, and the output is living situation data. Specifically, the server integrates the text data "I don't remember what I ate today" with the behavioral pattern of frequently going to the kitchen, and determines that the user tends to forget to eat.

[0698] Step 7:

[0699] The server will contact relatives or medical institutions as necessary. The input is living situation data and the output is notification data. Specifically, if the server detects text data such as "I have a headache today" and determines through emotion analysis that the pain is severe, it will contact a medical institution and report the user's poor health.

[0700] (Application example 3)

[0701] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0702] To ensure the safety and improve the quality of life of elderly people and people living alone, it is important to understand their health status and emotions from their daily conversations and behavioral patterns, and provide appropriate support as needed. However, conventional systems have had difficulty accurately analyzing users' emotions and connecting with relatives and security services in real time. Furthermore, there was a lack of means to comprehensively understand the user's living situation, making it difficult to provide appropriate support.

[0703] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for imitating a conversation with a human using natural language processing technology; means for generating a conversation including questions for determining the progression of dementia; means for analyzing the responses to the generated conversation and determining the progression of dementia; means for sharing the determination result with relatives, medical institutions, etc.; means for understanding the user's living situation from the content of the user's conversation and behavioral patterns, etc., and providing appropriate support as needed; means including an emotion engine for recognizing the user's emotions; means for analyzing emotions from the user's tone of voice, facial expressions, choice of words, etc.; and means for sending an alert to relatives or security services when the user's emotions are in a specific state. This makes it possible to ensure the safety of elderly people and people living alone and improve their quality of life.

[0704] "Natural language processing technology" is a technology that allows computers to process and understand human language.

[0705] "Questions to determine the progression of dementia" are questions designed to assess the cognitive function of the user.

[0706] A "means for generating a conversation" is a technology for automatically creating a dialogue with a user.

[0707] The "means for analyzing the content of responses" refers to a technique for analyzing the user's responses and evaluating their content.

[0708] The "means for determining the progression of dementia" is a technology for assessing the progression of dementia based on the user's responses.

[0709] "Means of communication with relatives, medical institutions, etc." refers to technology for notifying relatives and medical institutions of the results of the judgment.

[0710] "Means of understanding a user's living situation from the content of their conversations and patterns of behavior" refers to technology that analyzes the user's daily conversations and behavior to understand their living situation.

[0711] "Means for providing appropriate support" refers to technology that provides necessary assistance according to the user's living situation.

[0712] An "emotion engine" is a technology for analyzing user emotions.

[0713] "Means of analyzing emotions from tone of voice, facial expressions, choice of words, etc." is a technology for assessing emotions based on the user's tone of voice, facial expressions, and choice of words.

[0714] "Means for sending alerts to relatives or security services when a user's emotions are in a specific state" refers to technology for sending alerts to relatives or security services when a user's emotions are determined to be in a specific state.

[0715] "Smart glasses" are wearable devices that users can wear to display information and input voice.

[0716] As an example of an embodiment of this invention, we will explain the "Security Guardian" application installed on smart glasses. This system analyzes the user's conversation content, behavioral patterns, and even emotions, and issues security alerts as necessary.

[0717] Hardware and software used

[0718] Hardware

[0719] Smart glasses: Wearable devices with voice input and camera capabilities.

[0720] software

[0721] Python: A programming language used to implement programs.

[0722] speech_recognition library: A library for performing speech recognition.

[0723] emotion_recognition library: A library for performing emotion analysis.

[0724] requests library: A library for data integration.

[0725] Data processing and calculation

[0726] Voice input

[0727] The microphone in the smart glasses is used to capture the user's voice, which is then converted to text using the speech_recognition library.

[0728] Voice Recognition

[0729] To convert the voice data to text, we use the speech_recognition library, which will capture what the user is saying in text form.

[0730] Emotion analysis

[0731] The acquired text data is then subjected to emotion analysis using the emotion_recognition library, which evaluates the user's emotions based on their tone of voice, facial expressions, and word choice.

[0732] Data Linkage

[0733] If the emotion is in a certain state (e.g. "anger", "fear", "sadness"), the requests library is used to send an alert to relatives or security services. The alert includes the emotional state and text data.

[0734] Specific examples

[0735] For example, if an elderly person is alone and says "help," emotion analysis can detect "fear," and this information can be immediately sent as an alert to relatives or security services, allowing for a prompt response.

[0736] Prompt Sentence Examples

[0737] Write a Python program that, if the user says "help," will use emotion analysis to detect "fear" and send an alert to relatives or security services.

[0738] In this way, the mode for carrying out the invention can be concretely demonstrated. This system can ensure the safety of elderly people and people living alone and improve their quality of life.

[0739] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0740] Step 1:

[0741] A user puts on the smart glasses and starts voice input. The microphone of the smart glasses picks up the user's voice. The input is the user's voice data, and the output is the acquisition of the voice data.

[0742] Step 2:

[0743] The speech data acquired by the device is converted into text using the speech_recognition library. The input is speech data and the output is text data. Specifically, the speech recognition engine analyzes the speech data and generates the corresponding text.

[0744] Step 3:

[0745] The device performs emotion analysis on the generated text data using the emotion_recognition library. The input is text data, and the output is emotion data. Specifically, the emotion recognition engine analyzes the content of the text, tone of voice, word choice, etc. to evaluate the user's emotional state.

[0746] Step 4:

[0747] If the server detects a specific emotional state (e.g., "anger," "fear," or "sadness") based on the emotion data, it uses the requests library to send an alert to relatives or security services. The input is emotion data, and the output is sending an alert. Specifically, if the emotion data meets certain conditions, an alert message is generated and sent to the specified contacts.

[0748] Step 5:

[0749] The server logs the alert sending result. The input is the alert sending result and the output is the log data. Specifically, it checks whether the alert was sent successfully and records the result in the log file.

[0750] In this way, a system is realized that ensures the safety of users and allows for quick response when necessary.

[0751] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0752] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0753] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0754] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0755] [Second embodiment]

[0756] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0757] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0758] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0759] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0760] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0761] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0762] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0763] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0764] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0765] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0766] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0767] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0768] "Example 1"

[0769] As an example embodiment of the present invention, an interactive robot is provided. This robot uses natural language processing technology to imitate conversations with humans. Specifically, a natural language generation model such as a generative AI (generative pretrained transformer) is used to generate natural conversations with humans. The robot also generates conversations that include questions for determining the progression of dementia. For example, the robot asks questions such as "What day is it today?" and "What time is it now?" and analyzes the replies. This analysis determines the progression of dementia. The results of the determination are shared with relatives, medical institutions, etc.

[0770] "Example 2"

[0771] Furthermore, as an example of the present invention, a function is provided to assist users in using services such as e-commerce sites and arranging taxis. Specifically, the robot purchases products on e-commerce sites and arranges taxis based on the user's instructions. This makes it possible to support the daily lives of elderly people.

[0772] "Example 3"

[0773] In addition, as an example of the present invention, a function is provided that grasps the user's living situation from the content of the user's conversations and behavioral patterns, etc., and provides appropriate support as needed. Specifically, the function grasps the user's dietary intake and changes in physical condition from the content of the user's conversations, and connects with relatives, medical institutions, etc. as needed. This enables health management for the elderly.

[0774] The processing flow of each embodiment will be described below.

[0775] "Example 1"

[0776] Step 1: The conversational robot starts and starts a conversation with the user.

[0777] Step 2: The robot uses natural language processing technology to understand what the user is saying and generate an appropriate response.

[0778] Step 3: The robot generates a conversation containing questions to determine the progression of dementia and asks the user questions.

[0779] Step 4: The robot receives the user's response and analyzes it.

[0780] Step 5: Based on the analysis results, the progression of dementia is determined and the results are shared with relatives, medical institutions, etc.

[0781] "Example 2"

[0782] Step 1: The user instructs the robot to use a service such as purchasing a product on an e-commerce site or arranging a taxi.

[0783] Step 2: The robot understands the user's instructions and follows them to make purchases on an e-commerce site or order a taxi.

[0784] "Example 3"

[0785] Step 1: The robot analyzes the user's conversation content, behavioral patterns, etc. to understand their living situation.

[0786] Step 2: Based on the analysis results, the robot will provide appropriate support as needed. For example, it will understand the patient's dietary intake and changes in physical condition, and contact relatives or medical institutions as needed.

[0787] Example 1

[0788] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0789] In modern society, early detection of dementia and understanding its progression are important issues. However, conventional methods require diagnosis at specialized medical institutions, and there is a lack of easy ways to monitor the progression of dementia in daily life. Furthermore, systems for understanding users' living conditions and providing appropriate support are not well developed. This creates the risk of overlooking the progression of dementia and the problem of not being able to provide appropriate support.

[0790] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0791] In this invention, the server includes means for generating a program for an interactive robot using a generative artificial intelligence model, means for executing the generated program and starting a conversation with a user, means for generating questions for determining the progression of dementia, means for asking the generated questions to the user, means for receiving the user's responses and analyzing them using natural language processing technology, and means for transmitting the analysis results to the server and communicating with relatives and medical institutions. This makes it possible to easily monitor the progression of dementia in everyday life and provide appropriate support.

[0792] A "generative artificial intelligence model" is an artificial intelligence technology that is trained in advance with large amounts of data and has the ability to generate natural language.

[0793] An "interactive robot" is a robot that uses natural language processing technology to imitate conversations with humans, providing information and asking questions through dialogue.

[0794] The "means for generating a program" is a method for using a generative artificial intelligence model to create scripts and codes for execution by an interactive robot.

[0795] The "means for executing the program" refers to a method for loading the generated program into the interactive robot and for actually running it.

[0796] The "means for generating questions" is a method for using a generative artificial intelligence model to create questions necessary to determine the progression of dementia.

[0797] The "means of asking a question to a user" refers to a method by which the interactive robot presents the generated question to the user by voice or text.

[0798] The "means for analyzing responses" is a method for analyzing responses from users using natural language processing technology to understand their meaning and intent.

[0799] The "means for transmitting the analysis results" is a method for transmitting the analysis results to a server and further communicating them to relatives or medical institutions.

[0800] "Means of linking with relatives and medical institutions" refers to methods for communicating information about the progression of dementia to appropriate parties.

[0801] "Means to support the use of e-commerce sites" refers to methods that provide users with the support they need when shopping online.

[0802] "Transportation arrangement support" refers to a method that provides the user with the support they need to arrange a taxi or other means of transportation.

[0803] "Means for understanding the living situation" refers to a method for analyzing the content of a user's conversations and behavioral patterns to understand the situation of daily life.

[0804] "Means for providing appropriate support" refers to methods for providing necessary assistance and services according to the user's living situation.

[0805] This invention is a system that uses a generative artificial intelligence model to generate a program for an interactive robot, determines the progression of dementia through natural conversation with the user, and communicates the results to relatives and medical institutions. Specific embodiments of this system are described below.

[0806] Program Generation

[0807] Subject: Server

[0808] The server generates a conversational robot program using a generative artificial intelligence model (e.g., GPT-3). The server uses a natural language processing library (e.g., spaCy or NLTK) to design the program so that the robot can have a natural conversation. Specifically, the server inputs the following prompt sentence into the generative artificial intelligence model:

[0809] Example prompt sentence:

[0810] "Generate a scenario in which the robot asks the user questions to determine the progression of dementia. Example questions include, 'What day is it today?' and 'What time is it now?'"

[0811] Running the program

[0812] Subject: Robot

[0813] The robot receives and executes the program provided by the server. As an initial setting, the robot prepares to start a conversation with the user. Specifically, the robot greets the user with a greeting such as "Hello, how are you doing today?" and creates a flow of conversation.

[0814] Question Generation

[0815] Subject: Robot

[0816] The robot uses a generative artificial intelligence model to generate questions to assess the progression of dementia, such as "What day is it today?" or "What time is it now?" This allows it to collect information to assess the user's cognitive function.

[0817] User interaction

[0818] Subject: Robot

[0819] The robot then asks the user the generated question. For example, the robot asks, "What day is it today?" and the user replies, "Today is Monday." In this way, the robot collects the necessary information through dialogue with the user.

[0820] Analysis of responses

[0821] Subject: Robot

[0822] The robot receives the user's responses and analyzes them using natural language processing technology. Specifically, it analyzes the content of the responses and calculates indicators to determine the progression of dementia. For example, it evaluates whether the user's responses are accurate and how long it takes to respond.

[0823] Sending the results

[0824] Subject: Robot

[0825] The robot sends the analysis results to a server, which then shares the results with relatives and medical institutions. For example, the server may notify relatives that the user's dementia is at a mild stage. This allows the relevant parties to take appropriate action.

[0826] Specific examples

[0827] Hardware and software used

[0828] Hardware: Interactive robot, server

[0829] Software: Generative AI models (GPT-3, etc.), natural language processing libraries (spaCy, NLTK)

[0830] This system makes it possible to easily monitor the progression of dementia in everyday life and provide appropriate support.

[0831] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0832] Step 1: Generate the program

[0833] The server generates a program for the interactive robot using a generative artificial intelligence model (such as GPT-3). The server inputs a prompt statement into the generative artificial intelligence model and generates a script for the interactive robot to execute. Specifically, the server inputs the prompt statement, "Please generate a scenario in which the robot asks the user questions to determine the progression of dementia." The input is the prompt statement, and the output is the script for the interactive robot.

[0834] Step 2: Run the program

[0835] The robot receives and executes a program provided by the server. As an initial setting, the robot prepares to start a conversation with the user. Specifically, the robot greets the user with a greeting such as "Hello, how are you doing today?" and creates a flow of conversation. The input is the program provided by the server, and the output is the initial conversation with the user.

[0836] Step 3: Generate questions

[0837] The robot uses a generative AI model to generate questions to determine the progression of dementia. For example, it generates questions such as "What day is it today?" or "What time is it now?" The input is the generative AI model and user information, and the output is the generated question.

[0838] Step 4: User interaction

[0839] The robot asks the user the generated question. For example, the robot asks "What day is it today?" and the user replies "Today is Monday." The input is the generated question, and the output is the user's response.

[0840] Step 5: Analyze the responses

[0841] The robot receives the user's response and analyzes it using natural language processing technology. Specifically, it analyzes the content of the response and calculates indicators to determine the progression of dementia. For example, it evaluates whether the user's response is accurate and the time it takes to respond. The input is the user's response, and the output is the analysis result.

[0842] Step 6: Sending the results

[0843] The robot sends the analysis results to a server, which then shares the results with relatives and medical institutions. For example, the server may notify relatives that the user's dementia is at a mild stage. The input is the analysis results, and the output is a notification to relatives and medical institutions.

[0844] In this way, the interactive robot can assess the progression of dementia through natural conversation and provide the necessary information to those involved.

[0845] (Application example 1)

[0846] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0847] There is a need to detect the progression of dementia in the elderly early and take appropriate measures. However, it is difficult to continuously monitor the progression of dementia in the elderly in their daily lives, and there is a lack of means to respond quickly when abnormalities occur. Therefore, an effective system is needed to ensure the safety of the elderly and to detect the progression of dementia early.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0849] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for sending a notification when an abnormality is detected, and means for being installed on a smartphone. This makes it possible to continuously monitor the progression of dementia in elderly people and to respond quickly when an abnormality occurs.

[0850] "Natural language processing technology" is a technology that enables computers to understand, generate, and respond to human language.

[0851] "Questions to determine the progression of dementia" are questions asked to elderly people to assess the progression of dementia.

[0852] The "means for analyzing the content of replies to the generated conversation" is a means for analyzing the content of replies to the generated conversation and determining the progression of dementia.

[0853] "Means of communicating the results of the assessment to relatives, medical institutions, etc." refers to means of notifying relatives and medical institutions of the results of the assessment of the progression of dementia.

[0854] The "means for sending a notification when an abnormality is detected" is a means for sending a notification to relatives or medical institutions when an abnormality is detected in the progression of dementia.

[0855] "Means for being installed on a smartphone" refers to means for being installed as an application on a smartphone.

[0856] "Means for understanding a user's living situation from the content of their conversations and patterns of behavior, etc." refers to means for analyzing the content of a user's conversations and patterns of behavior to understand their living situation.

[0857] A "security conversational assistant" is a conversational assistant that monitors the user's safety and provides appropriate support when necessary.

[0858] A system for implementing this invention is configured as an application installed on a smartphone. The system uses natural language processing technology to mimic human conversation and generate conversations including questions for assessing the progression of dementia. The system analyzes the responses in the generated conversation to determine the progression of dementia. The results of the assessment are shared with relatives, medical institutions, etc., and a notification is sent if an abnormality is detected.

[0859] Hardware and software used

[0860] Hardware: Smartphone

[0861] Software: OpenAI API, Python, smtplib (email sending library)

[0862] Data processing and calculation

[0863] The server processes the data in the following steps:

[0864] 1. Prepare a list of questions to ask the elderly, such as "What day is it today?" and "What time is it now?"

[0865] 2. Conversation generation: Using OpenAI's API, natural responses are generated to questions. A generative AI model (Generative Pretrained Transformer) is used to generate responses based on the prompt sentence.

[0866] 3. Analysis of response content: The generated response content is analyzed to determine the progression of dementia. For example, responses such as "I don't know" or "I forgot" are considered abnormal.

[0867] 4. Notification: If an abnormality is detected, a notification is sent to relatives or medical institutions by email. The notification is sent to the specified email address using smtplib.

[0868] Specific examples

[0869] Example questions and answers

[0870] Question: "What day is it today?"

[0871] Response: "Today is Monday."

[0872] Analysis result: normal

[0873] Prompt Sentence Examples

[0874] Question: What day is it today?

[0875] A: Today is Monday.

[0876] This system makes it possible to continuously monitor the progression of dementia in elderly people and respond quickly when abnormalities occur. It also analyzes the user's conversation content and behavioral patterns to understand their living situation and provide appropriate support as needed. As a security dialogue assistant, it monitors the user's safety and provides necessary support.

[0877] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0878] Step 1:

[0879] The server sets a question list. The question list contains questions to be asked to the elderly. For example, questions such as "What day is it today?" and "What time is it now?" are set. The input is the question list, and the output is the set question list. The server saves these questions for use in subsequent processing.

[0880] Step 2:

[0881] The server selects a question from a configured question list and generates a conversation using OpenAI's API. The input is the question selected from the question list, and the output is the generated response. The server uses a generative AI model (Generative Pretrained Transformer) to generate natural responses based on the prompt sentence. Specifically, the server sends an API request and receives a response.

[0882] Step 3:

[0883] The server analyzes the generated response content. The input is the generated response, and the output is the analysis result. The server analyzes the response content and determines the progression of dementia. For example, if the response includes "I don't know" or "I forgot," it is determined to be abnormal. Specifically, the server performs text analysis of the response content and determines whether or not there is an abnormality.

[0884] Step 4:

[0885] The server shares the analysis results with relatives, medical institutions, etc. The input is the analysis results, and the output is a notification message. If the server detects an abnormality, it sends a notification to relatives or medical institutions by email. Specifically, the server uses smtplib to send the notification to the configured email address.

[0886] Step 5:

[0887] The device operates as an application installed on a smartphone. Input is instructions from the server, and output is dialogue with the user. The device presents questions received from the server to the user and sends the user's responses to the server. Specifically, the device displays questions through a user interface and accepts voice and text input.

[0888] Step 6:

[0889] The user responds to questions presented by the terminal. The input is the question from the terminal, and the output is the user's response. The user responds by voice or text, and the response is sent to the server via the terminal. Specifically, the user inputs the response using the terminal's microphone or keyboard.

[0890] Example 2

[0891] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0892] In order to support the lives of elderly people and dementia patients, it is important to reduce the effort required to use various daily services and provide appropriate support. However, previous systems lacked the functionality to automatically perform specific operations based on user instructions, which placed a heavy burden on users. In addition, there were insufficient means to determine the progression of dementia and provide appropriate support.

[0893] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of responses to the generated conversation and determining the progression of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for receiving and analyzing user instructions, means for inputting prompt sentences into the generation AI model and generating specific operation procedures, means for transmitting the generated procedures to a robot and performing operations through the API of an e-commerce site or a taxi dispatch service, and means for notifying the user of the operation results. This makes it possible to efficiently support the lives of elderly people and dementia patients, reduce the burden on users, and provide appropriate support.

[0894] "Natural language processing technology" is a technology that uses computers to understand, interpret, and generate human language.

[0895] "Means for imitating conversation" refers to means for reproducing dialogue with humans using natural language processing technology.

[0896] "Questions to determine dementia progression" are questions designed to assess the progression of dementia.

[0897] The "means for analyzing the content of replies to the generated conversation" is a means for analyzing replies from users and evaluating their content.

[0898] The "means for determining the progression of dementia" is a means for evaluating the degree of progression of dementia based on the responses in the generated conversation.

[0899] "Means of communicating the results of the assessment to relatives, medical institutions, etc." refers to means of notifying relatives and medical institutions of the results of the assessment of the progression of dementia.

[0900] The "means for receiving and analyzing user instructions" refers to a means for receiving instructions from a user and analyzing the contents of those instructions.

[0901] A "generative AI model" is a model that uses artificial intelligence to generate responses or steps for a specific task.

[0902] A "prompt sentence" is an instruction sentence to be input into a generative AI model.

[0903] The "means for generating specific operating procedures" refers to a means for inputting a prompt sentence into a generative AI model and generating specific operating procedures.

[0904] A "robot" is a machine that automatically performs operations based on programmed instructions.

[0905] An "EC site" is a website that sells products and services over the Internet.

[0906] A "taxi dispatch service" is an online service for arranging a taxi.

[0907] "API" stands for Application Program Interface, an interface for exchanging functions and data between different software programs.

[0908] The "means for notifying the user of the operation result" is a means for notifying the user of the result of the operation performed by the robot.

[0909] MODE FOR CARRYING OUT THE INVENTION

[0910] The present invention is a system for supporting the daily lives of users, particularly elderly people and people with dementia. A specific embodiment of this system will be described below.

[0911] System configuration

[0912] This system consists of a server, user devices (PCs, smartphones, tablets), and a robot. The server uses natural language processing technology to analyze user instructions and generates specific operating procedures using a generative AI model. The robot performs operations based on the instructions received from the server through the API of an e-commerce site or taxi dispatch service.

[0913] Hardware and software used

[0914] Hardware: Servers, user devices (PCs, smartphones, tablets), robots

[0915] Software: Natural language processing technology, generative AI models (e.g., GPT-4), APIs for e-commerce sites, APIs for taxi dispatch services

[0916] Program processing

[0917] When the server receives instructions from the user, it analyzes them using natural language processing technology. Based on the analysis results, it inputs prompt statements into the generative AI model to generate specific operating procedures. The generated procedures are sent to the robot, which then performs the operations via the API of the e-commerce site or taxi dispatch service. The results of the operations are notified to the user via the server.

[0918] Specific examples

[0919] Example 1: Purchasing a product on an e-commerce site

[0920] User Instruction: "I want you to buy this product"

[0921] Server analysis: Extract product name and quantity

[0922] Prompt for generative AI model: "What are the steps to purchase a product on an e-commerce site? The product name is 'XYZ' and the quantity is 1."

[0923] Robot operation: Calls the API of the e-commerce site, searches for products, adds them to the cart, and completes the purchase process

[0924] Notify the user: "Your purchase of product 'XYZ' has been completed. Delivery is expected in 3 days."

[0925] Example 2: Arranging a taxi

[0926] User instruction: "Please arrange a taxi for me."

[0927] Server analysis: Extracting origin and destination

[0928] Prompt for generative AI model: "What are the steps to order a taxi? From point A to point B."

[0929] Robot operation: Call the API of a taxi dispatch service, specify the departure and destination points, and arrange a taxi.

[0930] Tell the user: "Your taxi has been booked and is expected to arrive in 10 minutes."

[0931] In this way, the server, user terminal, and robot work together to perform specific operations based on the user's instructions, making it a system that supports the user's daily life. This system can efficiently support the daily lives of elderly people and dementia patients, reduce the burden on the user, and provide appropriate support.

[0932] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0933] Step 1:

[0934] The user inputs instructions into the system using a terminal (PC, smartphone, tablet). For example, the user inputs "Please arrange a taxi." This instruction is sent to the server.

[0935] Input: User instructions (e.g., "Please arrange a taxi for me")

[0936] Output: User instructions sent to the server

[0937] Step 2:

[0938] The server analyzes the instructions received from the user, using natural language processing technology to understand the intent of the instructions and extract the necessary information (e.g., departure and destination points).

[0939] Input: User instructions

[0940] Data processing: Analyze instructions using natural language processing technology and extract necessary information

[0941] Output: Analysis results (e.g., origin and destination)

[0942] Step 3:

[0943] The server inputs a prompt sentence into the generative AI model based on the analysis results. For example, it generates a prompt sentence such as, "Please tell me the steps to order a taxi. The starting point is 'Point A' and the destination is 'Point B'."

[0944] Input: Analysis results

[0945] Data processing: Input prompts into the generative AI model to generate specific operating procedures

[0946] Output: Generated operating instructions

[0947] Step 4:

[0948] The server sends the generated operation instructions to the robot, which then follows the instructions received from the server and performs operations via the API of the e-commerce site or taxi dispatch service.

[0949] Input: Generated operating instructions

[0950] Data processing: Send instructions to the robot and execute operations through API

[0951] Output: Operation result

[0952] Step 5:

[0953] The robot calls the API of an e-commerce site or a taxi dispatch service to perform the specified operation, for example, calling the API of a taxi dispatch service to order a taxi based on the departure point and destination.

[0954] Input: Instructions sent to the robot

[0955] Specific behavior: Call the API and perform the specified operation

[0956] Output: Response from the API (e.g., taxi booking completed)

[0957] Step 6:

[0958] The server receives the operation result from the robot and notifies the user. For example, it sends a message to the user's terminal saying, "Taxi arrangements have been completed. Estimated arrival time is in 10 minutes."

[0959] Input: Operation result from the robot

[0960] Data processing: Converts the operation result into a message that notifies the user

[0961] Output: A notification message to the user

[0962] In this way, the server, terminal, and robot work together to perform specific operations based on the user's instructions, creating a system that supports the user's daily life.

[0963] (Application example 2)

[0964] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0965] In order to provide support for the elderly and dementia patients, there is a need for a means to easily access the services they need in their daily lives. However, these users often have difficulty with technical operations, especially when using services such as purchasing products on e-commerce sites or ordering a taxi. It is also important to properly assess the progression of dementia and provide the necessary support, but current systems have difficulty in comprehensively resolving these issues.

[0966] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the responses to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for searching for products on an e-commerce site based on the user's voice instructions and completing the purchase procedure, means for learning the user's preferences and past purchase history and recommending optimal products, and means for arranging a taxi based on the user's voice instructions. This enables elderly people and dementia patients to easily use services they need in their daily lives, and makes it possible to appropriately determine the progression of dementia and provide necessary support.

[0967] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0968] "Means for mimicking conversation" refers to technologies and algorithms that mimic human interactions and generate natural conversations.

[0969] "Dementia progression questions" are specific questions designed to assess the progression of dementia.

[0970] "Means for analyzing the content of responses generated from conversations" refers to technology for analyzing the user's responses and understanding their meaning and intent from their content.

[0971] "Means for determining the progression of dementia" refers to algorithms or systems for assessing the progression of dementia based on the user's responses.

[0972] "Means of communicating the assessment results to relatives, medical institutions, etc." refers to a means of communication for notifying relatives and medical institutions of the assessment results of the progression of dementia.

[0973] "Means for searching for products on an e-commerce site and completing the purchase procedure based on the user's voice instructions" refers to a system that receives the user's voice instructions, searches for products on an e-commerce site, and automatically completes the purchase procedure.

[0974] "A means of learning a user's preferences and past purchase history and recommending the most suitable products" is an algorithm that recommends the most suitable products to a user based on the user's past behavioral data.

[0975] The "means for dispatching a taxi based on a user's voice instructions" is a system for automatically dispatching a taxi based on a user's voice instructions.

[0976] To implement the present invention, the following system configuration and program are required.

[0977] System Configuration

[0978] The system is configured using the following hardware and software.

[0979] Hardware: Smartphone, microphone

[0980] Software: Python, Google Cloud Speech-to-Text API, Selenium

[0981] Program processing explanation

[0982] Voice Recognition

[0983] When a user speaks into the smartphone's microphone, the smartphone converts the voice into text using the Google Cloud Speech-to-Text API. This speech recognition technology allows the smartphone to accurately understand the user's commands.

[0984] Product search and purchase on e-commerce sites

[0985] Based on the text data obtained through speech recognition, the smartphone uses Selenium to access e-commerce sites and search for the specified product. The most suitable product is selected from the search results and the purchase process is automatically carried out. This makes it easy for even elderly or technology-inexperienced users to purchase products.

[0986] Product recommendation

[0987] It uses a generative AI model to learn users' past purchase history and preferences. This model analyzes user behavior data and recommends the best products, allowing users to easily find the products that suit them.

[0988] Taxi arrangements

[0989] When a user wants to order a taxi, they can access the taxi dispatch service based on their voice commands and call a taxi to the specified location. This function is also implemented using Selenium.

[0990] Specific examples

[0991] For example, if a user says, "Buy milk," the smartphone converts this voice command into text, searches for "milk" on an e-commerce site, and completes the purchase. Or, if a user says, "Order a taxi," the smartphone accesses a taxi dispatch service and calls a taxi to the specified location.

[0992] Prompt Sentence Examples

[0993] "Buy some milk."

[0994] "Arrange a taxi."

[0995] In this way, a system can be realized that uses voice instructions to support the lives of elderly people and users who are unfamiliar with technology.

[0996] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0997] Step 1:

[0998] The user issues a voice command into the smartphone's microphone. The input is the user's voice, and the output is audio data, which is captured by the smartphone's microphone.

[0999] Step 2:

[1000] The device sends the captured voice data to the Google Cloud Speech-to-Text API for conversion to text data. The input is voice data, and the output is text data. In this conversion process, the voice data is analyzed and the corresponding text is generated.

[1001] Step 3:

[1002] The device analyzes the converted text data and understands the user's instructions. The input is the text data, and the output is the analysis result. Specifically, it checks whether the text data contains keywords such as "purchase" or "taxi."

[1003] Step 4:

[1004] If the user's instruction is related to "purchase," the device uses Selenium to access the e-commerce site and search for the specified product. The input is the analysis result and product name, and the output is a search result page. Specifically, Selenium is used to operate the browser, enter the product name in the search box, and click the search button.

[1005] Step 5:

[1006] The terminal selects the most suitable product from the search results and carries out the purchase procedure. The input is the search result page, and the output is confirmation of purchase completion. Specifically, the user clicks on the most suitable product from the search result page and presses the purchase button to complete the purchase procedure.

[1007] Step 6:

[1008] If the user's instruction is related to "taxi," the device will use Selenium to access a taxi dispatch service and call a taxi to the specified location. The input is the analysis result and the specified location, and the output is confirmation of the taxi dispatch. Specifically, the device will access the taxi dispatch service page, enter the specified location, and click the button to dispatch a taxi.

[1009] Step 7:

[1010] The device uses a generative AI model to learn the user's past purchase history and preferences and recommend the most suitable products. The input is the user's past behavioral data, and the output is the recommended products. Specifically, the generative AI model analyzes the user's past purchase history and recommends the most suitable products to the user.

[1011] Step 8:

[1012] The server generates a conversation including questions to assess the progression of dementia and presents it to the user. The input is the user's past conversation data, and the output is the generated conversation. Specifically, natural language processing technology is used to generate questions to assess the progression of the user's dementia.

[1013] Step 9:

[1014] The server analyzes the generated conversation responses and determines the progression of dementia. The input is the user's responses, and the output is the evaluation result of the dementia progression. Specifically, the server analyzes the user's responses and applies an algorithm to evaluate the dementia progression.

[1015] Step 10:

[1016] The server then shares the results of the assessment with relatives, medical institutions, etc. The input is the assessment result of the dementia progression, and the output is a notification message. Specifically, a notification message is sent to relatives and medical institutions based on the assessment result.

[1017] Example 3

[1018] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1019] It is important to understand the living conditions of elderly people and dementia patients and provide appropriate support, but conventional methods have had difficulty understanding the situation in real time or detecting abnormalities. In addition, communication with relatives and medical institutions had to be done manually, making it difficult to respond quickly.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1021] In this invention, the server includes means for imitating human conversation using natural language processing technology, means for converting voice data into text data, means for analyzing the text data and extracting keywords for understanding the living situation, means for evaluating the living situation based on the extracted keywords, means for sharing the evaluation results with relatives, medical institutions, etc., and means for providing appropriate support as needed. This makes it possible to understand the user's living situation in real time and to promptly notify relatives and medical institutions if an abnormality is detected.

[1022] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1023] "Voice data" means data that is a digital recording of a user's speech.

[1024] "Text data" refers to data obtained by converting voice data into character information.

[1025] "Keywords" are words or phrases extracted from text data that are important for understanding a person's living situation.

[1026] "Lifestyle" is information that indicates the user's daily activities and health status.

[1027] "Evaluation" is the process of analyzing the user's living situation based on the extracted keywords and detecting abnormalities or specific patterns.

[1028] "Family" refers to the User's family or close relatives.

[1029] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1030] "Support" refers to the assistance and advice provided to users based on their life situation.

[1031] "Collaboration" is the process of notifying relatives and medical institutions of the evaluation results and sharing information.

[1032] The present invention is a system for understanding a user's living situation and providing appropriate support as needed. A specific embodiment of this system will be described below.

[1033] First, a user engages in everyday conversation using a smartphone or other device with a microphone. The device collects this conversation data in real time and sends it to a server. The server then converts the received voice data into text data using the Google Speech-to-Text API. This API provides highly accurate speech recognition and accurately transcribes the user's speech into text.

[1034] The server then analyzes the text data using spaCy, a natural language processing library. Specifically, it performs morphological analysis to extract keywords related to dietary intake and changes in physical condition. For example, keywords such as "eating," "physical condition," and "healthy" are included.

[1035] The server evaluates the user's lifestyle based on the extracted keywords. It uses a machine learning model with TensorFlow to compare the data with past data and detect abnormal patterns. For example, if the keyword "eat" does not appear repeatedly, it determines that there is something abnormal with the user's dietary intake.

[1036] If an abnormality is detected, the server will send a notification to relatives and medical institutions. The device will display an interface to provide appropriate support to the user. For example, if the user says, "I haven't had much of an appetite recently," the device will send that information to the server, which will then send a notification to relatives.

[1037] As a concrete example, consider the case where a user says, "I didn't eat anything yesterday." This utterance is collected through the smartphone's microphone and converted into text using the Google Speech-to-Text API. The text is then analyzed using spaCy, and the keyword "didn't eat" is extracted. Based on this information, the server determines that there is something abnormal with the user's dietary intake and sends a notification to their relatives.

[1038] Example prompts to input to a generative AI model:

[1039] User says: "I didn't eat anything yesterday."

[1040] Prompt: "The user said they didn't eat anything yesterday. Use this information to assess whether their dietary intake is abnormal and notify family members or healthcare providers as needed."

[1041] In this way, the system supports the user's health management. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1042] Step 1:

[1043] Users use smartphones or other devices with microphones to carry out everyday conversations. The devices collect this conversation data in real time and send it to a server. The input is the user's voice data, and the output is the voice data sent to the server.

[1044] Step 2:

[1045] The server converts the received voice data into text data using the Google Speech-to-Text API. The input is voice data and the output is text data. Specifically, the voice recognition engine analyzes the voice data and generates the corresponding text.

[1046] Step 3:

[1047] The server analyzes the text data using spaCy, a natural language processing library. The input is text data, and the output is extracted keywords. Specifically, it performs morphological analysis and extracts keywords related to dietary intake and changes in physical condition.

[1048] Step 4:

[1049] The server evaluates the user's living situation based on the extracted keywords. The input is the extracted keywords, and the output is the evaluation result of the living situation. Specifically, it uses a machine learning model using TensorFlow to compare past data and detect abnormal patterns.

[1050] Step 5:

[1051] If an abnormality is detected, the server sends a notification to relatives or a medical institution. The input is the evaluation result of the living situation, and the output is a notification message. Specifically, the notification system sends a message to relatives or a medical institution based on the evaluation result.

[1052] Step 6:

[1053] The terminal displays an interface to provide appropriate support to the user. The input is a notification message from the server, and the output is a support message that is displayed to the user. Specifically, the terminal displays a message on the screen saying, "There has been something unusual with your recent diet. Your relatives have been notified."

[1054] In this way, the system supports the user in managing their health.

[1055] (Application example 3)

[1056] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1057] Health management for the elderly is an important issue, and early detection of changes in health status, particularly the progression of dementia and changes in daily life, is required. However, current systems have difficulty analyzing the content of users' conversations and behavioral patterns in detail and providing appropriate support. Furthermore, there is a lack of means to quickly notify relatives and medical institutions. This makes it difficult to effectively manage the health status of the elderly.

[1058] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1059] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for analyzing the content of the user's conversation and behavioral patterns and monitoring the health condition, means for recording information about the health condition in a log, and means for sending notifications to relatives and medical institutions as necessary. This makes it possible to grasp the health condition of elderly people in detail and provide appropriate support.

[1060] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1061] "Conversational imitation methods" refers to technologies and algorithms that simulate human interaction.

[1062] "Questions to determine the progression of dementia" are questions designed to assess the cognitive function of the user.

[1063] "Means for analyzing the content of responses generated from conversations" refers to technology for analyzing the user's responses and understanding their meaning and intent from their content.

[1064] The "means for determining the progression of dementia" is a technology for assessing the progression of dementia based on the user's responses.

[1065] "Means of linking with relatives, medical institutions, etc." refers to technologies and methods for notifying relatives and medical institutions of the user's health condition and the progression of dementia.

[1066] "Means for analyzing user conversation content and behavioral patterns" refers to technology that analyzes users' everyday conversations and behavioral data and understands those patterns.

[1067] "Means for monitoring health status" refers to technology for continuously monitoring a user's health status and detecting abnormalities.

[1068] "Means for logging health information" refers to technology that records data about a user's health status for later reference.

[1069] "Means for sending notifications to relatives and medical institutions as necessary" refers to technology for sending notifications to relatives and medical institutions when an abnormality in the user's health condition is detected.

[1070] A system for carrying out the present invention includes a program for analyzing the content of a user's conversations and behavioral patterns and monitoring the health condition of the user. A specific embodiment of this system will be described below.

[1071] System configuration

[1072] The system includes a smartphone, a server, and a network infrastructure for sending notifications to relatives and medical institutions. The smartphone is installed with an application that uses voice recognition software and natural language processing technology. The server runs a program that analyzes the user's health status and sends notifications as needed.

[1073] Hardware and software used

[1074] Hardware: Smartphone microphone, server

[1075] Software: speech_recognition library, requests library, natural language processing technology

[1076] Data processing and calculation

[1077] 1. Acquiring voice data: Capture the user's voice in real time using the smartphone microphone.

[1078] 2. Speech Recognition: Use the speech_recognition library to convert the captured speech data into text.

[1079] 3. Text analysis: Using natural language processing technology, the converted text is analyzed to understand changes in dietary intake and physical condition.

[1080] 4. Health status recording: Send the analysis results to the server and record the health status information in a log.

[1081] 5. Send notifications: Call APIs to notify relatives and medical institutions, if necessary.

[1082] Specific examples

[1083] For example, if a user says, "I don't have an appetite today," the smartphone's microphone picks up the speech and the speech_recognition library converts it into text. Natural language processing technology then analyzes the text and extracts information about the meal. The server logs this information and notifies relatives or medical institutions as needed.

[1084] Example prompt sentence:

[1085] "If a user says, 'I don't have an appetite today,' the app will analyze that and send a notification to their relatives."

[1086] In this way, the system can gain a detailed understanding of the health status of elderly people and provide appropriate support.

[1087] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1088] Step 1:

[1089] The smartphone's microphone picks up the user's voice.

[1090] Input: User's voice

[1091] Output: Audio data

[1092] What it does: When a user speaks into their smartphone, the microphone captures their voice.

[1093] Step 2:

[1094] The smartphone's voice recognition software converts the voice data into text.

[1095] Input: Audio data

[1096] Output: Text data

[1097] What it does: It uses the speech_recognition library to analyze audio data and generate corresponding text.

[1098] Step 3:

[1099] The smartphone's natural language processing technology analyzes the text data to understand changes in dietary intake and physical condition.

[1100] Input: Text data

[1101] Output: Health status information

[1102] Specific operation: Using natural language processing technology, keywords related to diet and physical condition are extracted from text data and health status is evaluated.

[1103] Step 4:

[1104] The smartphone sends information about the health status to a server.

[1105] Input: Health information

[1106] Output: Data sent to the server

[1107] Specific operation: The smartphone sends data to a server via the Internet.

[1108] Step 5:

[1109] Logs health information received by the server.

[1110] Input: Data sent to the server

[1111] Output: Data recorded in the log file

[1112] Specific operation: The server analyzes the data received and records it in a log file along with the date and time.

[1113] Step 6:

[1114] The server will notify relatives and medical institutions as needed.

[1115] Input: Data recorded in the log file

[1116] Output: Notification sent to relatives and medical facilities

[1117] What it does: The server monitors the log file and sends notifications using the requests library if an anomaly is detected.

[1118] In this way, the system can gain a detailed understanding of the user's health status and provide appropriate support.

[1119] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1120] "Example 1"

[1121] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system imitates a conversation with a user and generates a conversation including questions for determining the progression of dementia. The replies in the generated conversation are analyzed to determine the progression of dementia. The determination result is shared with relatives, medical institutions, etc. Furthermore, this system includes an emotion engine that recognizes the user's emotions. This emotion engine analyzes emotions from the user's tone of voice, facial expressions, choice of words, etc. For example, if the user is feeling lonely, this information can be shared with relatives, medical institutions, etc., to provide appropriate support.

[1122] "Example 2"

[1123] Another embodiment of the present invention provides a system incorporating an emotion engine. This system supports users in using services such as e-commerce sites and taxi booking. The system also includes an emotion engine that recognizes the user's emotions. This emotion engine analyzes the user's emotions from the user's tone of voice, facial expressions, choice of words, etc. For example, if the user is feeling stressed, this information can be shared with relatives, medical institutions, etc., to provide appropriate support.

[1124] "Example 3"

[1125] As yet another embodiment of the present invention, a system incorporating an emotion engine is provided. This system understands the user's living situation from the content of their conversations, behavioral patterns, etc., and provides appropriate support as needed. Furthermore, this system includes an emotion engine that recognizes the user's emotions. This emotion engine analyzes the user's emotions from the user's tone of voice, facial expressions, choice of words, etc. For example, if the user is feeling happy, this information can be shared with relatives, medical institutions, etc., so that appropriate support can be provided.

[1126] The processing flow of each embodiment will be described below.

[1127] "Example 1"

[1128] Step 1: Generate a conversation with the user, including questions to determine the progression of dementia.

[1129] Step 2: Analyze the responses generated from the conversation to determine the progression of dementia.

[1130] Step 3: Communicate the results of the decision to relatives, medical institutions, etc.

[1131] Step 4: Recognize the user's emotions Using an emotion engine, analyze the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1132] Step 5: Share the analysis results with relatives, medical institutions, etc., and provide appropriate support.

[1133] "Example 2"

[1134] Step 1: Support users in using e-commerce sites and services such as arranging taxis.

[1135] Step 2: Recognize the user's emotions Using an emotion engine, analyze the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1136] Step 3: Share the analysis results with relatives, medical institutions, etc., and provide appropriate support.

[1137] "Example 3"

[1138] Step 1: Understand the user's living situation based on their conversations and behavioral patterns, and provide appropriate support as needed.

[1139] Step 2: Recognize the user's emotions Using an emotion engine, analyze the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1140] Step 3: Share the analysis results with relatives, medical institutions, etc., and provide appropriate support.

[1141] Example 1

[1142] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1143] Conventional interactive robot systems have difficulty imitating natural conversations with users, and have had issues with being unable to adequately analyze questions and responses to determine the progression of dementia. Furthermore, there was a lack of means to properly analyze the user's emotions and share the results with relatives and medical institutions. This made it difficult to properly understand the user's health and emotional state and provide the necessary support.

[1144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1145] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for recognizing the user's voice and converting it into text data, means for generating appropriate replies using a generative AI model, means for analyzing the user's emotions using an emotion engine, and means for sharing the analysis results with relatives, medical institutions, etc. This makes it possible to determine the progression of dementia through natural conversation with the user, appropriately analyze the user's emotional state, and provide necessary support.

[1146] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1147] "Questions for determining the progression of dementia" are questions designed to assess the cognitive function of the user.

[1148] A "generative AI model" is an artificial intelligence model that has been trained in advance with large amounts of data and is designed to generate natural language.

[1149] An "emotion engine" is a technology for analyzing emotions from a user's tone of voice, facial expressions, choice of words, etc.

[1150] "Means of communicating with relatives, medical institutions, etc." refers to means for notifying relatives and medical institutions of the results of analysis and judgment.

[1151] "Speech recognition" is a technology that converts a user's voice into text data.

[1152] "Text data" is character information converted using voice recognition technology.

[1153] The "means for analyzing the content of the reply" is a technique for analyzing the user's reply and evaluating its content.

[1154] The "means for generating an appropriate response" is a technique for generating a natural response to a user's input.

[1155] The "analysis results" are the results of analyzing the user's responses and emotional state.

[1156] "Means for providing support" refers to means for providing appropriate support and services according to the user's condition.

[1157] This invention uses an interactive robot system to realize natural conversation with the user and determine the progression of dementia. The system is composed of a combination of natural language processing technology, a generative AI model, and an emotion engine.

[1158] The server uses natural language processing technology to mimic conversations with users. Specifically, it uses a generative AI model (e.g., a Generative Pretrained Transformer) to generate appropriate responses to user input. This generative AI model has learned a large amount of data in advance and is capable of natural language generation.

[1159] The device uses speech recognition technology to recognize the user's voice and convert it into text data. For example, the Google Speech-to-Text API can be used as the speech recognition software. The converted text data is sent to a server and analyzed by a generative AI model.

[1160] The server generates questions to assess the progression of dementia, such as "What day is it today?" and "What time is it now?" These questions are designed to assess the user's cognitive function.

[1161] The device converts the generated question into speech using speech synthesis software (e.g., Amazon Polly) and outputs it to the user through the speaker. When the user responds to the question, the device again uses speech recognition technology to convert the response into text data and sends it to the server.

[1162] The server analyzes the responses and determines the progression of the user's dementia. For example, it checks whether the user correctly answered the day of the week. It also uses an emotion engine (for example, Microsoft Azure's emotion recognition API) to analyze the user's emotions from their tone of voice and facial expressions. The analysis results are shared with relatives and medical institutions.

[1163] As a specific example, when a user says "Hello" to an interactive robot, the device captures the audio and converts it into text data using the Google Speech-to-Text API. The server inputs the text data into GPT-3 and generates a response such as "Hello, how was your day?". The server then generates a question to determine the progression of dementia, such as "What day of the week is it today?". The device uses Amazon Polly to convert this question into speech and outputs it to the user through the speaker.

[1164] If the user responds with "Today is Monday," the device again uses the Google Speech-to-Text API to convert the response into text data. The server then analyzes the text data and verifies whether the user correctly answered the day of the week. It also uses Microsoft Azure's emotion recognition API to analyze the user's emotions from their tone of voice and facial expressions. For example, if the user sounds lonely, it will analyze their emotions. Finally, the server communicates the results of its judgment and emotion analysis to relatives and medical institutions via email or a dedicated app.

[1165] Example prompt sentence:

[1166] User: "What day is it today?"

[1167] Robot: "Today is Monday."

[1168] Server: "Analyze the user's responses and determine the progression of dementia."

[1169] In this way, the interactive robot can assess the progression of dementia through natural conversation with the user, properly analyze the user's emotional state, and provide the necessary support.

[1170] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1171] Step 1:

[1172] The user speaks to the interactive robot.

[1173] Specifically, the user speaks to the interactive robot saying "hello." The input is the user's voice, and the output is voice data.

[1174] Step 2:

[1175] The device recognizes the user's voice and converts it into text data.

[1176] Specifically, the device captures the user's voice through a microphone and converts the voice into text data using speech recognition software (e.g., Google Speech-to-Text API). The input is voice data and the output is text data.

[1177] Step 3:

[1178] The server uses a generative AI model to generate an appropriate response.

[1179] Specifically, the server inputs the received text data into a generative AI model (e.g., GPT-3) to generate an appropriate response. The input is text data, and the output is the generated response text data.

[1180] Step 4:

[1181] The server generates questions to determine the progression of dementia.

[1182] Specifically, the server generates questions to determine the progression of dementia. For example, it generates a question such as, "What day is it today?" The input is the user's basic information and past dialogue history, and the output is the question text data.

[1183] Step 5:

[1184] The terminal outputs the generated question to the user by voice.

[1185] Specifically, the device converts text data into speech using speech synthesis software (e.g., Amazon Polly) and outputs it to the user through a speaker. The input is the text data of the question, and the output is speech data.

[1186] Step 6:

[1187] The user responds to the question.

[1188] As a specific operation, the user responds, "Today is Monday." The input is the user's voice, and the output is voice data.

[1189] Step 7:

[1190] The terminal converts the user's response into text data.

[1191] Specifically, the terminal again uses voice recognition software to convert the user's response into text data, with the input being voice data and the output being text data.

[1192] Step 8:

[1193] The server analyzes the responses and determines the progression of dementia.

[1194] Specifically, the server analyzes the response and checks whether the user correctly answered the day of the week. The input is the text data of the response, and the output is the judgment result regarding the progression of dementia.

[1195] Step 9:

[1196] The server analyzes the user's emotions using an emotion engine.

[1197] Specifically, the server uses an emotion engine (for example, Microsoft Azure's emotion recognition API) to analyze the user's emotions from their tone of voice and facial expressions. The input is the user's voice data and video data, and the output is the emotion analysis results.

[1198] Step 10:

[1199] The server will share the judgment results and emotion analysis results with relatives and medical institutions.

[1200] Specifically, the server communicates the judgment results and emotion analysis results to relatives and medical institutions via email or a dedicated app. The input is the judgment results and emotion analysis results, and the output is notification data.

[1201] (Application example 1)

[1202] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1203] In modern society, it is important to detect the progression of dementia in the elderly early and provide appropriate support. However, there are limited means to determine the progression of dementia, and there is no system that can simultaneously analyze user emotions and security risks, making it difficult to provide comprehensive support.

[1204] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for analyzing the user's emotions, and means for determining security risks based on the emotion analysis results and sharing the results with relatives and security services. This makes it possible to determine the progression of dementia while simultaneously analyzing the user's emotions and security risks, thereby providing comprehensive support.

[1205] "Natural language processing technology" is a technology that enables computers to understand, generate, and respond to human language.

[1206] "Questions to assess dementia progression" are questions designed to assess a user's cognitive function, such as "What day is it today?" and "What time is it now?"

[1207] The "means for analyzing the content of responses generated from conversations" refers to technology that analyzes the responses provided by users and evaluates the progression of dementia from their content.

[1208] "Means of connecting with relatives, medical institutions, etc." refers to technology for notifying the user's relatives and medical institutions of the results of the system's judgment.

[1209] "Means for analyzing user emotions" refers to technology for analyzing emotions from the user's tone of voice, facial expressions, choice of words, etc.

[1210] "Means for determining security risks based on the results of emotion analysis" is a technology for evaluating security risks such as fear and anxiety felt by users based on the results of emotion analysis.

[1211] "Means for linking with relatives and security services" refers to technology that notifies the user's relatives and security services when a security risk is detected.

[1212] A system for implementing this invention uses natural language processing technology to mimic human conversations and generate conversations that include questions for assessing the progression of dementia. The system analyzes the responses in the generated conversations to determine the progression of dementia. The system also analyzes the user's emotions, determines security risks based on the emotion analysis results, and connects with relatives and security services.

[1213] Hardware and software used

[1214] Hardware: Smartphones, robots

[1215] Software: OpenAI API, IBM Watson Tone Analyzer API

[1216] Data processing and calculation

[1217] natural language generation

[1218] The server uses OpenAI's GPT-3 model to generate a conversation with the user, using prompts such as:

[1219] Example prompt sentence:

[1220] Hello, how was your day today?

[1221] Based on this prompt, the server receives responses from the user and generates natural conversation.

[1222] Emotion analysis

[1223] The server uses IBM Watson Tone Analyzer to analyze the emotions of the generated conversation, based on the user's tone of voice, facial expressions, and word choice, and produces results such as the following:

[1224] Example of sentiment analysis results:

[1225] We have detected that you are feeling anxious.

[1226] Determining security risks

[1227] The server evaluates the user's fear, anxiety, and other security risks based on the results of emotion analysis, and if a risk is detected, it notifies the user's relatives and security services.

[1228] Specific examples

[1229] 1. Talk to your users:

[1230] Server: "Hello, how was your day today?"

[1231] User: "I was a little tired today, but nothing too bad."

[1232] 2. Emotion analysis:

[1233] The server sends the user's response to the IBM Watson Tone Analyzer to analyze the emotion.

[1234] Result: "We have detected that you are feeling anxious."

[1235] 3. Determining security risks:

[1236] The server determines security risks based on the results of emotion analysis and, if necessary, connects with relatives or security services.

[1237] In this way, the system can simultaneously analyze the user's emotions and security risks while determining the progression of dementia, and provide comprehensive support.

[1238] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1239] Step 1:

[1240] The server generates a prompt to start a conversation with the user. Specifically, it uses OpenAI's GPT-3 model to generate a prompt such as "Hello, how was your day today?". This prompt is then presented to the user.

[1241] Input: None

[1242] Output: Prompt statement (e.g. "Hello, how was your day today?")

[1243] Step 2:

[1244] The user responds to prompts provided by the server, and the responses are sent to the server in text format.

[1245] Input: prompt statement

[1246] Output: User response (e.g., "I was a little tired today, but nothing too bad.")

[1247] Step 3:

[1248] The server receives the user's responses and generates a conversation using natural language processing technology, specifically OpenAI's GPT-3 model, to generate appropriate responses to the user's responses.

[1249] Input: User response

[1250] Output: The generated conversation (e.g., "That's tough. Is there anything I can help you with?")

[1251] Step 4:

[1252] The server analyzes the generated conversation responses to determine the progression of dementia. Specifically, it analyzes the user's responses and evaluates whether there are any signs of dementia progression.

[1253] Input: Generated conversation

[1254] Output: Assessment result of dementia progression (e.g., "No progression of dementia is observed.")

[1255] Step 5:

[1256] The server sends the user's response to the IBM Watson Tone Analyzer, which analyzes their emotions based on their tone of voice, facial expressions, and choice of words.

[1257] Input: User response

[1258] Output: Sentiment analysis results (e.g., "We have detected that the user is feeling anxious.")

[1259] Step 6:

[1260] The server determines security risks based on the results of emotion analysis. Specifically, it evaluates security risks such as fear and anxiety felt by the user based on the results of emotion analysis.

[1261] Input: Sentiment analysis results

[1262] Output: Security risk assessment result (e.g. "Security risk detected.")

[1263] Step 7:

[1264] If a security risk is detected, the server notifies the user's relatives and security services of the information by generating a notification message and sending it to them.

[1265] Input: Security risk assessment results

[1266] Output: Notification message (e.g. "A security risk has been detected due to user concerns.")

[1267] In this way, the system can simultaneously analyze the user's emotions and security risks while determining the progression of dementia, and provide comprehensive support.

[1268] Example 2

[1269] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1270] Elderly people and people with dementia face difficulties in their daily lives, making it difficult to use e-commerce sites or order taxis. It is also necessary to properly understand the stress and anxiety these users feel and provide the necessary support. It is also important to determine the progression of dementia early and take appropriate measures.

[1271] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for receiving instructions from the user and supporting the use of services, means for analyzing the user's emotions using an emotion engine, and means for providing appropriate support based on the analysis results. This enables elderly people and dementia patients to live their daily lives smoothly, and makes it possible to appropriately understand the user's emotional state and provide necessary support.

[1272] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1273] "Questions to determine dementia progression" are questions designed to assess the progression of dementia.

[1274] The "means for analyzing the content of responses to the generated conversation" refers to a means for analyzing responses provided by users and evaluating their content.

[1275] The "means for determining the progression of dementia" is a means for evaluating the degree of progression of dementia based on the responses in the generated conversation.

[1276] "Means of coordinating with relatives, medical institutions, etc." refers to the means of notifying relatives and medical institutions of the results of the judgment and encouraging them to take necessary measures.

[1277] "Means for receiving user instructions and supporting service usage" refers to means for receiving instructions from a user and providing a service based on those instructions.

[1278] The "emotion engine" is an engine that analyzes emotions from the user's tone of voice, facial expressions, choice of words, etc.

[1279] "Means for analyzing user emotions" refers to means for assessing the emotional state of a user using an emotion engine.

[1280] "Means for providing appropriate support" refers to means for providing necessary assistance and services to users based on the analysis results.

[1281] MODE FOR CARRYING OUT THE INVENTION

[1282] This invention is a system that supports users in using services such as e-commerce sites and taxi bookings, and further analyzes the user's emotions using an emotion engine to provide appropriate support. Specific embodiments of this system are described below.

[1283] Hardware and software used

[1284] Hardware: Servers, devices (smartphones, tablets, PCs), robots

[1285] Software: Emotion engine, e-commerce API, taxi booking API, generative AI model

[1286] System Overview

[1287] The user uses the device to input instructions, such as purchasing a product on an e-commerce site or ordering a taxi. The device then sends these instructions to the server, which then executes them using the appropriate API. The server also uses an emotion engine to analyze the user's emotions and contacts relatives or medical institutions as needed.

[1288] Specific examples

[1289] 1. Receive user instructions

[1290] The user uses their smartphone to say, "Buy this item," and the device uses voice recognition software to convert the command into text.

[1291] 2. Send instructions to the server

[1292] The terminal sends a text instruction to the server, "Purchase this product." The server receives this instruction and starts processing the product purchase using the EC site API.

[1293] 3. Purchasing products on an e-commerce site

[1294] The server calls the EC site API, adds the specified product to the cart, then completes the purchase and notifies the user of the purchase completion.

[1295] 4. Arranging a taxi

[1296] When a user requests a taxi, the server calls the taxi dispatch API and dispatches a taxi to the user's current location. The server then notifies the user of the estimated arrival time of the taxi.

[1297] 5. Emotion analysis using an emotion engine

[1298] If a user says, "I've been feeling stressed lately," the server inputs the voice data into the emotion engine and analyzes the user's stress level. The analysis result is, "The user is feeling high stress."

[1299] 6. Providing appropriate support

[1300] Based on the analysis results, the server notifies the user's relatives that the user is experiencing high levels of stress, and if necessary, contacts a medical institution and arranges for appropriate support.

[1301] Prompt Sentence Examples

[1302] "Please enter instructions for users to purchase products on your e-commerce site."

[1303] "Users can enter instructions to order a taxi."

[1304] "Analyze the user's emotions and provide instructions to provide appropriate support."

[1305] The above is a specific embodiment for carrying out the present invention. This system enables elderly people and dementia patients to smoothly lead their daily lives, and it is possible to properly grasp the user's emotional state and provide necessary support.

[1306] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1307] Step 1:

[1308] A user uses a terminal to input instructions for purchasing a product on an e-commerce site or ordering a taxi. Specifically, the user uses a smartphone to give a voice command such as "purchase this product." The terminal uses voice recognition software to convert this command into text. The input is voice data, and the output is text data.

[1309] Step 2:

[1310] The terminal sends a text instruction to the server. Specifically, the terminal sends the text data "Purchase this product" to the server. The input is text data, and the output is data sent to the server.

[1311] Step 3:

[1312] The server uses the EC site API to purchase the product specified by the user. Specifically, the server calls the API, adds the specified product to the cart, and completes the purchase. The input is text data, and the output is a notification that the purchase is complete.

[1313] Step 4:

[1314] When a user issues a taxi request, the server uses the taxi dispatch API to dispatch a taxi to the user's specified location. Specifically, the server calls the API to dispatch a taxi to the user's current location or a specified location. The input is text data, and the output is a notification of the estimated arrival time of the taxi.

[1315] Step 5:

[1316] The server uses an emotion engine to analyze emotions based on the user's tone of voice, facial expressions, and choice of words. Specifically, the server inputs voice data and image data into the emotion engine to analyze the user's emotional state. The input is voice data and image data, and the output is the emotion analysis results.

[1317] Step 6:

[1318] The server provides appropriate support based on the analysis results. Specifically, the server communicates the analysis results to relatives and medical institutions and arranges appropriate support for the user. The input is the emotion analysis results, and the output is notifications to relatives and medical institutions.

[1319] (Application example 2)

[1320] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1321] Elderly people and people with dementia often have difficulty using services such as e-commerce sites, arranging taxis, and ordering food delivery in their daily lives. Furthermore, when these users experience stress or anxiety, they lack the means to properly recognize their emotions and notify their relatives or medical institutions. This makes it difficult for them to receive appropriate support.

[1322] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1323] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing replies to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for ordering food delivery based on the user's voice instructions, means for analyzing the user's tone of voice and facial expressions to recognize emotions, and means for notifying relatives or medical institutions if the user is feeling stressed or anxious. This not only enables elderly people and dementia patients to easily use services necessary for their daily lives, but also makes it possible to appropriately recognize changes in emotions and quickly provide necessary support.

[1324] "Natural language processing technology" is a technology that allows computers to process and understand human language.

[1325] "Conversational imitation methods" refers to technologies and algorithms that simulate human interaction.

[1326] "Questions to assess dementia progression" are questions designed to assess the symptoms and progression of dementia.

[1327] The "means for analyzing the content of responses" refers to a technology for analyzing the user's responses and extracting specific information from the content.

[1328] "Means of communicating with relatives, medical institutions, etc." refers to technologies and methods for notifying relatives and medical institutions of analysis results and important information.

[1329] "Means for placing food delivery orders based on voice instructions" refers to technology that recognizes a user's voice commands and places food delivery orders based on them.

[1330] "Means for recognizing emotions by analyzing tone of voice and facial expressions" refers to technology that analyzes the tone of a user's voice and facial expressions to determine their emotional state.

[1331] "Means for notifying relatives or medical institutions when a user is feeling stressed or anxious" refers to technology that notifies relatives or medical institutions of the information when it is determined that the user is feeling stressed or anxious.

[1332] A system for carrying out this invention has the following configuration: The system includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the replies in the generated conversation and determining the progression of dementia, means for communicating the determination results to relatives, medical institutions, etc., means for ordering food delivery based on the user's voice instructions, means for analyzing the user's tone of voice and facial expressions and recognizing emotions, and means for notifying relatives or medical institutions if the user is feeling stressed or anxious.

[1333] Hardware and software used

[1334] Hardware: smartphone, microphone, camera

[1335] Software: Natural language processing libraries (e.g., speech_recognition), emotion recognition libraries (e.g., emotion_recognition), HTTP request libraries (e.g., requests), email sending libraries (e.g., smtplib)

[1336] Data processing and calculation

[1337] 1. Speech recognition: Use the smartphone's microphone to capture the user's voice and convert it into text using the speech_recognition library.

[1338] 2. Emotion analysis: The acquired voice data is analyzed using the emotion_recognition library to determine the user's emotional state.

[1339] 3. Food delivery order: Based on voice instructions, an order is sent to the food delivery API using an HTTP request.

[1340] 4. Notification: If the user is feeling stressed or anxious, the smtplib library is used to send notification emails to relatives or medical institutions.

[1341] Specific examples

[1342] When a user says "I want to order pizza," the smartphone's microphone picks up the audio and converts it into text using the speech_recognition library. Next, the emotion_recognition library is used to analyze the user's emotions from the audio data. If the user is feeling stressed or anxious, that information is sent via email to their relatives or a medical institution. Finally, an HTTP request is used to send the order to a food delivery API, and the results are displayed to the user.

[1343] Prompt Sentence Examples

[1344] If a user says "I want to order a pizza," convert that speech to text and send the order to a food delivery API. Also, analyze the user's tone of voice to determine their emotions and send an email notification to their relatives if they are feeling stressed or anxious.

[1345] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1346] Step 1:

[1347] The user issues a voice command into the smartphone's microphone.

[1348] Input: User's voice commands

[1349] Output: Audio data

[1350] Specific operation: The user says, "I want to order a pizza."

[1351] Step 2:

[1352] The device receives the voice data and converts it to text using the speech_recognition library.

[1353] Input: Audio data

[1354] Output: Text data

[1355] What it does: Your smartphone records the voice data and uses the speech_recognition library to convert it into text: "I'd like to order a pizza."

[1356] Step 3:

[1357] The device inputs the text data into the emotion_recognition library to analyze the user's emotions.

[1358] Input: Text data

[1359] Output: Emotion data (e.g., stress, anxiety, joy, etc.)

[1360] Specific behavior: Analyze text data and determine whether the user is experiencing stress.

[1361] Step 4:

[1362] If the device detects that the user is feeling stressed or anxious based on emotional data, it uses the smtplib library to send a notification email to relatives or medical institutions.

[1363] Input: Emotion data

[1364] Output: Notification email

[1365] Specific operation: If it is determined that the user is feeling stressed, an email stating "The user is feeling stressed" is sent to the user's relatives.

[1366] Step 5:

[1367] The device uses the text data to send an order to the food delivery API using an HTTP request.

[1368] Input: Text data

[1369] Output: Order result data

[1370] Specific operation: Sends text data to the food delivery API and makes a request to "order pizza."

[1371] Step 6:

[1372] The terminal displays the order result data from the food delivery API to the user.

[1373] Input: Order result data

[1374] Output: User notification

[1375] Specific behavior: Receives a response from the food delivery API and notifies the user whether the order was successful or not.

[1376] Example 3

[1377] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1378] There is a need to understand the living conditions and health status of elderly people and dementia patients in real time and provide appropriate support. However, conventional systems have the problem of being unable to fully analyze the user's emotions and behavioral patterns, making it difficult to provide appropriate support. In addition, there is a lack of means to quickly notify relatives or medical institutions of changes in the user's physical condition or emotions, which can delay response in emergencies.

[1379] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of responses to the generated conversation and determining the progression state of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for analyzing the content of the user's conversation and behavioral patterns and understanding the living situation, means for analyzing the user's emotions from the user's tone of voice and facial expressions, and means for contacting relatives or medical institutions as necessary. This makes it possible to understand the living situation and health condition of elderly people and dementia patients in real time and provide appropriate support.

[1380] "Natural language processing technology" is a technology that uses computers to understand, interpret, and generate human language.

[1381] "Means to mimic conversation" refers to technology that reproduces a dialogue with the user in a natural way.

[1382] "Questions to determine the progression of dementia" are questions designed to assess the cognitive function of the user.

[1383] The "means for analyzing the content of responses generated from conversations" refers to technology that analyzes the user's responses and evaluates the progression of dementia from their content.

[1384] "Means for determining the progression of dementia" refers to technology that evaluates the progression of dementia based on the user's responses and behavior.

[1385] "Means of connecting with relatives, medical institutions, etc." refers to technology for notifying relatives and medical institutions of the user's health condition and living situation.

[1386] "Means for analyzing user conversation content and behavioral patterns" refers to technology that analyzes users' statements and behavior to understand their living situation.

[1387] "Means for understanding living conditions" refers to technology that analyzes the user's daily life situation and provides the necessary support.

[1388] "Means for analyzing emotions from a user's tone of voice and facial expression" refers to a technology that analyzes a user's tone of voice and facial expression and evaluates their emotions.

[1389] "Means of contacting relatives or medical institutions as needed" refers to technology that allows for quick contact with relatives or medical institutions depending on the user's condition.

[1390] This invention is a system that analyzes the content of a user's conversations and behavioral patterns, grasps their living situation, and provides appropriate support as needed. Furthermore, by combining it with an emotion engine, it can recognize the user's emotions and provide more accurate support.

[1391] Hardware and software used

[1392] Hardware: microphones, cameras, servers

[1393] Software: speech recognition software (e.g., Google Speech-to-Text), sentiment analysis engines (e.g., Affectiva), database management systems (e.g., MySQL)

[1394] Data processing and calculation flow

[1395] 1. Audio data collection:

[1396] The device collects the user's conversation in real time through a microphone.

[1397] The collected voice data is immediately sent to the server.

[1398] 2. Analysis of audio data:

[1399] The server uses voice recognition software to convert the voice data into text data.

[1400] The converted text data is stored in a database.

[1401] 3. Behavioral Pattern Analysis:

[1402] The device uses a camera to capture the user's actions.

[1403] The captured video data is transmitted to a server.

[1404] The server uses video analysis software to extract user behavior patterns.

[1405] 4. Emotion Analysis:

[1406] The server uses an emotion analysis engine to analyze the user's emotions from their tone of voice and facial expressions.

[1407] The analysis results are stored in a database.

[1408] 5. Understanding your living situation:

[1409] The server integrates the voice data, behavioral patterns, and emotional data collected to understand the user's living situation.

[1410] For example, it detects changes in food intake and physical condition.

[1411] 6. Providing Support:

[1412] The server will contact relatives and medical institutions as needed.

[1413] For example, if a user complains of feeling unwell, a medical institution will be notified.

[1414] Specific examples

[1415] Example 1: When a user says, "I have a headache today"

[1416] The server uses voice recognition software to generate text data such as "I have a headache today."

[1417] An emotion analysis engine analyzes the level of pain from the user's tone of voice.

[1418] The server uses this information to contact medical institutions and provide appropriate support.

[1419] Example 2: When a user smiles and says, "Today was fun"

[1420] The server uses voice recognition software to generate text data such as "Today was fun."

[1421] The emotion analysis engine analyzes the user's smile to determine their joy.

[1422] The server then communicates this information to relatives, sharing the user's positive feelings.

[1423] Prompt Sentence Examples

[1424] Prompt 1: "Please explain how you would respond if a user complained of feeling unwell."

[1425] Prompt 2: "If the user experiences joy, explain how they would communicate this information to their next of kin."

[1426] This system makes it possible to grasp the living conditions and health conditions of elderly people and dementia patients in real time and provide appropriate support. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1427] Step 1:

[1428] The device collects the user's conversation in real time through a microphone. The input is the user's voice, and the output is digital voice data. Specifically, when a user says, "I don't remember what I ate today," the device captures this voice and sends it to the server as digital voice data.

[1429] Step 2:

[1430] The server uses speech recognition software to convert the voice data into text data. The input is digital voice data, and the output is text data. Specifically, the server converts the voice data "I can't remember what I ate today" into text, and stores the converted text "I can't remember what I ate today" in a database.

[1431] Step 3:

[1432] The device uses a camera to capture the user's actions. The input is the user's actions, and the output is video data. Specifically, the camera captures the user walking around the room and sends the video data to the server.

[1433] Step 4:

[1434] The server uses video analysis software to extract the user's behavioral patterns. The input is video data, and the output is behavioral pattern data. Specifically, the server analyzes the video data and detects that the user frequently visits the kitchen.

[1435] Step 5:

[1436] The server uses an emotion analysis engine to analyze emotions from the user's tone of voice and facial expression. The input is audio and video data, and the output is emotional data. In concrete terms, when a user says with a smile, "Today was fun," the server analyzes the tone of voice and facial expression and determines that the user is feeling happy.

[1437] Step 6:

[1438] The server integrates collected voice data, behavioral patterns, and emotional data to understand the user's living situation. The input is text data, behavioral pattern data, and emotional data, and the output is living situation data. Specifically, the server integrates the text data "I don't remember what I ate today" with the behavioral pattern of frequently going to the kitchen, and determines that the user tends to forget to eat.

[1439] Step 7:

[1440] The server will contact relatives or medical institutions as necessary. The input is living situation data and the output is notification data. Specifically, if the server detects text data such as "I have a headache today" and determines through emotion analysis that the pain is severe, it will contact a medical institution and report the user's poor health.

[1441] (Application example 3)

[1442] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1443] To ensure the safety and improve the quality of life of elderly people and people living alone, it is important to understand their health status and emotions from their daily conversations and behavioral patterns, and provide appropriate support as needed. However, conventional systems have had difficulty accurately analyzing users' emotions and connecting with relatives and security services in real time. Furthermore, there was a lack of means to comprehensively understand the user's living situation, making it difficult to provide appropriate support.

[1444] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for imitating a conversation with a human using natural language processing technology; means for generating a conversation including questions for determining the progression of dementia; means for analyzing the responses to the generated conversation and determining the progression of dementia; means for sharing the determination result with relatives, medical institutions, etc.; means for understanding the user's living situation from the content of the user's conversation and behavioral patterns, etc., and providing appropriate support as needed; means including an emotion engine for recognizing the user's emotions; means for analyzing emotions from the user's tone of voice, facial expressions, choice of words, etc.; and means for sending an alert to relatives or security services when the user's emotions are in a specific state. This makes it possible to ensure the safety of elderly people and people living alone and improve their quality of life.

[1445] "Natural language processing technology" is a technology that allows computers to process and understand human language.

[1446] "Questions to determine the progression of dementia" are questions designed to assess the cognitive function of the user.

[1447] A "means for generating a conversation" is a technology for automatically creating a dialogue with a user.

[1448] The "means for analyzing the content of responses" refers to a technique for analyzing the user's responses and evaluating their content.

[1449] The "means for determining the progression of dementia" is a technology for assessing the progression of dementia based on the user's responses.

[1450] "Means of communication with relatives, medical institutions, etc." refers to technology for notifying relatives and medical institutions of the results of the judgment.

[1451] "Means of understanding a user's living situation from the content of their conversations and patterns of behavior" refers to technology that analyzes the user's daily conversations and behavior to understand their living situation.

[1452] "Means for providing appropriate support" refers to technology that provides necessary assistance according to the user's living situation.

[1453] An "emotion engine" is a technology for analyzing user emotions.

[1454] "Means of analyzing emotions from tone of voice, facial expressions, choice of words, etc." is a technology for assessing emotions based on the user's tone of voice, facial expressions, and choice of words.

[1455] "Means for sending alerts to relatives or security services when a user's emotions are in a specific state" refers to technology for sending alerts to relatives or security services when a user's emotions are determined to be in a specific state.

[1456] "Smart glasses" are wearable devices that users can wear to display information and input voice.

[1457] As an example of an embodiment of this invention, we will explain the "Security Guardian" application installed on smart glasses. This system analyzes the user's conversation content, behavioral patterns, and even emotions, and issues security alerts as necessary.

[1458] Hardware and software used

[1459] Hardware

[1460] Smart glasses: Wearable devices with voice input and camera capabilities.

[1461] software

[1462] Python: A programming language used to implement programs.

[1463] speech_recognition library: A library for performing speech recognition.

[1464] emotion_recognition library: A library for performing emotion analysis.

[1465] requests library: A library for data integration.

[1466] Data processing and calculation

[1467] Voice input

[1468] The microphone in the smart glasses is used to capture the user's voice, which is then converted to text using the speech_recognition library.

[1469] Voice Recognition

[1470] To convert the voice data to text, we use the speech_recognition library, which will capture what the user is saying in text form.

[1471] Emotion analysis

[1472] The acquired text data is then subjected to emotion analysis using the emotion_recognition library, which evaluates the user's emotions based on their tone of voice, facial expressions, and word choice.

[1473] Data Linkage

[1474] If the emotion is in a certain state (e.g. "anger", "fear", "sadness"), the requests library is used to send an alert to relatives or security services. The alert includes the emotional state and text data.

[1475] Specific examples

[1476] For example, if an elderly person is alone and says "help," emotion analysis can detect "fear," and this information can be immediately sent as an alert to relatives or security services, allowing for a prompt response.

[1477] Prompt Sentence Examples

[1478] Write a Python program that, if the user says "help," will use emotion analysis to detect "fear" and send an alert to relatives or security services.

[1479] In this way, the mode for carrying out the invention can be concretely demonstrated. This system can ensure the safety of elderly people and people living alone and improve their quality of life.

[1480] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1481] Step 1:

[1482] A user puts on the smart glasses and starts voice input. The microphone of the smart glasses picks up the user's voice. The input is the user's voice data, and the output is the acquisition of the voice data.

[1483] Step 2:

[1484] The speech data acquired by the device is converted into text using the speech_recognition library. The input is speech data and the output is text data. Specifically, the speech recognition engine analyzes the speech data and generates the corresponding text.

[1485] Step 3:

[1486] The device performs emotion analysis on the generated text data using the emotion_recognition library. The input is text data, and the output is emotion data. Specifically, the emotion recognition engine analyzes the content of the text, tone of voice, word choice, etc. to evaluate the user's emotional state.

[1487] Step 4:

[1488] If the server detects a specific emotional state (e.g., "anger," "fear," or "sadness") based on the emotion data, it uses the requests library to send an alert to relatives or security services. The input is emotion data, and the output is sending an alert. Specifically, if the emotion data meets certain conditions, an alert message is generated and sent to the specified contacts.

[1489] Step 5:

[1490] The server logs the alert sending result. The input is the alert sending result and the output is the log data. Specifically, it checks whether the alert was sent successfully and records the result in the log file.

[1491] In this way, a system is realized that ensures the safety of users and allows for quick response when necessary.

[1492] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1494] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1495] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1496] [Third embodiment]

[1497] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1498] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1499] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1500] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1501] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1503] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1504] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1507] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1508] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1509] "Example 1"

[1510] As an example embodiment of the present invention, an interactive robot is provided. This robot uses natural language processing technology to imitate conversations with humans. Specifically, a natural language generation model such as a generative AI (generative pretrained transformer) is used to generate natural conversations with humans. The robot also generates conversations that include questions for determining the progression of dementia. For example, the robot asks questions such as "What day is it today?" and "What time is it now?" and analyzes the replies. This analysis determines the progression of dementia. The results of the determination are shared with relatives, medical institutions, etc.

[1511] "Example 2"

[1512] Furthermore, as an example of the present invention, a function is provided to assist users in using services such as e-commerce sites and arranging taxis. Specifically, the robot purchases products on e-commerce sites and arranges taxis based on the user's instructions. This makes it possible to support the daily lives of elderly people.

[1513] "Example 3"

[1514] In addition, as an example of the present invention, a function is provided that grasps the user's living situation from the content of the user's conversations and behavioral patterns, etc., and provides appropriate support as needed. Specifically, the function grasps the user's dietary intake and changes in physical condition from the content of the user's conversations, and connects with relatives, medical institutions, etc. as needed. This enables health management for the elderly.

[1515] The processing flow of each embodiment will be described below.

[1516] "Example 1"

[1517] Step 1: The conversational robot starts and starts a conversation with the user.

[1518] Step 2: The robot uses natural language processing technology to understand what the user is saying and generate an appropriate response.

[1519] Step 3: The robot generates a conversation containing questions to determine the progression of dementia and asks the user questions.

[1520] Step 4: The robot receives the user's response and analyzes it.

[1521] Step 5: Based on the analysis results, the progression of dementia is determined and the results are shared with relatives, medical institutions, etc.

[1522] "Example 2"

[1523] Step 1: The user instructs the robot to use a service such as purchasing a product on an e-commerce site or arranging a taxi.

[1524] Step 2: The robot understands the user's instructions and follows them to make purchases on an e-commerce site or order a taxi.

[1525] "Example 3"

[1526] Step 1: The robot analyzes the user's conversation content, behavioral patterns, etc. to understand their living situation.

[1527] Step 2: Based on the analysis results, the robot will provide appropriate support as needed. For example, it will understand the patient's dietary intake and changes in physical condition, and contact relatives or medical institutions as needed.

[1528] Example 1

[1529] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1530] In modern society, early detection of dementia and understanding its progression are important issues. However, conventional methods require diagnosis at specialized medical institutions, and there is a lack of easy ways to monitor the progression of dementia in daily life. Furthermore, systems for understanding users' living conditions and providing appropriate support are not well developed. This creates the risk of overlooking the progression of dementia and the problem of not being able to provide appropriate support.

[1531] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1532] In this invention, the server includes means for generating a program for an interactive robot using a generative artificial intelligence model, means for executing the generated program and starting a conversation with a user, means for generating questions for determining the progression of dementia, means for asking the generated questions to the user, means for receiving the user's responses and analyzing them using natural language processing technology, and means for transmitting the analysis results to the server and communicating with relatives and medical institutions. This makes it possible to easily monitor the progression of dementia in everyday life and provide appropriate support.

[1533] A "generative artificial intelligence model" is an artificial intelligence technology that is trained in advance with large amounts of data and has the ability to generate natural language.

[1534] An "interactive robot" is a robot that uses natural language processing technology to imitate conversations with humans, providing information and asking questions through dialogue.

[1535] The "means for generating a program" is a method for using a generative artificial intelligence model to create scripts and codes for execution by an interactive robot.

[1536] The "means for executing the program" refers to a method for loading the generated program into the interactive robot and for actually running it.

[1537] The "means for generating questions" is a method for using a generative artificial intelligence model to create questions necessary to determine the progression of dementia.

[1538] The "means of asking a question to a user" refers to a method by which the interactive robot presents the generated question to the user by voice or text.

[1539] The "means for analyzing responses" is a method for analyzing responses from users using natural language processing technology to understand their meaning and intent.

[1540] The "means for transmitting the analysis results" is a method for transmitting the analysis results to a server and further communicating them to relatives or medical institutions.

[1541] "Means of linking with relatives and medical institutions" refers to methods for communicating information about the progression of dementia to appropriate parties.

[1542] "Means to support the use of e-commerce sites" refers to methods that provide users with the support they need when shopping online.

[1543] "Transportation arrangement support" refers to a method that provides the user with the support they need to arrange a taxi or other means of transportation.

[1544] "Means for understanding the living situation" refers to a method for analyzing the content of a user's conversations and behavioral patterns to understand the situation of daily life.

[1545] "Means for providing appropriate support" refers to methods for providing necessary assistance and services according to the user's living situation.

[1546] This invention is a system that uses a generative artificial intelligence model to generate a program for an interactive robot, determines the progression of dementia through natural conversation with the user, and communicates the results to relatives and medical institutions. Specific embodiments of this system are described below.

[1547] Program Generation

[1548] Subject: Server

[1549] The server generates a conversational robot program using a generative artificial intelligence model (e.g., GPT-3). The server uses a natural language processing library (e.g., spaCy or NLTK) to design the program so that the robot can have a natural conversation. Specifically, the server inputs the following prompt sentence into the generative artificial intelligence model:

[1550] Example prompt sentence:

[1551] "Generate a scenario in which the robot asks the user questions to determine the progression of dementia. Example questions include, 'What day is it today?' and 'What time is it now?'"

[1552] Running the program

[1553] Subject: Robot

[1554] The robot receives and executes the program provided by the server. As an initial setting, the robot prepares to start a conversation with the user. Specifically, the robot greets the user with a greeting such as "Hello, how are you doing today?" and creates a flow of conversation.

[1555] Question Generation

[1556] Subject: Robot

[1557] The robot uses a generative artificial intelligence model to generate questions to assess the progression of dementia, such as "What day is it today?" or "What time is it now?" This allows it to collect information to assess the user's cognitive function.

[1558] User interaction

[1559] Subject: Robot

[1560] The robot then asks the user the generated question. For example, the robot asks, "What day is it today?" and the user replies, "Today is Monday." In this way, the robot collects the necessary information through dialogue with the user.

[1561] Analysis of responses

[1562] Subject: Robot

[1563] The robot receives the user's responses and analyzes them using natural language processing technology. Specifically, it analyzes the content of the responses and calculates indicators to determine the progression of dementia. For example, it evaluates whether the user's responses are accurate and how long it takes to respond.

[1564] Sending the results

[1565] Subject: Robot

[1566] The robot sends the analysis results to a server, which then shares the results with relatives and medical institutions. For example, the server may notify relatives that the user's dementia is at a mild stage. This allows the relevant parties to take appropriate action.

[1567] Specific examples

[1568] Hardware and software used

[1569] Hardware: Interactive robot, server

[1570] Software: Generative AI models (GPT-3, etc.), natural language processing libraries (spaCy, NLTK)

[1571] This system makes it possible to easily monitor the progression of dementia in everyday life and provide appropriate support.

[1572] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1573] Step 1: Generate the program

[1574] The server generates a program for the interactive robot using a generative artificial intelligence model (such as GPT-3). The server inputs a prompt statement into the generative artificial intelligence model and generates a script for the interactive robot to execute. Specifically, the server inputs the prompt statement, "Please generate a scenario in which the robot asks the user questions to determine the progression of dementia." The input is the prompt statement, and the output is the script for the interactive robot.

[1575] Step 2: Run the program

[1576] The robot receives and executes a program provided by the server. As an initial setting, the robot prepares to start a conversation with the user. Specifically, the robot greets the user with a greeting such as "Hello, how are you doing today?" and creates a flow of conversation. The input is the program provided by the server, and the output is the initial conversation with the user.

[1577] Step 3: Generate questions

[1578] The robot uses a generative AI model to generate questions to determine the progression of dementia. For example, it generates questions such as "What day is it today?" or "What time is it now?" The input is the generative AI model and user information, and the output is the generated question.

[1579] Step 4: User interaction

[1580] The robot asks the user the generated question. For example, the robot asks "What day is it today?" and the user replies "Today is Monday." The input is the generated question, and the output is the user's response.

[1581] Step 5: Analyze the responses

[1582] The robot receives the user's response and analyzes it using natural language processing technology. Specifically, it analyzes the content of the response and calculates indicators to determine the progression of dementia. For example, it evaluates whether the user's response is accurate and the time it takes to respond. The input is the user's response, and the output is the analysis result.

[1583] Step 6: Sending the results

[1584] The robot sends the analysis results to a server, which then shares the results with relatives and medical institutions. For example, the server may notify relatives that the user's dementia is at a mild stage. The input is the analysis results, and the output is a notification to relatives and medical institutions.

[1585] In this way, the interactive robot can assess the progression of dementia through natural conversation and provide the necessary information to those involved.

[1586] (Application example 1)

[1587] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1588] There is a need to detect the progression of dementia in the elderly early and take appropriate measures. However, it is difficult to continuously monitor the progression of dementia in the elderly in their daily lives, and there is a lack of means to respond quickly when abnormalities occur. Therefore, an effective system is needed to ensure the safety of the elderly and to detect the progression of dementia early.

[1589] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1590] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of replies to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for sending a notification when an abnormality is detected, and means for being installed on a smartphone. This makes it possible to continuously monitor the progression of dementia in elderly people and to respond quickly when an abnormality occurs.

[1591] "Natural language processing technology" is a technology that enables computers to understand, generate, and respond to human language.

[1592] "Questions to determine the progression of dementia" are questions asked to elderly people to assess the progression of dementia.

[1593] The "means for analyzing the content of replies to the generated conversation" is a means for analyzing the content of replies to the generated conversation and determining the progression of dementia.

[1594] "Means of communicating the results of the assessment to relatives, medical institutions, etc." refers to means of notifying relatives and medical institutions of the results of the assessment of the progression of dementia.

[1595] The "means for sending a notification when an abnormality is detected" is a means for sending a notification to relatives or medical institutions when an abnormality is detected in the progression of dementia.

[1596] "Means for being installed on a smartphone" refers to means for being installed as an application on a smartphone.

[1597] "Means for understanding a user's living situation from the content of their conversations and patterns of behavior, etc." refers to means for analyzing the content of a user's conversations and patterns of behavior to understand their living situation.

[1598] A "security conversational assistant" is a conversational assistant that monitors the user's safety and provides appropriate support when necessary.

[1599] A system for implementing this invention is configured as an application installed on a smartphone. The system uses natural language processing technology to mimic human conversation and generate conversations including questions for assessing the progression of dementia. The system analyzes the responses in the generated conversation to determine the progression of dementia. The results of the assessment are shared with relatives, medical institutions, etc., and a notification is sent if an abnormality is detected.

[1600] Hardware and software used

[1601] Hardware: Smartphone

[1602] Software: OpenAI API, Python, smtplib (email sending library)

[1603] Data processing and calculation

[1604] The server processes the data in the following steps:

[1605] 1. Prepare a list of questions to ask the elderly, such as "What day is it today?" and "What time is it now?"

[1606] 2. Conversation generation: Using OpenAI's API, natural responses are generated to questions. A generative AI model (Generative Pretrained Transformer) is used to generate responses based on the prompt sentence.

[1607] 3. Analysis of response content: The generated response content is analyzed to determine the progression of dementia. For example, responses such as "I don't know" or "I forgot" are considered abnormal.

[1608] 4. Notification: If an abnormality is detected, a notification is sent to relatives or medical institutions by email. The notification is sent to the specified email address using smtplib.

[1609] Specific examples

[1610] Example questions and answers

[1611] Question: "What day is it today?"

[1612] Response: "Today is Monday."

[1613] Analysis result: normal

[1614] Prompt Sentence Examples

[1615] Question: What day is it today?

[1616] A: Today is Monday.

[1617] This system makes it possible to continuously monitor the progression of dementia in elderly people and respond quickly when abnormalities occur. It also analyzes the user's conversation content and behavioral patterns to understand their living situation and provide appropriate support as needed. As a security dialogue assistant, it monitors the user's safety and provides necessary support.

[1618] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1619] Step 1:

[1620] The server sets a question list. The question list contains questions to be asked to the elderly. For example, questions such as "What day is it today?" and "What time is it now?" are set. The input is the question list, and the output is the set question list. The server saves these questions for use in subsequent processing.

[1621] Step 2:

[1622] The server selects a question from a configured question list and generates a conversation using OpenAI's API. The input is the question selected from the question list, and the output is the generated response. The server uses a generative AI model (Generative Pretrained Transformer) to generate natural responses based on the prompt sentence. Specifically, the server sends an API request and receives a response.

[1623] Step 3:

[1624] The server analyzes the generated response content. The input is the generated response, and the output is the analysis result. The server analyzes the response content and determines the progression of dementia. For example, if the response includes "I don't know" or "I forgot," it is determined to be abnormal. Specifically, the server performs text analysis of the response content and determines whether or not there is an abnormality.

[1625] Step 4:

[1626] The server shares the analysis results with relatives, medical institutions, etc. The input is the analysis results, and the output is a notification message. If the server detects an abnormality, it sends a notification to relatives or medical institutions by email. Specifically, the server uses smtplib to send the notification to the configured email address.

[1627] Step 5:

[1628] The device operates as an application installed on a smartphone. Input is instructions from the server, and output is dialogue with the user. The device presents questions received from the server to the user and sends the user's responses to the server. Specifically, the device displays questions through a user interface and accepts voice and text input.

[1629] Step 6:

[1630] The user responds to questions presented by the terminal. The input is the question from the terminal, and the output is the user's response. The user responds by voice or text, and the response is sent to the server via the terminal. Specifically, the user inputs the response using the terminal's microphone or keyboard.

[1631] Example 2

[1632] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1633] In order to support the lives of elderly people and dementia patients, it is important to reduce the effort required to use various daily services and provide appropriate support. However, previous systems lacked the functionality to automatically perform specific operations based on user instructions, which placed a heavy burden on users. In addition, there were insufficient means to determine the progression of dementia and provide appropriate support.

[1634] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the content of responses to the generated conversation and determining the progression of dementia, means for communicating the determination result to relatives, medical institutions, etc., means for receiving and analyzing user instructions, means for inputting prompt sentences into the generation AI model and generating specific operation procedures, means for transmitting the generated procedures to a robot and performing operations through the API of an e-commerce site or a taxi dispatch service, and means for notifying the user of the operation results. This makes it possible to efficiently support the lives of elderly people and dementia patients, reduce the burden on users, and provide appropriate support.

[1635] "Natural language processing technology" is a technology that uses computers to understand, interpret, and generate human language.

[1636] "Means for imitating conversation" refers to means for reproducing dialogue with humans using natural language processing technology.

[1637] "Questions to determine dementia progression" are questions designed to assess the progression of dementia.

[1638] The "means for analyzing the content of replies to the generated conversation" is a means for analyzing replies from users and evaluating their content.

[1639] The "means for determining the progression of dementia" is a means for evaluating the degree of progression of dementia based on the responses in the generated conversation.

[1640] "Means of communicating the results of the assessment to relatives, medical institutions, etc." refers to means of notifying relatives and medical institutions of the results of the assessment of the progression of dementia.

[1641] The "means for receiving and analyzing user instructions" refers to a means for receiving instructions from a user and analyzing the contents of those instructions.

[1642] A "generative AI model" is a model that uses artificial intelligence to generate responses or steps for a specific task.

[1643] A "prompt sentence" is an instruction sentence to be input into a generative AI model.

[1644] The "means for generating specific operating procedures" refers to a means for inputting a prompt sentence into a generative AI model and generating specific operating procedures.

[1645] A "robot" is a machine that automatically performs operations based on programmed instructions.

[1646] An "EC site" is a website that sells products and services over the Internet.

[1647] A "taxi dispatch service" is an online service for arranging a taxi.

[1648] "API" stands for Application Program Interface, an interface for exchanging functions and data between different software programs.

[1649] The "means for notifying the user of the operation result" is a means for notifying the user of the result of the operation performed by the robot.

[1650] MODE FOR CARRYING OUT THE INVENTION

[1651] The present invention is a system for supporting the daily lives of users, particularly elderly people and people with dementia. A specific embodiment of this system will be described below.

[1652] System configuration

[1653] This system consists of a server, user devices (PCs, smartphones, tablets), and a robot. The server uses natural language processing technology to analyze user instructions and generates specific operating procedures using a generative AI model. The robot performs operations based on the instructions received from the server through the API of an e-commerce site or taxi dispatch service.

[1654] Hardware and software used

[1655] Hardware: Servers, user devices (PCs, smartphones, tablets), robots

[1656] Software: Natural language processing technology, generative AI models (e.g., GPT-4), APIs for e-commerce sites, APIs for taxi dispatch services

[1657] Program processing

[1658] When the server receives instructions from the user, it analyzes them using natural language processing technology. Based on the analysis results, it inputs prompt statements into the generative AI model to generate specific operating procedures. The generated procedures are sent to the robot, which then performs the operations via the API of the e-commerce site or taxi dispatch service. The results of the operations are notified to the user via the server.

[1659] Specific examples

[1660] Example 1: Purchasing a product on an e-commerce site

[1661] User Instruction: "I want you to buy this product"

[1662] Server analysis: Extract product name and quantity

[1663] Prompt for generative AI model: "What are the steps to purchase a product on an e-commerce site? The product name is 'XYZ' and the quantity is 1."

[1664] Robot operation: Calls the API of the e-commerce site, searches for products, adds them to the cart, and completes the purchase process

[1665] Notify the user: "Your purchase of product 'XYZ' has been completed. Delivery is expected in 3 days."

[1666] Example 2: Arranging a taxi

[1667] User instruction: "Please arrange a taxi for me."

[1668] Server analysis: Extracting origin and destination

[1669] Prompt for generative AI model: "What are the steps to order a taxi? From point A to point B."

[1670] Robot operation: Call the API of a taxi dispatch service, specify the departure and destination points, and arrange a taxi.

[1671] Tell the user: "Your taxi has been booked and is expected to arrive in 10 minutes."

[1672] In this way, the server, user terminal, and robot work together to perform specific operations based on the user's instructions, making it a system that supports the user's daily life. This system can efficiently support the daily lives of elderly people and dementia patients, reduce the burden on the user, and provide appropriate support.

[1673] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1674] Step 1:

[1675] The user inputs instructions into the system using a terminal (PC, smartphone, tablet). For example, the user inputs "Please arrange a taxi." This instruction is sent to the server.

[1676] Input: User instructions (e.g., "Please arrange a taxi for me")

[1677] Output: User instructions sent to the server

[1678] Step 2:

[1679] The server analyzes the instructions received from the user, using natural language processing technology to understand the intent of the instructions and extract the necessary information (e.g., departure and destination points).

[1680] Input: User instructions

[1681] Data processing: Analyze instructions using natural language processing technology and extract necessary information

[1682] Output: Analysis results (e.g., origin and destination)

[1683] Step 3:

[1684] The server inputs a prompt sentence into the generative AI model based on the analysis results. For example, it generates a prompt sentence such as, "Please tell me the steps to order a taxi. The starting point is 'Point A' and the destination is 'Point B'."

[1685] Input: Analysis results

[1686] Data processing: Input prompts into the generative AI model to generate specific operating procedures

[1687] Output: Generated operating instructions

[1688] Step 4:

[1689] The server sends the generated operation instructions to the robot, which then follows the instructions received from the server and performs operations via the API of the e-commerce site or taxi dispatch service.

[1690] Input: Generated operating instructions

[1691] Data processing: Send instructions to the robot and execute operations through API

[1692] Output: Operation result

[1693] Step 5:

[1694] The robot calls the API of an e-commerce site or a taxi dispatch service to perform the specified operation, for example, calling the API of a taxi dispatch service to order a taxi based on the departure point and destination.

[1695] Input: Instructions sent to the robot

[1696] Specific behavior: Call the API and perform the specified operation

[1697] Output: Response from the API (e.g., taxi booking completed)

[1698] Step 6:

[1699] The server receives the operation result from the robot and notifies the user. For example, it sends a message to the user's terminal saying, "Taxi arrangements have been completed. Estimated arrival time is in 10 minutes."

[1700] Input: Operation result from the robot

[1701] Data processing: Converts the operation result into a message that notifies the user

[1702] Output: A notification message to the user

[1703] In this way, the server, terminal, and robot work together to perform specific operations based on the user's instructions, creating a system that supports the user's daily life.

[1704] (Application example 2)

[1705] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1706] In order to provide support for the elderly and dementia patients, there is a need for a means to easily access the services they need in their daily lives. However, these users often have difficulty with technical operations, especially when using services such as purchasing products on e-commerce sites or ordering a taxi. It is also important to properly assess the progression of dementia and provide the necessary support, but current systems have difficulty in comprehensively resolving these issues.

[1707] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the responses to the generated conversation and determining the progression of dementia, means for sharing the determination result with relatives, medical institutions, etc., means for searching for products on an e-commerce site based on the user's voice instructions and completing the purchase procedure, means for learning the user's preferences and past purchase history and recommending optimal products, and means for arranging a taxi based on the user's voice instructions. This enables elderly people and dementia patients to easily use services they need in their daily lives, and makes it possible to appropriately determine the progression of dementia and provide necessary support.

[1708] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1709] "Means for mimicking conversation" refers to technologies and algorithms that mimic human interactions and generate natural conversations.

[1710] "Dementia progression questions" are specific questions designed to assess the progression of dementia.

[1711] "Means for analyzing the content of responses generated from conversations" refers to technology for analyzing the user's responses and understanding their meaning and intent from their content.

[1712] "Means for determining the progression of dementia" refers to algorithms or systems for assessing the progression of dementia based on the user's responses.

[1713] "Means of communicating the assessment results to relatives, medical institutions, etc." refers to a means of communication for notifying relatives and medical institutions of the assessment results of the progression of dementia.

[1714] "Means for searching for products on an e-commerce site and completing the purchase procedure based on the user's voice instructions" refers to a system that receives the user's voice instructions, searches for products on an e-commerce site, and automatically completes the purchase procedure.

[1715] "A means of learning a user's preferences and past purchase history and recommending the most suitable products" is an algorithm that recommends the most suitable products to a user based on the user's past behavioral data.

[1716] The "means for dispatching a taxi based on a user's voice instructions" is a system for automatically dispatching a taxi based on a user's voice instructions.

[1717] To implement the present invention, the following system configuration and program are required.

[1718] System Configuration

[1719] The system is configured using the following hardware and software.

[1720] Hardware: Smartphone, microphone

[1721] Software: Python, Google Cloud Speech-to-Text API, Selenium

[1722] Program processing explanation

[1723] Voice Recognition

[1724] When a user speaks into the smartphone's microphone, the smartphone converts the voice into text using the Google Cloud Speech-to-Text API. This speech recognition technology allows the smartphone to accurately understand the user's commands.

[1725] Product search and purchase on e-commerce sites

[1726] Based on the text data obtained through speech recognition, the smartphone uses Selenium to access e-commerce sites and search for the specified product. The most suitable product is selected from the search results and the purchase process is automatically carried out. This makes it easy for even elderly or technology-inexperienced users to purchase products.

[1727] Product recommendation

[1728] It uses a generative AI model to learn users' past purchase history and preferences. This model analyzes user behavior data and recommends the best products, allowing users to easily find the products that suit them.

[1729] Taxi arrangements

[1730] When a user wants to order a taxi, they can access the taxi dispatch service based on their voice commands and call a taxi to the specified location. This function is also implemented using Selenium.

[1731] Specific examples

[1732] For example, if a user says, "Buy milk," the smartphone converts this voice command into text, searches for "milk" on an e-commerce site, and completes the purchase. Or, if a user says, "Order a taxi," the smartphone accesses a taxi dispatch service and calls a taxi to the specified location.

[1733] Prompt Sentence Examples

[1734] "Buy some milk."

[1735] "Arrange a taxi."

[1736] In this way, a system can be realized that uses voice instructions to support the lives of elderly people and users who are unfamiliar with technology.

[1737] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1738] Step 1:

[1739] The user issues a voice command into the smartphone's microphone. The input is the user's voice, and the output is audio data, which is captured by the smartphone's microphone.

[1740] Step 2:

[1741] The device sends the captured voice data to the Google Cloud Speech-to-Text API for conversion to text data. The input is voice data, and the output is text data. In this conversion process, the voice data is analyzed and the corresponding text is generated.

[1742] Step 3:

[1743] The device analyzes the converted text data and understands the user's instructions. The input is the text data, and the output is the analysis result. Specifically, it checks whether the text data contains keywords such as "purchase" or "taxi."

[1744] Step 4:

[1745] If the user's instruction is related to "purchase," the device uses Selenium to access the e-commerce site and search for the specified product. The input is the analysis result and product name, and the output is a search result page. Specifically, Selenium is used to operate the browser, enter the product name in the search box, and click the search button.

[1746] Step 5:

[1747] The terminal selects the most suitable product from the search results and carries out the purchase procedure. The input is the search result page, and the output is confirmation of purchase completion. Specifically, the user clicks on the most suitable product from the search result page and presses the purchase button to complete the purchase procedure.

[1748] Step 6:

[1749] If the user's instruction is related to "taxi," the device will use Selenium to access a taxi dispatch service and call a taxi to the specified location. The input is the analysis result and the specified location, and the output is confirmation of the taxi dispatch. Specifically, the device will access the taxi dispatch service page, enter the specified location, and click the button to dispatch a taxi.

[1750] Step 7:

[1751] The device uses a generative AI model to learn the user's past purchase history and preferences and recommend the most suitable products. The input is the user's past behavioral data, and the output is the recommended products. Specifically, the generative AI model analyzes the user's past purchase history and recommends the most suitable products to the user.

[1752] Step 8:

[1753] The server generates a conversation including questions to assess the progression of dementia and presents it to the user. The input is the user's past conversation data, and the output is the generated conversation. Specifically, natural language processing technology is used to generate questions to assess the progression of the user's dementia.

[1754] Step 9:

[1755] The server analyzes the generated conversation responses and determines the progression of dementia. The input is the user's responses, and the output is the evaluation result of the dementia progression. Specifically, the server analyzes the user's responses and applies an algorithm to evaluate the dementia progression.

[1756] Step 10:

[1757] The server then shares the results of the assessment with relatives, medical institutions, etc. The input is the assessment result of the dementia progression, and the output is a notification message. Specifically, a notification message is sent to relatives and medical institutions based on the assessment result.

[1758] Example 3

[1759] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1760] It is important to understand the living conditions of elderly people and dementia patients and provide appropriate support, but conventional methods have had difficulty understanding the situation in real time or detecting abnormalities. In addition, communication with relatives and medical institutions had to be done manually, making it difficult to respond quickly.

[1761] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1762] In this invention, the server includes means for imitating human conversation using natural language processing technology, means for converting voice data into text data, means for analyzing the text data and extracting keywords for understanding the living situation, means for evaluating the living situation based on the extracted keywords, means for sharing the evaluation results with relatives, medical institutions, etc., and means for providing appropriate support as needed. This makes it possible to understand the user's living situation in real time and to promptly notify relatives and medical institutions if an abnormality is detected.

[1763] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1764] "Voice data" means data that is a digital recording of a user's speech.

[1765] "Text data" refers to data obtained by converting voice data into character information.

[1766] "Keywords" are words or phrases extracted from text data that are important for understanding a person's living situation.

[1767] "Lifestyle" is information that indicates the user's daily activities and health status.

[1768] "Evaluation" is the process of analyzing the user's living situation based on the extracted keywords and detecting abnormalities or specific patterns.

[1769] "Family" refers to the User's family or close relatives.

[1770] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1771] "Support" refers to the assistance and advice provided to users based on their life situation.

[1772] "Collaboration" is the process of notifying relatives and medical institutions of the evaluation results and sharing information.

[1773] The present invention is a system for understanding a user's living situation and providing appropriate support as needed. A specific embodiment of this system will be described below.

[1774] First, a user engages in everyday conversation using a smartphone or other device with a microphone. The device collects this conversation data in real time and sends it to a server. The server then converts the received voice data into text data using the Google Speech-to-Text API. This API provides highly accurate speech recognition and accurately transcribes the user's speech into text.

[1775] The server then analyzes the text data using spaCy, a natural language processing library. Specifically, it performs morphological analysis to extract keywords related to dietary intake and changes in physical condition. For example, keywords such as "eating," "physical condition," and "healthy" are included.

[1776] The server evaluates the user's lifestyle based on the extracted keywords. It uses a machine learning model with TensorFlow to compare the data with past data and detect abnormal patterns. For example, if the keyword "eat" does not appear repeatedly, it determines that there is something abnormal with the user's dietary intake.

[1777] If an abnormality is detected, the server will send a notification to relatives and medical institutions. The device will display an interface to provide appropriate support to the user. For example, if the user says, "I haven't had much of an appetite recently," the device will send that information to the server, which will then send a notification to relatives.

[1778] As a concrete example, consider the case where a user says, "I didn't eat anything yesterday." This utterance is collected through the smartphone's microphone and converted into text using the Google Speech-to-Text API. The text is then analyzed using spaCy, and the keyword "didn't eat" is extracted. Based on this information, the server determines that there is something abnormal with the user's dietary intake and sends a notification to their relatives.

[1779] Example prompts to input to a generative AI model:

[1780] User says: "I didn't eat anything yesterday."

[1781] Prompt: "The user said they didn't eat anything yesterday. Use this information to assess whether their dietary intake is abnormal and notify family members or healthcare providers as needed."

[1782] In this way, the system supports the user's health management. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1783] Step 1:

[1784] Users use smartphones or other devices with microphones to carry out everyday conversations. The devices collect this conversation data in real time and send it to a server. The input is the user's voice data, and the output is the voice data sent to the server.

[1785] Step 2:

[1786] The server converts the received voice data into text data using the Google Speech-to-Text API. The input is voice data and the output is text data. Specifically, the voice recognition engine analyzes the voice data and generates the corresponding text.

[1787] Step 3:

[1788] The server analyzes the text data using spaCy, a natural language processing library. The input is text data, and the output is extracted keywords. Specifically, it performs morphological analysis and extracts keywords related to dietary intake and changes in physical condition.

[1789] Step 4:

[1790] The server evaluates the user's living situation based on the extracted keywords. The input is the extracted keywords, and the output is the evaluation result of the living situation. Specifically, it uses a machine learning model using TensorFlow to compare past data and detect abnormal patterns.

[1791] Step 5:

[1792] If an abnormality is detected, the server sends a notification to relatives or a medical institution. The input is the evaluation result of the living situation, and the output is a notification message. Specifically, the notification system sends a message to relatives or a medical institution based on the evaluation result.

[1793] Step 6:

[1794] The terminal displays an interface to provide appropriate support to the user. The input is a notification message from the server, and the output is a support message that is displayed to the user. Specifically, the terminal displays a message on the screen saying, "There has been something unusual with your recent diet. Your relatives have been notified."

[1795] In this way, the system supports the user in managing their health.

[1796] (Application example 3)

[1797] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1798] Health management for the elderly is an important issue, and early detection of changes in health status, particularly the progression of dementia and changes in daily life, is required. However, current systems have difficulty analyzing the content of users' conversations and behavioral patterns in detail and providing appropriate support. Furthermore, there is a lack of means to quickly notify relatives and medical institutions. This makes it difficult to effectively manage the health status of the elderly.

[1799] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1800] In this invention, the server includes means for imitating a conversation with a human using natural language processing technology, means for generating a conversation including questions for determining the progression of dementia, means for analyzing the replies to the generated conversation and determining the progression of dementia, means for sharing the determination results with relatives, medical institutions, etc., means for analyzing the content of the user's conversation and behavioral patterns and monitoring the health condition, means for recording information about the health condition in a log, and means for sending notifications to relatives and medical institutions as necessary. This makes it possible to grasp the health condition of elderly people in detail and provide appropriate support.

[1801] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1802] "Conversational imitation methods" refers to technologies and algorithms that simulate human interaction. ...

Claims

1. A means for mimicking human conversation using natural language processing techniques; A means for generating a conversation including questions for determining the progression of dementia; A means for recognizing a user's voice and converting it into text data; means for analyzing an emotional state of a user using an emotion engine; A means for generating an appropriate response using a generative AI model based on a prompt sentence using the generated response content of the conversation and the analyzed emotional state; A means for analyzing the conversation and determining the progression of dementia; A system including a means for communicating the result of the judgment to the user's relatives or a medical institution.

2. A means for mimicking human conversation using natural language processing techniques; A means for generating a conversation including questions for determining the progression of dementia; means for analyzing an emotional state of a user using an emotion engine; A means for analyzing the generated response content of the conversation and the emotional state to determine a progression state of dementia; A means for communicating the result of the judgment to the user's relatives or a medical institution; A means for placing a food delivery order based on a voice instruction from the user; means for notifying said relatives or said medical institution if said user is feeling stressed or anxious; A system including:

3. A means for mimicking human conversation using natural language processing techniques; A means for generating a conversation including questions for determining the progression of dementia; means for recognizing an emotional state of a user from the user's tone of voice, facial expression, and choice of words; A means of analyzing the responses generated from the conversation to determine the progression of dementia; and A means for communicating the result of the judgment to relatives or a medical institution; A means for understanding the user's living situation based on the content of the user's conversations and behavioral patterns and providing support; means for sending an alert to relatives or security services if said emotional state is in a particular state; A system including:

Images