Memory degradation detection and improvement

Abstract

Memory degradation detection and assessment includes capturing human utterances using a voice interface and generating a corpus of human utterances for a user, selecting human utterances from the corpus of human utterances based on a meaning of the human utterances determined by a computer processor through natural language processing. Contextual information for one or more of the corpus of human utterances is determined based on data generated in response to signals sensed by one or more sensing devices operatively coupled with the computer processor. Patterns in the corpus of human utterances are identified based on pattern recognition performed by the computer processor using one or more machine learning models. Changes in memory functioning of the user are identified based on the pattern recognition. The changes identified based on the contextual information are classified as to whether the changes are likely due to a memory impairment of the user.

Description

Background Technology

[0001] This disclosure relates to voice-based computer-human interaction, and more specifically, to voice-based computer-human interaction using artificial intelligence (AI).

[0002] Artificial intelligence computer systems are growing in both number and complexity, and frequently involve voice-based computer-human interaction. Voice-based interaction systems are commonly found in homes and other environments to control automated equipment and systems through voice prompts and commands. Voice-based interaction systems are used for countless purposes beyond automation control. For example, through voice-based interaction systems, users can check the weather, get driving directions, search for favorite songs, and so on. Users can even request reminders for things they've forgotten, such as where they left a set of car keys. Summary of the Invention

[0003] In one or more embodiments, a method includes capturing human speech with a voice interface and generating a corpus of human speech for a corresponding user, the corpus comprising human speech selected from a plurality of human speech based on the meaning of the human speech, the meaning being determined by a computer processor through natural language processing of the human speech. The method includes determining contextual information corresponding to one or more human speech items in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. The method includes identifying patterns in the corpus of human speech based on pattern recognition performed by the computer processor using one or more machine learning models. The method includes identifying changes in a user's memory function based on pattern recognition. The method includes classifying whether changes in memory function are likely due to a user's memory impairment based on the contextual information corresponding to one or more human speech items in the corpus.

[0004] In one or more embodiments, a system includes a processor configured to initiate operations. The operations include capturing a plurality of human utterances using a voice interface and generating a corpus of human utterances for a corresponding user, the corpus including human utterances selected from the plurality of human utterances based on the meaning of the human utterances, the meaning being determined by a computer processor through natural language processing of the human utterances. The operations include determining contextual information corresponding to one or more human utterances in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. The operations include identifying patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models. The operations include identifying changes in a user's memory function based on pattern recognition. The operations include classifying whether changes in memory function are likely due to a user's memory impairment based on contextual information corresponding to one or more human utterances in the corpus.

[0005] In one or more embodiments, a computer program product includes one or more computer-readable storage media having instructions stored thereon. The instructions are executable by a processor to initialize operations. The operations include capturing a plurality of human utterances using a voice interface and generating a corpus of human utterances for a corresponding user, the corpus including human utterances selected from the plurality of human utterances based on the meaning of the human utterances, the meaning being determined by a computer processor through natural language processing of the human utterances. The operations include determining contextual information corresponding to the one or more human utterances in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. The operations include identifying patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models. The operations include identifying changes in a user's memory function based on pattern recognition. The operations include classifying whether changes in memory function are likely due to a user's memory impairment based on contextual information corresponding to the one or more human utterances in the corpus.

[0006] This overview section is provided only to introduce certain concepts and not to identify any key or essential features of the claimed subject matter. Other features of the invention will become apparent from the accompanying drawings and the detailed description below. Attached Figure Description

[0007] The arrangement of the invention is illustrated in the accompanying drawings by way of example. However, the drawings should not be construed as limiting the invention. The arrangement is made only with respect to the specific implementation shown. Various aspects and advantages will become apparent upon review of the following detailed description and with reference to the accompanying drawings.

[0008] Figure 1 An example computing environment according to one embodiment is shown.

[0009] Figure 2 An example memory evaluator system according to one embodiment is shown.

[0010] Figure 3 This is a flowchart of a memory monitoring and evaluation method according to one embodiment.

[0011] Figure 4 A cloud computing environment according to one embodiment is shown.

[0012] Figure 5 An abstract model layer according to one embodiment is shown.

[0013] Figure 6 A cloud computing node according to one embodiment is shown. Detailed Implementation

[0014] Although this disclosure concludes with claims defining novel features, it is believed that the description of the various features described herein, in conjunction with the accompanying drawings, will be better understood from the following considerations. The processes, machines, manufactures, and any variations thereof described herein are provided for illustrative purposes. The specific structural and functional details described in this disclosure should not be construed as limiting, but merely as the basis for the claims and as a representative basis for teaching those skilled in the art to use the features described in virtually any appropriately detailed structure in different ways. Furthermore, the terminology and phrases used in this disclosure are not intended to be limiting, but rather to provide an understandable description of the described features.

[0015] This disclosure relates to voice-based computer-human interaction, and more specifically, to voice-based computer-human interaction using AI. Smart homes and similar environments typically include AI voice response systems that implement Natural Language Understanding (NLU) to create a human-machine interface through which the machine can determine the language structure and meaning from human speech. AI voice response systems can control various Internet of Things (IoT) devices in smart homes or similar environments through interfaces such as smart speakers that include speakers and intelligent virtual agents, via NLU processing of voice prompts and commands. AI voice response systems can also provide responses to user-specific queries (e.g., “Where are my glasses?”) and requests for general information (e.g., “What’s the date today?”).

[0016] One aspect of the systems, methods, and computer program products disclosed herein is the generation of a corpus of human discourse (e.g., user-specific and general requests) and the identification of patterns in the corpus that indicate a user's forgetfulness patterns. Forgetfulness patterns are identified using one or more machine learning models that perform pattern recognition. Based on incremental monitoring and contextualization of the content, frequency, and patterns of user discourse in the corpus, a user's memory impairment can be detected. As defined herein, "memory impairment" is a long-term or permanent change in the structure and function of an individual's brain due to a disease or condition, resulting in a decline in the individual's ability to store and retrieve information and data. Therefore, memory impairment differs from simple forgetfulness, which is caused by temporary conditions (e.g., stress, fatigue, and / or distraction) and fades as the condition improves. In contrast, memory impairment is caused by a disease (e.g., Alzheimer's disease) or condition (e.g., age, brain injury) that leads to a long-term (typically years) or permanent decline in an individual's ability to store and retrieve information and data.

[0017] User speech can be contextualized based on sensor information captured by one or more IoT devices. Contextual information can be used to distinguish between memory impairment and everyday forgetfulness. In some embodiments, human-computer dialogue can be conducted with the user using machine learning and generated based on patterns identified in a speech corpus to differentiate between memory impairment and everyday forgetfulness.

[0018] Another aspect of the systems, methods, and computer program products disclosed herein is the generation of improved action strategies. These strategies can also be determined based on the identification of patterns between utterances contained in a corpus and corresponding to contextual information. Improved action strategies may include automatically engaging the user in an interactive, machine-generated dialogue at a predetermined time (e.g., at night when the effects of an illness are more pronounced) to help the user perform one or more tasks (e.g., reminding the user to take medication and ensuring that household appliances are turned off and doors are locked). The dialogue can be machine-generated based on contextual data and can be structured to calm or soothe the user depending on the nature of their memory impairment.

[0019] Other aspects of the embodiments described in this disclosure are described in more detail with reference to the accompanying drawings. For purposes of simplicity and clarity, the components shown in the drawings are not necessarily drawn to scale. For example, the dimensions of some components may be enlarged relative to other components for clarity. Furthermore, repeated reference numerals in the drawings denote corresponding, similar, or analogous features where deemed appropriate.

[0020] Figure 1An example computing environment 100 is depicted. The computing environment 100 includes an example memory evaluation system (MES) 102 according to one embodiment. The MES 102 is illustratively implemented as software stored in memory and executed on one or more processors of a computer system 104. The computer system 104 may be part of a computing node (e.g., a cloud-based server) such as computing node 600, which includes computer systems such as example computer system 612. Figure 6 Computer system 104 provides a platform for the AI ​​voice response system. MES 102 is operatively coupled to or integrated therein with the AI ​​voice response system executing on computer system 104. In other embodiments, MES 102 executes on a separate computer system and is communicatively coupled to an AI voice response system running on a different computer system. The AI ​​voice response system is connected to voice interface 108 via data connection communication network 106. Data communication network 106 can be, and typically is, the Internet. Data communication network 106 can be or can include a wide area network (WAN), a local area network (LAN), or a combination of these and / or other types of networks, and can include wired, wireless, fiber optic, and / or other connections. Voice interface 108 can be, for example, a smart speaker, which includes a speaker and a smart virtual assistant that responds to human speech (e.g., voice prompts and commands) interpreted by the AI ​​voice response system.

[0021] An AI voice response system running on computer system 104 provides a Natural Language Understanding (NLU) human-machine interface, which allows the machine to determine the structure and meaning of language based on human speech. Using machine learning, the AI ​​voice response system can learn to recognize the user's unique voice characteristics (e.g., sound signal patterns) and voice commands and prompts transmitted through voice interface 108. The voice commands and prompts captured by voice interface 108 can select, create, and modify data stored in electronic memory to control one or more IoT devices, such as IoT devices 110a, 110b, 110c, and 110n. IoT devices 110a-110n can communicate with voice interface 108 via wired or wireless (e.g., Wi-Fi, Bluetooth). IoT devices 110a-110n may include, for example, smart appliances, climate controllers, lighting systems, entertainment systems, energy monitors, etc., which are part of a smart home or similar environment and can be located in the same location as them or remotely from computer system 104.

[0022] Figure 2Some components of a MES 200 according to one embodiment are shown in more detail. The MES 200 illustratively includes a corpus generator 202, a contextifier 204, a pattern recognizer 206, a recognizer 208, and a classifier 210. Each of the corpus generator 202, contextifier 204, pattern recognizer 206, recognizer 208, and classifier 210 can be implemented in software operatively coupled or integrated into an AI voice response system. In other embodiments, the corpus generator 202, contextifier 204, pattern recognizer 206, recognizer 208, and classifier 210 can be implemented in dedicated circuitry or a combination of circuitry and software configured to operate collaboratively with the AI ​​voice response system.

[0023] Corpus generator 202 generates a corpus corresponding to the user's human speech. The corpus is generated by corpus generator 202 from a variety of human speech captured by a voice interface (e.g., a smart speaker) operatively coupled to the AI ​​voice response system. The human speech is selected by corpus generator 202 based on the meaning of the speech. The meaning of the speech is determined by corpus generator 202 using natural language processing.

[0024] The context of the selected one or more human utterances is determined by contextifier 204 based on data generated in response to signals sensed by one or more sensing devices. The one or more sensing devices may include Internet of Things (IoT) devices operatively coupled to the voice interface and controlled via voice prompts and commands interpreted by an AI-powered voice response system.

[0025] Pattern recognizer 206 performs pattern recognition to identify patterns in a corpus of human speech. Pattern recognition is performed by pattern recognizer 206 based on one or more machine learning models. Based on pattern recognition, recognizer 208 identifies changes in the user's memory function. Classifier 210 applies contextual data to classify whether changes in the user's memory function are symptoms of memory impairment. Optionally, in response to classifier 210 classifying changes in the user's memory function as symptoms of memory impairment, actuator 212 may initiate one or more mitigation actions to alleviate the effects of the user's memory impairment.

[0026] User utterances can include single words, phrases, and / or sentences spoken by the user of the MES 200. The user is an individual whose voice is recognized by an AI voice response system that works in conjunction with the MES 200 and is trained using machine learning to identify the user based on unique voice features (e.g., acoustic signal patterns) corresponding to that user. These utterances are captured by a voice interface coupled to the AI ​​voice response system and transmitted to the MES 200.

[0027] Corpus generator 202 selects specific utterances from human discourse where users ask questions (e.g., “Where are my car keys?”), inquire about information (e.g., “How’s the weather outside?”), or involve identifiable aspects of the user’s memory or cognition (e.g., “I don’t remember my appointment time today”). Corpus generator 202 can also select user utterances from conversations between the user and an AI voice response system. A user might say, for example, “I’ve lost my glasses,” and the AI ​​voice response system might respond with a machine-generated audio response via a voice interface, “Are you wearing your glasses?” Statements such as “I’ve lost my glasses” are not direct questions. Only inferentially is the statement a question or a request for help. Nevertheless, corpus generator 202 can use machine learning to include utterances in a corpus.

[0028] In some embodiments, utterances may also be selected from dialogues in which the user verbally communicates with another person and is captured by a voice interface. In other embodiments where the MES 200 is coupled to a wireless and / or wired telephone system, the corpus generator 202 may select user utterances from dialogues conducted over a wireless or wired telephone connection. In other embodiments where the MES 200 is operatively coupled to a text messaging system (e.g., email), the corpus generator 202 may select text-based messages relating to questions, information requests, or other messages identified as relating to user cognition.

[0029] Corpus generator 202 selects utterances based on utterances with syntax and semantics determined using natural language processing. Corpus generator 202 obtains semantic understanding of utterances by applying one or more machine learning models to understand the content of utterances and selecting utterances based on the content.

[0030] In some embodiments, the corpus generator 202 applies a bidirectional long short-term model (Bi-LSTM). Bi-LSTM is, in a sense, a combination of two recurrent neural networks (RNNs). An RNN contains a network of nodes connected to form a directed graph, where each node (input, output, and hidden node) has a real-valued activation function that varies over time. Instead of treating all inputs and outputs as independent, RNNs “remember” information over time, enabling them to predict, for example, the next word in a sentence. Bi-LSTM processes ordered sequences in two directions: from past to future and from future to past. Running Bi-LSTM backward preserves information from the future, and using a combination of two hidden states allows it to retain information from the past and future at any given time. The Bi-LSTM model can determine the content of a utterance based on multi-hop reasoning across multiple utterances.

[0031] In other embodiments, corpus generator 202 uses other machine or statistical learning models, such as Non-Latent Similarity (NLS). For example, NLS uses a second-order similarity matrix to determine semantic similarity between words, phrases, and sentences. Linguistic units of utterances (e.g., words, phrases, sentences) are represented as vectors, and the similarity between utterances is measured by the cosine of the angle between each pair of vectors. Corpus generator 202 can use other machine or statistical learning models to determine the content and meaning of user utterances. Using machine or statistical learning, corpus generator 202 can also identify one or more topics in user utterances. Corpus generator 202 generates an utterance corpus 214 by storing user utterances selected based on determined content and meaning in electronic memory.

[0032] The corpus generator 202 can also use natural language processing to perform sentiment analysis. Based on sentiment analysis, the corpus generator 202 can identify the mental state (e.g., anger, sadness, frustration, happiness) of users corresponding to one or more selected user utterances.

[0033] Contextualizer 204 provides context for at least some selected user utterances. Contextual information may include the user's health status, cognitive state, attention, etc. Contextualizer 204 may determine contextual information based on data provided to a smart virtual assistant (e.g., a smart speaker including a speaker and a smart virtual assistant) via signal feeds (e.g., Wi-Fi, Bluetooth) from one or more IoT devices to the voice interface. Illustratively, contextualizer 204 receives contextual data obtained by an object tracker (e.g., an IoT camera) 216, and it also constructs a contextual database 218 by storing the contextual data in electronic memory. Other IoT devices, not explicitly shown, may also generate signals in response to various types of sensing performed by other IoT devices.

[0034] In some embodiments, contextifier 204 determines contextual information (e.g., one or more IoT cameras) based on contextual data including images and / or videos captured by object tracker 216. Using a convolutional neural network (CNN) trained to recognize and classify images and / or videos, contextifier 204 can provide contextual meaning for utterances captured by the voice interface. In some embodiments, the content and meaning of the user's utterance are determined by a synergistic application of a Bi-LSTM model (implemented as described above) and a region convolutional neural network (R-CNN) to identify one or more objects in the image corresponding to the user's utterance. R-CNN can identify objects in an image by identifying boundary regions in the image. Feature vectors are extracted from each region using a deep CNN. A classifier (e.g., a linear support vector machine (SVM)) applied to each feature vector is used to identify the object. Contextifier 204 can combine a Bi-LSTM model for determining the meaning or content of the user's utterance with a CNN for performing backward reasoning jumps relative to the captured image to generate contextual data corresponding to the utterance.

[0035] The relationship between user speech and images can be determined by contextifier 204 based on time and / or topic. For example, machine comparisons between timestamps corresponding to the time the user speech was recorded and the time the image was captured can reveal that the events captured in the user speech and the image occurred simultaneously or nearly simultaneously. Alternatively, bi-LSTM and R-CNN can both identify the topics of user speech and images, respectively. Contextifier 204 can then match user speech with images based on their respective topics.

[0036] Pattern recognizer 206 identifies patterns in selected user utterances. Pattern recognition can be based on various factors. For example, pattern recognizer 206 may identify patterns based on factors such as the nature of the utterance (e.g., question, information request, statement). A user may be looking for an object (e.g., glasses, keys). A user may ask about a date or time. A user may repeat such questions or statements. The number of times a question or statement is repeated within a predetermined interval may be accelerated. Pattern recognizer 206 may also identify patterns based on the frequency or number of times the same or semantically similar utterances are repeated within a predetermined time interval, using, for example, NLS or other NLU processing described above. Pattern recognizer 206 is trained using machine learning to recognize patterns. For example, using a supervised learning model, pattern recognizer 206 may be trained using a set of labeled feature vectors to recognize patterns forgotten by the user. The labeled feature vectors used to train pattern recognizer 206 may include a set of feature vectors corresponding to the user's past utterances. In other embodiments, the set of labeled feature vectors may correspond to the past utterances of multiple people who agree to share such data.

[0037] Not all forgetfulness is a symptom of memory impairment. For example, asking a voice response system, "Where did I put my umbrella?" is less likely to be due to memory impairment than asking "What time is it?" six times in an hour. Recognizer 208 combines the patterns recognized by pattern recognizer 206 with contextual information generated by the corresponding pattern contextifier 204 to identify the possible root cause of the forgetfulness patterns identified by pattern recognizer 206.

[0038] In some embodiments, if the nature of the utterance is ambiguous or its importance relative to the user's forgetfulness is uncertain, the MES 200 can automatically invoke a search for contextual information. Searching electronically stored contextual data can initiate the identification of contextual information to mitigate uncertainty. For example, a statement such as "What happened to my keys?" might not be easily identified as a question about the location of the user's keys. Using a cascaded model combining R-CNN and Bi-LSTM (each described above), the Bi-LSTM initially determines that the meaning is ambiguous, automatically invoking a search for contextual information, while R-CNN provides this by acquiring images showing the user moving around the house looking for something (timestamps consistent with the user's spoken statement). The fusion of the utterance and contextual information (images) can resolve ambiguity and prove that the nature of the utterance is equivalent to "Where are my keys?" Other IoT devices can provide contextual information beyond the images.

[0039] When performing a contextual information search, if the nature of the utterance is ambiguous or its importance relative to the user's memory is uncertain, the MES 200 performs an inference jump, bidirectionally moving contextual data (as described above) within the electronically stored information to resolve ambiguity or uncertainty. If the required information is not obtained through inference jumps within a corpus of contextual data, the MES 200 can search and retrieve the required data from one or more external Internet of Things (IoT) systems. If the probability of the MES 200 determining the meaning based on the retrieved data meets a predetermined confidence level (e.g., 60% or higher), the MES 200 will not search the contextual data to obtain additional information to resolve any ambiguity or uncertainty.

[0040] Based on the contextual information generated by contextifier 204, recognizer 208 can identify potential causes for one or more possible forgetfulness patterns. One possible cause might be an underlying health condition of the user, rather than a memory impairment. For example, a forgetfulness pattern identified within a given time period might be due to the user taking a doctor's prescription medication that induces drowsiness. The contextual data generated by contextifier 204 may include contextual data stored electronically in context database 218, based on input provided by the user or caregiver, including health-related parameters such as the medication the user is taking and the time of administration, such as the half-life (the time span during which the drug is effective), which can indicate temporal overlap with the utterance indicating forgetfulness. Therefore, recognizer 208 can identify that the possible cause of the forgetfulness pattern identified by pattern recognizer 206 might be due to a health condition (e.g., drowsiness-inducing drugs) rather than a memory impairment.

[0041] Similarly, the identified forgetfulness pattern might be due to temporary inattention, for example, if the user is busy with ongoing home repairs. The contextualizer 204 can generate contextual data that may include images captured by the object tracker 216 (e.g., one or more IoT cameras), which could show the user being repeatedly interrupted by the repairman. Based on the contextual data, the recognizer 208 can identify the likely cause of the forgetfulness pattern as inattention due to distraction during home repairs rather than memory impairment.

[0042] In contrast, users might repeatedly ask, “Where are my glasses?” The contextual data generated by the contextifier 204 can include images captured by the object tracker 216 (e.g., one or more IoT cameras) that show that the user is actually wearing glasses every time or most of the time when the user asks the question.

[0043] The contextifier 204 can generate contextual data that does not correspond to a specific utterance but still provides information to the recognizer 208 to identify possible underlying causes of the forgetfulness patterns identified by the pattern recognizer 206. For example, the contextual data generated by the contextifier 204 may include an image captured by the object tracker 216 (e.g., one or more IoT cameras) showing a user placing a pair of reading glasses in the refrigerator. The recognizer 208, also trained using machine learning, can identify unusual user behavior. When the image is combined with earlier or subsequent utterances, even just one or a few, such as saying “I’ve lost my glasses” or “Where are my glasses?”, the recognizer 208 does not attribute the forgetfulness patterns identified by the pattern recognizer 206 to health conditions, distraction, etc., but rather identifies possible symptoms of forgetfulness.

[0044] Contextual information based on different signals generated by different sensing devices (e.g., sensor-embedded IoT devices) can include the user's attention, cognitive state, and / or health status. Image-related contextual information can include the user's facial patterns, gestures, and / or movements in a given environment (e.g., a smart home). External noise, weather conditions, etc., can also provide contextual information.

[0045] Even contextual information that doesn't involve any specific utterance can be used by the recognizer 208 to identify forgetting patterns. For example, the contextifier 204 can generate contextual information from images showing unusual activity captured by the object tracker 216 (e.g., one or more IoT cameras). This activity, while not directly corresponding to a specific utterance, can correspond to, for example, a user attempting to use a tool or an object (e.g., an IoT device) that they wouldn't intend to use to perform the activity. For instance, an image taken by an IoT camera might show a user attempting to use an electric razor or placing a key in a microwave to open a pot of soup. Based on a set of labeled training data extracted from the experiences of patients with memory impairments, a machine learning model can identify behaviors that patients frequently perform. This type of contextual information can be used by the classifier 210 to classify whether changes in memory function (represented by more frequent forgetfulness) might be due to the user's memory impairment.

[0046] Classifier 210 categorizes changes in memory function demonstrated by accelerated repetition of user utterances (e.g., questions, information requests, or equivalents), which are contained in a corpus generated by corpus generator 202. Contextual information supporting the inference that identified forgetfulness patterns are attributable to general memory decline is detrimental to classifying the changes as due to memory impairment. Forgetfulness may be caused by stress, fatigue, distraction, etc. Contextual information indicating conditions such as stress or fatigue, such as heart rate, respiratory rate, or other biomarkers, can be captured by a device (e.g., a smartwatch) and wirelessly transmitted to MES 200 for electronic storage in context database 218. Context database 218 can also provide contextual information indicating distraction or user inattention, such as an image of a worker performing repairs at home or an outdoor weather-related event that might distract the user. Such contextual information can indicate events or conditions that support the inference that the user's forgetfulness is temporary rather than caused by memory impairment. However, the lack of contextual information to support the inference of temporary memory loss leads classifier 210 to categorize changes in memory function as due to memory impairment. In some embodiments, if there is no contextual information indicating that these changes only reflect temporary memory decline, the default classification of classifier 210 is to categorize changes in memory function as due to memory impairment.

[0047] In some embodiments, classifier 210 can implement a machine learning classification model, such as a deep learning neural network. The classification model can be trained using supervised learning, which is applied to a set of labeled feature vectors corresponding to different categories or classes of an individual. The model can be a binary classifier that classifies a user as having or not having a memory impairment based on the input vectors. In some embodiments, the classification model can be a multi-classifier model, including a non-diseased category and multiple categories, each corresponding to a different disease. Depending on the underlying feature vectors used to train the multi-class classification model, classifier 210 can classify changes in memory that may be attributable to the user's memory impairment based on the predicted severity of the impairment.

[0048] In some embodiments, the corpus generator 202 uses crowdsourcing to populate the discourse corpus 214. Various statistical analyses can be performed based on the population including the corpus and used as baseline functions to classify whether changes in a user's memory function are likely due to memory impairment (e.g., whether the change deviates from the population norm by more than a predetermined number of standard deviations). Iterative reinforcement learning can be used to generate machine learning classification models, which are then iteratively refined as the population size of the discourse corpus 214 increases.

[0049] Optionally, the MES 200 may include an improved motion actuator 212 designed to improve action strategies in response to symptoms that classify changes in a user's memory function as memory impairment. Improved action strategies include engaging the user in an interactive, machine-generated dialogue at predetermined times to help the user perform one or more tasks. For example, an improved action strategy might involve tracking (e.g., using an IoT camera) the user's movement and generating a voice alert before the user leaves the device (e.g., the stove) (via a voice interface) to prepare for sleep. The alert may be part of an interactive dialogue in which the user is asked if the front door is locked and / or reminded how to perform some other activity to ensure the user's safety. Improved action strategies may include issuing voice reminders when the user needs to take medication or take other actions. Improved action strategies may include determining the user's current health status and generating an electronic notification recommending that the user consult a health professional as needed. Improved action strategies may include generating and transmitting an electronic notification to caregivers to inform them of the user's potential memory impairment. The notification may be transmitted via a data communication network, a wired network, or a wireless communication network.

[0050] Figure 3 This is a flowchart of an example method 300 for memory monitoring and evaluation according to one embodiment. Method 300 can be used with reference to... Figure 1 and Figure 2The system is executed using the same or similar methods described. In block 302, the system captures multiple human utterances with a speech interface and generates a corpus of human utterances for the corresponding user. This corpus includes human utterances selected from multiple human utterances based on their meaning. The meaning of the human utterances is determined by the computer processor through natural language processing of the human utterances.

[0051] In block 304, the system determines contextual information corresponding to one or more human utterances in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to a computer processor. In block 306, the system identifies patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models.

[0052] In block 308, the system identifies changes in the user's memory function based on pattern recognition. Based on contextual information corresponding to one or more human utterances in the corpus, the system in block 310 is able to classify whether the changes in memory function may be due to memory impairment in the user.

[0053] In some embodiments, the system responds to symptoms that classify changes in memory function as memory impairment by engaging the user in an interactive, machine-generated dialogue.

[0054] In some embodiments, the system categorizes changes by performing user behavior analysis based on contextual information. In other embodiments, the system categorizes changes by performing sentiment analysis to determine the user's mental state. Sentiment analysis may determine, based on the tone of the speech and / or specific words spoken, that the user is angry, frustrated, or experiencing emotions that may, along with distraction or inattention, temporarily impair the user's memory, rather than indicating memory impairment.

[0055] In other embodiments, the system determines contextual information by performing a computer-based search of a context database in response to ambiguous utterances that may or may not correspond to changes in the user's memory function. If the nature of the utterance is ambiguous or its importance is uncertain relative to the user's forgetfulness, the system may automatically invoke a search of the context database to obtain contextual information.

[0056] Optionally, the system may construct an action strategy to improve memory function in response to classifying changes in memory function as symptoms of memory impairment. In some embodiments, the action strategy includes engaging the user in an interactive, machine-generated dialogue at a predetermined time to help the user perform one or more tasks. In other embodiments, the action strategy includes generating an electronic notification suggesting the user consult a healthcare professional. In other embodiments, the system may alternatively or additionally generate an electronic notification and transmit it to a caregiver to inform the user that the caregiver may have a memory impairment. This notification may be transmitted via a data communication network, a wired network, or a wireless network.

[0057] It is explicitly stated that while this disclosure includes a detailed description of cloud computing, the implementation of the teachings herein is not limited to cloud computing environments. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0058] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services), which can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0059] The features are as follows:

[0060] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0061] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0062] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0063] Rapid flexibility: The ability to provide capacity quickly and flexibly, automatically scaling down and up rapidly in some situations to scale up rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0064] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the providers and consumers of the services being utilized.

[0065] The service model is as follows:

[0066] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from different client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0067] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created by the consumer or acquired using programming languages ​​and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environment.

[0068] Infrastructure as a Service (IaaS): The capabilities offered to consumers are processing, storage, networking, and other basic computing resources that enable consumers to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0069] The deployment model is as follows:

[0070] Private cloud: A cloud infrastructure that operates solely for an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0071] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0072] Public cloud: Makes cloud infrastructure available to the public or large industry groups and is owned by an organization that sells cloud services.

[0073] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).

[0074] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0075] Now for reference Figure 4 An illustrative cloud computing environment 400 is shown. As illustrated, the cloud computing environment 400 includes one or more cloud computing nodes 410 to which local computing devices used by cloud consumers can communicate, such as personal digital assistants (PDAs) or mobile phones 440a, desktop computers 440b, laptop computers 440c, and / or automotive computer systems 440n. The nodes 410 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 400 to provide infrastructure, platform, and / or software as a service, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 4 The types of computing devices 440a-n shown are for illustrative purposes only, and computing node 410 and cloud computing environment 400 can communicate with any type of computing device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0076] Now for reference Figure 5 This demonstrates the 400 (cloud computing environment) Figure 4 This provides a set of functional abstractions. It should be understood beforehand that... Figure 5 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0077] The hardware and software layer 560 includes hardware and software components. Examples of hardware components include: a host 561; a server 562 based on a RISC (Reduced Instruction Set Computer) architecture; a server 563; a blade server 564; a storage device 565; and a network and network components 566. In some embodiments, the software components include network application server software 567 and database software 568.

[0078] The virtualization layer 570 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 571; virtual storage 572; virtual network 573, including virtual private network; virtual application and operating system 574; and virtual client 575.

[0079] In one example, management layer 580 can provide the functions described below: Resource Provisioning 581 Provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 582 Provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security Provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 583 Provides access to the cloud computing environment for consumers and system administrators. Service Level Management 584 Provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 585 Provides pre-scheduling and procurement of cloud resources based on anticipated future needs according to the SLA.

[0080] Workload layer 590 provides examples of functionalities that can leverage a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 591; software development and lifecycle management 592; virtual classroom education delivery 593; data analytics and processing 594; transaction processing 595; and MES 596.

[0081] Figure 6 A schematic diagram illustrating an example of compute node 600 is shown. In one or more embodiments, compute node 600 is an example of a suitable cloud computing node. Compute node 600 is not intended to impose any limitation on the scope of use or functionality of the embodiments of the invention described herein. Compute node 600 is capable of performing any part of the functionality described in this disclosure.

[0082] Computing node 600 includes computer system 612, which operates in conjunction with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for computer system 612 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computers, and distributed cloud computing environments including any of the aforementioned systems or devices.

[0083] Computer system 612 can be described in the general context of computer system executable instructions, such as program modules, executed by the computer system. Typically, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer system 612 can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0084] like Figure 6 As shown, computer system 612 is illustrated as a general-purpose computing device. Components of computer system 612 may include, but are not limited to, one or more processors 616, memory 628, and a bus 618 coupling various system components, including memory 628, to processor 616. As defined herein, a "processor" means at least one piece of hardware circuitry configured to execute instructions. Hardware circuitry may be an integrated circuit. Examples of processors include, but are not limited to, central processing units (CPUs), array processors, vector processors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), application-specific integrated circuits (ASICs), programmable logic circuits, and controllers.

[0085] Instructions for executing a computer program by a processor include executing or running the program. As defined herein, “running” and “executing” include a series of actions or events performed by the processor according to one or more machine-readable instructions. As defined herein, “running” and “executing” refer to processor activity performing actions or events. The terms “running,” “executing,” and “executing” are used as synonyms herein.

[0086] Bus 618 represents one or more of several types of bus architectures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. As an example and not a limitation, such architectures include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, Peripheral Component Interconnect (PCI) buses, and PCI Express (PCIe) buses.

[0087] Computer system 612 typically includes various computer system readable media. Such media can be any available media accessible to computer system 612, and can include volatile and non-volatile media, removable and non-removable media.

[0088] Memory 628 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 630 and / or cache memory 632. Computer system 612 may also include other removable / non-removable, volatile / non-volatile computer system storage media. For example, storage system 634 may be provided for reading and writing from non-removable, non-volatile magnetic media and / or solid-state drives (not shown and generally referred to as "hard disk drives"). Although not shown, disk drives and optical disk drives for reading and writing from removable non-volatile disks (e.g., "floppy disks") may provide software for reading or writing to removable, non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media). In this case, each may be connected to bus 618 via one or more data media interfaces. As will be further described and depicted below, memory 628 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the present invention.

[0089] A program / utility 640 having a set (at least one) of program modules 642 may be stored, as an example and not a limitation, in memory 628, along with an operating system, one or more applications, other program modules, and program data. Each of the operating system, one or more applications, other program modules, and program data, or some combination thereof, may include an implementation of a network environment. Program modules 642 typically perform the functions and / or methods of embodiments of the invention as described herein. For example, one or more program modules may include MES 596 or portions thereof.

[0090] Program / utility 640 may be executed by processor 616. Program / utility 640 and any data items used, generated, and / or operated by computer system 612 are functional data structures, which are given functionality when used by computer system 612. As defined in this disclosure, a "data structure" is a physical implementation of the data organization of a data model in physical memory. Therefore, a data structure consists of specific electrical or magnetic structural elements in memory. Data structures impose physical organization on data stored in memory, as used by applications executed using the processor.

[0091] Computer system 612 can also communicate with one or more external devices 614, such as a keyboard, pointing device, display 624, etc.; one or more devices that enable a user to interact with computer system 612; and / or any device that enables computer system 612 to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). Such communication can occur via input / output (I / O) interface 622. Furthermore, computer system 612 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 620. As shown, network adapter 620 communicates with other components of computer system 612 via bus 618. It should be understood that, although not shown, other hardware and / or software components may be used in conjunction with computer system 612. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archiving storage systems.

[0092] Although compute node 600 is used to illustrate an example of a cloud computing node, it should be understood that computer systems use and combine... Figure 1 The same or similar architecture described. Figure 6 This can be used in non-cloud computing implementations to perform the various operations described herein. In this regard, the example embodiments described herein are not intended to be limited to cloud computing environments. Compute node 600 is an example of a data processing system. As defined herein, a “data processing system” means one or more hardware systems configured to process data, each hardware system including at least one processor and memory programmed to initialize operations.

[0093] Compute node 600 is an example of computer hardware. Compute node 600 may include more than Figure 6 Fewer components shown or Figure 6 Additional components not shown depend on the specific type of device and / or system implemented. The specific operating system and / or applications included may vary depending on the device and / or system type and the type of I / O devices included. Furthermore, one or more illustrative components may be incorporated into another component or otherwise form part of another component. For example, a processor may include at least some memory.

[0094] Compute node 600 is also an example of a server. As defined herein, a "server" refers to a data processing system configured to share services with one or more other data processing systems. As defined herein, a "client device" refers to a data processing system that requests shared services from a server and with which a user interacts directly. Examples of client devices include, but are not limited to, workstations, desktop computers, computer terminals, mobile computers, laptop computers, netbook computers, tablet computers, smartphones, personal digital assistants, smartwatches, smart glasses, gaming devices, set-top boxes, smart TVs, etc. In one or more embodiments, the various user devices described herein can be client devices. Network infrastructure such as routers, firewalls, switches, and access points are not client devices as defined herein by the term "client device".

[0095] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. Nevertheless, several definitions applicable throughout the document will now be introduced.

[0096] As defined herein, the singular forms “a,” “an,” and “the” also include the plural forms, unless the context clearly indicates otherwise.

[0097] As defined in this article, "another" means a second or more.

[0098] As defined herein, unless otherwise expressly stated, “at least one,” “one or more,” and “and / or” are open-ended expressions that are conjunctions and analytics in operation. For example, each expression “at least one of A, B, and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” and “A, B, and / or C” refers to a single A, a single B, a single C, A and B together, A and C together, B and C together, or A, B, and C together.

[0099] As defined in this article, “automatic” means without user intervention.

[0100] As defined herein, “includes,” “including,” “comprises,” and / or “comprising” specify the presence of features, integers, steps, operations, elements, and / or components of a statement, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0101] As defined herein, “if” means “in response to” or “responsive to,” depending on the context. Therefore, the phrase “if it is determined” can be interpreted as “in response to determining” or, depending on the context, “responsive to determining.” Similarly, the phrase “if [astated condition or event] is detected” can be interpreted as “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event]” or “responsive to detecting [the stated condition or event],” depending on the context.

[0102] As defined herein, the terms "one embodiment," "an embodiment," "in one or more embodiments," "in particular embodiment," or similar language indicate that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment described in this disclosure. Therefore, the appearance of the foregoing phrases and / or similar language throughout this disclosure may, but does not necessarily refer to the same embodiment.

[0103] As defined in this paper, the phrases "in response to" and "responsive to" refer to a receptiveness to an action or event. Therefore, if a second action is performed "in response to" or "in response to," a causal relationship exists between the occurrence of the first action and the occurrence of the second action. The phrases "in response to" and "responsive to" indicate a causal relationship.

[0104] As defined herein, “substantially” means that the listed characteristics, parameters, or values ​​do not need to be precisely achieved, but rather that deviations or variations, including, for example, tolerances, measurement errors, measurement accuracy limitations, and other factors known to those skilled in the art, may occur in quantities that do not preclude the effects that the characteristic is intended to provide.

[0105] As defined in this article, "user" and "indicidual" both refer to human beings.

[0106] The terms first, second, etc., may be used in this document to describe various elements. These elements should not be limited by these terms, as they are only used to distinguish one element from another unless otherwise stated or clearly indicated by the context.

[0107] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0108] Computer-readable storage media can be tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0109] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0110] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this invention.

[0111] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0113] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions indicated in the blocks may occur in a non-consecutive order as shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0115] The description of various embodiments of the present invention is given for illustrative purposes and is not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements to technologies found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for computer-human interaction, comprising: Multiple human utterances are captured using a voice interface, and a corpus of human utterances is generated for the corresponding user. The corpus of human utterances includes human utterances selected from multiple human utterances based on the meaning of the human utterances, the meaning of which is determined by a computer processor through natural language processing of the human utterances. Contextual information corresponding to one or more human utterances in the corpus is determined based on data generated in response to signals sensed by one or more sensing devices operatively coupled to a computer processor. Pattern recognition based on computer processors using one or more machine learning models to identify patterns in a corpus of human speech; Based on pattern recognition, changes in user memory function can be identified. as well as Based on contextual information corresponding to one or more human utterances in the corpus, the possibility that changes in memory function may be due to memory impairment in the user is classified.

2. The method of claim 1, wherein the classification includes engaging the user in an interactive machine-generated dialogue in response to classifying changes in memory function as symptoms of memory impairment.

3. The method of claim 1, wherein the classification further includes performing at least one of behavioral analysis or emotion analysis to determine the user's psychological state.

4. The method of claim 1, wherein the determined contextual information includes performing a computer-based search of the context database in response to ambiguity in the meaning of the utterance or uncertainty regarding the importance of the utterance that the user has forgotten.

5. The method of claim 1, further comprising constructing an action strategy to improve memory function in response to classifying changes in memory function as symptoms of memory impairment.

6. The method of claim 5, wherein the improved action strategy includes engaging the user in an interactive machine-generated dialogue at a predetermined time to help the user perform one or more tasks.

7. The method of claim 5, wherein the improvement action strategy includes at least one of: generating an electronic notification to advise the user to consult a health professional or generating an electronic notification and transmitting the notification to a caregiver to inform the caregiver of possible memory impairment in the user.

8. A system for computer-human interaction, comprising: The processor is configured for initialization operations, including: Multiple human utterances are captured using a voice interface, and a corpus of human utterances is generated for the corresponding user. The corpus of human utterances includes human utterances selected from multiple human utterances based on the meaning of the human utterances, the meaning of which is determined by a computer processor through natural language processing of the human utterances. Contextual information corresponding to one or more human utterances in the corpus is determined based on data generated in response to signals sensed by one or more sensing devices operatively coupled to a computer processor. Pattern recognition based on computer processors using one or more machine learning models to identify patterns in a corpus of human speech; Based on pattern recognition, changes in user memory function can be identified; and Based on contextual information corresponding to one or more human utterances in the corpus, the possibility that changes in memory function may be due to memory impairment in the user is classified.

9. The system of claim 8, wherein the classification includes engaging the user in an interactive, machine-generated dialogue in response to classifying changes in memory function as symptoms of memory impairment.

10. The system of claim 8, wherein the classification further includes performing at least one of behavioral analysis or emotion analysis to determine the user's psychological state.

11. The system of claim 8, wherein the determined context information includes performing a computer-based search of the context database in response to ambiguity in the meaning of the utterance or uncertainty regarding the importance of the utterance that the user has forgotten.

12. The system of claim 8, wherein the processor is configured to initialize operations, the operations further comprising constructing an improvement action strategy in response to classifying changes in memory function as symptoms of memory impairment.

13. The system of claim 12, wherein the improved action strategy includes engaging the user in an interactive machine-generated dialogue at a predetermined time to help the user perform one or more tasks.

14. A computer program product comprising: One or more computer-readable storage media and program instructions co-stored on the one or more computer-readable storage media, the program instructions being executable by a processor to initialize the processor, including: Multiple human utterances are captured using a voice interface, and a corpus of human utterances is generated for the corresponding user. The corpus of human utterances includes human utterances selected from multiple human utterances based on the meaning of the human utterances, the meaning of which is determined by a computer processor through natural language processing of the human utterances. Contextual information corresponding to one or more human utterances in the corpus is determined based on data generated in response to signals sensed by one or more sensing devices operatively coupled to a computer processor. Pattern recognition based on computer processors using one or more machine learning models to identify patterns in a corpus of human speech; Based on pattern recognition, changes in user memory function are identified; and Based on contextual information corresponding to one or more human utterances in the corpus, the possibility that changes in memory function may be due to memory impairment in the user is classified.

15. The computer program product of claim 14, wherein the classification includes engaging the user in an interactive, machine-generated dialogue in response to classifying changes in memory function as symptoms of memory impairment.

16. The computer program product of claim 14, wherein the classification further includes performing at least one of behavioral analysis or sentiment analysis to determine the user's mental state.

17. The computer program product of claim 14, wherein the determined context information includes performing a computer-based search of a context database in response to ambiguity in the meaning of the utterance or uncertainty regarding the importance of the utterance that the user has forgotten.

18. The computer program product of claim 14, wherein the program instructions are executable by the processor to initialize the processor, the operations further comprising constructing an improvement action strategy in response to classifying changes in memory function as symptoms of memory impairment.

19. The computer program product of claim 18, wherein the improved action strategy includes engaging the user in an interactive machine-generated dialogue at a predetermined time to help the user perform one or more tasks.

20. The computer program product of claim 18, wherein the improvement action strategy includes at least one of: generating an electronic notification to advise the user to consult a health expert or generating an electronic notification and transmitting the notification to a caregiver to inform the caregiver of the user's possible memory impairment.

Images