Detecting and improving memory decline

Abstract

The method for detecting and assessing memory decline includes capturing human speech using a voice interface and generating, for a user, a corpus of human speech including human utterances selected from a plurality of human utterances based on meanings of the human utterances determined by natural language processing by a computer processor. Contextual information corresponding to one or more human utterances in the corpus is determined based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. Patterns in the corpus of human speech are recognized based on pattern recognition performed by the computer processor using one or more machine learning models. Based on the pattern recognition, changes in the user's memory function are identified. The identified changes are classified as likely to be changes attributable to the user's memory impairment based on the contextual information.

Description

[Technical Field]

[0001] The present disclosure relates to voice-based computer-human interaction, and more particularly to voice-based computer-human interaction using artificial intelligence (AI). [Background technology]

[0002] AI-enabled computer systems are growing in number and complexity and often involve voice-based computer-human interaction. Voice-based dialogue systems are frequently found in homes and other environments, successfully controlling automated devices and systems through voice prompts and commands. Voice-based dialogue systems are used for a variety of purposes beyond automation control. For example, a voice-based dialogue system allows a user to ask about the weather, get driving directions, or search for a favorite song. A user can also request a reminder for something they forgot, such as where they left their set of car keys. Summary of the Invention

[0003] In one or more embodiments, a method includes capturing a plurality of human utterances using a speech interface and generating, for a corresponding user, a corpus of human utterances including human utterances selected from the plurality of human utterances based on meaning of the human utterances, the meaning being determined by natural language processing of the human utterances by a computer processor. The method includes determining contextual information corresponding to one or more human utterances in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. The method includes recognizing patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models. The method includes identifying a change in memory function of the user based on the pattern recognition. The method includes classifying whether the change in memory function is likely to be attributable to the user's memory impairment based on the contextual information corresponding to the one or more human utterances in the corpus.

[0004] In one or more embodiments, the system includes a processor configured to initiate operations including capturing a plurality of human utterances using a speech interface and generating, for a corresponding user, a corpus of human utterances including human utterances selected from the plurality of human utterances based on meanings of the human utterances, the meanings being determined by natural language processing of the human utterances by a computer processor. The operations include determining contextual information corresponding to one or more human utterances in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. The operations include recognizing patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models. The operations include identifying a change in the user's memory function based on the pattern recognition. The operations include classifying whether the change in memory function is likely to be attributable to the user's memory impairment based on the contextual information corresponding to the one or more human utterances in the corpus.

[0005] In one or more embodiments, a computer program product includes one or more computer-readable storage media having instructions stored thereon. The instructions are executable by a processor to initiate operations. The operations include capturing a plurality of human utterances using a speech interface and generating, for a corresponding user, a corpus of human utterances including human utterances selected from the plurality of human utterances based on meanings of the human utterances, the meanings being determined by natural language processing of the human utterances by a computer processor. The operations include determining contextual information corresponding to one or more human utterances in the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. The operations include recognizing patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models. The operations include identifying a change in the user's memory function based on the pattern recognition. The operations include classifying whether the change in memory function is likely to be attributable to the user's memory impairment based on the contextual information corresponding to the one or more human utterances in the corpus.

[0006] This Summary section is provided merely to introduce certain concepts and is not intended to identify all key or essential features of the claimed subject matter. Other features of the inventive arrangements will become apparent from the accompanying drawings and the detailed description that follows.

[0007] Arrangements of the present invention are illustrated by way of example in the accompanying drawings. However, these drawings should not be construed as limiting the arrangements of the present invention to only the particular implementations shown. Various aspects and advantages will become apparent by consideration of the following detailed description and by reference to the drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an exemplary computing environment according to one embodiment. [Figure 2] FIG. 1 illustrates an exemplary memory assessor system according to one embodiment. [Figure 3] 1 is a flowchart of a method for memory monitoring and assessment according to one embodiment. [Figure 4] FIG. 1 illustrates a cloud computing environment according to one embodiment. [Figure 5] FIG. 2 illustrates abstraction model layers according to one embodiment. [Figure 6] FIG. 1 illustrates a cloud computing node according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] While the present disclosure concludes with claims defining novel features, the various features described within this disclosure will be better understood by considering the description in conjunction with the drawings. The processes, machines, manufacture, and all variations thereof described herein are provided for illustrative purposes. The specific structural and functional details described within this disclosure should not be construed as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art to employ the described features in various ways in substantially all appropriately detailed structures. Furthermore, the terms and phrases used within this disclosure are not intended to be limiting, but rather to provide an understandable description of the described features.

[0010] This disclosure relates to voice-based computer-human interaction, and more specifically, to voice-based computer-human interaction using AI. Smart homes and similar environments commonly include AI voice response systems, implementing natural language understanding (NLU) to create a human-computer interface where linguistic structure and meaning can be mechanically determined from human speech. AI voice response systems operate through interfaces such as smart speakers with speakers and intelligent virtual agents, and can control various Internet of Things (IoT) devices within smart homes and similar environments through NLU processing of voice prompts and commands. AI voice response systems can provide responses to user-specific queries (e.g., "Where are my glasses?") and general information requests (e.g., "What day is it today?").

[0011] One aspect of the systems, methods, and computer program products disclosed herein generates a corpus of human speech (e.g., user-specific requests and general requests) and recognizes patterns in the corpus that indicate the user's forgetfulness. The forgetfulness patterns are recognized using one or more machine learning models that perform pattern recognition. Based on gradual monitoring and contextualization of the content, frequency, and patterns of the user's speech in the corpus, the user's memory impairment can be detected. As defined herein, a "memory impairment" is a decline in a person's ability to remember and recall information or data due to a disease or condition that causes long-term or permanent changes to the structure and function of the person's brain. Thus, a memory impairment is different from simple forgetfulness that is caused by a temporary condition (e.g., stress, fatigue, or distraction, or a combination thereof) and fades when the condition improves. In contrast, a memory impairment is caused by a disease (e.g., Alzheimer's disease) or a condition (e.g., aging, brain injury) that causes a long-term (usually several years) or permanent decline in a person's ability to remember and recall information or data.

[0012] The user's utterances may be contextualized based on sensor information captured by one or more IoT devices. The contextual information may be used to distinguish memory impairment from everyday forgetfulness. In some embodiments, to distinguish memory impairment from everyday forgetfulness, a human-computer interaction may be conducted with the user based on patterns recognized in a corpus of utterances generated using machine learning.

[0013] Another aspect of the systems, methods, and computer program products disclosed herein generates a remedial action plan. The plan may also be determined based on recognizing patterns between utterances contained in the corpus and in response to contextual information. The remedial action plan may include automatically engaging the user in an interactive, machine-generated dialogue at a predetermined time (e.g., at night when the effects of the illness are more noticeable) to assist the user in performing one or more tasks (e.g., reminding the user to take medication, ensure appliances are turned off, and ensure doors are locked). The dialogue may be machine-generated based on contextual data and may be configured to soothe or reassure the user depending on the nature of the user's memory impairment.

[0014] Further aspects of the embodiments described in this disclosure will be described in more detail with reference to the following figures. For clarity and convenience of the figures, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Furthermore, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding, similar, or like features.

[0015] FIG. 1 illustrates an exemplary computing environment 100. The computing environment 100 includes an exemplary memory evaluator system (MES) 102 according to one embodiment. The MES 102 is illustratively implemented in software stored in memory and executed on one or more processors of a computer system 104. The computer system 104 may be part of a computing node (e.g., a cloud-based server), such as a computing node 600, which includes a computer system such as exemplary computer system 612 (FIG. 6). The computer system 104 provides a platform for an AI voice response system. The MES 102 is operatively coupled or integrated with an AI voice response system running on the computer system 104. In other embodiments, the MES 102 runs on a remote computer system and is communicatively coupled to an AI voice response system running on another computer system. The AI ​​voice response system connects to a voice interface 108 via a data communications network 106. The data communications network 106 may be, and typically is, the Internet. The data communications network 106 may be or may include a wide area network (WAN), a local area network (LAN), or a combination thereof, and / or other types of networks, and may include wired, wireless, fiber optic, or other connections, or combinations thereof. The voice interface 108 may be, for example, a smart speaker with a speaker and an intelligent virtual assistant that responds to human speech (e.g., voice prompts and commands) that are interpreted by an AI voice response system.

[0016] The AI ​​voice response system running on the computer system 104 provides a natural language understanding (NLU) computer-human interface in which linguistic structure and meaning can be determined by a machine from human speech. Using machine learning, the AI ​​voice response system can learn to recognize a user's unique voice characteristics (e.g., acoustic signal patterns) and the voice commands and prompts communicated through the voice interface 108. The voice commands and prompts captured by the voice interface 108 can select, generate, and modify data stored in electronic memory to control one or more IoT devices, exemplified by IoT devices 110a, 110b, 110c, and 110n. The IoT devices 110a-110n can communicate with the voice interface 108 via wired or wireless connections (e.g., Wi-Fi, Bluetooth®). The IoT devices 110a-110n may include, for example, smart appliances, climate controllers, lighting systems, entertainment systems, energy monitors, etc., which are part of a smart home or similar environment and may be co-located with or remotely located from the computer system 104.

[0017] 2 illustrates in more detail certain elements of MES 200 according to one embodiment. MES 200 illustratively includes corpus generator 202, contextualizer 204, pattern recognizer 206, identifier 208, and classifier 210. Each of corpus generator 202, contextualizer 204, pattern recognizer 206, identifier 208, and classifier 210 may be implemented in software operatively coupled with or integrated into an AI voice response system. In other embodiments, corpus generator 202, contextualizer 204, pattern recognizer 206, identifier 208, and classifier 210 may be implemented in dedicated circuitry or a combination of circuitry and software configured to operate in coordination with the AI ​​voice response system.

[0018] The corpus generator 202 generates a corpus of human utterances corresponding to a user. The corpus is generated by the corpus generator 202 selecting utterances of a particular user from various human utterances captured by a voice interface (e.g., a smart speaker) operatively coupled to the AI ​​voice response system. The human utterances are selected by the corpus generator 202 based on the meaning of the utterance. The meaning of the utterance is determined by the corpus generator 202 using natural language processing.

[0019] One or more contexts of the selected human utterance are determined by the contextualizer 204 based on data generated in response to signals sensed by one or more sensing devices that may be operatively coupled to a voice interface to configure an IoT device controlled with voice prompts and commands interpreted by an AI voice response system.

[0020] The pattern recognizer 206 performs pattern recognition to recognize patterns in a corpus of human speech. The pattern recognition is performed by the pattern recognizer 206 based on one or more machine learning models. Based on the pattern recognition, the identifier 208 identifies a change in the user's memory function. The classifier 210 applies contextual data to classify the identified change in the user's memory function as a symptom of memory impairment. In response to the classifier 210 classifying the identified change in the user's memory function as a symptom of memory impairment, the actuator 212 can optionally initiate one or more remedial actions to mitigate the effects of the user's memory impairment.

[0021] User utterances may include individual words, phrases, or sentences, or combinations thereof, spoken by a user of MES 200. A user is an individual whose voice is recognized by an AI voice response system operating in coordination with MES 200 and trained using machine learning to identify users based on unique voice characteristics (e.g., acoustic signal patterns) corresponding to the user. The utterances are captured and communicated to MES 200 by a voice interface communicatively coupled to the AI ​​voice response system.

[0022] The corpus generator 202 selects specific human utterances from among those in which the user poses a question (e.g., "Where are my car keys?"), requests information (e.g., "What's the weather like outside?"), or contains features that can determine the user's memory or cognition (e.g., "I can't remember what time my appointment is today."). The corpus generator 202 can select user utterances from a conversation between the user and an AI voice response system. The user may say, for example, "I lost my glasses," to which the AI ​​voice response system may deliver a machine-generated voice response through a voice interface, such as, "Are you wearing your glasses?" An utterance such as "I lost my glasses" is not a direct question; it merely asks a question or requests assistance by inference. Nevertheless, the corpus generator 202 can be trained to include this utterance in the corpus using machine learning.

[0023] In some embodiments, the utterances may also be selected from a conversation in which a user verbally engages with another individual and is captured by a voice interface. In other embodiments in which the MES 200 is communicatively coupled to a wireless and / or wired telephone system, the corpus generator 202 may select user utterances from conversations conducted over a wireless or wired telephone connection. In still other embodiments in which the MES 200 is operatively coupled to a text messaging system (e.g., email), the corpus generator 202 may select text-based messages, including questions, requests for information, or other messages, identified as cognitively relevant to the user.

[0024] The selection of utterances by the corpus generator 202 is based on the syntax and semantics of the utterance determined using natural language processing. The corpus generator 202 obtains a semantic understanding of the utterance by applying one or more machine learning models to understand the content of the utterance and select utterances based on the content.

[0025] In some embodiments, the corpus generator 202 applies a bidirectional long-short-term model (Bi-LSTM). Bi-LSTM is, in a sense, a combination of two recurrent neural networks (RNNs). An RNN includes a network of nodes connected to form a directed graph, with each node (input, output, and hidden) having a time-varying, real-valued activation function. Rather than treating all inputs and outputs independently, an RNN "remembers" information over time, allowing the RNN to predict, for example, the next word in a sentence. Bi-LSTM processes ordered sequences in two directions: from past to future and from future to past. Bi-LSTM retains future information by running backward, and can retain both past and future information at any point in time by using a combination of two hidden states. Bi-LSTM models can determine the content of an utterance based on performing multi-hop inference across multiple utterances.

[0026] In other embodiments, the corpus generator 202 determines semantic understanding using other machine or statistical learning models, such as non-latent similarity (NLS). For example, NLS uses a quadratic similarity matrix to determine semantic similarity between words, phrases, and sentences. The verbal units of an utterance (e.g., word, phrase, sentence) are represented as vectors, and the similarity between utterances is measured by the cosine of the angle between each part of the vector. The corpus generator 202 can use other machine or statistical learning models to determine the content and meaning of a user's utterance. Using machine learning or statistical learning, the corpus generator 202 can also identify one or more topics of the user's utterance. The corpus generator 202 generates the utterance corpus 214 by storing selected user utterances based on the determined content and meaning in an electronic memory.

[0027] The corpus generator 202 can also perform sentiment analysis using natural language processing. Based on the sentiment analysis, the corpus generator 202 can identify a user's psychological state (e.g., anger, sadness, disappointment, joy) corresponding to one or more selected user utterances.

[0028] The contextualizer 204 provides context to at least a portion of a selected user utterance. Contextual information may include the user's health status, the user's cognitive state, attention level, etc. The contextualizer 204 can determine the contextual information based on data provided via a signal feed (e.g., Wi-Fi, Bluetooth®) from one or more IoT devices to the voice-interface intelligent virtual assistant (e.g., a smart speaker with a speaker and an intelligent virtual assistant). Illustratively, the contextualizer 204 receives contextual data obtained by an object tracker (e.g., an IoT camera) 216, which also builds a context database 218 by storing the contextual data in electronic memory. Other IoT devices not explicitly shown can also generate signals in response to various types of sensing performed by other IoT devices.

[0029] In some embodiments, the contextualizer 204 determines contextual information based on context data including images and / or video captured by the object tracker 216 (e.g., one or more IoT cameras). Using a convolutional neural network (CNN) trained to recognize and classify images and / or videos, the contextualizer 204 can provide contextual meaning to utterances captured by the voice interface. In particular embodiments, the content and meaning of a user's utterance are determined by jointly applying a Bi-LSTM model (implemented as described above) to determine the content of the utterance and a regional-convolutional neural network (R-CNN) to identify one or more objects in the image that correspond to the user's utterance. The R-CNN can identify objects in the image by constraining regions within the image. A feature vector is extracted from each region by the deep CNN. The classifier identifies the object using a classifier (e.g., a linear support vector machine (SVM)) applied to each feature vector. The contextualizer 204 can combine a Bi-LSTM model, which determines the meaning or content of a user's utterance, with an R-CNN, which performs reverse reasoning hops on the captured image, to generate context data corresponding to the utterance.

[0030] The relationship between the user's speech and the image can be determined by the contextualizer 204 based on time, topic, or both. For example, a machine comparison between the timestamps corresponding to the time the user's speech was recorded and the time the image was captured can reveal that the user's speech and the event captured in the image occurred at or near the same time. Alternatively, both Bi-LSTM and R-CNN can identify the topic of the user's speech and the image, respectively. The contextualizer 204 can associate the user's speech with the image according to their respective topics.

[0031] The pattern recognizer 206 recognizes patterns among selected user utterances. Pattern recognition can be based on various factors. For example, the pattern recognizer 206 recognizes patterns based on factors such as the nature of the utterance (e.g., question, information request, statement). The user may be looking for an object (e.g., glasses or keys). The user may be asking about the date or time. Such questions or statements may be repeated by the user. The number of times a question or statement is repeated within a predetermined interval may accelerate. The pattern recognizer 206 can recognize patterns based on, for example, how frequently identical or semantically similar utterances are repeated or the number of times within a predetermined time interval, as determined using the NLS or other NLU processes described above. The pattern recognizer 206 is trained to recognize patterns using machine learning. For example, using a supervised learning model, the pattern recognizer 206 can be trained to recognize a user's forgetfulness patterns using a collection of labeled feature vectors. The labeled feature vectors used to train the pattern recognizer 206 may include a collection of feature vectors corresponding to the user's past utterances. In other embodiments, the collection of labeled feature vectors corresponds to the past utterances of multiple people who have agreed to jointly share such data.

[0032] Not all forgetfulness is a symptom of memory loss. For example, asking a voice response system "Where did I put my umbrella?" for three consecutive rainy days is less likely to be caused by memory loss than asking, for example, "What time is it?" six times an hour. The identification unit 208 combines the patterns recognized by the pattern recognition unit 206 with the corresponding context information generated by the contextualizer 204 to identify likely underlying causes of the forgetfulness patterns recognized by the pattern recognition unit 206.

[0033] In some embodiments, the MES 200 can automatically conduct a search for contextual information when the nature of the utterance is ambiguous or the significance of the utterance regarding the user's forgetfulness is unknown. A search of electronically stored contextual data can be initiated to identify contextual information and mitigate the uncertainty. For example, a declarative sentence such as "What about my keys?" may not be easily recognized as a question about the location of the user's keys. Using the above-described cascade model combining R-CNN and Bi-LSTM, if the Bi-LSTM initially determines that the meaning is ambiguous, a search for contextual information can be automatically conducted. The contextual information is provided by the R-CNN by acquiring images (timestamped simultaneously with the user's utterance of the declarative sentence) showing the user moving around the house looking for something. The combination of the utterance and the contextual information (images) can resolve the ambiguity and indicate that the nature of the utterance is equivalent to "Where are my keys?" Contextual information other than images can be provided by other IoT devices.

[0034] When performing a search for context information, if the nature of the utterance is ambiguous or the significance of the utterance regarding the user's forgetfulness is unknown, the MES200 performs inference hops, moving bidirectionally (as described above) between electronically stored context data to resolve the ambiguity or uncertainty. If the required information is not obtained by inference hops within the corpus of context data, the MES200 can search for and retrieve the required data from one or more external IoT devices. If, based on the retrieved data, the MES200 determines that the likelihood of the meaning meets a predetermined confidence level (e.g., 60% or greater), the MES200 does not search for additional information in the context data to resolve the ambiguity or uncertainty.

[0035] The identification unit 208 can identify one or more possible underlying reasons for the forgetfulness pattern based on the context information generated by the contextualizer 204. The possible reason may be the user's underlying health condition rather than memory impairment. For example, a pattern of forgetfulness over a period of time may be due to the user taking a sleep-inducing medication prescribed by a doctor. The contextual data generated by the contextualizer 204 may include electronically stored contextual data in the context database 218 based on input provided by the user or a caregiver, including health-related parameters such as the medication the user is taking and its half-life (the period during which the medication is effective), which may indicate a temporal overlap with utterances indicating forgetfulness. Thus, the identification unit 208 can identify that the possible reason for the forgetfulness pattern recognized by the pattern recognition unit 206 is likely due to a health condition (e.g., a sleep-inducing medication) rather than memory impairment.

[0036] Similarly, a recognized pattern of forgetfulness may be due to a momentary lapse in attention, such as when a user is preoccupied with ongoing home repairs. The contextual data generated by the contextualizer 204 may include images captured by the object tracker 216 (e.g., one or more IoT cameras) that may indicate the user is repeatedly interrupted by a worker engrossed in performing repairs. Based on this contextual data, the identifier 208 may identify that the likely reason for the forgetfulness pattern is a lack of attention due to being distracted by the home repairs, rather than a memory impairment.

[0037] In contrast, a user may repeatedly ask, "Where are my glasses?" The contextual data generated by the contextualizer 204 may include images captured by the object tracker 216 (e.g., one or more IoT cameras) that indicate that the user was in fact wearing their glasses whenever, or most of the time, the user asked that question.

[0038] The contextualizer 204 can generate context data that does not correspond to a specific utterance but may still provide the identifier 208 with information to identify possible underlying reasons for the forgetfulness pattern recognized by the pattern recognizer 206. For example, the contextual data generated by the contextualizer 204 may include an image captured by the object tracker 216 (e.g., one or more IoT cameras) that shows the user placing their reading glasses in the refrigerator. The identifier 208 can also be trained using machine learning to identify anomalous user behavior. If the image is combined with even just one or a few previous or subsequent utterances such as "I lost my glasses" or "Where are my glasses?", the identifier 208 will treat the forgetfulness pattern recognized by the pattern recognizer 206 as a likely symptom of forgetfulness rather than attributing it to a health condition, distraction, or the like.

[0039] Contextual information based on various signals generated by various sensing devices (e.g., sensor-embedded IoT devices) may include a user's attention, cognitive state, or health status, or a combination thereof. Contextual information related to images may include a user's facial patterns, gestures, or movements within a given environment (e.g., a smart home), or a combination thereof. External noise and weather conditions may also provide contextual information.

[0040] Contextual information unrelated to any particular utterance can also be used by the identifier 208 to identify patterns of forgetfulness. For example, the contextualizer 204 can generate contextual information from images captured by the object tracker 216 (e.g., one or more IoT cameras) showing anomalous behavior. Such behavior does not directly correspond to a particular utterance, but may correspond, for example, to a user attempting to perform an action using a tool or object (e.g., an IoT device) that was not intended to be used to perform that action. For example, footage captured by an IoT camera may show a user attempting to open a can of soup using an electric shaver or leaving keys in a microwave. Based on a set of labeled training data extracted from the experiences of individuals with memory impairment, a machine learning model can identify actions frequently performed by the patient. This type of contextual information can be used by the classifier 210 to classify whether changes in memory function (indicated by more frequent forgetfulness) are likely to be caused by the user's memory impairment.

[0041] The classifier 210 classifies changes in memory function evidenced by an accelerating repetition of user utterances (e.g., questions, information requests, declarative sentences equivalent to questions or information requests) contained in the corpus generated by the corpus generator 202. Corresponding contextual information supporting the inference that the recognized forgetfulness pattern is due to normal memory decline will adversely affect the classification of the change as due to memory impairment. Forgetfulness may be due to stress, fatigue, distraction, etc. Contextual information, such as heart rate, respiratory rate, or other biomarkers indicative of conditions such as stress or fatigue, may be captured by a device (e.g., a smartwatch) and wirelessly transmitted to the MES 200 for electronic storage in the context database 218. The context database 218 may similarly be supplied with contextual information indicative of distraction or user inattention, such as images of workers performing repairs in the home or weather-related events outdoors that may distract the user. Such contextual information may indicate events or circumstances that support the inference that the user's forgetfulness is temporary and not caused by memory impairment. However, in the absence of contextual information to support an inference of temporary memory loss, the classifier 210 will classify the change in memory function as being due to memory impairment. The default classification of the classifier 210, in some embodiments, is to classify the change in memory function as being due to memory impairment in the absence of contextual information indicating that the change simply reflects a temporary memory decline.

[0042] In some embodiments, the classifier 210 may implement a machine learning classification model, such as a deep learning neural network. The classification model may be trained using supervised learning applied to a collection of labeled feature vectors corresponding to various classes or categories of individuals. The model may be a binary classifier that classifies a user as suffering from or not suffering from a memory impairment based on the input vector. In some embodiments, the classification model may be a multi-class classification model that includes a non-affected class and multiple classes, each corresponding to a different disorder. Depending on the underlying feature vector used to train the multi-class classification model, the classifier 210 may classify changes in memory function likely attributable to the user's memory impairment according to the predicted severity of the impairment.

[0043] In some embodiments, the corpus generator 202 uses crowdsourcing to add to the speech corpus 214. Various statistical analyses can be performed based on the population comprising the corpus and used as criteria to classify whether changes in the user's memory function are likely due to memory impairment (e.g., whether the changes deviate from the population mean by more than a predetermined number of standard deviations). The machine learning classification model can be generated using iterative reinforcement learning, whereby the model is iteratively improved as the size of the speech corpus 214 population increases.

[0044] The MES 200 may optionally include a remediation actuator 212 that devise a remediation action plan in response to classifying changes in the user's memory function as symptoms of memory impairment. The remediation action plan may include engaging the user in a timed, interactive, machine-generated dialogue to assist the user in performing one or more tasks. For example, the remediation action plan may include tracking the user's movements (e.g., using an IoT camera) and generating an audio alert (via a voice interface) if the user leaves an appliance (e.g., a stove) on before getting ready for bed. The reminder may be part of an interactive dialogue in which the user is asked if the front door is locked, or may remind the user how to perform some other activity to ensure the user's safety, or both. The remediation action plan may include issuing an audio reminder when the user needs to take medication or take another action. The remediation action plan may include determining the user's current health status and, if necessary, generating an electronic notification advising the user to consult a health professional. The remedial action plan may include generating an electronic notification and transmitting the notification to a caregiver informing the caregiver that the user is likely to have memory impairment. The notification may be transmitted via a data communications network, a wired network, or a wireless communications network.

[0045] 3 is a flowchart of an exemplary method 300 for memory monitoring and assessment according to one embodiment. Method 300 may be implemented using the same or similar systems as those described with reference to FIGS. 1 and 2. In block 302, the system captures a plurality of human utterances using a voice interface and generates, for a corresponding user, a corpus of human utterances including human utterances selected from the plurality of human utterances based on the meaning of the human utterances. The meaning of the human utterances is determined by natural language processing of the human utterances by a computer processor.

[0046] In block 304, the system determines contextual information corresponding to one or more human utterances of the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor. In block 306, the system recognizes patterns in the corpus of human utterances based on pattern recognition performed by the computer processor using one or more machine learning models.

[0047] In block 308, the system identifies a change in the user's memory function based on pattern recognition. Based on contextual information corresponding to one or more human utterances in the corpus, the system in block 310 can classify whether the change in memory function is likely due to the user's memory impairment.

[0048] In some embodiments, upon classifying an alteration in memory function as a symptom of memory impairment, the system classifies the alteration by engaging the user in an interactive machine-generated dialogue.

[0049] In some embodiments, the system classifies the change by performing a behavioral analysis of the user based on the contextual information. In yet other embodiments, the system classifies the change by performing a sentiment analysis to determine the user's state of mind. The sentiment analysis can determine, based on tone of speech and / or specific words spoken, that the user is not exhibiting memory impairment, but is instead experiencing emotions such as anger, depression, or feelings that may be consistent with distraction or inattention, causing the user's temporary memory impairment.

[0050] In other embodiments, the system determines the context information by performing a computer-based search of the context database in response to ambiguous utterances that may or may not correspond to changes in the user's memory function. When the nature of the utterance is ambiguous or the significance of the utterance with respect to the user's forgetfulness is unknown, the system can automatically conduct a search of the context database for context information.

[0051] In response to classifying the changes in memory function as symptoms of memory impairment, the system can optionally develop a remedial action plan. In some embodiments, the remedial action plan includes engaging the user in an interactive, machine-generated dialogue at predetermined times to assist the user in performing one or more tasks. In other embodiments, the remedial action plan includes generating an electronic notification advising the user to consult a health professional. In still other embodiments, the system alternatively or additionally generates and transmits an electronic notification to a caregiver informing the caregiver of the user's possible memory impairment. The notification can be transmitted via a data communications network, a wired network, or a wireless network.

[0052] It is expressly noted that although this disclosure includes detailed descriptions of cloud computing, implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention may be practiced in conjunction with any other type of computing environment now known or later developed.

[0053] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0054] The features are as follows.

[0055] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, automatically as needed without human interaction with the service provider.

[0056] Wide network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0057] Resource Pooling: Provider computing resources are pooled to serve multiple consumers using a multi-tenant model with various physical and virtual resources dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge regarding the exact location of the resources provided, although there is a sense of location independence in that it may be possible to specify location at a higher level of abstraction (e.g., country, state, or data center).

[0058] Rapid Elasticity: Capacity can be provisioned quickly and elastically, sometimes automatically, quickly scaled out, quickly released, and quickly scaled in. To the consumer, the capacity available for provisioning often appears unlimited, and they can purchase any quantity at any time.

[0059] Thoughtful Services: Cloud systems automatically control and optimize resource usage by leveraging instrumentation capabilities at levels of abstraction depending on the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of utilized services.

[0060] The service model is as follows:

[0061] Software as a Service (SaaS): The consumer is offered the ability to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). With the possible exception of limited user-specific application configuration settings, the consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application capabilities.

[0062] Platform as a Service (PaaS): The ability offered to consumers is to deploy consumer-generated or -collected applications, generated using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does control the deployed applications and, in some cases, the application hosting environment configuration.

[0063] Infrastructure as a Service (IaaS): The ability offered to consumers is to provision processing, storage, network, and other basic computing resources on which they can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does control the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0064] The deployment model is as follows:

[0065] Private Cloud: Cloud infrastructure is operated solely for one organization and can be managed by that organization or a third party and can reside on-premise or off-premise.

[0066] Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with shared priorities (e.g., mission, security requirements, policies, and compliance considerations). It may be managed by the organization or a third party and may reside on- or off-premises.

[0067] Public Cloud: Cloud infrastructure is made available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0068] Hybrid Cloud: A cloud infrastructure consisting of two or more clouds (private, community, or public) that continue as distinct entities but are bound together by standardized or proprietary technologies that allow data and application portability (e.g., cloud bursting for load balancing between clouds).

[0069] Cloud computing environments are service-oriented with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that comprises a network of interconnected nodes.

[0070] Referring now to FIG. 4, an exemplary cloud computing environment 400 is shown. As shown, the cloud computing environment 400 includes one or more cloud computing nodes 410 with which local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or cellular phone 440a, a desktop computer 440b, a laptop computer 440c, or an automobile computer system 440n, or combinations thereof, may communicate. The computing nodes 410 may communicate with each other. They may be physically or virtually grouped in one or more networks (not shown), such as private, community, public, or hybrid clouds, or combinations thereof, as described hereinabove. This enables the cloud computing environment 400 to provide infrastructure, platform, and / or software as a service without the need for cloud consumers to maintain resources on their local computing devices. It will be understood that the types of computing devices 440a-n shown in FIG. 4 are for illustrative purposes only, and that computing node 410 and cloud computing environment 400 may communicate with any type of computerized device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0071] Referring now to Figure 5, there is shown a series of functional abstraction layers provided by cloud computing environment 400 (Figure 4). It should be understood that the components, layers, and functions shown in Figure 5 are for illustrative purposes only, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0072] Hardware and software layer 560 includes hardware and software components. Examples of hardware components include mainframe 561, reduced instruction set computer (RISC) architecture-based server 562, server 563, blade server 564, storage device 565, and network and networking components 566. In some embodiments, software components include network application server software 567 and database software 568.

[0073] The virtualization layer 570 provides an abstraction layer at which the following examples of virtual entities can be provided: virtual servers 571, virtual storage 572, virtual networks including virtual private networks 573, virtual applications and operating systems 574, and virtual clients 575.

[0074] In one example, management layer 580 may provide the functions described below. Resource provisioning 581 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 582 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks and protection for data and other resources. User portal 583 provides access to the cloud computing environment to consumers and system administrators. Service level management 584 allocates and manages cloud computing resources to ensure required service levels are met. Service level agreement (SLA) planning and fulfillment 585 provides proactive provisioning and procurement of cloud computing resources in anticipation of future requirements according to SLAs.

[0075] The workload layer 590 provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this layer include mapping and navigation 591, software development and lifecycle management 592, virtual classroom education delivery 593, data analytics processing 594, transaction processing 595, and MES 596.

[0076] 6 illustrates a schematic diagram of an example computing node 600. In one or more embodiments, computing node 600 is an example of a suitable cloud computing node. Computing node 600 is not intended to suggest any limitation regarding the scope of use or functionality of the embodiments of the invention described herein. Computing node 600 is capable of performing any of the functions described within this disclosure.

[0077] Computing node 600 includes computer system 612 operable with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, may be suitable for use with computer system 612, including, but not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0078] Computer system 612 may be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer system 612 may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0079] As shown in FIG. 6, computer system 612 is illustrated in the form of a general-purpose computing device. Components of computer system 612 may include, but are not limited to, one or more processors 616, memory 628, and a bus 618 coupling various system components, including memory 628, to processor 616. As defined herein, a "processor" means at least one hardware circuit configured to execute instructions. The hardware circuit may be an integrated circuit. Examples of processors include, but are not limited to, a central processing unit (CPU), an array processor, a vector processor, a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic array (PLA), an application specific integrated circuit (ASIC), a programmable logic circuit, and a controller.

[0080] The execution of computer program instructions by a processor includes executing or running the program. As defined herein, "run" and "execute" include a sequence of actions or events performed by a processor in accordance with one or more machine-readable instructions. As defined herein, "running" and "executing" refer to the active performance of actions or events by a processor. As used herein, the terms run, running, execute, and executing are used interchangeably.

[0081] Bus 618 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures, including, by way of example only and not limitation, an Industry Standard Architecture (ISA®) bus, a Micro Channel Architecture (MCA®) bus, an Enhanced ISA (EISA®) bus, a Video Electronics Standards Association (VESA®) local bus, a Peripheral Component Interconnect (PCI®) bus, and a PCI Express® (PCIe®) bus.

[0082] Computer system 612 typically includes a variety of computer system readable media. Such media can be any available media that can be accessed by computer system 612 and can include both volatile and nonvolatile media, removable and non-removable media.

[0083] Memory 628 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 630 and / or cache memory 632. Computer system 612 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example, storage system 634 may be provided for reading from and writing to non-removable, non-volatile magnetic media and / or solid state drives (not shown, commonly referred to as "hard drives"). Although not shown, a magnetic disk drive may be provided for reading from and writing to removable, non-volatile magnetic disks (e.g., "floppy disks"), and an optical disk drive may be provided for reading from and writing to removable, non-volatile optical disks, such as CD-ROMs, DVD-ROMs, or other optical media. In such cases, each may be connected to bus 618 by one or more data media interfaces. As further shown and described below, memory 628 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the present invention.

[0084] A program / utility 640 having at least one set of program modules 642 may be stored in memory 628, as well as an operating system, one or more application programs, other program modules, and program data, by way of example and not limitation. Each of the operating system, one or more application programs, other program modules, and program data, or a combination of portions thereof, may comprise an implementation of a networking environment. The program modules 642 generally perform the functions and / or methods of embodiments of the present invention described herein. For example, one or more of the program modules may include MES 596 or portions thereof.

[0085] The programs / utilities 640 are executable by the processor 616. All items used, created, and / or manipulated by the programs / utilities 640 and the computer system 612 are functional data structures that impart functionality when employed by the computer system 612. As defined within this disclosure, a "data structure" is a physical implementation of the organization of data in a data model in physical memory. As such, a data structure is formed of specific electrical or magnetic structural elements in memory. The data structure imposes a physical organization on data stored in memory for use by application programs executed using the processor.

[0086] The computer system 612 may also communicate with one or more external devices 614, such as a keyboard, a pointing device, a display 624, one or more devices that allow a user to interact with the computer system 612, or any device that allows the computer system 612 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), or a combination thereof. Such communication may occur via an input / output (I / O) interface 622. The computer system 612 may also communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter 620. As shown, the network adapter 620 communicates with other components of the computer system 612 via a bus 618. It should be understood that other hardware and / or software components, not shown, may be used with the computer system 612. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0087] While computing node 600 is used to describe an example of a cloud computing node, it should be understood that computer systems using the same or similar architecture as described in connection with FIG. 6 may be used in non-cloud computing implementations to perform various operations described herein. In this regard, the exemplary embodiments described herein are not intended to be limited to cloud computing environments. Computing node 600 is an example of a data processing system. As defined herein, "data processing system" means one or more hardware systems configured to process data, each including at least one processor and memory programmed to initiate operations.

[0088] Computing node 600 is an example of computer hardware. Computing node 600 may include fewer components than those shown in FIG. 6 or additional components not shown, depending on the particular type of device and / or system in which it is implemented. The particular operating systems and / or applications included may vary depending on the type of device and / or system, as may the types of I / O devices included. Furthermore, one or more of the example components may be incorporated into or otherwise form part of another component. For example, a processor may include at least some memory.

[0089] Computing node 600 is also an example of a server. As defined herein, a "server" refers to a data processing system configured to share services with one or more other data processing systems. As defined herein, a "client device" refers to a data processing system that requests shared services from a server and with which a user directly interacts. Examples of client devices include, but are not limited to, workstations, desktop computers, computing terminals, mobile computers, laptop computers, netbook computers, tablet computers, smart phones, personal digital assistants, smart watches, smart glasses, gaming devices, set-top boxes, smart televisions, etc. In one or more embodiments, the various user devices described herein may be client devices. Network infrastructure such as routers, firewalls, switches, and access points are not client devices, as the term "client device" is defined herein.

[0090] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. However, here are some definitions that apply throughout this document.

[0091] As defined herein, the singular forms "a," "an," and "the" include the plural forms as well, unless the context clearly dictates otherwise.

[0092] As defined herein, "another" means at least a second or more.

[0093] As defined herein, "at least one," "one or more," and "and / or" are open-ended expressions that are conjunctive and disjunctive in effect unless expressly stated otherwise. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, or C, or combinations thereof" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.

[0094] As defined herein, "automatically" means without user intervention.

[0095] As defined herein, "includes," "including," "comprises," or "comprising," or combinations thereof, specify the presence of stated features, integers, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0096] As defined herein, "if" can mean "in response to" or "responsive to," depending on the context. Thus, the phrase "if it is determined" can be interpreted to mean "in response to determining" or "responsive to determining," depending on the context. Similarly, the phrase "if [a stated condition or event] is detected" can be interpreted as "upon detecting [a stated condition or event]," or "in response to [a stated condition or event]," or "responsive to [a stated condition or event]," depending on the context.

[0097] As defined herein, the terms "in one embodiment," "one embodiment," "in one or more embodiments," "in a particular embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment described within the disclosure. Thus, throughout this disclosure, appearances of the foregoing phrases and / or similar language may, but do not necessarily, all refer to the same embodiment.

[0098] As defined herein, the phrases "in response to" and "responsive to" mean to respond or react readily to an action or event. Thus, when a second action occurs "in response to" or "in response to" a first action, there is a causal relationship between the occurrence of the first action and the occurrence of the second action. The phrases "in response to" and "responsive to" indicate a causal relationship.

[0099] As defined herein, "substantially" means that the referenced characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including, for example, tolerances, measurement errors, measurement accuracy limits, and other factors known to those skilled in the art, may occur in an amount that does not interfere with the effect that the characteristic is intended to impart.

[0100] As defined herein, "user" and "individual" each refer to a human being.

[0101] As used herein, terms such as first, second, etc. may be used to describe various elements, and these elements should not be limited by these terms, unless otherwise stated or clearly indicated by context, as they are only used to distinguish one element from another.

[0102] The present invention may be a system, method, or computer program product, or a combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium having thereon a computer-readable program for causing a processor to carry out aspects of the present invention.

[0103] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes®, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash® memory), static random access memory (SRAM), portable compact disc® read-only memory (CD®-ROM), digital versatile disc (DVD®), memory sticks®, floppy disks, punch cards or mechanically encoded devices such as raised structures in grooves having instructions recorded thereon, and any suitable combination of the above. As used herein, computer-readable storage media should not be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over electrical wires.

[0104] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, fiber optic transmission cables, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to a computer-readable storage medium within the respective computing / processing device for storage.

[0105] Computer-readable program instructions for carrying out operations of the present invention may be either source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk®, C++®, and procedural programming languages ​​such as the “C®” programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server, as a standalone software package. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, to carry out aspects of the present invention, electronic circuitry including, for example, programmable logic circuitry, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer-readable program instructions and personalize the electronic circuitry by utilizing state information of the computer-readable program instructions.

[0106] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0107] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to manufacture a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, generate means for implementing the function(s) / act(s) specified in a block or blocks of the flowchart(s) and / or block diagram(s). These computer-readable program instructions may also be stored in a computer-readable storage medium that may instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture including instructions for performing aspects of the function(s) / act(s) specified in a block or blocks of the flowchart(s) and / or block diagram(s).

[0108] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operable steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-executed process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / acts specified in a block or blocks of the flowcharts and / or block diagrams.

[0109] The flowchart diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart diagrams or block diagrams may represent a module, segment, or portion of an instruction, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be executed concurrently, substantially concurrently, partially, or fully overlapping in time as a single step, or the blocks may sometimes be executed in reverse order depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs specific functions or acts or executes a combination of dedicated hardware and computer instructions.

[0110] The description of various embodiments of the present invention has been presented for illustrative purposes and is not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer processor executed method comprising: capturing a plurality of human utterances using a speech interface; and generating, for a user selected from the plurality of human utterances, a corpus of human utterances including utterances of the user selected from the plurality of human utterances based on meaning of the human utterances, the meaning being determined by natural language processing of the human utterances by the computer processor; determining context information corresponding to one or more human utterances of the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor; recognizing patterns in the corpus of human speech based on pattern recognition performed by the computer processor using one or more machine learning models; identifying a forgetfulness pattern of the user based on the pattern recognition; classifying whether the user's forgetfulness pattern is likely to be caused by the user's memory impairment based on the context information corresponding to one or more human utterances of the corpus; and A method comprising:

2. 10. The method of claim 1, wherein the classifying comprises engaging the user in an interactive machine-generated dialogue in response to classifying the user's forgetfulness pattern as a symptom of memory impairment.

3. The method of claim 1 , wherein the classifying further comprises performing at least one of a behavioral analysis or a sentiment analysis to determine the user's state of mind.

4. 10. The method of claim 1, wherein determining the context information comprises performing a computer-based search of a context database in response to an utterance whose meaning is ambiguous or whose significance with respect to the user's forgetfulness is unknown.

5. The method of claim 1, further comprising formulating a remedial action plan in response to classifying the user's forgetfulness pattern as a symptom of memory impairment.

6. 6. The method of claim 5, wherein the remedial action plan includes engaging the user in interactive machine-generated dialogue at predetermined times to assist the user in performing one or more tasks.

7. 6. The method of claim 5, wherein the remedial action plan includes at least one of generating an electronic notification advising the user to consult a health professional, or generating an electronic notification and communicating the electronic notification to a caregiver informing the caregiver of the user's possible memory impairment.

8. 1. A system comprising: a processor configured to perform operations, the operations comprising: capturing a plurality of human utterances using a speech interface; and generating, for a user selected from the plurality of human utterances, a corpus of human utterances including utterances of the user selected from the plurality of human utterances based on meaning of the human utterances, the meaning being determined by natural language processing of the human utterances by a computer processor; determining context information corresponding to one or more human utterances of the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor; recognizing patterns in the corpus of human speech based on pattern recognition performed by the computer processor using one or more machine learning models; identifying a forgetfulness pattern of the user based on the pattern recognition; classifying whether the user's forgetfulness pattern is likely to be caused by the user's memory impairment based on the context information corresponding to one or more human utterances of the corpus; and Including, the system.

9. 10. The system of claim 8, wherein the classifying comprises engaging the user in an interactive machine-generated dialogue in response to classifying the user's forgetfulness pattern as a symptom of memory impairment.

10. The system of claim 8 , wherein the classifying further comprises performing at least one of a behavioral analysis or a sentiment analysis to determine the user's state of mind.

11. 10. The system of claim 8, wherein determining the context information comprises performing a computer-based search of a context database in response to an utterance whose meaning is ambiguous or whose significance with respect to the user's forgetfulness is unknown.

12. 10. The system of claim 8, wherein the processor is configured to perform operations further including developing a remedial action plan in response to classifying the user's pattern of forgetfulness as a symptom of memory impairment.

13. 13. The system of claim 12, wherein the remedial action plan includes engaging the user in an interactive machine-generated dialogue at predetermined times to assist the user in performing one or more tasks.

14. A computer program comprising: executable by a processor to cause the processor to perform operations, the operations comprising: capturing a plurality of human utterances using a speech interface; and generating, for a user selected from the plurality of human utterances, a corpus of human utterances including utterances of the user selected from the plurality of human utterances based on meaning of the human utterances, the meaning being determined by natural language processing of the human utterances by a computer processor; determining context information corresponding to one or more human utterances of the corpus based on data generated in response to signals sensed by one or more sensing devices operatively coupled to the computer processor; recognizing patterns in the corpus of human speech based on pattern recognition performed by the computer processor using one or more machine learning models; identifying a forgetfulness pattern of the user based on the pattern recognition; classifying whether the user's forgetfulness pattern is likely to be caused by the user's memory impairment based on the context information corresponding to one or more human utterances of the corpus; and a computer program comprising:

15. 15. The computer program product of claim 14, wherein the classifying comprises engaging the user in an interactive machine-generated dialogue in response to classifying the user's forgetfulness pattern as a symptom of memory impairment.

16. 15. The computer program product of claim 14, wherein the classifying further comprises performing at least one of a behavioral analysis or an emotional analysis to determine the user's state of mind.

17. 15. The computer program product of claim 14, wherein determining the context information comprises performing a computer-based search of a context database in response to an utterance whose meaning is ambiguous or whose significance with respect to the user's forgetfulness is unknown.

18. A computer program as described in claim 14, executable by the processor and causing the processor to perform operations further including formulating a remedial action plan in response to classifying the user's forgetfulness pattern as a symptom of memory impairment.

19. 20. The computer program product of claim 18, wherein the remedial action plan includes engaging the user in interactive machine-generated dialogue at predetermined times to assist the user in performing one or more tasks.

20. 20. The computer program product of claim 18, wherein the remedial action plan includes at least one of generating an electronic notification advising the user to consult a health professional, or generating an electronic notification and communicating the electronic notification to a caregiver informing the caregiver of the user's possible memory impairment.

Images