System and method for automated analysis of text in psychotherapy, counseling, and other mental health care activities - Patents.com

Abstract

A method for analyzing a patient's mental state and generating real-time results and predictions by automatically processing, ingesting and analyzing text-based and voice-based sources from patients and clinical staff.A system for processing, analyzing and managing patient input including data pools, model hubs, search services, topic modeling services, mental health related prediction services and analytics, all within electronic communications.A method for processing, analyzing and managing patient and clinical staff text and voice input and analyzing patients by informing and extending diagnostic and prognostic progress, identifying improvements and deteriorations in a patient's mental state, and identifying adverse events in psychotherapy, counseling and other mental health management activities.A system for processing, analyzing and managing clinical and diagnostic text and voice transcripts.

Description

[Technical field]

[0001] 2. Background of the Invention 1.Technical Field The present invention relates to a system and method for analyzing the mental state of a patient. [Background technology]

[0002] 2.Background technology The mental status of a patient may be evaluated by medical personnel during a medical appointment. General observations may be made, as well as specific tests of attention, executive function, cognition, language, memory, orientation, execution, prosody, thought content, thought processes, and visuospatial abilities.

[0003] 2. Description of the Related Art The processes and methods currently employed in psychotherapy, counseling, and other mental health care activities generate excessive amounts of text- and voice-based data input from both patients and clinical staff.

[0004] Data originating from patients is typically in the form of free and structured text responses to diagnostic assignments and tasks, patient diaries, patient journals, worry scripts, transcripts of patient / clinician discussions from patient treatment and therapy, written and transcribed responses to a structured set of questions, and audio recordings. Data originating from clinical staff is typically in the form of clinical staff clinical notes, initial assessment notes, progress notes, non-clinical notes from patient treatment and therapy, medication management notes, non-clinical and research staff notes, treatment plans, prescriptions, audio recordings, and release documents.

[0005] While all of the above data sources hold enormous clinical, diagnostic, and prognostic value, it is currently not possible to efficiently process these large unstructured data sources into comprehensive, clinically relevant output.

[0006] Several systems have been developed to assess mental states. For example, US Patent Application Publication No. 20170119297 discloses a computer-implemented method for assessing a mental state of a subject (106) that includes receiving (302) a heart rate record (200) of the subject as an input. The heart rate record includes a series of heart rate data samples acquired over a time span including a pre-sleep period (208), a sleep period (209) having a sleep onset time (224) and a sleep termination time (226), and a post-sleep period (210). At least a sleep onset time and a sleep termination time are identified (304) within the heart rate record. Next, a knowledge base (124) is accessed (306) that includes data acquired via expert evaluation of a training set of subjects and embodies a computer model of the relationship between mental state features and heart rate features. Using information within the knowledge base, the computer model is applied (308) to calculate at least one metric related to the mental state of the subject and to generate an index of the mental state based on the metric. An indication of mental state is provided as an output (310).

[0007] US Patent Application Publication No. 20130297536 discloses a system and method for collecting data and monitoring a user's mental health. The user's use of electronic devices such as the use of mobile phones, tablets and web activity is tracked. The invention "learns" each patient's unique behavioral pattern, which is used as a "baseline" representing the patient's steady state (chronic phase). An algorithmic processing unit detects any irregularities in the patient's behavioral pattern and generates an exacerbation prediction. If it is determined that a threshold is exceeded, an alert is sent to a health professional.

[0008] There remains a need for a system that can effectively monitor a patient's mental health and that can incorporate text-based sources of information. Summary of the Invention [Means for solving the problem]

[0009] Summary of the Invention The present invention provides a method for analyzing a patient's mental state and generating real-time results and predictions by automatically processing, capturing and analyzing text-based and voice-based sources from the patient and clinical staff.

[0010] The present invention provides a system for processing, analyzing, and managing patient input that includes a data pool, a model HUB, a search service, a topic modeling service, and mental health related predictive services and analytics, all within electronic communications.

[0011] The invention also provides a method for analyzing patients by processing, analysing and managing patient and clinical staff text and voice input and informing and extending diagnostic and prognostic progress, identifying improvements and deteriorations in a patient's mental state, and identifying adverse events in psychotherapy, counselling and other mental health management activities.

[0012] The present invention also provides a system for processing, analyzing and managing clinical and diagnostic text and audio transcripts including data pools, model hubs, search services, topic modeling services, mental health related prediction services and analytics all within electronic communications, said system being able to identify and highlight recurrent topics, critical moments and clinically valuable moments within clinical staff free text notes to manage the content / information load of clinical professionals and to present retrospective views or summaries of patients' treatment progress.

[0013] The present invention provides a method for analyzing clinical and diagnostic text by processing, analysing and managing clinical and diagnostic text, identifying and highlighting recurrent topics, critical moments and clinically valuable moments in the free text notes of clinical staff, managing the content / information load of clinical experts and presenting retrospective views or summaries of patient treatment progress.

[0014] Description of the drawings Other advantages of the present invention will be readily appreciated as the same becomes better appreciated by reference to the following detailed description when considered in conjunction with the accompanying drawings. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram of the system architecture. [Diagram 2] FIG. 1 is a diagram of an active learning and data collection system. [Diagram 3] A diagram of the assessment (classification) of pathological and non-pathological mental states from texts. [Figure 4] FIG. 1 is a diagram of specific psychiatric disorder severity and progression assessments from text. [Diagram 5] FIG. 1 is a line diagram highlighting portions of text with high mental disorder severity. [Figure 6] FIG. 1 is a diagram of a topic analysis of psychotherapeutic texts and patient reports. [Figure 7] 1 is a diagram projecting a psychotherapeutic session context onto a relational topic map. [Figure 8] FIG. 13 is a diagram predicting topic and keyword occurrence in the next session based on their presence (frequency / criticality) in the previous session. [Figure 9] FIG. 1 is a diagram of a search for patients reporting similar behavioral, psychological, or emotional symptoms, indicators, or conditions. [Figure 10] A diagram of the distillation of a large amount of psychotherapeutic text into shorter summaries. [Figure 11] A diagram of questions and answers regarding psychotherapeutic texts. [Figure 12] FIG. 1 is a diagram of text retrieval on psychotherapy transcripts based on text model embedding constrained by clinically relevant topics, systems or indicators. [Figure 13] FIG. 1 is a diagram that pairs critical psychological events with clinical relevance (or topics) with specific entities within the text identified via an entity recognition model. [Figure 14] Therapy notes are shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Detailed Description of the Invention The present invention provides a system for processing, analyzing, and managing patient input. The purpose of the system is to inform and enhance diagnostic and prognostic progression, to identify improvements and deteriorations in a patient's mental state, and to identify adverse events. The present invention automatically processes, ingests, analyzes, and generates real-time results and predictions with clinical values ​​from text-based sources. The system allows clinicians to i) efficiently search all generated content, ii) use diagnostic and prognostic metrics derived by the system from the content in therapy and treatment, and iii) detect increased risk of adverse events during a patient's progression.

[0017] The system is generally indicated at 10 in FIG. 1 and includes a data pool 12, a model HUB 14, a search service 16, a topic modeling service 18, a mental health related prediction service 20 and analytics 22, all in electronic communication.

[0018] A search service 16 uses the data pool 12 contents to queue queries and generate search results. A topic modeling service 18 uses the data pool 12 to define topics for topic extraction.

[0019] The model hub 14 provides retrained models for client facing features and services (such as a search service 16, a topic modeling service 18, and a mental health-related predictive service 20).

[0020] The Search Service 16 receives the query, text and metadata from the Analysis 22 and returns the most relevant existing content data pool 12. The Search Service 16 transforms the query and text metadata from the Analysis 22 through the model version coming from the Model HUB 14 and uses the new representation of the current text to search the data pool 12.

[0021] The topic modeling service 18 receives video stream data (e.g., clinician notes) and journals from analysis 22 and returns topic modeling relevant data. The topic modeling service 18 searches for the most relevant keywords and phrases that describe the topics of the input data. The topic modeling service 18 utilizes pre-computed data from the data pool 12 to perform its functions.

[0022] The mental health related prediction service 20 receives the video stream data, clinician notes and diaries from the analysis 22 and returns ratings and highlights in the text. The mental health related prediction service 20 selects the correct model type or version and makes a prediction for a particular user-facing feature or metric.

[0023] The analytics 22 represent user-facing metrics, features and indicators (visualizations, notifications, lists, counters, charts, etc.) The analytics 22 receives input from the search service 16, the topic modeling service 18 and the mental health-related predictive service 20 and provides feedback thereto.

[0024] The active learning model and data collection system of the system is shown in FIG. 2 at 30. While FIG. 1 depicts the functional architecture of the production system, FIG. 2 describes the functional architecture of the subsystems, in particular the architecture of the continuous machine learning model training. The FIG. 2 subsystems enable the system described in FIG. 1 to improve its diagnostic and prognostic performance over time. The system includes virtual treatment sessions 32, on-site treatment sessions 34, clinical staff 36, raw database 38, training dataset 40, trained machine learning model 42, data collection between successive sessions 44, analysis platform 46, and annotation environment 48.

[0025] The clinical staff 36 may receive information from the patient during the virtual treatment session 32 or the in-person treatment session 34 .

[0026] Video stream data and written notes are transmitted from the clinical staff database 36 to an internal live database 38 .

[0027] Training data is selected and transmitted from an internal raw database 38 to a training data set database 40. The database 38 also continuously receives video stream data and text-based data (e.g., diary) from the patient during the treatment session 44.

[0028] Model training is performed against the training dataset database 40 and generates a new version of the machine learning model 42.

[0029] The analytics platform 46 and annotation environment 48 receive data from the internal raw database 38 and the trained machine learning model 42. The analytics platform 46 sends active feedback to the annotation environment 48 for re-annotation. The analytics platform 46 and annotation environment 48 send new data annotations, re-annotated data based on model feedback, and re-annotated data based on active feedback from the analytics platform to the raw database 38.

[0030] The systems and methods of the present invention may be used with any type of therapy for any type of disorder in a patient and in combination with any type of medication.

[0031] Sources of input from patients can be, but are not limited to, free and structured text responses to diagnostic missions / tasks, patient diaries, journals, concern scripts, transcripts of patient / clinical staff discussions, written / transcribed responses to a structured set of questions, and analysis of audio recordings.

[0032] The system 10 may include the assessment (classification) of pathological and non-pathological mental states from patient-generated text and speech (shown in FIG. 3). The model input may be a random length text field (or speech transcript). The model type is Mental State NLP Classifier. The model output is a probability distribution over selected mental states related to the text. The training data are patient texts and transcripts from therapy sessions, diaries, psychotherapeutic interventions, with annotations of a given mental state (pathological or non-pathological) or bootstrapped public data with high correlation with a particular mental state. The user flow is text and speech generated by a first user (patient) that is enabled as input to the Mental State NLP Classifier model. The model processes the provided text in near real-time and identifies correlations and indicators of pathological and non-pathological mental states expressed in text by its author. The results are presented and visualized in a configurable interface to a second user (clinical staff) (alternatively to the first user (patient)). The output of the system is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0033] The system 10 may include assessment of specific mental disorder severity and progression from patient-generated text and speech (as shown in FIG. 4). The model input is a random length text field (or speech transcript). The model type is Mental Disorder Severity NLP Classifier. The model output is a score in the range 0-100 representing the severity of the specific mental disorder and its progression over time. The training data are text and transcripts from therapy sessions, diaries, psychotherapeutic interventions, along with severity annotations of a given mental condition (pathological or non-pathological) or bootstrapped public data with high correlation with specific mental condition severity. The user flow is text and speech generated by a first user (patient) that is enabled as input to the Mental Disorder Severity NLP Classifier model. The model processes the provided text in near real-time and identifies correlations and indicators of the severity of the specific mental disorder and / or its progression detected in the text. The results are presented and visualized in a configurable interface to a second user (clinical staff) (alternatively to the first user (patient)). The output of the system may be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0034] The system 10 may include highlighting portions of text with high mental disorder severity (as shown in FIG. 5). The model input is a random length text field (or audio transcript) output from the mental disorder severity text classifier and / or the mental condition text classifier. The model type is an explainability framework of models. The model output is (for example) color-coded portions of text with high severity, high relevance text extraction, severity score per topic, list of n-grams most correlated with specific mental disorders, automatic alerts. The training data is text and transcripts from therapy sessions, diaries, psychotherapeutic interventions, with severity annotations of a given mental condition (pathological or non-pathological), or bootstrap public data with high correlation with specific mental condition severity. The user flow is text and audio generated by a first user (patient) that is enabled as input to the explainability framework of NLP models. The framework plots the text in near real time and visually highlights correlations and indicators of severity of a specific mental disorder and / or its progression detected within the broader body of text. The results are presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to a first user (patient)). The output of the system 10 is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0035] The system 10 may include topic analysis of psychotherapy texts, audio transcripts and patient reports (as shown in FIG. 6). The model input is a random length text field (or audio transcript). The model type is a generic NLP model with the following hyperparameters: level of keyword diversity, number of keywords per topic, minimum relevance score of keywords. The model output is keywords with assigned scores representing importance. The training data is a model trained on a generic text corpus. The user flow is text and audio generated by a first user (patient) that is enabled as input to the generic NLP model. The model processes the text in near real time and identifies clinically and therapeutically relevant topics mentioned by the author. These topics are represented by a set of relevant keywords. The topics and keywords are presented and visualized in a configurable interface to a second user (clinical staff) (alternatively to the first user (patient)). Topics can be ordered by importance or by: specific topic criticality, relevance, frequency and repetition of a topic, relationship to other topics, relationship to diagnosis, relationship to disorder progression, relationship to treatment course or stage, embedding of all elements (keywords, topics, paragraphs and other metrics). The output of the system 10 can be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0036] The system 10 may include projecting psychotherapy session content into a topic map of relationships (as shown in FIG. 7). The model input is the output of topic analysis of psychotherapy texts and patient reports with embeddings of all elements (keywords, topics, paragraphs and other metrics). There may be a visual architecture of plots and related interactive features. The model output is each text / paragraph of text visualized into a multi-dimensional topic map with distances representing semantic similarity and highlighted areas representing characteristic topics. This may be combined with disorder severity and progress indicators to plot trends. Output features are visual tracking of topic evolution over time on the semantic map (select topic and time range and watch animation), tracking mental condition / topic severity over time, highlighting critical topics for a specific patient over the entire treatment time range, highlighting semantically related topics by variables (similarity threshold), highlighting only detected topics related to a specific pathological or non-pathological mental condition, mental disorder severity or type. Model variables are the minimum amount of text belonging to a topic to actually form a topic, the minimum probability of text belonging to a given topic, clustering algorithm variables (e.g. HDBSCAN algorithm properties), and n-gram size represents how many vocabulary words can form one text entity. User flows are text and speech generated by a first user (patient) that are enabled as input to a visualization framework and plotting architecture. The system processes text variables in near real-time and identifies correlations and indicators of severity of a particular psychiatric disorder and / or its progression for a particular patient or group of patients extracted from a broader body of text. The results are plotted on a multi-dimensional topic map with configurable characteristics (see model output). An interface to a second user (clinical staff) (alternatively to the first user (patient)) enables visual analysis of the extracted and mapped topics and their relationship to diagnosis, progression, other topics and therapeutic interventions. The output of the system 10 is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0037] The system 10 includes predicting the occurrence of topics and keywords in the next treatment session based on their presence (frequency and criticality) in the previous session (shown in FIG. 8). The model input is the previous session (random length text field or audio transcript) with timestamp. The model types are a general NLP model (for generating topics), a general model for predicting next session topics. The model output is a probability distribution for the next topic. The user flow is the text and audio generated by the first user (patient) that is enabled as input to the general NLP model and the general prediction model. The system processes the text in near real time and identifies the topics and keywords that are most likely to occur in future therapeutic interactions (psychotherapeutic dialogues, monologues and future patient generated texts). The results are presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)). The identified topics and keywords can be presented by degree of their predicted probability of reoccurrence or by their criticality to the treatment course (or both as indicators). The output of the system 10 may be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0038] The system 10 may include a patient-generated text search of patients reporting similar behavioral, psychological, or emotional symptoms, disorder indicators, or mental conditions (as shown in FIG. 9). The model input is a random length text field (or audio transcript). The model type is a generic NLP model with the following hyperparameters: minimum level of similarity, and distance metric. The model output is a pointer to sessions or patients with similar content / topics. The training data is in the first iteration a pre-trained model is used, and another model can be fine-tuned on the same dataset as that of the classification of mental health problems. The user flow is the text and audio generated by the first user (patient) that is enabled as input to a generic NLP model with hyperparameters of minimum level of similarity and distance metric. The system processes the text in near real-time and identifies commonalities (by keyword co-occurrence, topic co-occurrence, and their relationships). The next result is i) an ordered couplet (or group) of patients, behaviors, psychological indicators, emotional indicators reported in the processed text; ii) a set of pointers to therapeutic events (e.g., sessions) to patients or groups of patients with system-detected similarities in the reported content or topic, presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)) as text snippets of configurable length, text extracts or in-text highlights. The automatic result retains text with high therapeutic and clinical relevance and groups entities and events with similar reported clinical indicators and experiences (e.g., trauma). The output of the system 10 is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0039] The system 10 may synthesize large amounts of patient-generated text into shorter summaries with clinical and diagnostic relevance (as shown in FIG. 10). The model input is a random-length patient-generated text field (or audio transcript). The model type is an NLP text summarization model. The model output is a larger text summary that describes the content at a high level by highlighting critical aspects of the larger psychotherapeutic text input. The training data is session transcripts (e.g., summaries and clinical summaries from psychotherapeutic sessions). The user flow is text and audio generated by a first user (patient) that is validated as input to the NLP text summarization model. The system processes the text in near real-time and summarizes large bodies of text to compress summaries and extract the most relevant moments in the text. The results are presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to the first user (patient)). The automatically generated summaries retain the text with high therapeutic and clinical relevance. The output of the system 10 is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0040] The system 10 may include answering questions on patient-generated text (as shown in FIG. 11). The model input is a patient-generated pool of text used as a source for generic questions and answers and questions. The model type is a generic NLP question-answering model. The model output is a pointer to a pool of text where answers may reside. The training data is a dataset of tuples of text, questions, and pointers to answers. The user flow is text and speech generated by a first user (patient) that is enabled as input to the generic NLP question-answering model. The user formulates a question (speech or in text) as input to a configurable interface with search constraints of clinical relevance, therapeutic indication, time scale, and other types of search constraints. The system processes the text in near real-time. The result is a set of pointers to locations in the pool of text where the most likely answers may reside, presented and visualized in a configurable interface to a second user (clinical staff) (alternatively to the first user (patient)) as text snippets of configurable length, text extracts, or in-text highlights. The automatic results retain text with high therapeutic and clinical relevance. The output of the system 10 can be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0041] The system 10 may include text search across patient-generated texts and transcripts based on text model embeddings constrained by clinically relevant topics, symptoms or indicators (as shown in FIG. 12). The model input is a pool of texts that the user wants to search, which may be indexed by using a query (and optionally the severity of the mental health problem related to the query) in terms of paragraph embeddings, words or sentences. The model type is a general NLP embedding model. The model output is the most semantically similar data found in the text pool, represented by paragraphs, keywords or sentences. The training data are texts and transcripts from therapy sessions, diaries, psychotherapeutic interventions, with severity annotations of a given mental condition (pathological or non-pathological), or bootstrap public data with high correlation with specific mental condition severity, for embedding fine-tuning into the next interaction. The user flow is the text and speech generated by the first user (patient) that is validated as input to the general NLP embedding model. A user inputs a search query into a configurable interface with clinical relevance, therapeutic indications, time-scale search constraints, and other types of search constraints. The system processes the text in near real-time. The result is a list of the most semantically similar data found in the text pool, presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to a first user (patient)) as text snippets and text extractions of configurable length. The automatically generated text snippets retain text with high therapeutic and clinical relevance. The output of the system 10 is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0042] The system 10 may include pairing critical psychological events of clinical relevance with concrete entities in patient-generated text identified via an entity recognition model (shown in FIG. 13). The model input is a random length patient-generated text field (or audio transcript). The model types are NER model, mental disorder severity NLP classifier and / or mental state NLP classifier. The model output is the entities (people, organizations, places) most related to the source of anxiety in the given text, their emotional relationships and impact in the text, and the evolution of the relationships over time. The training data are texts and transcripts from therapy sessions, diaries, psychotherapeutic interventions, with annotations of given entities in the text. The user flow is the text and audio generated by the first user (patient) that is enabled as input to the NER model, mental disorder severity NLP classifier or / and mental state NLP classifier. The system processes the text in near real-time and identifies topics (defined by a set of keywords), events, entities (e.g. people, institutions, organizations, groups, objects, etc.) that have clinical relevance or high value for the treatment process. The results are presented and visualized in a configurable interface to a second user (clinical staff) (alternatively to a first user (patient)). The identified events, topics, entities and keywords are contextually paired to generate a problem space network represented by nodes (events, entities, topics) and edges (their relationship with directions, capabilities or qualities) or can be presented in order of their criticality to the treatment process. The output of the system 10 is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0043] The invention also provides a method for analyzing patients by processing, analyzing, and managing patient and clinical staff text and voice input and informing and extending diagnostic and prognostic progress, identifying improvements and deteriorations in a patient's mental state, and identifying adverse events in psychotherapy, counseling, and other mental health management activities, the steps of which may be performed as described above and may include the components described above.

[0044] The present invention also provides a system 10 for processing, analyzing, and managing clinical and diagnostic text and audio recordings. The goal of the system is to identify and highlight recurrent topics, critical moments, and clinically valuable moments in the free text notes of clinical staff to manage the content / information load for clinical professionals and to present a retrospective view or summary of a patient's treatment progress (within and between sessions).

[0045] Sources of input from the patient may be, but are not limited to, clinical staff clinical notes, initial assessment notes, progress notes, non-clinical notes from the patient's treatment and therapy, medication management notes, clinical and research staff notes, treatment plans, prescriptions, audio recordings, and release documents.

[0046] The system may include the assessment (classification) of pathological and non-pathological mental conditions from clinical and diagnostic text and audio. The model input is a random length text field (or audio transcript). The model type is Mental State NLP Classifier. The model output is a probability distribution over the selected mental conditions associated with the text. The training data are patient text and transcripts from therapy sessions, diaries, psychotherapeutic interventions, clinical staff diagnostic notes from patient treatment and therapy, non-clinical notes from patient treatment and therapy, medication management notes, clinical and research staff notes, prescriptions, release documents, along with annotations of a given mental condition (pathological or non-pathological) or bootstrapped public data with high correlation with a particular mental condition. The user flow is text and audio generated by a second user (clinical staff) that is validated as input to the Mental State NLP Classifier model. The model processes the provided text in near real time and identifies correlates and indicators of pathological and non-pathological mental conditions expressed in text by its author. The results are presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to a first user (patient)). The output of the system is used in the diagnostic process, in psychotherapy or for continuous patient monitoring.

[0047] The system may include assessment of specific mental disorder severity and progression from clinical and diagnostic text and audio. Model inputs are random length clinical and diagnostic text fields (or audio transcripts). Model type is Mental Disorder Severity NLP Classifier. Model output is a score in the range 0-100 representing the severity of the specific mental disorder and its progression over time. Training data are patient text and transcripts from therapy sessions, diaries, psychotherapeutic interventions, clinical staff diagnostic notes from patient treatment and therapy, non-clinical notes from patient treatment and therapy, medication management notes, clinical and research staff notes, prescriptions, release documents, along with severity annotations of a given mental condition (pathological or non-pathological) or bootstrapped public data with high correlation with specific mental condition severity. User flow is text and audio generated by a second user (clinical staff) that is validated as input to the Mental Disorder Severity NLP Classifier model. The model processes the provided text in near real time and identifies correlations and indicators of the severity of the specific psychiatric disorder and / or its progression detected in the text. The results are presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to a first user (patient)). The output of the system is used in the diagnostic process, in psychotherapy or for continuous patient monitoring.

[0048] The system may include highlighting portions of clinical and diagnostic text with high mental disorder severity. The model input is a random length text field (or audio transcript) output from the mental disorder severity text classifier and / or the mental condition text classifier. The model type is the model explainability framework. The model output is (for example) color-coded portions of text with high severity, high relevance text extraction, severity score per topic, list of n-grams most correlated with specific mental disorders, and automatic alerts. The training data is text and transcripts from therapy sessions, diaries, psychotherapeutic interventions, clinical staff diagnostic notes from patient treatment and therapy, non-clinical notes from patient treatment and therapy, medication management notes, clinical and research staff notes, prescriptions, release documents, along with severity annotations of a given mental condition (pathological or non-pathological) or bootstrapped public data with high correlation with specific mental condition severity. The user flow is text and audio generated by a second user (clinical staff) that is enabled as input to the NLP model explainability framework. The framework plots the text in near real-time and visually highlights correlations and indicators of the severity of a particular psychiatric disorder and / or its progression detected within the broader body of text. The results are presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to a first user (patient)). The output of the system is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0049] The system may include topic analysis of clinical and diagnostic texts, audio transcripts, and clinical staff notes and reports. Model input is a random length text field (or audio transcript). Model type is a generic NLP model with the following hyperparameters: level of keyword diversity, number of keywords per topic, and minimum relevance score of keywords. Model output is keywords with assigned scores representing importance. Training data is a model trained on a generic text corpus. User flow is text and audio generated by a second user (clinical staff) that is enabled as input to the generic NLP model. The model processes the text in near real time and identifies clinically and therapeutically relevant topics mentioned by the author. These topics are represented by a set of relevant keywords. The topics and keywords are presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)). Topics can be ordered by importance or by: specific topic criticality, relevance, frequency and repetition of a topic, relationship to other topics, relationship to diagnosis, relationship to disorder progression, relationship to treatment course or stage, embedding of all elements (keywords, topics, paragraphs and other metrics). The output of the system can be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0050] The system may include projecting clinical and diagnostic texts and audio generated in psychotherapy sessions into a topic map of relationships. The model input is the output of topic analysis of psychotherapy texts and patient reports with embeddings of all elements (keywords, topics, paragraphs and other metrics). There may be a visual architecture of plots and related interactive features. The model output is each text / paragraph of text visualized into a multi-dimensional topic map where distances represent semantic similarity and highlighted areas represent characteristic topics. This may be combined with disorder severity and progress indicators to plot trends. Output features are visual tracking of topic evolution over time on the semantic map (select topic and time range and watch animation), tracking mental condition / topic severity over time, highlighting critical topics for a specific patient over the entire treatment time range, highlighting semantically related topics with variables (similarity threshold), highlighting only detected topics related to a specific pathological or non-pathological mental condition, mental disorder severity or type. Model variables are the minimum amount of text belonging to a topic to actually form a topic, the minimum probability of text belonging to a given topic, clustering algorithm variables (e.g. HDBSCAN algorithm properties), and n-gram size represents how many vocabulary words can form one text entity. User flows are text and speech generated by a second user (clinical staff) that are enabled as input to a visualization framework and plotting architecture. The system processes the text variables in near real time and identifies correlations and indicators of severity of a particular patient or patient group, a particular psychiatric disorder and / or its progression extracted from a broader body of text. The results are plotted on a multidimensional topic map with configurable characteristics (see model output). An interface for the second user (clinical staff) (or alternatively for the first user (patient)) enables visual analysis of the extracted and mapped topics and their relationship to diagnosis, progression, other topics, or particular past therapeutic interventions.The output of the system may be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0051] The system includes predicting the occurrence of topics and the occurrence of keywords in the next therapeutic session based on their presence (frequency and criticality) in the previous session. Model inputs are previous sessions (random length text fields or audio transcripts) with timestamps, psychotherapeutic interventions, clinical staff diagnostic notes from the patient's treatment and therapy, non-clinical notes from the patient's treatment and therapy, medication administration notes, clinical and laboratory staff notes, prescriptions, and release documents. Model types are general NLP model (for generating topics), general model for predicting next session topics. Model outputs are probability distributions over the next topics. User flows are text and audio generated by a second user (clinical staff) that are enabled as input to the general NLP model and the general prediction model. The system processes the text in near real time and identifies topics and keywords (psychotherapeutic dialogue, monologues, and future patient generated text) that are most likely to occur in future therapeutic interactions. Results are presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)). The identified topics and keywords can be presented according to their predicted probability of reoccurrence or according to their criticality to the course of treatment (or both as indicators). The output of the system can be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0052] The system may include clinical and diagnostic text searches of patients reporting similar behavioral, psychological or emotional symptoms, disorder indicators or mental conditions. Model input is a random length text field (or audio transcript). Model type is a generic NLP model with the following hyperparameters: minimum level of similarity and distance metric. Model output is a pointer to sessions or patients with similar content / topics. Training data is in the first iteration a pre-trained model is used, another model can be fine-tuned on the same dataset as that of the classification of mental health problems. User flow is text and audio generated by a second user (clinical staff) that is enabled as input to a generic NLP model with hyperparameters of minimum level of similarity and distance metric. The system processes the text in near real-time and identifies commonalities (by keyword co-occurrence, topic co-occurrence and their relationships). The results are: i) an ordered couplet (or group) of patient, behavioral, psychological and emotional indicators reported in the processed text; ii) a set of pointers to therapeutic events (e.g. sessions) to patients or groups of patients with system-detected similarities in the reported content or topic, presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)) as text snippets of configurable length, text extracts or in-text highlights. The automatic results retain texts with high therapeutic and clinical relevance and group entities and events with similar reported clinical indicators and experiences (e.g. trauma). The output of the system is used in the diagnostic process, in psychotherapy or for continuous patient monitoring.

[0053] The system can synthesize large amounts of clinical and diagnostic text into short summaries with clinical and diagnostic relevance. Model inputs are random-length patient-generated text fields (or audio transcripts). Model type is an NLP text summarization model. Model output is summaries of larger texts that describe the content at a high level by highlighting critical aspects of the larger psychotherapeutic text input. Training data are transcripts of therapy sessions (e.g., summaries and clinical summaries from psychotherapeutic sessions, psychotherapeutic interventions, clinical staff diagnostic notes from patient treatment and therapy, non-clinical notes from patient treatment and therapy, medication management notes, clinical and laboratory staff notes, prescriptions, and release documents). User flow is text and audio generated by a second user (clinical staff) that is enabled as input to the NLP text summarization model. The system processes the text in near real-time and summarizes large bodies of text to condense summaries by extracting the most relevant moments in the text. Results are presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)). The automatically generated summaries retain text with high therapeutic and clinical relevance. The output of the system can be used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0054] The system may include question-answering on clinical and diagnostic text. The model input is a patient-generated pool of text used as a source for generic questions and answers and questions. The model type is a generic NLP question-answering model. The model output is a pointer to a pool of text where answers may exist. The training data is a dataset of tuples of text, questions, and pointers to answers. The user flow is text and speech generated by a second user (clinical staff) that is enabled as input to the generic NLP question-answering model. The user formulates a question (speech or in text) as input to a configurable interface with search constraints of clinical relevance, therapeutic indications, time scale, and other types of search constraints. The system processes the text in near real-time. The result is a set of pointers to locations in the pool of text. Here, the most likely answer is the one that is presented and visualized in a configurable interface to the second user (clinical staff) (alternatively to the first user (patient)) as a text snippet of configurable length, a text extract, or an in-text highlight. The automatic results retain text with high therapeutic and clinical relevance. The output of the system is used in the diagnostic process, in psychotherapy or for continuous patient monitoring.

[0055] The system may include text search across clinical and diagnostic texts and transcripts based on text model embeddings constrained by clinically relevant topics, symptoms or indicators. The model input is the pool of text that the user wants to search, indexed by using a query in terms of paragraph embeddings, words or sentences (and optionally the severity of the mental health problem related to the query). The model type is a general NLP embedding model. The model output is the most semantically similar data found in the text pool represented by paragraphs, keywords or sentences. The training data are text and transcripts from therapy sessions, diaries, psychotherapeutic interventions, clinical staff diagnostic notes from patients' treatment and therapy, non-clinical notes from patients' treatment and therapy, medication management notes, clinical and research staff notes, prescriptions, release documents, or bootstrapped public data with high correlation to specific mental conditions, for embedding fine-tuning into the next interaction. The user flow is text and speech generated by a second user (clinical staff) that is validated as input to the general NLP embedding model. A user inputs a search query into a configurable interface with search constraints of clinical relevance, therapeutic indications, time scale, and other types of search constraints. The system processes the text in near real-time. The result is a list of the most semantically similar data found in the text pool that is presented and visualized in a configurable interface to a second user (clinical staff) (or alternatively to a first user (patient)) as text snippets of configurable length and text extractions. The automatically generated text snippets retain text with high therapeutic and clinical relevance. The output of the system is used in the diagnostic process, in psychotherapy, or for continuous patient monitoring.

[0056] The system may include pairing critical psychological events with clinical relevance with concrete entities in clinical and diagnostic texts identified via entity recognition models. Model inputs are random length patient generated text fields (or audio transcripts). Model types are NER model, mental disorder severity NLP classifier and / or mental state NLP classifier. Model outputs are entities (people, organizations, places) most relevant to the source of anxiety in a given text, their emotional associations and impacts in the text, and the evolution of the associations over time. Training data are texts and transcripts from therapy sessions, psychotherapeutic interventions, clinical staff diagnostic notes from patient treatment and therapy, non-clinical notes from patient treatment and therapy, medication management notes, clinical and research staff notes, prescriptions, release documents. User flows are texts and audio generated by a second user (clinical staff) that are enabled as inputs to the NER model, mental disorder severity NLP classifier and / or mental state NLP classifier. The system processes the text in near real-time and identifies topics (defined by a set of keywords), events, entities (e.g. people, institutions, organizations, groups, objects, etc.) that have clinical relevance or high value for the treatment process. The results are presented and visualized in a configurable interface to a second user (clinical staff) (alternatively to a first user (patient)). The identified events, topics, entities and keywords are contextually paired to generate a problem space network represented by nodes (events, entity topics) and edges (their relationship with directions, capabilities or qualities) or can be presented in order of their criticality to the treatment process. The output of the system is used in the diagnostic process, in psychotherapy or for continuous patient monitoring.

[0057] The present invention provides a method for analyzing clinical and diagnostic text by processing, analysing and managing clinical and diagnostic text, identifying and highlighting recurrent topics, critical moments and clinically valuable moments in free text notes of clinical staff, managing the content / information load for clinical professionals and presenting a retrospective view or summary of a patient's treatment progress. These steps may be performed as described above and may include the components described above.

[0058] The present invention will be described in more detail by referring to the following experimental examples. These examples are provided for illustration only and are therefore not intended to be limiting unless otherwise specified. Therefore, the present invention should not be interpreted as being limited to the following examples in any way, but rather as encompassing any variations that become evident as a result of the teachings provided herein.

[0059] Example 1 14 shows an example of a personal therapy note, which is a common example of machine learning model input (i.e., text with potential clinical value that is processed by system 10).

[0060] Throughout this application, various publications, including several U.S. patents, are referenced by author, year and patent number. Full citations for these publications are listed below. The disclosures of these publications and patents in their entireties are incorporated by reference into this application in order to more fully describe the state of the art to which this invention pertains.

[0061] The invention has been described in an illustrative manner, and it is to be understood that the terminology which has been used is intended to be words of description rather than of limitation.

[0062] Obviously, many modifications and variations of the present invention are possible in light of the above teachings, and it is therefore to be understood that within the scope of the appended claims the invention may be practiced otherwise than as specifically described.

Claims

1. A method for analyzing a patient's mental state, A process for automatically processing, capturing, and analyzing text-based and audio-based sources from the aforementioned patients and clinical staff; A step of inputting the text-based and / or audio-based source into a mental state text classifier and outputting a text field and / or audio transcript, wherein the text field and / or audio transcript shows a probability distribution across the selected mental states related to the text field and / or audio transcript, and / or inputting the text-based and / or audio-based source into a mental severity text classifier and outputting a text field and / or audio transcript showing a score representing the severity of a specific mental disorder and its progression over time; A step of highlighting portions of text having a high severity of mental disorder using an explainability framework, wherein the text originates from the text field and / or audio transcript, the explainability framework plots the text, and visually highlights the correlations and / or indicators of the severity of the specific mental disorder and / or its progression detected within the text; and A method comprising the step of generating real-time results and predictions of the highlighting step.

2. The method according to claim 1, wherein the real-time results include at least one of the color-coded portions of text having high severity, highly relevant text extraction, severity score per topic, a list of n-grams most correlated with the specific mental disorder, and an automatic alert.

3. The method according to claim 1, wherein the text-based and audio-based sources are selected from a group consisting of analyses of free-text and structured text responses to diagnostic tasks, patient diaries, journals, concern scripts, transcripts of patient / clinical staff discussions, written / transcribed answers to a structured set of questions, and analyses of audio recordings.

4. The method according to claim 1, further comprising the step of providing an assessment of pathological and non-pathological mental states from patient-generated text and voice.

5. The method according to claim 1, wherein the generation step further includes presenting the results within a user's visual interface.

6. The method according to claim 1, further comprising the step of providing an assessment of the severity and progression of a specific mental disorder from patient-generated text and voice.

7. The method according to claim 1, further comprising the step of providing topic analysis of psychiatric texts, audio transcripts, and patient reports.

8. The method according to claim 1, further comprising the step of projecting the content of a psychotherapy session into a relationship topic map.

9. The method according to claim 1, further comprising the step of predicting the occurrence of topics and keywords in the next therapeutic session based on their presence in the previous session.

10. The method according to claim 1, further comprising the step of providing a patient-generated text search for patients reporting similar behavioral, psychological, or emotional symptoms, disorder indicators, or mental states.

11. The method according to claim 1, further providing a process for summarizing a large amount of patient-generated text into shorter summaries based on clinical and diagnostic relevance.

12. The method according to claim 11, further comprising the step of answering questions regarding patient-generated text.

13. The method according to claim 1, further comprising the step of performing a text search across patient-generated text and transcripts based on a text model embedding constrained by clinically relevant topics, signs, or indicators.

14. The method according to claim 1, further comprising the step of pairing critical psychological events having specific entities in patient-generated text identified via an entity recognition model.

15. A system for processing, analyzing, and managing patient input, which includes a data pool, model hub, search service, topic modeling service, mental health-related prediction service, and analysis, all within electronic communication. The mental health-related prediction service is a system that receives video stream data, clinician notes, and diaries from the analysis, inputs text-based and / or audio-based sources into a mental state text classifier, and outputs text fields and / or audio transcripts, wherein the text fields and / or audio transcripts output text fields and / or audio transcripts showing a probability distribution across selected mental states related to the text fields and / or audio transcripts, and / or inputs the text-based and / or audio-based sources into a mental severity text classifier, outputs text fields and / or audio transcripts showing scores representing the severity of specific mental disorders and their progression over time, returns evaluations and highlights in the text using an explainability framework, wherein the text originates from the text fields and / or audio transcripts, the explainability framework plots the text, and visually highlights correlations and / or indicators of the severity of specific mental disorders and / or their progression detected in the text.

16. The system according to claim 15, wherein the search service uses data pool content to save queries and generate search results.

17. The system according to claim 15, wherein the topic modeling service uses the data pool to define topics for topic extraction.

18. The system according to claim 17, wherein the topic modeling service receives video stream data and a log from the analysis and returns topic modeling-related data.

19. The system according to claim 15, wherein the model hub provides retrained models for clients facing features and services.

20. The system according to claim 15, wherein the search service receives queries, text, and metadata from the analysis, returns the most relevant existing content data pool, transforms the queries and text metadata from the analysis via a model version coming from the model hub, and uses the new representation of the current text to search the data pool.

21. The system according to claim 15, wherein the analysis includes user-facing metrics including visualizations, notifications, lists, counters, and charts, and receives input from the search service, the topic modeling service, and the mental health-related prediction service, and provides feedback to them.

22. A method for analyzing patients, The process of processing, analyzing, and managing text and voice input from patients and clinical staff; A step of inputting the aforementioned text and / or audio input into a mental state text classifier and outputting a text field and / or audio transcript, wherein the text field and / or audio transcript shows a probability distribution across the selected mental states related to the text field and / or audio transcript, and / or inputting the aforementioned text and / or audio input into a mental severity text classifier and outputting a text field and / or audio transcript showing a score representing a specific mental disorder and the severity of its progression over time; A step of notifying and expanding on the diagnosis and prognosis, and identifying the improvement and deterioration of the patient's mental state; A step of highlighting portions of text having a high severity of mental disorder using an explainability framework, wherein the text originates from the text field and / or audio transcript, and the explainability framework plots the text and visually highlights the correlations and / or indicators of the severity of specific mental disorders and / or their progression detected within the text; and A method comprising the step of identifying adverse events in psychotherapy, counseling, and mental health management activities.

23. A system for processing, analyzing, and managing clinical and diagnostic text and audio transcripts, which includes a data pool, model hub, search service, topic modeling service, mental health-related prediction service, and analysis, all within electronic communications, the system is capable of identifying and highlighting recurring topics, critical moments, and clinically valuable moments within free text notes of clinical staff in order to manage the content / information load for clinical professionals and to present a retrospective view or summary of the patient's treatment progress. The mental health-related prediction service receives video stream data, clinician notes, and diaries from the analysis, inputs text-based and / or audio-based sources into a mental state text classifier, and outputs text fields and / or audio transcripts, wherein the text fields and / or audio transcripts output text fields and / or audio transcripts showing a probability distribution across selected mental states related to the text fields and / or audio transcripts, and / or inputs text-based and / or audio-based sources into a mental severity text classifier, outputs text fields and / or audio transcripts showing scores representing the severity of specific mental disorders and their progression over time, returns evaluations and highlights in the text using an explainability framework, wherein the text originates from the text fields and / or audio transcripts, the explainability framework plots the text, and visually highlights correlations and / or indicators of the severity of specific mental disorders and / or their progression detected in the text. system.

24. A method for analyzing clinical and diagnostic texts, Processes for processing, analyzing, and managing clinical and diagnostic texts; The process of identifying and highlighting relapse topics, critical moments, and clinically valuable moments within free-text notes of clinical staff; The process involves inputting a text-based and / or audio-based source into a mental state text classifier and outputting a text field and / or audio transcript, wherein the text field and / or audio transcript shows a probability distribution across selected mental states related to the text field and / or audio transcript; and / or inputting a text-based and / or audio-based source into a mental severity text classifier and outputting a text field and / or audio transcript showing a score representing the severity of a specific mental disorder and its progression over time; A step of highlighting portions of text having a high degree of mental disorder severity using an explainability framework, wherein the text originates from the text field and / or audio transcript, and the explainability framework plots the text and visually highlights the correlations and / or indicators of the severity of specific mental disorders and / or their progression detected within the text; Processes for managing the content / information load of clinical professionals; and A method comprising the step of presenting a retrospective view or summary of the patient's treatment progress.

25. The method according to claim 24, wherein the input source is selected from a group consisting of clinical notes from clinical staff, initial assessment notes, progress notes from patient treatment and therapy, non-clinical notes from patient treatment and therapy, medication management notes, clinical and research staff notes, treatment plans, prescriptions, audio recordings, release documents, and combinations thereof.

26. The method according to claim 24, further comprising the step of evaluating pathological and non-pathological mental states from clinical and diagnostic texts and audio.

27. The method according to claim 24, further comprising the step of evaluating the severity and progression of a specific mental disorder from clinical and diagnostic texts and audio.

28. The method according to claim 24, further comprising the step of providing topic analysis of clinical and diagnostic texts, audio transcripts, and clinical staff notes and reports.

29. The method according to claim 24, further comprising the step of projecting clinical and diagnostic texts and audio generated in a psychotherapy session onto a relational topic map.

30. The method according to claim 24, further comprising the step of predicting the occurrence of topics and keywords in the next therapeutic session based on their presence in the previous session.

31. The method according to claim 24, further comprising the step of searching for clinical and diagnostic texts of patients reporting similar behavioral, psychological or emotional signs, disorder indicators or mental states.

32. The method according to claim 24, further comprising the step of summarizing a large volume of clinical and diagnostic texts into shorter summaries based on their clinical and diagnostic relevance.

33. The method according to claim 32, further comprising the step of answering questions regarding the aforementioned clinical and diagnostic texts.

34. The method according to claim 23, further comprising the step of performing a text search across the entire clinical and diagnostic text and transcript based on a text model embedding constrained by clinically relevant topics, signs, or indicators.

35. The method according to claim 24, further comprising the step of pairing critical psychological events having specific entities in the clinical and diagnostic text identified via an entity recognition model.

36. The method according to claim 24, further comprising presenting the results within a user's visual interface.

37. A method for improving the treatment of a patient by analyzing the patient's mental state, wherein the patient is receiving treatment for a mental disorder, and the method is A process of training a machine learning model using a dataset, wherein the dataset includes data from a group of patients who have previously received treatment for mental disorders; A step of acquiring patient information during a treatment session, wherein the patient information includes patient information from at least one text-based source; A step of analyzing patient information using the trained machine learning model, wherein the model is a mental state text classifier that takes the text base source as input and outputs text fields and / or audio transcripts, the fields and / or audio transcripts including, as input, text fields and / or audio transcripts output from a mental state text classifier and / or a mental severity text classifier that takes the text base source as input and outputs text fields and / or audio transcripts representing specific mental disorders and the severity of their progression over time; A step of outputting a score that quantifies the mental state of the patient; and A step of determining whether the treatment should be modified based on the score. Methods that include...

38. The method according to claim 37, further comprising presenting the score via a visual interface.

39. The method according to claim 37, wherein the information of the at least one text-based source includes the patient's diary, journal, concern script, transcript of patient / clinical staff discussion, written / transcribed answers to a structured set of questions, clinical notes from clinical staff from patient treatment and therapy, initial assessment notes, progress notes, non-clinical notes from patient treatment and therapy, medication management notes, clinical and research staff notes, treatment plans, prescriptions, or a combination thereof.

40. The method according to claim 37, wherein the acquired patient data further comprises at least one voice-based source.

41. The method according to claim 37, wherein the acquired patient information and scores become part of the dataset used to train the machine learning model.

Images