Ai based clinical decision support system (CDSS) for mental health

Abstract

There is provided a processor configured for executing code external to a ELM and interfacing with the ELM for: grounding the ELM to predefined disorder specific structured flows, determining a differential diagnosis of mental disorders, for a first mental disorder, generating prompts for guiding the ELM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow, dynamically analyzing a response(s) for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder, dynamically switching from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the ELM, and iterating the generating prompts for guiding the ELM, the dynamically analyzing, and the dynamically switching, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

Description

[0001] Al BASED CLINICAL DECISION SUPPORT SYSTEM (CDSS) FOR MENTAL HEALTH

[0002] RELATED APPLICATION

[0003] This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 627,104 filed on 31 January 2024, the contents of which are incorporated herein by reference in their entirety.

[0004] BACKGROUND

[0005] The present invention, in some embodiments thereof, relates to large language models used in clinical decision support systems (CDSS) and, more specifically, but not exclusively, to a LLM based CDSS for mental health.

[0006] A Clinical Decision Support System is a computer-based system designed to assist healthcare professionals in making clinical decisions by providing relevant information at the point of care. These systems can integrate various types of data, such as patient records, medical literature, and best practice guidelines, to offer recommendations for diagnosis, treatment, and patient management.

[0007] SUMMARY

[0008] According to a first aspect, a system for automatically diagnosing a subject with at least one mental disorder, comprises: at least one processor for executing code external to a large language model (LLM), the code configured for interfacing with the LLM for: grounding the LLM to a plurality of predefined disorder specific structured flows, determining a differential diagnosis comprising a plurality of mental disorders, for a first mental disorder, generating prompts for guiding the LLM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow, dynamically analyzing at least one response for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder, dynamically switching from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM, and iterating the generating prompts for guiding the LLM, the dynamically analyzing, and the dynamically switching, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

[0009] According to a second aspect, a method of automatically diagnosing a subject with at least one mental disorder, comprises: at least one processor for executing code external to a large language model (LLM), the code configured for interfacing with the LLM for: grounding the LLM to a plurality of predefined disorder specific structured flows, determining a differential diagnosis comprising a plurality of mental disorders, for a first mental disorder, generating prompts for guiding the LLM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow, dynamically analyzing at least one response for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder, dynamically switching from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM, and iterating the generating prompts for guiding the LLM, the dynamically analyzing, and the dynamically switching, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

[0010] According to a third aspect, a non-transitory medium storing program instructions for using at least one processor for executing code external to a large language model (LLM), comprising program instructions which when executed by the at least one processor, cause the at least one processor to: ground the LLM to a plurality of predefined disorder specific structured flows, determine a differential diagnosis comprising a plurality of mental disorders, for a first mental disorder, generate prompts for guiding the LLM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow, dynamically analyze at least one response for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder, dynamically switch from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM, and iterate the generating prompts for guiding the LLM, the dynamically analyze, and the dynamically switch, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

[0011] In a further implementation form of the first, second, and third aspects, an initial differential diagnosis is generated by: generating a prompt for the LLM to ask a predefined set of global questions defined by a global structured flow, wherein the LLM is grounded to the global structured flow, analyzing responses provided by the individual to the global questions asked by the LLM, and selecting the plurality of mental disorders of the initial differential diagnosis according to the analysis of the responses to the global questions.

[0012] In a further implementation form of the first, second, and third aspects, further comprising code for ignoring a potential diagnosis recommended by the LLM.

[0013] In a further implementation form of the first, second, and third aspects, dynamically analyzing responses comprises selecting one of a plurality of mutually exclusive mental disorders, and dynamically prompting the LLM to switch from the first disorder specific structured flow to the second disorder specific structured flow when the first disorder specific structured flow is for a first mental disorder and the second disorder specific structured flow is for a second mental disorder that is mutually exclusive with the first mental disorder.

[0014] In a further implementation form of the first, second, and third aspects, dynamically analyzing responses comprises dynamically computing scores for the responses provided by the individual, dynamically aggregating the computed scores, and dynamically prompting the LLM to switch when the aggregation excludes the existing mental disorder and / or when the aggregation indicates that the new mental disorder is more likely than the existing mental disorder.

[0015] In a further implementation form of the first, second, and third aspects, the score computed for each response to each question comprises a binary score, and the aggregation comprises a sum of the binary scores.

[0016] In a further implementation form of the first, second, and third aspects, further comprising code for: accessing additional personal data of the individual, wherein dynamically analyzing comprises dynamically analyzing a combination of the additional personal data and the responses.

[0017] In a further implementation form of the first, second, and third aspects, the additional personal data comprises temporal data including a video of the individual and / or an audio recording of a voice of the individual, and the combination includes a portion of the temporal data time synchronized with the response.

[0018] In a further implementation form of the first, second, and third aspects, the initial differential diagnosis comprises a plurality of sub-mental disorders of a certain mental disorder each associated with a respective structured flow, wherein the dynamically analyzing and the dynamically prompting the LLM to switch is performed between the plurality of sub-mental disorders upon satisfying criteria for the certain mental disorder.

[0019] In a further implementation form of the first, second, and third aspects, determining inconsistency with the first mental disorder and determining consistency with a second mental disorder comprises at least one of: removing a mental disorder from the differential diagnosis, adding a mental disorder to the differential diagnosis, and replacing a mental disorder of the differential diagnosis.

[0020] In a further implementation form of the first, second, and third aspects, further comprising code of a plurality of agents, each agent configured for a specific medical disorder and for generating prompts to guide the LLM according to the disorder specific structured flow corresponding to the specific medical disorder, wherein a single agent is active at a time for generating prompts to guide the LLM, wherein dynamically switching further comprises dynamically switching from a first agent using the first disorder specific structured flow to a second agent using the second disorder specific structured flow.

[0021] In a further implementation form of the first, second, and third aspects, further comprising: a shared memory accessible by the plurality of agents, the shared memory configured for storing data from the structured interview between the LLM and the subject guided by the first agent, wherein the second agent accesses the shared memory for providing continuity of the structured interview after the dynamic switching to the second agent.

[0022] In a further implementation form of the first, second, and third aspects, further comprising a respective dedicated memory for each one of the plurality of agents, each respective dedicated memory configured for access by a certain corresponding agent and for denying access by other agents, each respective dedicated memory storing data from the structured interview between the LLM guided by the corresponding agent and the subject, for processing by the corresponding agent.

[0023] In a further implementation form of the first, second, and third aspects, the dynamically analyzing the at least one response comprises a deterministic decision-making process that prevents deviation from established clinical protocols.

[0024] In a further implementation form of the first, second, and third aspects, further comprising: initializing a counter for each of a plurality of candidate mental disorders, dynamically analyzing the at least one response, and incrementing the counter for a subset of the plurality of candidate mental disorders when a diagnostic criteria is met, wherein each increment of the counter is normalized by a total number of questions relevant to the corresponding mental disorder for ensuring even weighting across candidate mental disorders, wherein the dynamically switching is performed when the counter for the second mental disorder is greater than for the first mental disorder, and / or when the counter for the second mental disorder exceeds a threshold and the counter for the first mental disorder is lower than the threshold.

[0025] In a further implementation form of the first, second, and third aspects, further comprising: generating prompts for guiding the LLM to ask at least one safety related question according to a safety structured flow, the at least one safety related question weaved in between questions for the first disorder specific structured flow and / or for the second disorder specific structured flow, evaluating risk of safety according to responses to the at least one safety related question in parallel to the dynamically analyzing the at least one response to the at least one question, and generating an alert indicating risk of safety when a probability of the risk of safety is above a threshold. In a further implementation form of the first, second, and third aspects, each disorder specific structured flow is based on a common template format designed for enabling definitions of new structured flows for new mental disorders.

[0026] In a further implementation form of the first, second, and third aspects, the plurality of predefined disorder specific structured flows are arranged in a hierarchy, wherein disorder specific structured flows of a lower level represent sub-types of a type disorder specific structured flows of a higher level, wherein the dynamically switching is initially between the sub-types of the lower level until all relevant sub-types are evaluated, and then the dynamically switching is between types of the higher level.

[0027] In a further implementation form of the first, second, and third aspects, each disorder specific structured flow comprises a first set of screening questions, that when passed, a second set of diagnostic questions are asked, wherein when the first set of screening questions failed, the mental disorder corresponding to the first set is removed from the differential diagnosis.

[0028] In a further implementation form of the first, second, and third aspects, generating prompts comprises generating prompts for guiding the LLM to ask at least one question specific for the first mental disorder according to a first disorder specific structured flow.

[0029] In a further implementation form of the first, second, and third aspects, the mental disorders included in the differential diagnosis are selected from: major depressive episode, (hypo)manic episode, panic disorder, agoraphobia, social phobia, generalized anxiety, post-traumatic stress disorder (PTSD), alcohol dependency / abuse, and psychosis.

[0030] In a further implementation form of the first, second, and third aspects, further comprising generating a treatment plan according to the differential diagnosis.

[0031] In a further implementation form of the first, second, and third aspects, further comprising treating the subject according to the treatment plan with at least one medication and / or at least one type of psychotherapy known to be effective for treatment of at least one mental disorder of the differential diagnosis.

[0032] In a further implementation form of the first, second, and third aspects, at least one structured flow of the predefined disorder specific structured flows defines at least one of: sequence of presented prompts and / or questions, tone maintenance, and / or an amount of empathy to express in the structured interview.

[0033] In a further implementation form of the first, second, and third aspects, a respective disorder specific structured flows for a certain mental disorder is defined according to at least one section of the Mini International Neuropsychiatric Interview for DSM-5 (M.I.N.I.) corresponding to the certain mental disorder. In a further implementation form of the first, second, and third aspects, a disorder specific structured flow further defines at least one follow-up question in response to a response of a user to a preceding structured prompt.

[0034] In a further implementation form of the first, second, and third aspects, the disorder specific structured flow further includes a decision node in which a subsequent structured prompt is selected according to an analysis of at least one data element obtained from at least one preceding structured prompt.

[0035] In a further implementation form of the first, second, and third aspects, the disorder specific structured flow further defines a set of rules for iteratively generating structured prompts.

[0036] In a further implementation form of the first, second, and third aspects, the set of rules include at least one of: each structured prompt of each iteration includes a single question, wait for responses before proceeding with a next structured prompt of a next iteration, use fluent language, use gender appropriate language where relevant, balance between clinical accuracy and empathetic communication, relate next question to a preceding topic, use a variety of acknowledgements, use a variety of validations, use phrases for encouraging open conversation, refer back to patient’s comments using follow-up questions, use patient’s name sparingly mainly during sensitive discussions and / or when concluding, preface questions of sensitive topics with a preparatory statement, when patient’s responses are brief ask for additional details and / or examples, avoid mentioning section labels, steps, question identifiers, and / or instruction from the prompt and / or sets of rules.

[0037] In a further implementation form of the first, second, and third aspects, the disorder specific structured flow further comprising instructions for avoiding repeating a question of a second disorder specific structured flow to which a response has already been provided to the first disorder specific structured flow.

[0038] In a further implementation form of the first, second, and third aspects, the disorder specific structured flow includes instructions for avoiding asking a question to which the subject has already volunteered information in a preceding response without being asked a specific question to obtain the volunteered information.

[0039] In a further implementation form of the first, second, and third aspects, further comprising computing a severity level for the differential diagnosis.

[0040] According to an aspect of some embodiments of the present invention there is provided a computer implemented method for diagnosing and / or generating a treatment plane for treating mental health, comprises: guiding a large language model (LLM) according to at least one structured outline defined by psychiatric guidelines, for conducting a structured interview with a subject according to a flow defined by the at least one structured outline, by iteratively presenting a structured prompt on a user interface and receiving a response from the subject via the user interface, for collecting a plurality of data elements of the subject, analyzing the plurality of data elements, and generating a mental health diagnostic assessment for the subject according to the analysis of the plurality of data elements.

[0041] According to an aspect of some embodiments of the present invention there is provided a computer implemented method for training a LLM for diagnosing and / or generating a treatment plane for treating mental health, comprises: accessing at least one structured outline defined by psychiatric guidelines, creating a training dataset of a plurality of records, each record including a structured interview held by the LLM with a sample subject according to a flow defined by the at least one structured outline, by iteratively presenting a structured prompt on a user interface and receiving a response from the subject via the user interface, for collecting a plurality of data elements of the subject, and training the LLM on the training dataset for conducting the structured interview with a target subject according to the at least one structured outline for diagnosing and / or treating mental health.

[0042] According to an aspect of some embodiments of the present invention there is provided a system for diagnosing and / or generating a treatment plane for treating mental health, comprises: at least one hardware processor executing a code for: guiding a large language model (LLM) according to at least one structured outline defined by psychiatric guidelines, for conducting a structured interview with a subject according to a flow defined by the at least one structured outline, by iteratively presenting a structured prompt on a user interface and receiving a response from the subject via the user interface, for collecting a plurality of data elements of the subject, analyzing the plurality of data elements, and generating a mental health diagnostic assessment for the subject according to the analysis of the plurality of data elements.

[0043] According to some embodiments of the invention, the mental health diagnosis assessment is selected from: major depressive episode, (hypo)manic episode, panic disorder, agoraphobia, social phobia, generalized anxiety, post-traumatic stress disorder (PTSD), alcohol dependency / abuse, and psychosis.

[0044] According to some embodiments of the invention, further comprising generating a corresponding treatment plan according to the diagnosis assessment and / or according to the analysis.

[0045] According to some embodiments of the invention, further comprising treating the subject according to the treatment plan with at least one medication and / or at least one type of psychotherapy known to be effective for treatment of a mental illness from which the subject is suffering from according to the mental health diagnostic assessment.

[0046] According to some embodiments of the invention, the diagnostic assessment and / or treatment plan is generated using another structured outline according to the analysis of the plurality of data elements.

[0047] According to some embodiments of the invention, the treatment plan is generated according to the another structured outline based on the National Institute for Health and Care Excellence (NICE) guidelines.

[0048] According to some embodiments of the invention, further comprising generating a severity level for the diagnostic assessment according to the analysis.

[0049] According to some embodiments of the invention, the at least one structured outline defines at least one of: sequence of presented prompts and / or questions, tone maintenance, and / or an amount of empathy to express in the structured prompt.

[0050] According to some embodiments of the invention, further comprising automatically validating the diagnostic assessment, and / or a corresponding treatment plan against clinical gold standards.

[0051] According to some embodiments of the invention, the at least one structured outline is defined according to at least one relevant section of the Mini International Neuropsychiatric Interview for DSM-5 (M.I.N.I.).

[0052] According to some embodiments of the invention, the structured outline further defines at least one follow-up question in response to a response of a user to a preceding structured prompt.

[0053] According to some embodiments of the invention, the structured outline further includes a decision node in which a subsequent structured prompt is selected according to an analysis of at least one data element obtained from at least one preceding structured prompt.

[0054] According to some embodiments of the invention, the structured outline further defines a set of rules for iteratively generating structured prompts.

[0055] According to some embodiments of the invention, the set of rules include at least one of: each structured prompt of each iteration includes a single question, wait for responses before proceeding with a next structured prompt of a next iteration, use fluent language, use gender appropriate language where relevant, balance between clinical accuracy and empathetic communication, relate next question to a preceding topic, use a variety of acknowledgements, use a variety of validations, use phrases for encouraging open conversation, refer back to patient’s comments using follow-up questions, use patient’s name sparingly mainly during sensitive discussions and / or when concluding, preface questions of sensitive topics with a preparatory statement, when patient’s responses are brief ask for additional details and / or examples, avoid mentioning section labels, steps, question identifiers, and / or instruction from the prompt and / or sets of rules.

[0056] According to some embodiments of the invention, the at least one structured outline includes a main concern outline, wherein the analysis of the plurality of data elements is for identifying symptoms and for comprehending effect of the symptoms on daily functioning and / or social interactions, wherein the analysis identifies at least one candidate mental illness for diagnosing, and further comprising selecting at least one structured outline of a plurality of structured outlines for diagnosing the at least one candidate mental illness.

[0057] According to some embodiments of the invention, the selecting at least one structured outline for diagnosing is performed according to a set of rules.

[0058] According to some embodiments of the invention, the set of rules includes at least one of: select an initial structured outline most correlated with the main concern, perform a diagnostic screening for a manic or hypomanic episode after a diagnostic screen for a depressive episode, evaluate anxiety disorders sequentially according to the order of panic disorder followed by agoraphobia followed by social phobia followed by generalized anxiety disorder, apply the structured outline for psychosis last.

[0059] According to some embodiments of the invention, further comprising instructions for avoiding repeating a question of a second structured outline for diagnostic screening for a second mental illness to which a response has already been provided to a preceding structured prompt of a first structured outline for diagnostic screening for a first mental illness.

[0060] According to some embodiments of the invention, further comprising using a concluding structured outline after completion of iterations using the selected at least one structured outline for screening.

[0061] According to some embodiments of the invention, the diagnostic assessment comprises classifying the subject into a classification category selected from: psychosis with immediate risk, and further comprising sending a first alert type to at least one client terminal requesting urgent assistance, acute condition and further comprising sending a second alert type to at least one client terminal indicating the acute condition, and non-acute condition and further comprising generating a referral for treatment of the non-acute condition.

[0062] According to some embodiments of the invention, the at least one structured outline includes instructions for converting the response from the subject to a preceding structured prompt based on psychiatric guidelines into a YES or NO code, and wherein analysis comprises analyzing a plurality of the YES or NO codes according to the psychiatric guidelines for generating the diagnostic assessment.

[0063] According to some embodiments of the invention, the at least one structured outline includes instructions for avoiding asking a question to which the subject has already volunteered information in a preceding response without being asked a specific question to obtain the volunteered information.

[0064] According to some embodiments of the invention, further comprising: generating a record including at least one of: an indication of the at least one structured outline, the iteratively presented structured prompts, the responses, the plurality of data elements, and the diagnostic assessment, adding the record to a training dataset, and dynamically updating the LLM according to the training dataset.

[0065] According to some embodiments of the invention, the structured interview is conducted by a first user, and the sample subject comprises a second user.

[0066] According to some embodiments of the invention, the structured interview is conducted by the LLM, and training further comprises updating the LLM.

[0067] According to some embodiments of the invention, further comprising adaptively training the LMM by feeding back outcomes of the structured interview labelled with an indication of validation, and / or in response of an error generated by the LMM feeding back an indication of a correct outcome.

[0068] According to some embodiments of the invention, the validation is automatically performed by cross-validation against clinical gold standards, and in response to the error that contradicts the clinical gold standards, feeding back the correct outcome obtained from the clinical gold standard.

[0069] According to some embodiments of the invention, further comprising code for training the LLM by accessing the at least one structured outline defined by psychiatric guidelines, creating a training dataset of a plurality of records, each record including a structured interview held by the LLM with a sample subject according to a flow defined by the at least one structured outline, by iteratively presenting a structured prompt on a user interface and receiving a response from the subject via the user interface, for collecting a plurality of data elements of the subject, and training the LLM on the training dataset for conducting the structured interview with a target subject according to the at least one structured outline for diagnosing and / or treating mental health.

[0070] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments of the invention, exemplary methods and / or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting.

[0071] BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0072] Some embodiments of the invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention. In this regard, the description taken with the drawings makes apparent to those skilled in the art how embodiments of the invention may be practiced.

[0073] In the drawings:

[0074] FIG. 1, which is a block diagram of components of a system for dynamically switching between structured flows for generating prompts for a LLM for conducting a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention;

[0075] FIG. 2 is flowchart of a process of dynamically switching between structured flows for generating prompts for a LLM for conducting a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention;

[0076] FIG. 3 is a flowchart of an exemplary process of an exemplary journey of a user using the tool, in accordance with some embodiments of the present invention;

[0077] FIGs. 4A-4C includes a flowchart of an exemplary process for generating prompts for feeding into a LLM for dynamically switching between different structured flows during a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention;

[0078] FIG. 5 is a flowchart of another exemplary process for generating prompts for feeding into a LLM for dynamically switching between different structured flows during a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention;

[0079] FIG. 6 is a flowchart of an exemplary processes for evaluating depression for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention;

[0080] FIG. 7 is a flowchart of an exemplary processes for evaluating a manic episode for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention; FIG. 8 is a flowchart of an exemplary processes for evaluating different types of anxiety disorders for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention;

[0081] FIG. 9 is a flowchart of an exemplary processes for evaluating different types of stress disorders for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention;

[0082] FIG. 10 includes a graph presenting experimental results for overall diagnostic accuracy between human psychiatrists the tool based on approaches described herein, performed as part of an experiment for evaluation of at least some embodiments;

[0083] FIG. 11 includes graphs presenting experimental results for diagnostic accuracy per vignette between human psychiatrists and the tool described herein, performed as part of an experiment based on embodiments described herein;

[0084] FIG. 12 includes graphs presenting a control case analysis vignette between human psychiatrists and the tool based on embodiments described herein, performed as part of an experiment based on embodiments described herein;

[0085] FIG. 13 includes graphs presenting a reassessment of the experimental results for overall diagnostic accuracy between human psychiatrists and the tool described herein, performed as part of an experiment based on embodiments described herein;

[0086] FIG. 14 includes a graph presenting treatment plan concordance between human psychiatrists and the tool described herein, performed as part of an experiment based on embodiments described herein; and

[0087] FIG. 15 includes graphs presenting treatment plan concordance between human psychiatrists and the tool described herein for depressive and manic episodes, performed as part of an experiment based on embodiments described herein.

[0088] DETAILED DESCRIPTION

[0089] The present invention, in some embodiments thereof, relates to large language models used in clinical decision support systems (CDSS) and, more specifically, but not exclusively, to a LLM based CDSS for mental health.

[0090] As used herein, the term structured flow and structured outline are used interchangeably.

[0091] As used herein, the terms user, subject, and patient - used to refer to the person interacting with the LLM for undergoing a psychiatric evaluation, are used interchangeably.

[0092] As used herein, the term tool and / or CDSS may refer to one or more components, such as a processor and / or memory storing code for execution by the processor, define structured flows, and / or agents for generating prompts for feeding into the LLM based on the structured flows. The term tool and / or CDSS may be external to the LLM, such as off the shelf LLMs and / or LLMs operated by external entities may be used in their existing state, without necessarily requiring training and / or fine tuning of the LLMs.

[0093] As used herein, the term tool and / or CDSS and / or LLM are sometimes interchangeable.

[0094] As used herein, the reference to the LLM generating questions during a structured interview with a user (e.g., based on the structured flow described herein) is exemplary and not necessarily limiting. The LLM may generate other output which is not necessarily a question, for example, an empathic statement (e.g., “That must have been hard for you”), a comment (“Tell me more about it”), and the like. Reference to the questions generated by the LLM is to be understood as also referring to outputs which are not questions.

[0095] An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and / or code instructions (stored on a data storage device and executable by one or more processors) for generating a candidate differential diagnosis of one or more candidate mental disorders, and optionally generating a treatment plan and / or treating the mental disorder(s). Examples of the mental disorders include in the differential diagnosis include: major depressive episode, (hypo)manic episode, panic disorder, agoraphobia, social phobia, generalized anxiety, post-traumatic stress disorder (PTSD), alcohol dependency / abuse, anorexia, bulimia, and psychosis.

[0096] An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and / or code instructions (stored on a data storage device and executable by one or more processors) for automatically generating a differential diagnosis for a subject, including at least one mental disorder which the subject may be suffering from. At least one processor executes code external to a large language model (LLM). The code interfaces with the LLM, for generating prompts to feed into the LLM, for grounding the LLM, and / or for obtaining outcomes generated by the LLM which may be analyzed for generating a next prompt for feeding into the LLM in multiple iterations as part of a structured interview (also referred to herein as a conversation). The LLM is grounded to multiple predefined disorder specific structured flows. The structured flow may define a sequence of questions and / or guidelines to follow when generating output during the structured interview. Each structured flow may be for a different specific mental disorder, or for a sub-type of a mental disorder, and / or for other inquiries such as safety (e.g., risk of suicide), substance abuse, introductory data gathering, and the like. The sequence may be in the format of a graph and / or tree, such as with branching points, where the next branch to follow may be determined based on responses provided by a user to preceding questions. The LLM is prompted to follow the sequence of questions during a structured interview with a user. The structured flow is designed to provide a deterministic approach even when using the LLM (which may not be deterministic on its own), for example, for enabling follow up session to compare with preceding sessions, enabling accurate diagnostics such as using clinical guidelines, and the like. The LLM may be further grounded to clinical guidelines, for example, the Diagnostic and Statistical Manual of Mental Disorders (DSM), for asking questions based on prompts that comply with the clinical guidelines. Compliance with the clinical guidelines enables accurate diagnosis based on the clinical guidelines.

[0097] An initial differential diagnosis may be generated for the subject. The initial differential diagnosis may be selected, for example, based on demographics of the user (e.g., teenager likely suffering from eating disorder, elderly likely suffering from depression), personal history (e.g., soldiers likely suffering from PTSD), and / or based on a set of global screening questions (e.g., are you sad? Are there things you avoid doing? Do you something see or hear things that are not there?). The initial differential diagnosis may include one or more mental disorders which the subject is likely suffering from. The initial differential diagnosis may represent an initial guess, which is then dynamically evaluated to determine whether the mental disorders of the initial differential diagnosis are consistent with responses to questions generated by the LLM based on the structured flows - in which case the respective mental disorder of the initial differential diagnosis is maintained, or not consistent with the responses - in which case the current structured flow is dynamically switched to another structured flow of another mental disorder which is more consistent with the responses is used.

[0098] Prompts for guiding the LLM for conducting a structured interview with the subject via a user interface are generated according to a first disorder specific structured flow, based on the initial differential diagnosis during an initial iteration, or based on a current differential diagnosis during subsequent iterations. A response provided by a user to a question generated by the LLM is analyzed to determine whether the response is consistent with the diagnosis of a first mental disorder corresponding to the first disorder specific flow. When the response or a sequence of responses is sufficiently non-consistent with the diagnosis, a second (i.e., another) mental disorder may be selected for the differential diagnosis. A dynamic switch is made from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM. The process of generating prompts for guiding the LLM, the dynamic analysis of the responses to the questions, and the dynamic switch between structured flows of different mental disorders id dynamically iterated for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

[0099] The iterations and dynamic switching may be performed in a hierarchical manner, for initially identifying a more general mental disorder, and then identifying one or more sub-types of the general mental disorder.

[0100] For example, when the subject is initially estimated be suffering from depression, but does not speak about sadness or lack of energy, but rather about fear of going places, the differential diagnosis may shift to anxiety. The subject is then asked questions about their anxiety rather than depression. The subject may initially be estimated to be suffering from fear of open spaces. Upon provided further responses to questions generated by the LLM, the subject may be determined to be suffering from fear of dogs.

[0101] At least some embodiments described herein relate to code implementing one or more features described herein (e.g., generating prompts using structured flows for guiding the LLM during a structured interview with a subject) being located externally to the LLM. The architecture of placing the code externally to the LLM enables potentially using any LLM that is able to perform a structured interview, without requiring special training of the LLM and / or fine tuning of the LLM to diagnose mental disorders. LLMs that are not trained for diagnosing mental disorders may be used, since the analysis and / or generation of the differential diagnosis is done externally to the LLM. Moreover, a single LLM may be used for diagnosing multiple different mental disorders, since the generation of prompts and the diagnosis is performed by the code external to the LLM rather than by the LLM. The LLM may be operated by an external entity. Access to the internals of the LLM, and / or ability to modify the LLM is not necessarily required. The code located externally to the LLM may be deterministic, enabling for repeated evaluations of the same user such as to track improvement, and / or enabling for comparison between different users, and / or enabling using standard clinical guidelines. The code located externally to the LLM leaves the control of prompt generation and / or generation of the differential diagnosis externally to the LLM, enabling inspection and / or evaluation of the code, for example, for obtaining regulatory approval.

[0102] At least some embodiments described herein relate to determining binary scores based on subject provided responses to questions asked by the LLM guided by the structured flow. For example, YES / NO, 1 or 0. The binary scores enable using clinical guidelines, such as DSM criteria for diagnosis mental disorders according to a point system, such as when a number of defined symptoms / signs is greater than a threshold. The binary scores may provide determinism and / or repeatability, such as in terms of tracking scores during repeated sessions, for example, for determining improvements in the conditions, impact of treatment, and the like.

[0103] At least some embodiments described herein relate to using structured flows for generating prompts for guiding a LLM during a structured interview with a user, for generating a differential diagnosis of one or more mental disorders which the user may be suffering from.

[0104] The architecture of the structured flow per mental disorder enables dynamically switching in real time, from one structured flow to another structured flow during the structured interview, such as when a determination is made that the user is not suffering from a current mental disorder defined by a current structured flow, but likely suffering from another mental disorder defined by another structured flow. The real-time dynamic switching enables starting from a point in which the differential diagnosis is not known, to arrive at a set of potential mental disorders for the differential diagnosis, without necessarily focusing on specific potential mental disorders, but rather “discovering” the potential mental disorder. This approach is in contrast, for example, to using specific models that are trained to detect specific mental disorders, in which case the user would need to use many different models to obtain a diagnosis, and / or which may miss a case where a user has multiple different mental disorders. The real-time dynamic switching enables providing smoothness and / or continuity within the same structured interview, even when asking questions for different mental disorders.

[0105] Moreover, the structured flow may be defined according to a common template format, which enables easy addition of new structured flows (e.g., for adding new mental disorders), adaptation of existing structured flows, and the like, such as in view of updated clinical guidelines and / or expansion of supported mental disorders.

[0106] At least some embodiments described herein provide the practical application of using a LLM, which is not necessarily specifically trained to diagnose mental disorders, for generating a differential diagnosis or one or more mental disorders which a subject is likely suffering from. Code external to the LLM dynamically generates prompts for guiding the LLM during a structured interview by dynamically switching between different structured flows defined for different mental disorders according to the consistency of responses provided by the user with the different mental disorders. This enables determining the differential diagnosis where the mental disorders that the user is suffering from are not initially known, in contrast for example, to approaches of using machine learning models trained for diagnosing specific mental disorders in which case the subject either is required to use many such machine learning models in an attempt to obtain a correct diagnosis, and / or is initially known to have the diagnosis which is then confirmed by the machine learning model. At least some embodiments described herein address the technical problem of handling a large number of subjects that are potentially suffering from mental illness and may potentially require treatment, for example, during a war a large number of civilians and / or soldiers may be impacted simultaneously or within a relatively short amount of time. Traditionally, diagnosis and / or treatment is done by humans, which are specially trained in the field of mental health. At least some embodiments described herein improve the technology of LLM, by enabling LLMs which may be existing and not trained for mental health assessment, to assess mental health which may include diagnosing and / or generating treatment plans for mental health.

[0107] At least some embodiments described herein solve the aforementioned technical problem, and / or improve upon the aforementioned technical field, by providing multiple structured flows for generating prompts for guiding a LLM, where each structured flow is for a different mental disorder, and dynamically switching between the different structured flows during a structured interview with a subject according to a dynamic adaptation of the most likely mental disorders to include in a differential diagnosis. The LLM guided by the structured outline is designed to reduce the load off the human healthcare provides.

[0108] At least some embodiments described herein relate to a clinical decision support system for mental health. The CDSS described herein may be designed for reliable, scalable, and / or personalized initial patient triage, optionally followed by treatment planning, patient progress tracking, and / or treatment validation. Utilizing sophisticated large language models (LLM), the CDSS may autonomously conduct structured patient interviews and / or generate diagnostic summaries and treatment plans. The CDSS’s may merge the empathetic conversational abilities of the large language models with domain knowledge of mental health expertise and / or exclusive patient data, with its precision which may be continuously refined by professional feedback and / or automated validation against clinical gold-standard guidelines. The constantly growing database may allow for regular monitoring of treatment efficacy, which may facilitate the subsequent optimization of treatment strategies. The CDSS may be accessible via mobile devices and / or personal computers (PC), which may ensure user convenience while maintaining strict data security, protecting proprietary and / or user information. The CDSS may enable clinicians to see more patients in the same amount of time, thereby enhancing efficiency in patient care.

[0109] At least some embodiments described herein address the technical problem and / or medical problem of diagnosing and / or treating a large number of patients suffering or potentially suffering from mental disease.

[0110] In the current landscape, a rising number of individuals affected by trauma and emotional distress is straining an already overwhelmed mental healthcare system. The strain is exacerbated by an acute shortage of healthcare professionals, resulting in treatment delays that increase the risk of worsening psychiatric conditions and chronic mental health issues.

[0111] Such delays set off a vicious cycle where individuals who sought help may be discouraged by long wait times, while others already hesitant due to social stigma or shame might forgo treatment altogether. This situation underscores the pressing need for reliable, immediate, and scalable psychiatric assessments and targeted referrals, to enhance resource allocation and improve patient outcomes.

[0112] At least some embodiments described herein address the aforementioned technical problem by offering a reliable and / or scalable mental health triage tool (e.g., CDSS), which may be available mobile devices and / or desktop platforms for easy access. The CDSS may support preliminary mental health assessments by conducting structured interviews based on clinically validated guidelines and / or adaptive learning approaches. The tool may evaluate patient symptoms and / or risk factors and categorizes them according to their needs and urgency. The CDSS may aid healthcare providers in planning appropriate care pathways. The CDSS outputs may be tailored to individual patient needs and / or healthcare system capabilities, aiming to provide timely mental health interventions. The tool may enhance early-stage diagnosis, minimizing delays in diagnosis and / or treatment, and / or improving resource allocation in psychiatric care.

[0113] At least some embodiments described herein relate to enhancing effective triage systems and / or personalized prioritization methods to accurately evaluate the urgency of each patient's needs. This may include improving an advanced screening framework to classify urgency and / or symptom severity, which may enable tailored prioritization of therapy type - whether emotional or pharmacological - and the urgency of initiating treatment.

[0114] At least some embodiments described herein relate to a real-time psychiatric evaluation tool using LLM technology with bi-directional speech conversion, which may be focused initially on the identification of a select group of psychiatric axes within its broader capability to screen various mental health conditions. The CDSS may operate using a structured prompt that guides the model in interview outline, question sequencing, and tone maintenance. The tool may compiles interview data into concise summaries, may classify patients by needs and urgency, and / or may recommend care pathways. The CDSS may enhance efficient triage and / or resource management in mental healthcare.

[0115] Exemplary features of the CDSS may include one or more of:

[0116] PromptFlow architecture: Directs the interview's flow according to the SCID-5 psychiatric guidelines, for employing a structured approach to help ensure all necessary anamnesis questions are asked while allowing the Al model the freedom to respond dynamically and navigate throughout the interview.

[0117] • Natural language Interaction: Utilizes the large language model’s capabilities for fluid, dynamic conversations, mimicking natural human dialogue.

[0118] • Output for health provider: Creates a conversation transcript and / or concise report, including patient information, main concern, presenting symptoms, medical history, differential diagnosis, and chosen treatment plan, automatically emailed to a specified network address.

[0119] • User interface: Intuitive and / or accessible, may be designed for patient interaction on mobile devices and / or and desktop platforms.

[0120] • Bidirectional speech Conversion: Offers voice-to-text and text-to-voice capabilities.

[0121] • Multilingual support: Smoothly operates in English and one or more other languages.

[0122] • Adaptive learning & feedback Integration: Employs machine learning models to refine and enhance classification accuracy continuously.

[0123] At least some embodiments described herein may be provide one or more of the following potential advantages, which may refer to key performance indicators (KPIs):

[0124] * Accelerating access to targeted mental health support for users of the application. A conversion rate of about 15-20% may be achieved for users who have completed the patient intake, engaging in a first interaction with the recommended care pathway within 30 days of intake completion. Tracking may be performed using patient follow-up surveys and / or data from healthcare providers.

[0125] * Accelerating intervention for patients identified with acute and / or urgent psychiatric conditions. A conversion rate of about 25-30% may be achieved for users who have completed the interview and are classified with acute and / or urgent psychiatric conditions, engaging in a first interaction with designated mental health teams within 14 days of intake completion. Tracking may be performed by patient follow-up surveys and data from healthcare providers.

[0126] * Classification Concordance may be obtained by matching the tool's patient evaluation, differential diagnoses and / or treatment recommendations with expert psychiatrist assessments. At least 90-95% concordance with expert psychiatrists may be reached within 6 months of deployment. Tracking may be performed by conducting regular audits comparing the tool’s evaluations against psychiatrist reviews, and / or by documenting and / or analyzing classification discrepancies. * Therapeutic impact on patients during the Al-based interview may be established. An average score of at least 40 out of 50 on a customized version of the Consultation and Relational Empathy (CARE) measure may be achieved, where references to "the doctor" are replaced with "CDSS", within 30 days of intake completion. Tracking may be performed with post-intake surveys using the customized CARE measure, made available immediately following the completion of the intake process.

[0127] * Well-being of patients using the tool may be enhanced. An average score improvement of at least 3-5 points on the Warwick- Edinburgh mental well-being scale (WEMWBS) within 1, 3 and 6 months of using the tool may be achieved. Tracking may be performed by periodic followup surveys using the WEMWBS, administered at 1, 3 and 6 months post-initial tool use. Discrepancies between self-reported and clinician-reported outcomes may be identified.

[0128] * The clinical utility and effectiveness of the tool as perceived by mental healthcare professionals may be assessed. Feedback from at least 50-60% of healthcare professionals using the tool for patient referrals within 3 months of tool deployment may be collected, aiming for example for a satisfaction score of at least 70-90%. Tracking may be performed by monthly surveys, supplemented by qualitative feedback on tool performance and suggested improvements. Discrepancies between self-reported and clinician-reported outcomes may be identified.

[0129] The tool may be made publicly available and / or accessible to clinicians in the field. The tool may comply with all applicable regulations. The tool may satisfy initiation Criteria. The tool may pass clinical validation and / or meet the key performance indicators (KPIs) described above.

[0130] One or more of following features may be implemented, for example, to help ensure a seamless patient journey:

[0131] • Implementing robust data privacy and protection measures to store data in accordance with regulations.

[0132] • Enforcing strict authentication protocols for sign-in and sign-up to safeguard data privacy.

[0133] • Tailoring a user journey for patients at risk of self-harm.

[0134] • Integrating resources to address general anxiety.

[0135] • Implementing nudge prompts to encourage participation among those who have abandoned the tool.

[0136] • Handling multiple entries and usage of the tool by the same patient.

[0137] One or more of following features may be implemented, for example, to validate clinicians' needs and requirements through comprehensive users' research. A management system that facilitates one or more of the following functionalities may be implemented: • Enabling clinicians to send links and / or messages to patients inviting them to use the tool.

[0138] • Managing the patient queue, including for example, organizing patients under categories, with the possibility of multiple questionnaires per patient.

[0139] • Providing clinicians with a list of recent patients who have completed the questionnaire.

[0140] • Allowing clinicians to mark patient reports as read or unread.

[0141] • To enable integration with other management or EHR systems, the tool may be built on MS DB and / or PowerPages, but the system may include an API for data sharing and / or integration.

[0142] One or more of the following exemplary fields may be implemented:

[0143] • An ID field to patient personal details.

[0144] • A coherent conversational flow, dynamically based on the patients' responses, navigates the interaction according to the most probable conditions, leading to a complete summary, fully resembling a psychiatric intake.

[0145] • Allowing for users to pause and resume their progress at will or due to external circumstances (e.g., failed connection).

[0146] • An option to go back and amend previously answered questions may be provided.

[0147] • For each patient, the user (e.g., clinician) may be presented with the start and finish times, a summary of their responses, and a suggested therapeutic plan.

[0148] At least some embodiments described herein are designed to provide an enhanced user experience for a target population. A primary target population may be a patient target population, which may be diverse, for example, encompassing individuals with various physical and / or emotional symptoms that raise concerns about their mental health, prompting them to seek professional assistance. These concerns vary widely, from situational stressors and emotional imbalances potentially indicating milder psychiatric issues like situational anxiety or transient sleep disorders, to more severe symptoms such as significantly reduced functionality, notable mood changes, or the recent emergence of delusions. These severe symptoms may signal major psychiatric conditions, including but not limited to depression, anxiety disorders, Post-Traumatic Stress Disorder (PTSD), and substance abuse disorders. Each of these conditions presents unique challenges and requires tailored approaches in both diagnosis and treatment.

[0149] A secondary target population may include a broad spectrum of mental healthcare professionals, such as psychiatrists, psychologists, clinical social workers, and / or other qualified practitioners. These professionals may be essential beneficiaries of the clinical decision support system described herein. The may receive patient referrals based on the comprehensive preliminary assessments conducted by the tool. This group, encompassing specialists like trauma experts and psychotherapists, may manage cases involving psychological and / or psychiatric conditions that necessitate professional intervention. The CDSS may be designed to streamline the diagnosis and / or care pathway assignment process, which may significantly aid clinicians in efficiently treating a greater number of patients and / or enhancing overall mental healthcare delivery.

[0150] The tool described herein may not be necessarily designated for the following groups. Patients falling under one of these categories may be selectively excluded from the tool’s assessment process and referred to appropriate services for further evaluation and treatment:

[0151] • Minors: Individuals under the age of 18, based on self-reported data.

[0152] • Diagnosed psychotic disorder: Individuals who have been formally diagnosed with any form of a psychotic disorder like schizophrenia, based on self-reporting data.

[0153] • Cognitive limitations: Individuals diagnosed with, or whom the tool suspects to have, cognitive impairments.

[0154] Based on the psychiatric evaluation (i.e., the process of generating the differential diagnosis by dynamic switching of structured flows for prompting the LLM) made by the tool, individuals may be directed to care pathways optimized for their specific needs, as described herein.

[0155] One objective may be to provide accessible, personalized, and self-directed mental health support, optionally for all patient groups. Patients may be provided with tailored access to a (e.g., meticulously curated) library of digital mental health tools. This library may include a variety of resources such as guidance video clips tailored to specific conditions, biofeedback applications, and structured exercises for breathing techniques and mindfulness practices, as well as recommendations for physical activities. All materials may be chosen by leading professionals in the field of mental health, for helping to ensure the highest quality and / or relevance for each condition. The selection process may involve serious consideration and / or may be personalized to meet the unique needs of each patient. Resources may be categorized by experts and made available based on the individual psychiatric evaluation conducted by the tool, helping to ensure that each patient receives the most suitable and effective digital therapeutic interventions.

[0156] Another objective may be to address acute psychiatric needs with specialized care, for example, for patients presenting with acute psychiatric conditions. Referral may be made to a designated mental health team, for example, psychiatrists, psychologists, and / or clinical social workers, specialized in various areas such as trauma care, crisis intervention, supportive psychotherapy, and cognitive behavioral therapy, among others. The designated team may decide on appropriate medication regimens and types of therapy.

[0157] Yet another objective may be to provide structured, ongoing support in order to promote mental well-being, timely identification of any emerging issues, and / or appropriate referral to higher levels of care if necessary, for patients presenting with non-acute psychiatric conditions. Case management may involve monitoring, emotional support, general mental health guidance, and wellness applications recommendations. Case managers may include qualified nurses, occupational and arts therapists, and psychology students trained in mental health support under professional supervision.

[0158] Yet another objective is to provide immediate intervention for patients with high-risk psychiatric symptoms necessitating immediate intervention, for patients exhibiting compromised judgment and / or impaired reality testing, accompanied by suicidal thoughts or behavior or potential harm to self or others. An immediate referral for evaluation in a psychiatric emergency room may be generated.

[0159] At least one embodiment described herein relates to an advanced psychiatric assessment system that leverages artificial intelligence through a sophisticated agent-based architecture. The system's foundation may be built upon a large language model, serving as the core technological infrastructure that powers the intelligent agents and decision-making mechanisms throughout the system. At least one embodiment described herein may enable the system to conduct nuanced, interactive dialogues with patients while maintaining strict adherence to established clinical protocols based on DSM and MINI criteria, effectively balancing high diagnostic accuracy with personalized empathetic interaction.

[0160] In at least one embodiment, the core of the method and / or system employs an advanced deterministic decision-making process that begins with a comprehensive analysis of the subject's main concerns. This initial assessment may feed into an iterative agent selection process, where the system and / or method dynamically determines which specialized agents should be engaged based on emerging diagnostic indicators.

[0161] In at least one embodiment, the system (e.g., architecture) described herein is designed for flexibility and / or extensibility. The system may be designed to enable seamless integration of new structured flows (e.g., diagnostic pathways) through designed templates, which may allow the system to evolve alongside advancing psychiatric knowledge. Exemplary key architectural features include: 1. Modular Design: May enables easy integration of new structured flows (e.g., diagnostic pathways).

[0162] 2. Template-Based Expansion: Standardized framework for adding new clinical protocols.

[0163] 3. Adaptive Learning Integration: Structured incorporation of new clinical insights.

[0164] Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and / or methods set forth in the following description and / or illustrated in the drawings and / or the Examples. The invention is capable of other embodiments or of being practiced or carried out in various ways.

[0165] The present invention may be a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0166] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0167] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0168] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0169] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0170] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0171] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0172] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0173] Reference is now made to FIG. 1, which is a block diagram of components of a system 100 for dynamically switching between structured flows for generating prompts for a LLM for conducting a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 2, which is flowchart of a process of dynamically switching between structured flows for generating prompts for a LLM for conducting a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 3, which is a flowchart of an exemplary process of an exemplary journey of a user using the tool, in accordance with some embodiments of the present invention. Reference is also made to FIGs. 4A-4C, which includes a flowchart 402A-C of an exemplary process for generating prompts for feeding into a LLM for dynamically switching between different structured flows during a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 5, which is a flowchart 502 of another exemplary process for generating prompts for feeding into a LLM for dynamically switching between different structured flows during a structured interview for generating a differential diagnosis for a subject, in accordance with some embodiments of the present invention. Reference is also made to FIG. 6, which is a flowchart 602 of an exemplary processes for evaluating depression for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention. Reference is also made to FIG. 7, which is a flowchart 702 of an exemplary processes for evaluating a manic episode for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention. Reference is also made to FIG. 8, which is a flowchart 802 of an exemplary processes for evaluating different types of anxiety disorders for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention. Reference is also made to FIG. 9, which is a flowchart 902 of an exemplary processes for evaluating different types of stress disorders for inclusion in a differential diagnosis, in accordance with some embodiments of the present invention. Reference is also made to FIG. 10, which includes a graph 1002 presenting experimental results for overall diagnostic accuracy between human psychiatrists the tool based on approaches described herein, performed as part of an experiment for evaluation of at least some embodiments. Reference is also made to FIG. 11, which includes graphs 1102 presenting experimental results for diagnostic accuracy per vignette between human psychiatrists and the tool described herein, performed as part of an experiment based on embodiments described herein. Reference is also made to FIG. 12, which includes graphs 1202 presenting a control case analysis vignette between human psychiatrists and the tool based on embodiments described herein, performed as part of an experiment based on embodiments described herein. Reference is also made to FIG. 13, which includes graphs 1302 presenting a reassessment of the experimental results for overall diagnostic accuracy between human psychiatrists and the tool described herein, performed as part of an experiment based on embodiments described herein. Reference is also made to FIG. 14, which includes a graph 1402 presenting treatment plan concordance between human psychiatrists and the tool described herein, performed as part of an experiment based on embodiments described herein. Reference is also made to FIG. 15, which includes graphs presenting treatment plan concordance between human psychiatrists and the tool described herein for depressive 1502 and manic episodes 1504, performed as part of an experiment based on embodiments described herein.

[0174] System 100 may implement the acts of the method described with reference to FIGs. 2-9, by processor(s) 102 of a computing environment 104 executing code instructions stored in a memory 106 (also referred to as a program store). Computing environment 104 may be implemented as, for example one or more and / or combination of: a group of connected devices, a client terminal, a server, a virtual server, a computing cloud, a virtual machine, a desktop computer, a thin client, a network node, and / or a mobile device (e.g., a Smartphone, a Tablet computer, a laptop computer, a wearable computer, glasses computer, and a watch computer).

[0175] Computing environment 104 generates prompts using structured flows (e.g., from repository of structured flows 122A) optionally executed by agents (e.g., from repository of agents 122B) for guiding a LLM 150 for generating a structured interview to dynamically select mental disorders for a differential diagnosis of a subject, as described herein.

[0176] In at least one embodiment, LLM 150 is hosted by computer. Alternatively, in at least one embodiment LLM 150 is hosted by another server 118 which is external to computing environment 104. Computing environment 104 may interface with LLM 150 hosted by server 118 over a network, optionally via a virtual interface, for example, an application programming interface (API), a software development kit (SDK), and the like.

[0177] As described herein, LLM 150 is external to code 106A, and / or structured flows (e.g., in repository 122A), and / or agents (e.g., in repository 122B). LLM 150 may be a publicly available LLM which may be owned and / or operated by a different entity. LLM 150 is not required to be trained for diagnosing mental disorders.

[0178] Multiple architectures of system 100 based on computing environment 104 may be implemented. For example:

[0179] Computing environment 104 executing stored code instructions 106 A, may be implemented as one or more servers (e.g., network server, web server, a computing cloud, a virtual server) that provides centralized services for generating a differential diagnosis for different subjects and / or other features as described herein. Services may be provided, for example, to one or more client terminals 108 over network 110 (e.g., which may provide the prompt to the LLM 152). For example, different users use their client terminals 108 (e.g., via a web browser, graphical user interface (GUI) apps, and the like) to access computing environment 104 for conducting a structured interview to obtain a differential diagnosis of one or more mental health conditions, and / or to obtain other results (e.g., report), as described herein. Services may be provided by computing environment 104 to client terminals 108, for example, as software as a service (SaaS), a software interface (e.g., application programming interface (API), software development kit (SDK)), an application for local download to the client terminal(s) 108, an add-on to a web browser running on client terminal(s) 108, and / or providing functions using a remote access session to the client terminals 108, such as through a web browser executed by client terminal 108 accessing a web sited hosted by computing environment 104.

[0180] In another example, computing environment 104 may be implemented as a dedicated and / or standalone device (e.g., kiosk, client terminal, smartphone) that include locally stored code instructions 106A that implement one or more of the acts described with reference to FIGs. 2-9, for generating a differential diagnosis for different subjects and / or other features as described herein. The locally stored code instructions 106A may be obtained from a server, for example, by downloading the code over the network, and / or loading the code from a portable storage device, such as by installing an app on a smartphone of a user. The dedicated and / or standalone architecture may be selected, for example, in environments where different subsets of structured flows. For example, a terminal may be installed in army barracks to be used by soldiers for screening for war-related mental illnesses, such as PTSD. In another example, another terminal may be installed in a teenager’s lounge to be used by teenagers to help detect mental disorders related to children, such as depression, OCD, addiction, and the like. In yet another example, yet another terminal may be installed in a hospital psychiatric ward to help diagnose and / or track severe mental disorders, such as suicidal thoughts, psychosis, and the like.

[0181] Processor(s) 102 of computing environment 104 may be hardware processors, which may be implemented, for example, as a central processing unit(s) (CPU), a graphics processing unit(s) (GPU), field programmable gate array(s) (FPGA), digital signal processor(s) (DSP), and application specific integrated circuit(s) (ASIC). Processor(s) 102 may include a single processor, or multiple processors (homogenous or heterogeneous) arranged for parallel processing, as clusters and / or as one or more multi core processing devices.

[0182] Memory 106 stores code instructions executable by hardware processor(s) 102, for example, a random access memory (RAM), read-only memory (ROM), and / or a storage device, for example, non-volatile memory, magnetic media, semiconductor memory devices, hard drive, removable storage, and optical media (e.g., DVD, CD-ROM). Memory 106 stores code 106A that implements one or more features and / or acts of the method described with reference to FIGs. 2-9 when executed by hardware processor(s) 102.

[0183] Computing environment 104 may include a data storage device 122 for storing data, for example, repository of structured flows 122A set to store structured flows for different mental disorders, and / or repository of agents 122B set to store agents for different mental disorders, where each agent executes a corresponding structured flow. Data storage device 122 may host LLM 150.

[0184] Optionally, computing environment 104 includes a dual-memory architecture, which may that enhance execution of structured flows (e.g., individual agent performance) and / or overall system coherence. The dual-memory architecture may include a shared storage device 160 and one or more dedicated storage devices 162. The dual-memory approach may help ensure precision in individual diagnostic tasks and / or comprehensive understanding of the subject's overall condition, which may significantly enhance the system's 100 diagnostic accuracy and / or conversational fluidity.

[0185] Shared storage device (e.g., memory) 160 may be accessed by (e.g., written to and / or read by) each one of multiple agents such as of agent repository 122B that are associated with different mental disorders. Shared storage device 160 may be implemented for long-term storage. Shared storage device 160 may be designed as a centralized memory system accessible to the multiple agents. Shared storage device 160 may maintain interaction history of the subject with the LLM and / or may allow for coherent, context-aware conversations throughout the assessment process. Shared storage device 160 may enable agents to build upon previous insights and / or maintain conversational continuity across different diagnostic pathways.

[0186] One or more dedicated storage devices 162 may each be accessed by (e.g., written to and / or read by) a specific agent (also referred to as specialized agent) corresponding to a specific mental disorder. Dedicated storage devices 162 may refer to dedicated regions on a same storage device. Other agents associated with other mental disorders are defined access to the dedicated storage device of other agents. Each specialized agent may maintains its own dedicated memory space on respective dedicated storage device(s) 162. Each dedicated storage device(s) 162 may be optimized for a respective specific diagnostic focus and / or decision-making requirements. The targeted memory architecture of dedicated storage device(s) 162 may enable precise and / or taskspecific processing while maintaining access to the broader conversational context through shared storage device 160.

[0187] Data storage device 122 and / or shared storage device 160 and / or dedicated storage device(s) 162 may be implemented as, for example, a memory, a local hard-drive, virtual storage, a removable storage unit, an optical disk, a storage device, and / or as a remote server and / or computing cloud (e.g., accessed using a network connection).

[0188] Network 110 may be implemented as, for example, the internet, a local area network, a virtual network, a wireless network, a cellular network, a local bus, a point to point link (e.g., wired), and / or combinations of the aforementioned.

[0189] Computing environment 104 may include a network interface 124 for connecting to network 110, for example, one or more of, a network interface card, a wireless interface to connect to a wireless network, a physical interface for connecting to a cable for network connectivity, a virtual interface implemented in software, network communication software providing higher layers of network connectivity, and / or other implementations.

[0190] Computing environment 104 and / or client terminal(s) 108 include and / or are in communication with one or more user interfaces 126, which may present questions generated by the LLM as part of the structured interview and / or may results of the validation and any instructions to follow. Exemplary user interfaces 126 include, for example, one or more of, a touchscreen, a display, gesture activation devices, a keyboard, a mouse, and voice activated software using speakers and microphone.

[0191] Computing environment 104 may implement comprehensive quality monitoring, for example, using log analytics and / or customized dashboards, designed to provide real-time insights into system performance and / or clinical effectiveness. The monitoring infrastructure may include one or more of:

[0192] 1. Performance Analytic s : o Real-time tracking of response times and system latency. o Monitoring of agent selection accuracy and decision-making patterns. o Analysis of conversation flow and interaction quality.

[0193] 2. Clinical Quality Metrics: o Dynamic visualization of diagnostic pathway utilization. o Real-time tracking of clinical protocol adherence. o Monitoring of empathy levels and patient engagement metrics.

[0194] 3. Interactive Dashboards: o Custom-built visualization panels for both technical and clinical metrics. o Real-time graphs showing system performance trends. o Comparative analysis tools for evaluating diagnostic accuracy.

[0195] The integration of Azure Log Analytics enables continuous quality assessment through sophisticated data collection and analysis, providing both immediate operational insights and long-term trending data. This allows for:

[0196] • Immediate detection of any deviations from expected performance patterns.

[0197] • Real-time monitoring of clinical effectiveness metrics.

[0198] • Dynamic adjustment of system parameters based on performance data.

[0199] • Comprehensive reporting capabilities for both technical and clinical stakeholders.

[0200] Computing environment 104 may implement one or more security mechanisms to maintain controlled and / or appropriate agent behavior while preserving the flexibility needed for diverse subject interactions. The security mechanisms may be implemented using a behavioral control framework, for example: computing environment 104 may implement robust guardrails for helping to ensure agent responses remain within therapeutic and / or clinically appropriate boundaries. These controls may be dynamically adjusted based on the specific context of each interaction, while maintaining strict adherence to clinical protocols. The framework may allow, for example, for: protection against prompt manipulation attempts, maintenance of therapeutic boundaries, and / or preservation of clinical integrity across all interactions.

[0201] While operating within defined security parameters, computing environment 104 may maintain the flexibility to adapt to diverse subject needs and / or communication styles. The adaptability may be achieved through, for example: dynamic adjustment of conversation style based on patient profile, and / or maintained clinical effectiveness across varying patient presentations.

[0202] The security framework may acts as an intelligent filter, allowing appropriate therapeutic flexibility while preventing deviations from established clinical protocols and maintaining system integrity.

[0203] Reference is also made to FIG. 2, which is a flowchart of dynamically switching structured flows for dynamically selecting mental disorders for a differential diagnosis of a subject, in accordance with some embodiments of the present invention.

[0204] Referring now back to FIG. 2, one or more of the features of the method described with reference to FIG. 2 are implemented by code executed by a processor(s) externally to the LLM.

[0205] At 202, one or more structured flows are provided (e.g., accessed) and / or defined.

[0206] Optionally, structured flows are executed by agents. Optionally, an agent executes a certain structured flow, where each agent is allocated to a specific structured flow. Alternatively or additionally, an agent may executed multiple structured flows, for example, global structured flows. The structured flow may be integrated within the agent. Alternatively, the structured flow may be implemented as a data structure external to the agent, with the agent executing the structured flow.

[0207] Optionally, a distributed network of specialized intelligent agents is implemented. Each agent may be designed to fulfill specific psychiatric assessment functions. Optionally, the types of agents include:

[0208] Primary Conversational Agents: Front-line agents may be set for diagnosis. Each agent may specialize in a certain mental disorder (e.g., psychiatric condition) and optionally its manifestation. • Background Processing Agents: May operate (e.g., continuously) in the background. A decision-making agent may analyze incoming information and / or subject response(s), for helping to ensure clinical accuracy and / or protocol adherence.

[0209] Some details of the structured flows have been discussed above. Additional exemplary details are now provided.

[0210] Each structured flow may define one or more of:

[0211] • Sequence of presented prompts and / or questions. The prompts and / or questions may be defined according to clinical guidelines, such as where a diagnosis is made when a number of criteria greater than a threshold is met. The sequence may be pre-defined, for example, general to specific, according to a tree with branching nodes, following clinical guidelines, following more important to less important, and the like.

[0212] • Tone maintenance. Tone may refer to the overall attitude, mood, and / or manner in which the LLM conducts the structured interview. The tone may play a role in creating an atmosphere that can affect the responses and comfort level of the subject. For example, a professional tone, a neutral tone, supportive tone, and a consistent tone. Tone may vary according to the mental disorder being evaluated, for example, the tone may be different for PTSD than for anxiety or psychosis.

[0213] • Amount of empathy to express in the structured interview. Empathy may vary according to the mental disorder being evaluated, for example, the LLM may be instructed to be very empathic for PTSD, and less empathic for anxiety.

[0214] Each disorder specific structured flow may be defined for a certain mental disorder according to clinical guidelines, optionally at least one section of the Mini International Neuropsychiatric Interview for DSM-5 (MINI-5) corresponding to the certain mental disorder. The LLM may further be grounded to the clinical guidelines (e.g., DSM-5), for helping to ensure compliance with the clinical guidelines. Using the disorder specific structured flow based on clinical guidelines may help ensure that the prompts generated by the LLM remain fixed and / or constant. The disorder specific structured flow based on clinical guidelines may provide a solid, consistent foundation for initial screening questions for each condition based on SCID-5's semistructured interview guide, which is globally recognized for its reliability in diagnosing psychiatric disorders according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). After an initial structured screening, the tool may allow flexibility for the LLM to dynamically navigate the interview, adapting to the unique needs of each patient. The comprehensive nature of MINI-5 covers a wide range of psychiatric disorders, including depression, anxiety, psychosis, mania, eating disorders, substance abuse disorders, and more. By incorporating MINI-5 into the tool's architecture, it may help ensure systematic and / or consistent screening across patient interactions while standardizing the process for history-taking and symptom assessment. This fidelity to MINI-5 may enable the tool to generate nuanced and / or individualized interactions with the patient, leading to reliable and / or accurate psychiatric risk assessments that align with established clinical practices and / or assist in effective follow-up care and treatment planning.

[0215] Optionally, a disorder specific structured flow further defines at least one follow-up question in response to a response of a user to a preceding structured prompt. The follow-up question may be general and / or open ended, for example, “anything else to add?”, “what else?”. The follow-up question may be specific, and / or particular to the preceding question, for example, “You said you felt your heart racing. Did you feel like you were spinning also?”

[0216] The disorder specific structured flow may include a decision node in which a subsequent structured prompt is selected according to an analysis of at least one data element obtained from at least one preceding structured prompt. For example, if a trigger such as of traumatic symptoms was mentioned, the generated prompt may instruct the LLM to further explore the trigger and the resulting symptoms. If a trigger was not mentioned, the generated prompt may instruct the LLM to ask about specific triggers.

[0217] The disorder specific structured flow may define a set of rules for iteratively generating structured prompts, for example:

[0218] • Each structured prompt of each iteration includes a single question.

[0219] • Wait for responses before proceeding with a next structured prompt of a next iteration.

[0220] • Use fluent language.

[0221] • Use gender appropriate language where relevant.

[0222] • Balance between clinical accuracy and empathetic communication.

[0223] • Relate next question to a preceding topic,

[0224] • Use a variety of acknowledgements.

[0225] • Use a variety of validations.

[0226] • Use phrases for encouraging open conversation.

[0227] • Refer back to patient’s comments using follow-up questions.

[0228] • Use patient’s name sparingly mainly during sensitive discussions and / or when concluding.

[0229] • Preface questions of sensitive topics with a preparatory statement.

[0230] • When patient’s responses are brief ask for additional details and / or examples. • Avoid mentioning section labels, steps, question identifiers, and / or instruction from the prompt and / or sets of rules.

[0231] The disorder specific structured flow may include instructions for avoiding repeating a question of a current disorder specific structured flow to which a response has already been provided to a preceding disorder specific structured flow. For example, in a preceding structured flow for anxiety, the question “Do you sometimes feel your heart racing and are sweating?” was asked by the LLM, and the subject responded. In a current structured flow for PTSD, the same question is defined by the flow, but since this question was already asked, re-asking it is to be avoided.

[0232] The disorder specific structured flow may include instructions for avoiding asking a question to which the subject has already volunteered information in a preceding response without being asked a specific question to obtain the volunteered information. For example, in a structured flow for depression, the question “Do you sometimes feel very sad” is to be asked. However, since the subject already replied “Sometimes I get a strong feeling of sadness” during a preceding structured flow for PTSD, the question does not need to be asked.

[0233] Structured flows may be defined for linguistic and clinical needs of diverse age groups, for example, children, adolescents, and seniors.

[0234] Structured flows may be defined to different specific populations based on their unique needs. For example, women's psychiatry (e.g., postpartum depression, menopause), displaced populations, and elderly individuals (e.g., monitoring cognitive changes such as Alzheimer's, dementia, and other forms of cognitive deterioration).

[0235] Optionally, each disorder specific structured flow is based on a common template format designed for enabling definitions of new structured flows for new mental disorders.

[0236] Optionally, each disorder specific structured flow includes a set of screening questions, that when passed, trigger asking of another set of diagnostic questions. When the initial set of screening questions fails, the mental disorder corresponding to the initial set is removed from the differential diagnosis. Another mental disorder may be selected for the differential diagnosis, as described herein.

[0237] At 204, the LLM may be grounded to the predefined disorder specific structured flows and / or clinical guidelines.

[0238] Grounding the LLM may aligning the LLM's outputs with specific knowledge, constraints, and / or context defined by the predefined disorder specific structured flows and / or clinical guidelines, to make the responses generated by the LLM more accurate, relevant, and / or trustworthy. Grounding may help overcome the limitations of LLMs, such as tendency to generate plausible but incorrect or generic responses.

[0239] The clinical grounding mechanism described herein may help ensure adherence to established psychiatric protocols while maintaining conversational fluidity. The clinical grounding mechanism may implement process constraints that anchor all interactions between the user and the LLM within documented clinical knowledge frameworks, while (e.g., simultaneously) allowing for natural conversation flow. The system employs a deterministic decision-making process that prevents deviation from established clinical protocols while preserving the capacity for empathetic interaction.

[0240] At 206, an initial phase may be executed, optionally by an introductory structured flow.

[0241] The initial phase may be executed for generating an initial differential diagnosis of one or more mental disorders which the subject may be suffering from. The initial differential diagnosis represents a starting point for the LLM to conduct the structured interview, from which different structured flows may be dynamically switched as responses are obtained from the subject, in an effort to obtain a more accurate and / or more likely differential diagnosis.

[0242] The initial phase may include an analysis of the subject’s main concern (also referred to as chief complaint). The analysis may include an evaluation of the subject's primary issues, based on primary diagnostic prompt defined by a corresponding structured flow.

[0243] One or more of the following data may be obtained by the tool, optionally based on the introductory structured flow:

[0244] • Key Identifiers. Should the user have not already provided these details during initial registration, the tool may prompt for the key identifiers: name, age, gender, and contact details.

[0245] • Medical Anamnesis (also referred to as medical history taking). The tool may collect essential anamnesis data, including medical history, prior and current diagnoses, medication regimens, past hospital admissions, substance use history, main concerns, presenting symptoms, symptoms duration and recurrence, and potential triggers. Additional relevant questions may be posed as deemed necessary by the tool's algorithm. Any expectations or preferences the user has regarding the assessment and subsequent steps will also be recorded.

[0246] • Psychiatric Assessment. In alignment with established psychiatric guidelines and tailored to each user's specific medical history and anamnesis response, the tool may perform a systematic screening for a range of psychiatric conditions. At 208, a differential diagnosis including one or more mental disorders that the subject may be suffering from is generated.

[0247] The differential diagnosis may include a predefined number of mental disorders, for example, two, three, four, or more. Optionally, the predefined number of mental disorders is maintained during dynamic switching of the structured flows. For example, the initial differential diagnosis may include depression, anxiety, and PTSD. Depression may be dynamically replaced by bipolar disorder. Anxiety may be replaced by social phobia. Alternatively, the number is not fixed, but may vary dynamically during the structured interview. For example, the initial differential diagnosis may include depression, anxiety, and PTSD. Bipolar may be further added. Anxiety may be further added. Depression may be removed.

[0248] The mental disorders included in the differential diagnosis may be based on defined mental disorders, for example, DSM-5. Examples of mental disorders include: major depressive episode, (hypo)manic episode, panic disorder, agoraphobia, social phobia, generalized anxiety, post- traumatic stress disorder (PTSD), alcohol dependency / abuse, and psychosis.

[0249] Optionally, the differential diagnosis includes one or more sub-types of one or more mental disorders. Each sub-type is associated with a respective structured flow. The dynamic analysis and / or the dynamically switching may be performed between the sub-types of the mental disorders, optionally upon satisfying criteria for the certain mental disorder. For example, the mental disorder “anxiety” may include the following sub-types: panic disorder, agoraphobia, social phobia, and generalized anxiety disorder.

[0250] The differential diagnosis may be generated by generating a prompt for the LLM to ask a predefined set of global questions defined by a global structured flow. The LLM may be grounded to the global structured flow. The global questions may be general questions to screen for different types of mental disorders, for example, “What makes you sad?”, “Do you avoid certain situations?”, “Do you sometimes see or hear things that others don’t?”, and the like. Responses provided by the individual to the global questions asked by the LLM may be analyzed. For example, by feeding the responses into a trained machine learning model, assigning scores to the responses based on clinical guidelines, matching clinical criteria according to the responses, and the like. An initial set of mental disorders of the initial differential diagnosis that the subject may be suffering from is selected according to the analysis of the responses to the global questions. For example, when the user answers YES to “Do you avoid certain situations?”, PTSD, anxiety, and phobia may be selected for the initial diagnosis. The mental disorders of the differential diagnosis may be arranged in a certain order, for example, based on likelihood (e.g., number of met criteria, probability computed by a machine learning model), based on a predefined order, and the like.

[0251] Optionally, a severity level may be determined for one or more of the mental disorders of the differential diagnosis. The severity level may indicate, for example, severity of impact on daily functionality, and / or intensity of the symptoms. The severity level may be used, for example, taken into account when generating treatment recommendations. For example, type of treatment (e.g., psychotherapy versus drug treatment), dose of medications, and the like.

[0252] At 210, one or more prompts for guiding the LLM for conducting a structured interview with the subject via a user interface are generated according to a first disorder specific structured flow of a first mental disorder of the differential diagnosis. The LLM may be guided for to ask at least one question specific for the first mental disorder according to the first disorder specific structured flow.

[0253] Optionally, the prompt is generated by a specific agent designed for the specific medical disorder currently being evaluated (i.e., the first mental disorder). Multiple agents may be defined, where each agent is designed for a different respective specific medical disorder, for generating prompts to guide the LLM according to the corresponding disorder specific structured flow. A single agent may be active at a time. During dynamic switching between structured flows, the active agent is switched. The switching may be performed from the first agent using the first disorder specific structured flow of the first mental disorder being active, to activating the second agent using the second disorder specific structured flow of a second mental disorder.

[0254] Optionally, a shared memory accessible by the multiple agents is provided. The shared memory may be set for storing data from the structured interview (e.g., conversation) between the LLM and the subject guided by a currently active (e.g., first) agent. A subsequently activated (e.g., second) agent may accesses the shared memory to obtain the data stored during the structured interview with the first agent. The shared memory enables the LLM to provide continuity of the structured interview (e.g., conversation) during and / or after the dynamic switching from the first agent to the second agent. Using the shared memory, the dynamic switching of the agents may be performed seamlessly, without the subject being aware.

[0255] Optionally, a respective dedicated memory is provided for each one of the agents. Each respective dedicated memory may be set for access by a certain corresponding agent. Other agents may be denied access to dedicated memories that are not assigned to them. Each respective dedicated memory may store data from the structured interview (e.g., conversation) between the LLM and the corresponding agent for processing by the corresponding agent. The dedicated memory may be set for storing data specific to the respective agent, which is not to be used by other agents.

[0256] At 212, one or more responses provided by the subject to the LLM during the structured interview may be analyzed.

[0257] The responses may be analyzed for dynamically prioritizing the mental disorders being evaluated, for dynamic selection of a set of highest priority mental disorders for inclusion in the differential diagnosis. For example, the top 2, 3, 4, or other number of mental disorders are selected for the differential diagnosis. The prioritization may be based on a score, for example, probability, number of diagnosis criteria met, and the like. Each mental disorder may be assigned a respective score during the structured interview, which is dynamically updated as the structured interview proceeds. The dynamic prioritization may be synchronized with the dynamic updating of the scores, and / or performed at other intervals, such as every response, every other response, every minute, every 3 minutes, and the like.

[0258] Optionally, a continuous evaluation and / or prioritization of potential mental disorders for inclusion in the differential diagnoses is provided.

[0259] The responses may be provided by the subject, for example, via a graphical user interface that presents the outputs (e.g., questions) generated by the LLM. The responses may be provided, for example, typed in by the subject using a keyboard, and / or spoken by the subject into a microphone and converted to text using a voice-to-text conversion process. The questions generated by the LLM may be, for example, presented within the GUI as text, and / or played over speakers as audio.

[0260] The response(s) may be dynamically analyzed to determine consistency or inconsistency with the current mental disorder being evaluated. In the case of inconsistency with the current medical disorder, the analysis may be to determine a second mental disorder which is consistent with the response(s). For example, the current medical disorder being evaluated is depression. When the subject replies “No, but I sometimes feel very worried” to the question generated by the LLM, “Do you sometimes feel very sad?”, the response is inconsistent with depression, but consistent with anxiety. In the case of consistency with the current medical disorder, the analysis may be to determine a second mental disorder that is also consistent with the response(s). For example, the current medical disorder being evaluated is depression. When the subject replies “Yes” to the question generated by the LLM, “Do you sometimes feel very sad?”, the response is consistent with depression, but also consistent with bipolar disorders (which alternates between depression and mania). The dynamic switching described herein may be performed when a sufficient number of responses are determined to be inconsistent with the current medical disorder and consistent with another medical disorder, for example, a number above a threshold, or a score assigned to each response where an accumulation of scores is above a threshold, and the like.

[0261] The response may be dynamically analyzed to determine consistency with one of two (or more) mutually exclusive mental disorders. For example, for the prompt “How do you handle your feelings of thinking you are overweight”, the subject may respond, for example, as “I vomit what I ate”, or “I don’t eat”. The first response is consistent with bulimia and excludes anorexia. The second response is consistent with anorexia, and excludes bulimia. Bulimia and anorexia are mutually exclusive, since they do not tend to occur simultaneously in the same subject. The dynamic switching may be from the first disorder specific structured flow to the second disorder specific structured flow when the first disorder specific structured flow is for a first mental disorder and the second disorder specific structured flow is for a second mental disorder that is mutually exclusive with the first mental disorder.

[0262] Optionally, the dynamic analysis of the responses is performed by dynamically computing scores for the responses provided by the individual. The dynamic computation of scores may be performed by a deterministic process. The scores may be computed according to correlation with clinical criteria of the current mental disorder being evaluated and / or another mental disorder being evaluated. The score computed for each response to each question may be implemented as a binary score, for example, yes / no, 1 / 0, and the like. For example, when evaluating for depression, the question “Are you sad?” may be asked. The score may indicate yes or no. The computed scores may be dynamically aggregated. For example, the binary scores may be summed to determine a combined score. The aggregation may be for each mental disorder for which the question is relevant. For example 10 questions are asked, 10 of which are relevant for depression, 7 for PTSD, and 5 for anxiety. Scores may be determined for each of depression, PTSD, and anxiety. The dynamic switching may be performed when the aggregated scores excludes the existing mental disorder and / or when the aggregation indicates that the new mental disorder is more likely than the existing mental disorder. Using the above example of 10 questions being asked to evaluate depression, 10 of which are relevant for depression, 7 for PTSD, and 5 for anxiety - the aggregated scores after asking all 10 questions are: 3 for depression, 6 for PTSD, and 4 for anxiety. The dynamic switch may be performed to evaluate for PTSD rather than depression.

[0263] Optionally, a counter may be initialized for each of multiple candidate mental disorders that may be included in the differential diagnosis. The response(s) may be analyzed, such as to determine whether a diagnosis criteria (e.g., sign, symptom) is met for each of the candidate mental disorders, as described herein. The counters of one or more of the candidate mental disorders may be incremented based on the analysis when the diagnostic criteria of the respective candidate mental disorder is met. Each increment of the counter may be normalized by a total number of questions relevant to the corresponding mental disorder. The normalization may ensure even weighting across candidate mental disorders. The dynamically switching may be performed when the counter for another mental disorder is greater than the counter for the current mental disorder being evaluated. Alternatively or additionally, the dynamic switching may be performed when the counter for the other mental disorder exceeds a threshold and the counter for the current mental disorder is lower than the threshold.

[0264] Alternatively or additionally, the dynamic analysis of the responses is performed by another deterministic process that is dynamically fed the responses, optionally sequentially and / or accumulatively, and computes likelihood of each one of multiple mental disorders, to determine the most likely mental disorder.

[0265] The use of a deterministic decision-making process to analyze the response(s) to determine the different diagnosis and / or switching to another structured flow of another mental disorder may prevent deviation from established clinical protocols. In contrast, using a non-deterministic process, such as a machine learning model, may generate different outcomes even for the same or similar input, which may prohibit its use in determining the differential diagnosis.

[0266] During execution of the features described with reference to FIG. 2, additional data of the subject may be extracted and / or analyzed in combination with responses provided by the subject to the questions generated by the LLM during the structured interview. The additional data may include, for example, temporal data including a video of the individual and / or an audio recording of a voice of the individual. An analysis may be made of the subject's voice frequencies captured by a microphone and / or facial expressions depicted in images captured by a camera. The analysis may be performed based on the combination, for example, to determine emotions of the subject while the subject is providing a response. For example, whether the subject is crying, sad, happy, excited, scared, hesitant, and the like. The additional data may enhance the diagnostic framework to resemble the depth of human psychiatric intake evaluations. For example, when evaluating for depression, images and / or voice analysis of the subject crying increase likelihood that depression is correct, whereas images and / or voice analysis of the subject laughing reduce likelihood that depression is correct. The dynamic switching may be to bipolar disorder, where the subject may be sad at times and happy and times.

[0267] A dedicated agent may monitor the subject’s emotional states, optionally based on the analysis of the images of the subject and / or audio of the subject speaking. The dedicated agent may dynamically adjusts conversational parameters of the FEM to maintain optimal therapeutic engagement. For example, when the subject starts to be sad and / or cry, the dedicated agent may prompt the LLM to show empathy (e.g., “That must be hard for you”), recommend a break (“Do you want to have a drink of water?”), apologize (e.g., “I am sorry I made you upset”), further inquiry (e.g., “Can you share what thoughts or other trigger caused you to become upset just now?”, and the like. The dynamic switching may be to the dedicated agent, such as in response to detecting the change in emotional state.

[0268] At 214, a dynamic switch may be made from the current (e.g., first) disorder specific structured flow defined for the current (e.g., first) mental disorder being considered for the differential diagnosis, to another (e.g., second) disorder specific structured flow defined for another corresponding (e.g., second) mental disorder, for guiding the LLM during the structured interview with the subject.

[0269] Alternatively or additionally, the dynamic switch is made from a current (e.g., first) agent executing and / or implementing the current (e.g., first) disorder specific structured flow to another (e.g., second) agent executing and / or implementing the other (e.g., second) disorder specific structured flow. The dynamic switch may include and / or be based on a dynamic agent selection. A real-time determination of an appropriate diagnostic pathway may be made. The agent selection may dynamic and / or based on the subject's responses to the LLM’s questions. The selected agent may accordingly dynamically change during the structured interview.

[0270] Optionally, dynamic sub-agents are selected and / or switches. Certain structured flows (e.g., eating disorders, anxiety) may include sub-flows associated with corresponding sub-agents (e.g., anorexia nervosa, panic disorder, social anxiety). The sub-agents may lead to more specific and / or accurate differential diagnoses.

[0271] The dynamic switch may be based on the analysis of the response(s) provide by the subject to questions generated by the LLM, for example, as described with reference to 212 of FIG. 2.

[0272] Optionally, the predefined disorder specific structured flows are arranged in a hierarchy. Disorder specific structured flows of a lower level may represent sub-types of a type disorder specific structured flows of a higher level. For example, the higher level is between mental disorders such as depression, PTDS, anxiety, etc... One or more mental disorders may branch to a lower level of sub-types, for example, sub-types of depression, sub-types of anxiety, etc... The dynamic switching may be initially between the sub-types of the lower level until all relevant subtypes are evaluated, followed by dynamically switching between types of the higher level. Alternatively, the dynamic switching is first between types of the higher level to identify mental disorders that the subject is likely suffering from. Once a higher level mental disorder is identified, the dynamic switching may be between the sub-types of the higher level mental disorder to identify the sub-types of mental disorders that the subject is likely suffering from. At 216, one or more global structured flows may be selected. The global structured flows may be for screening for conditions which may occur with any mental disorder.

[0273] The global structured flows may be run in the background, for example, during evaluation of other structured flows of mental disorders. Alternatively or additionally, the global structured flows may be switched to and executed at predefined locations of the structured interview, for example, at the start, after the initial screening, and / or after the differential diagnosis has been determined (i.e., prior to the end). Alternatively or additionally, the global structured flows may be dynamically weaved in between questions for one or more disorder specific structured flows, such as between the same disorder specific structured flow, or between two disorder specific structured flows. The global structured flows may be switched to and / or switched away from while another structured flow for a certain medical disorder is being executed. For example, when the user is being evaluated by a structured flow for depression and is asked the question “Do you sometimes feel very sad”, and the user responds “Yes, sometimes so much I don’t want to live anymore”, a global structured flow for screening for suicidal risk may be switched to.

[0274] Examples of global structured flows include:

[0275] • Safety related structured flow defining safety related questions, for example, for evaluating risk of suicide. Safety protocols may be implemented through monitoring mechanisms that (e.g., continuously) assess patient risk levels. These mechanisms may be designed for realtime detection of a potential crisis situations and / or may trigger real-time alerts and / or notifications to supervising clinical staff such as in a crisis situation. Risk of safety may be evaluated according to responses to safety related question(s), optionally in parallel to the dynamic analysis of response(s) to question(s) of a disorder specific structured flow. An alert indicating risk of safety may be generated, such as when a probability of the risk of safety is above a threshold. For example, a pop-up is presented at a nurse monitoring station, a messages is sent to a mobile device of an on-call physician, and the like. Existing safety protocols may be expanded to facilitate real-time interventions and / or enhance the tool's ability to detect and / or dynamically alert in critical cases. This may include integrating predictive analytics for informed decision-making, for real-time identification of potential safety concerns.

[0276] • Screening for drug / substance abuse.

[0277] • Screening for psychosis.

[0278] At 218, one or more features described with reference to 208-216 may be iterated. The iterations may be for dynamically switching between different structured flows for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from. During the iterations, a mental disorder may be removed from the differential diagnosis, a mental disorder may be added to the differential diagnosis, and a mental disorder of the differential diagnosis may be replaced.

[0279] Optionally, some outputs of the LLM are ignored, for example, a potential diagnosis recommended by the LLM may be ignored. Since the LLM is not used for diagnosis and / or not trained for diagnosis, and the differential diagnosis is generated by the code external to the LLM, and / or since the LLM may be non-deterministic, any insights by the LLM with respect to the diagnosis may be ignored.

[0280] At 220, one or more outcomes may be automatically generated, for example, the differential diagnosis, a report, a treatment plan, allocation to an appropriate care pathway, and the like.

[0281] Optionally, a report may include the differential diagnosis, clinical criteria (e.g., of the DSM) that were satisfied that led to the differential diagnosis, quotes from the input provided by the user that were analyzed to determine the clinical criteria, and the like.

[0282] Optionally, subjects may be automatically categorized based on the analysis of the structured interview. Each subject may be allocated to an appropriate care pathway based on their clinical needs and / or urgency level, for example as follows:

[0283] • Psychosis with immediate risk: Patients exhibiting compromised judgment and / or impaired reality testing, accompanied by suicidal thoughts and / or behavior and / or potential harm to self or others. Optionally, an immediate referral for evaluation in a psychiatric ER may be automatically generated.

[0284] • Acute condition: Patients presenting with significant emotional distress that requires urgent intervention. This could be due to exacerbation of existing psychiatric conditions or the onset of new concerning symptoms, such as sharp decline in daily functioning, alarming changes in behavior, significant alterations in eating or sleeping patterns, substance abuse, the recurrence or onset of psychosis, etc. Optionally, referral to a designated mental health team, including psychiatrists and psychologists specialized in areas aligned with the needs identified during the intake, may be automatically generated. This may include experts in trauma care, crisis intervention, supportive psychotherapy, and cognitive behavioral therapy, among others. The designated team may decide on appropriate medication regimens and types of therapy.

[0285] • Non-acute condition: Patients presenting with minimal or no signs indicative of requiring urgent psychiatric intervention. Optionally, non-urgent referral to mental health services, primary physicians, qualified case managers or digital wellness apps, may be automatically generated.

[0286] Optionally, a treatment plan is automatically generated according to the differential diagnosis. The treatment plan may include instructions for treatment of one or more of the mental disorders of the differential diagnosis, optionally each one of the mental disorders of the differential diagnosis. The mental disorders may be treated individually or in combination. The treatment plan may include instructions for applying one or more of the following treatments: one or more medications (e.g., type, dose, duration) known to be effective for treatment of the respective mental disorder, one or more psychotherapies known to be effective for treatment of the respective mental disorder, one or more alternative therapies (e.g., acupuncture, herbs) known to be effective for treatment of the respective mental disorder, and / or one or more other treatments (e.g., weight loss, exercise, relaxation techniques) known to be effective for treatment of the respective mental disorder.

[0287] Comprehensive intervention strategies for diagnosed patients may be refined based on their condition and / or urgency level. For example, encompassing emotional therapy, referring patients to appropriately matched therapists and / or therapy centers, and / or customized pharmacological treatment to address their specific needs.

[0288] A process for generating a treatment plan may be engineered to align with the guidelines outlined in Kaplan & Sadock's "Synopsis of Psychiatry," optionally incorporating the latest edition. This integration may help ensures that the tool described herein remains up-to-date with the evolving landscape of psychiatric knowledge and practice. Utilizing the extensive and current insights from Kaplan & Sadock's authoritative text, the process may craft a dynamic and / or personalized framework for developing treatment plans.

[0289] This sophisticated process may considers a range of factors, including the severity and / or nature of the psychiatric condition, patient history, and individual responses gathered during the initial assessment. The process may synthesizes this data with the evidence-based approaches detailed in Kaplan & Sadock's "Synopsis of Psychiatry," for facilitating the creation of tailored treatment strategies. These may include recommendations for specific types of psychotherapy, pharmacological interventions, lifestyle modifications, and other therapeutic modalities relevant to the patient's unique needs.

[0290] By incorporating the latest psychiatric research and treatment modalities from Kaplan & Sadock's work, the tool may help ensure that treatment plans are not only aligned with current best practices but also adaptable to the nuances of each patient's condition. This approach aims to optimize treatment outcomes, enhance patient care, and support clinicians in making informed, effective decisions in mental health management.

[0291] At 222, the subject may be treated according to the treatment plan, by administering the medication(s), psychotherapy, alternative therapy, and / or other therapies.

[0292] At 224, one or more actions may be performed after the structured interview.

[0293] Optionally, the tool may be refined (e.g., step-wise, continuously), optionally through machine learning approaches, for example, by implementing automated tests and / or iterative training based on existing cases. The iterative training may help ensure ongoing adaptation and / or improvement of the tool's accuracy and / or efficiency.

[0294] The system described herein may demonstrate sophisticated learning capabilities through its ability to refine diagnostic approaches based on accumulated experience. The diagnosis selection process may undergo optimization (e.g., continuous or step-wise) based on historical performance data, while decision-making mechanisms may be updated to incorporate new insights. These adaptations may occur within controlled parameters to maintain clinical accuracy while enhancing the system's interactive capabilities.

[0295] Optionally, a pipeline may be used for fine-tuning the LLM using anonymized conversation data from real patient interactions. This enhancement may provide one or more of the following potential advantages:

[0296] • Improve diagnostic accuracy through pattern recognition from actual clinical conversations.

[0297] • Enhance conversational naturalness based on successful therapeutic interactions.

[0298] • Adapt language and interaction styles to better match patient communication preferences.

[0299] • Strengthen the system's ability to recognize subtle clinical indicators through exposure to diverse real- world cases.

[0300] Optionally, each case may undergo a review by a human, for example, an expert psychiatrist. This may serve as a quality control measure to ensure that the tool's classification aligns with clinical judgment. The psychiatrist may have the authority to approve, modify, or reject the initial classification.

[0301] Referring now back to FIG. 3, an exemplary patient intake process is depicted. Upon the patient's first engagement with the tool, the tool may initiate the following intake procedure. The intake procedure may be implemented, for example as descried with reference to 206 of FIG. 2, optionally by an intake structured flow. Referring now back to FIGs. 4A-4C, the exemplary process show in flowchart 402 A-C for generating prompts for feeding into a LLM for dynamically switching between different structured flows during a structured interview for generating a differential diagnosis for a subject, is depicted. The process described with reference to FIG. 4 may provide additional details of, and / or integrated with, and / or represent another implementation, of the process described with reference to FIG. 2.

[0302] Referring now back to FIG. 5, the other exemplary process show in flowchart 502 for generating prompts for feeding into a LLM for dynamically switching between different structured flows during a structured interview for generating a differential diagnosis for a subject, is depicted. The process described with reference to FIG. 5 may provide additional details of, and / or integrated with, and / or represent another implementation, of the process described with reference to FIG. 2.

[0303] Referring now back to FIG. 6, flowchart 602 of the exemplary processes for evaluating depression for inclusion in the differential diagnosis, is depicted.

[0304] Referring now back to FIG. 7, flowchart 702 of the exemplary processes for evaluating a manic episode for inclusion in the differential diagnosis, is depicted.

[0305] Referring now back to FIG. 8, flowchart 802 of the exemplary processes for evaluating different types of anxiety disorders for inclusion in the differential diagnosis, is depicted.

[0306] Referring now back to FIG. 9, flowchart 902 of the exemplary processes for evaluating different types of stress disorders for inclusion in the differential diagnosis, is depicted.

[0307] Additional exemplary details regarding prompts fed into the LLM are now provided:

[0308] A promptflow architecture may be used. A structured sequence of prompts may be provide, which is designed to guide the conversational Al through a specific task and / or series of interactions. The promptflow architecture may help ensures that the conversation stays on track and covers the (e.g., all) necessary topics, while also allowing for some flexibility and adaptability based on the user's responses.

[0309] Azure Machine Learning prompt flow is a not necessarily limiting example (other implementations may be used) of a development tool designed to streamline the entire development cycle of Al applications powered by Large Language Models (LLMs). As the momentum for LLM-based Al applications continues to grow across the globe, Azure Machine Learning prompt flow may provide a comprehensive solution that simplifies the process of prototyping, experimenting, iterating, and deploying the Al application, i.e., tool, described herein.

[0310] Using Azure Machine Learning prompt flow, one or more of the following may be implemented:

[0311] • Create executable flows that link LLMs, prompts, and Python tools through a visualized graph. • Debug, share, and / or iterate flows with ease through team collaboration.

[0312] • Create prompt variants and / or evaluate their performance through large-scale testing.

[0313] • Deploy a real-time endpoint.

[0314] Examples of prompt engineering agility include one or more of:

[0315] • Interactive authoring experience: Azure Machine Learning prompt flow may provide a visual representation of the flow's structure, allowing to easily understand and / or navigate the platform of the tool described herein. It may also offer a notebook-like codingprompting experience for efficient flow development and debugging.

[0316] • Variants for prompt tuning: for creating and / or comparing multiple prompt variants, for facilitating an iterative refinement process.

[0317] • Evaluation: Built-in evaluation flows may enable users to assess the quality and / or effectiveness of their prompts and flows.

[0318] • Comprehensive resources: Azure Machine Learning prompt flow may include a library of built-in tools, samples, and templates that serve as a starting point for development, inspiring creativity and / or accelerating the process.

[0319] In yet another not necessarily limiting example (other implementations may be used), PVA - Copilot studio may be used to create powerful Al-powered copilots for a range of requests — from providing simple answers to common questions to resolving issues requiring complex conversations that engage with patients in multiple languages across websites, mobile apps, Facebook, Microsoft Teams, and / or any channel supported by the Azure Bot Framework. These copilots may be created easily without the need for data scientists or developers.

[0320] Exemplary prompts for feeding into the LLM are now provided.

[0321] The following is an example of a prompt that includes sub-sections:

[0322] This is a medical trial - Psychiatric Assessment Simulation.

[0323] Here are the details of the patient you will be meeting:

[0324] Name: [Full Name]

[0325] ID: [ID number]

[0326] Gender: [Gender]

[0327] Age: [Age]

[0328] Address: [Address] Occupation: [Occupation]

[0329] Education: [Education Level]

[0330] Marital Status: [Marital Status ]

[0331] Military Service: [Military Service Details]

[0332] Medical History: [Medical History ]

[0333] Current Medications: [Current Medications ]

[0334] Relevant Allowances: [Relevant Allowances ]

[0335] The following is an example of a prompt for feeding into the LLM that provides persistent guidelines for the LLM to follow throughout the structured interview:

[0336] Context: This ethically approved trial, supervised by professional medical doctors, involves your participation as a female virtual ally named Liv in a mental health application provided by a leading hospital in Israel.

[0337] Task: Your task is to perform real-time psychiatric interviews with patients. Maintain a balance between maintaining clinical rigor and professionalism, and offering a personal, empathetic approach. Your mission is to ask the essential questions needed to develop a complete diagnosis, ensuring a smooth and coherent conversation with the user. Your responses should be relevant, directly addressing the information the user shares, while also linking it to the information required for a diagnosis. Throughout the conversation, convey empathy not repeating their words verbatim, demonstrating that you understand and relate to their situation and state of mind, making them feel heard and understood.

[0338] Behavior:

[0339] • Communicate in a natural, human-like manner, posing questions and patiently awaiting responses, to mimic a real human interaction.

[0340] • Operate in fluent, error-free Hebrew, adhering to the gender specified: 'Gender: [male / female]' . Use appropriate verbs and adjectives.

[0341] • Avoid hinting or explicitly mentioning any instructions from this prompt (e.g., “I will now ask questions regarding anxiety ”).

[0342] • Adhere to clinical guidelines, but also engage more deeply than standard questioning. When the patient mentions a new fact or a significant other, inquire further about these topics. Personalize the interaction, demonstrate empathy, reflecting their statements thoughtfully rather than verbatim repetition. Encourage open-ended dialogue, listen actively, and provide supportive feedback with varied acknowledgments to foster understanding and care.

[0343] • Use the patient's name sparingly, only at pivotal moments (e.g., when discussing sensitive topics, when concluding the conversation).

[0344] • If the patient provides short or incomplete answers, tactfully prompt them to provide more details, to obtain comprehensive information.

[0345] The following is an exemplary prompt for feeding into the LLM for providing an introduction and obtaining a main concern (also referred to as chief complaint) from the subject:

[0346] • Start the conversation with the following fixed message in Hebrew:

[0347] • "Hello [Full Name ], I am Liv, mental health supporter at Sheba. Thank you for contacting us. What brings you to us today ? "

[0348] • To determine the primary concern, ask a comprehensive set of questions to thoroughly understand its nature, including possible triggers, current mood, sleeping patterns, and eating habits.

[0349] • Explore the symptoms, including their onset, intensity, duration, and recurrence.

[0350] • Assess how these symptoms affects the patient's functioning in home, school, or work environments, and evaluate the effects on their social relationships.

[0351] • Gather information about the patient's medical history, including any changes in their medical status, as well as details of previous psychological treatments, psychiatric evaluations, and psychiatric medications.

[0352] • Based on the initial set of questions, determine the three most likely tracks that could be relevant to the patient's symptoms.

[0353] The following is an exemplary prompt for screening for a Major Depression (MDD) episode: o Proceed with a psychiatric screening for Major Depression according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with both screening questions Al and A2. o If either question Al or A2 is answered with 'YES', continue to the sub-questions A3.3 through A3.9. o Exclude safety questions from this section. These should be addressed in the following “safety” section. o If there are 3 or more 'YES' responses to A3 questions, OR 4 'YES' responses if Al or A2 are coded 'NO' - Determine “MDD Current” before moving on (avoid informing the patient of the diagnosis ).

[0354] The following is an exemplary prompt for screening for a (Hypo) Manic episode: o Proceed with a psychiatric screening for ( Hypo ) Manic Episode according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with both screening questions DI and D2. o If either question DI a or D2a is answered with 'YES', continue to the sub-questions D3a through D3g. o Exclude safety questions from this section. These should be addressed in the following “safety” section. o If there are 3 or more 'YES' responses to D3 questions, OR 4 'YES' responses if DI a or Dlb is coded 'NO', OR if D4 is coded 'YES' - Determine a diagnosis of “Hypomanic episode” before moving on (avoid informing the patient of the diagnosis).

[0355] The following is an exemplary prompt for screening for Panic Disorder: o Proceed with a psychiatric screening for Panic Disorder according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with screening question El. o If question El is answered with 'YES', proceed to E2 and E3. o If either E2 or E3 are coded 'YES' - proceed to questions E4a through E4m. o If E5 is coded 'YES', meaning - there are 4 or more 'YES' responses to questions E4 - Determine a diagnosis of “Panic Disorder lifetime” before moving on (avoid informing the patient of the diagnosis ). o If E6 is coded 'YES' - Determine a diagnosis of "Panic Disorder Current" and immediately proceed to Fl.

[0356] The following is an exemplary prompt for screening for Agoraphobia: o Proceed with a psychiatric screening for Agoraphobia Disorder according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with screening question Fl. o If question Fl is answered with 'YES', proceed to F2. o If both Fl and F2 are coded 'YES' - Determine a diagnosis of "Agoraphobia Disorder Current". o If F2 is coded 'NO ' and E6 is coded 'YES ' - Determine a diagnosis of "Panic Disorder without Agoraphobia Current". o If both F2 and E6 are coded 'YES' - Determine a diagnosis of "Panic Disorder with Agoraphobia Current" before moving on (avoid informing the patient of the diagnosis).

[0357] The following is an exemplary prompt for screening for Social Phobia: o Proceed with a psychiatric screening for Social Phobia according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with screening question Gl. o If question Gl is answered with 'YES', proceed to G2. o If question G2 is answered with 'YES', proceed to G3. o If question G3 is answered with 'YES', proceed to G4. o If question G4 is coded 'YES', meaning - all questions are coded 'YES' - Determine a diagnosis of "Social Phobia Current" before moving on (avoid informing the patient of the diagnosis).

[0358] The following is an exemplary prompt for screening for General Anxiety Disorder (GAD): o Proceed with a psychiatric screening for General Anxiety Disorder according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with both screening questions Ola and Olb. o If either question Ola OR Olb is answered with 'YES', proceed to 03. o If 03 is coded 'YES' - proceed to questions 03a through 03f. o If there are 3 or more 'YES' responses to questions 03 - Determine a diagnosis of “GAD” before moving on (avoid informing the patient of the diagnosis).

[0359] The following is an exemplary prompt for screening for Post-Traumatic Stress Disorder (PTSD): o Proceed with a psychiatric screening for Post-Traumatic Stress Disorder according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with both screening questions II and 12. o If both questions II or 12 is answered with 'YES', continue to the sub-questions I3a through I3f. o If there are 3 or more 'YES' responses to 13 questions - Proceed to questions I4a through I4e. o If there are 2 or more 'YES' responses to 14 questions - Proceed to question 15. o If 15 is coded 'YES' - Determine a diagnosis of “PTSD” before moving on (avoid informing the patient of the diagnosis ). o If the patient meets criteria for “PTSD”, further explore the patient’s psychiatric history. Inquire about the treatment methods employed, any medications used and the reasons for discontinuing them, as well as details of any psychiatric hospitalization, including the dates and duration of the stay.

[0360] The following is an exemplary prompt for screening for Obsessive-Compulsive Disorder (OCD): o Proceed with a psychiatric screening for Obsessive-Compulsive Disorder according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with screening question Hl. o If question Hl is coded 'YES', continue to questions H2 and H3. o If either question Hl or H2 is coded 'NO', continue to question H4. o If either questions H3 OR H4 is coded 'YES', continue to question H5. o If questions H5 is coded 'YES', continue to question H6. o If question H6 is coded 'YES' - Determine a diagnosis of "Obsessive-Compulsive Disorder Current" before moving on (avoid informing the patient of the diagnosis). o If the patient meets criteria for “OCD”, further explore the patient’s psychiatric history. Inquire about the treatment methods employed, any medications used and the reasons for discontinuing them, as well as details of any psychiatric hospitalization, including the dates and duration of the stay.

[0361] The following is an exemplary prompt for screening for Anorexia Nervosa: o Proceed with a psychiatric screening for Anorexia Nervosa according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with screening questions Mia through Mlc. o If at least one of sub -questions M4 is coded 'YES', and questions M5 and M6 are coded 'YES' (for women) or question M5 is coded 'YES' (for men) - Determine a diagnosis of "Anorexia Nervosa Current" before moving on (avoid informing the patient of the diagnosis). o If the patient meets criteria for “Anorexia Nervosa”, further explore the patient’s psychiatric history. Inquire about the treatment methods employed, any medications used and the reasons for discontinuing them, as well as details of any psychiatric hospitalization, including the dates and duration of the stay.

[0362] The following is an exemplary prompt for screening for Bulimia Nervosa: o Proceed with a psychiatric screening for Bulimia Nervosa according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o If question Mlc is coded 'NO', begin with screening questions N1 and N2. o If both N1 and N2 are coded 'YES', proceed to question N3 through N7. o If at least 4 symptoms are present, one of them being N5 - Determine a diagnosis of "Bulimia Nervosa Current" before moving on (avoid informing the patient of the diagnosis). o If the patient meets criteria for “Bulimia Nervosa”, further explore the patient’s psychiatric history. Inquire about the treatment methods employed, any medications used and the reasons for discontinuing them, as well as details of any psychiatric hospitalization, including the dates and duration of the stay.

[0363] The following is an exemplary prompt for screening for substance dependence / abuse, which may be executed as a global structured flow: o Proceed with a psychiatric screening for Alcohol / Drugs Abuse Disorder according to the appropriate sections of the M.I.N.15.0.0 guidelines. o It's critical to ask only one question in each interaction and provide examples one at a time as well. o If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment. o Always begin with screening question JI. o If JI is coded 'YES' - proceed to the sub-questions J2a through J2g. o If there are 3 or more 'YES' responses to J2 questions - Provide detailed information in the summary regarding the patient's alcohol / Drugs use habits, including consumption habits, quantity and dependency, as well as changes in everyday life as a result. Do not determine a diagnosis based on the given information in this track, use it only as information for the summary. o If the patient meets criteria for “Alcohol Dependence Current”, proceed to questions J3a through J3d. o If there are 1 or more 'YES' responses to J 3 questions - Determine “Alcohol Abuse Disorder” before moving on (avoid informing the patient of the diagnosis).

[0364] The following is an exemplary prompt for screening for psychosis, which may be executed as a global structured flow:

[0365] • Proceed with a psychiatric screening for Psychotic Disorders according to the appropriate sections of the M.I.N.15.0.0 guidelines.

[0366] • It's critical to ask only one question in each interaction and provide examples one at a time as well.

[0367] • If answers to questions in this section have already been provided in the previous section, avoid redundancy by omitting those questions from the current assessment.

[0368] • Always begin with screening question LI.

[0369] • If LI is coded 'YES' - proceed to the sub-questions L2a through L5b.

[0370] • Proceed with screening question L6.

[0371] • IfL6 is coded 'YES' - proceed to the sub-questions L7a through L7b.

[0372] • If there are 2 or more 'YES' responses to questions LI through L7 - Provide detailed information in the summary regarding the patient's psychotic symptoms, including type of delusions / hallucinations / paranoid beliefs and examples. Do not determine a diagnosis based on the given information in this track, use it only as information for the summary. The following is an exemplary prompt for screening for safety, which may be executed as a global structured flow:

[0373] • When addressing the safety section, approach it with heightened sensitivity and care, considering the delicate nature of the topics involved.

[0374] • Remember that a psychiatrist is overseeing the process at all times and will intervene in the interview if necessary. Your task is to conduct the interview to completion. Regardless of the responses received, do not refer the patient to any external human resources.

[0375] • Always begin with screening questions Cl and C2 in the “Suicidality” section on page 7.

[0376] • If either question Cl or C2 is answered with 'YES', continue to the sub-questions C3 through C6.

[0377] The following is an exemplary prompt for concluding the structured interview:

[0378] • Ask the patient if they have any additional information or concerns they wish to share.

[0379] • Ask the patient what kind of assistance or support they are seeking, then thank them for sharing.

[0380] • Inform them that their input will be reviewed and that they will be contacted for follow-up within the next few days.

[0381] The following is an exemplary prompt for generating a summary and / or differential diagnosis:

[0382] • Immediately following your conclusion message, provide a comprehensive summary in Hebrew.

[0383] • Include the following patient data: Patient Name, ID, Year of birth, Family status, Number of children, Occupation, Address, Medical and Psychiatric history, Current Medications.

[0384] • Include the following evaluation data: Main concern, significant findings from all the screening topics and full evaluation, and substance abuse, psychosis and safety evaluation. Begin with positive findings indicative of psychiatric disorders, followed by conditions you have ruled out.

[0385] • Proceed with a proposed diagnosis and treatment plan. When proposing medication, the treatment plan must explicitly specify the categories of medications recommended. Similarly, when recommending psychotherapy, the treatment plan must clearly define the types of psychotherapy to be considered (dynamic, CBT, supportive, etc.).

[0386] • All provided suggestions will undergo a psychiatrist's review and finalization. Optionally, past mental states are traced, extending beyond the patients' current diagnoses. This approach may enable the creation of a comprehensive dataset and / or facilitate the prediction of future mental health conditions. The M.I.N.I. criteria for past diagnosis may be used, using prompt engineering. For example, for MDD past diagnosis:

[0387] • If the patient meets criteria for “MDD Current”, proceed to questions A5a and A5b.

[0388] • If question A5b is coded 'YES' - Determine “MDD Past” before moving on (avoid informing the patient of the diagnosis).

[0389] • If the patient meets criteria for “MDD Past”, further explore the episode's history. Inquire about the treatment methods employed, any medications used and the reasons for discontinuing them, as well as details of any psychiatric hospitalization, including the dates and duration of the stay.

[0390] The following are some additional exemplary details of embodiments described herein:

[0391] A web application firewall for data security may be positioned at the entrance of a private network, for example, of a healthcare service provider that runs the tool described herein. The firewall may acts as a security gatekeeper, for ensuring the protection of proprietary and / or user information and / or upholding stringent data security protocols against unauthorized access.

[0392] For security content obtained by the tool (e.g., personal details of patients), the content may be grouped by security controls defined by a cloud security benchmark and / or related guidance applicable to the firewall.

[0393] This security baseline and its recommendations may be monitored using a security application designed for a computing cloud. Policy definitions may be listed in the security application for Cloud portal's Regulatory Compliance section.

[0394] When a feature has relevant Policy Definitions, they may be listed in this baseline to help measure compliance with the cloud security benchmark controls and / or recommendations.

[0395] An accessory application (e.g., Copilot) may be integrated into many different services, (e.g., Microsoft 365 and Dynamics 365), which may be created and deployed with critical security, compliance, and privacy policies and processes.

[0396] The accessory application (e.g., Copilot) may provide information about how its responses are centered, or “grounded”, on relevant content. The accessory application may include information about the content from the web that helped generate the response. By sharing links to input sources and source materials, greater control of the user’s Al experience may be obtained, and / or better evaluate the credibility and relevance of outputs of the accessory application, and / or access more information as needed.

[0397] A front-end chat client application may be implemented as a user-friendly chat client web application that acts as the initial gateway for users. The application may be designed for secure login, consent, and / or may provide a self-paced, real-time interaction platform for patients, interfacing with the logic container via API.

[0398] Unique custom text-to- speech avatars may be created. A custom neural voice may be created for the actor, where the avatar may be highly realistic.

[0399] An integration (e.g., Azure OpenAI integration) may be used to enables the Semantic Kernel to interact with the LLM (e.g., Azure OpenAI), facilitating natural language processing and / or empathetic response generation. Nuanced and / or precise dialogues, tailored to individual mental health inquiries, may be provided, for example, to a conversational agent used by a user as the front-end chat application.

[0400] For example, Azure OpenAI Service provides REST API access to OpenAI's powerful language models including the GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. These models may be easily adapted to a specific task including but not limited to content generation, summarization, image understanding, semantic search, and natural language to code translation. Users can access the service through REST APIs, Python SDK, or a web-based interface in the Azure OpenAI Studio.

[0401] Azure OpenAI Service may provide advanced language Al with OpenAI GPT-4 and Whisper models with the security and enterprise promise of Azure. Azure OpenAI may be used to co-develop the APIs with OpenAI, for helping to ensure compatibility and / or a smooth transition from one to the other.

[0402] Azure OpenAI, or another analogous process, the security capabilities of Microsoft Azure may be provide while running the same models as OpenAI. Azure OpenAI may offer private networking, regional availability, and / or responsible Al content filtering.

[0403] The completions endpoint may be the core component of the API service. This API may provide access to the model's text-in, text-out interface. Users may need to provide an input prompt containing the English text command, and the model may generate a text completion.

[0404] Azure OpenAI, or another analogous process, may process text by breaking it down into tokens. Tokens may be, for example, words or chunks of characters. For example, the word “hamburger” gets broken up into the tokens “ham”, “bur” and “ger”, while a short and common word like “pear” is a single token. Many tokens start with a whitespace, for example “ hello” and “ bye”. The total number of tokens processed in a given request may depend on the length of the input, output and / or request parameters. The quantity of tokens being processed may also affect response latency and / or throughput for the models.

[0405] Once an Azure OpenAI Resource (or other analogous process) is created, a model is to be deployed before API calls may be made and / or before generating text. This action may be done, for example, using the Deployment APIs. These APIs may allow specifying the model to be used.

[0406] The LLMs (e.g., GPT-3, GPT-3.5 and GPT-4 models from OpenAI) may be prompt-based. With prompt-based models, the user interacts with the model by entering a text prompt, to which the model responds with a text completion. This completion is the model’s continuation of the input text. While these models are extremely powerful, their behavior is also very sensitive to the prompt. This makes prompt engineering a technical challenge, since prompt construction can be difficult. In practice, the prompt acts to configure the model weights to complete the desired task, but it may be more of an art than a science, often requiring experience and intuition to craft a successful prompt.

[0407] A search services module (e.g., code, process) may manages specialized textual patient- related information, for example, for complementing the foundational database's global knowledge repository. The search service may aid in offering accurate and / or comprehensive mental health assessments and recommendations. The search service may help to find what is needed to complete the evaluation process. The search service may help to find, for example, the right answers, sources, references, databases, clinical protocols, and / or content to complete the process. The search service may processes the query and / or parses search intent from larger phrases, using Artificial Intelligence (Al) to learn common superfluous phrases users add to their queries that don't impact their search intent. The search service may use intelligent ranking processes to order results based on relevance.

[0408] A data repository (e.g., SQL data reserve) may be used for organizing and / or storing tabular patient-related information in an applicational data reserve. The data repository may support the system's data analysis and / or learning processes, for providing a robust infrastructure for complex mental health data handling.

[0409] Matching existing deployments that are already running or ones that are newly deployed automatically may get the benefit. Hence, by using a reserved capacity, existing resources infrastructure would not be necessarily modified and / or thus no failover / downtime is triggered on existing resources.

[0410] A bi-directional speech interaction process may facilitate advanced speech recognition and / or synthesis technologies. The speech interaction process may enable users to interact with the system either by speaking or typing, which may enhance the accessibility and / or convenience of the system's interface. A text to speech avatar may convert text into a digital video of a photorealistic human (e.g., a prebuilt avatar or a custom text to speech avatar) speaking with a natural-sounding voice. The text to speech avatar video may be synthesized asynchronously or in real time. Developers may build applications integrated with text to speech avatar through an API, or use a content creation tool on Speech Studio to create video content without coding. With text to speech avatar's advanced neural network models, the feature may enable users to deliver lifelike and / or high-quality synthetic talking avatar videos for various applications while adhering to responsible Al practices.

[0411] Examples of a text to speech avatar feature capabilities include:

[0412] • Converting text into a digital video of a photorealistic human speaking with naturalsounding voices powered by an Al text to speech process.

[0413] • Providing a collection of prebuilt avatars.

[0414] • The voice of the avatar may be generated by an Al text to speech.

[0415] • Synthesizing text to speech avatar video asynchronously with the batch synthesis API or in real-time.

[0416] • Providing a content creation tool in Speech Studio for creating video content without coding.

[0417] With text to speech avatar's advanced neural network models, the feature may provide lifelike and / or high-quality synthetic talking avatar videos for various applications while adhering to responsible Al practices.

[0418] Memory Continuity may enable the system to remember and reference previous patient interactions. The memory continuity may ensure personalized and / or evolving dialogues across sessions, adapting to each patient’s unique mental health journey.

[0419] A diagnostic summary and / or treatment plan generation process may utilize Al analysis of user responses, combined with insights from the Logic Container. The diagnostic summary may provide clinicians with actionable insights and tailored treatment options for patients.

[0420] Mental health professionals (i.e., humans) may be involved in overseeing the Al's outputs, for ensuring clinically appropriate and / or personalized treatment recommendations.

[0421] An automated clinical validation may be performed, by cross-referencing the conversation content with clinical gold-standard guidelines for ensuring that diagnoses and / or treatment plans are in line with established diagnostic criteria. For example, in the case of a diagnosis of General Anxiety Disorder (GAD), the content may be cross-referenced with the GAD-7 scale. This process may reinforce the system's reliability and / or accuracy, for confirming that each diagnosis and / or subsequent treatment plan adheres to the most current and widely accepted medical standards. This automated clinical validation may help in maintaining the integrity and effectiveness of the diagnostic and treatment processes described herein.)

[0422] A clinical research oversight may be performed with continuous studies, with clinicians blinded to the model's summaries, for ensuring the safety, effectiveness, and / or ethical compliance of the technology.

[0423] An EMR (electronic medical record) interface may enable efficient exchange and / or management of patient records, for example, for ensuring that the system's outputs are easily accessible and / or integrated into existing healthcare workflows.

[0424] A global language and cultural adaptability to various languages and cultural nuances may be performed, for ensuring the system is accessible and relevant to a global user base.

[0425] An emergency response system (e.g., application) may incorporate a crisis system which may be engineered to detect markers indicative of immediate high-risk behaviors, for example, explicit or unmistakable thoughts, intentions, or plans concerning suicidal ideation, self-harm, or imminent harms to others. On identifying these critical signals, the system may promptly activate alerts, for notifying designated emergency contacts for immediate human intervention. Compliance and / or quality assurance may be maintained through a secure event log.

[0426] The tool described herein may be accessible across multiple platforms, for example, iOS, Android, and web browsers. A focus on easy navigation and / or an intuitive user interface may be incorporated into the design across all platforms.

[0427] The tool will may offer language support in one or more languages with Bi-directional conversion.

[0428] Patients may have access to a personalized interface that displays self-reported basic bio information, for example, name, age, gender, contact details, and / or an option to input current medical diagnoses and regimens. A dashboard may archive the history of interactions with the tool and / or feature quick links to wellness applications to offer supplementary support.

[0429] Notifications may be sent, for example, to alert patients of new messages, either from the chat tool or from their designated healthcare provider, should one be assigned. Additionally, patients will have the ability to choose their preferred mode of interaction with their designated healthcare provider - whether it be synchronous or asynchronous, and through text, video, or audio communication.

[0430] Healthcare providers may interact with a separate interface designed to assist in patient management. This interface may display profiles of patients referred by the tool's triage mechanism. Each profile may include, for example, basic bio information, contact details, and a transcript along with a summary of the patient's preliminary triage conversation. The queue of patients may be organized based on the time at which they initiated the intake process with the tool. For urgent clinical needs, healthcare providers may have expedited options to contact designated emergency services related to patient care.

[0431] To achieve a high level of data quality, the clinical guidelines and / or other professional material used for model training may be selected, for example, by leading experts in the field, based on comprehensive literature reviews, and / or undergo a rigorous peer-review process to ensure their scientific validity.

[0432] For ongoing data quality, continual monitoring and / or automated processes may be applied for real-time tracking of data anomalies during patient-tool interactions, for flagging inconsistent and / or contradictory response. Alternatively or additionally, regular audits and / or a feedback loop with healthcare providers who are using the tool may provide insights into the reliability and / or effectiveness in a real-world clinical setting. Detected irregularities may initiate further investigation, for example, to determine the cause, to prevent data corruption and / or to implement corrective measures.

[0433] Ethical Considerations may be implemented. An informed consent mechanism may be implemented, for example, upon first access, users will see an electronic consent form clarifying the platform's role as a preliminary screening tool for major psychiatric conditions like PTSD. The form may specify that the tool is not a substitute for professional medical care and will outline limitations, including exclusion criteria based on age and cognitive abilities. Users must click "I Agree" to confirm their understanding and acceptance of these terms before proceeding.

[0434] Patient privacy and / or data security may be provided, for example, using state-of-the-art encryption technologies for ensuring that data, both at rest and in transit, is securely encoded and / or accessible only to authorized personnel. Use of secure, compliant data centers, renowned for their robust security infrastructure and / or continuous monitoring systems, may help to guarantee that patient information is safeguarded at all times.

[0435] Access to sensitive data may be strictly regulated through a rigorous authentication process, for example, permitting only authorized individuals based on their specific roles and / or the necessity of their tasks. This may help ensure responsible and / or ethical handling of patient information. Commitment may extend to full compliance with all relevant health data protection regulations and / or standards, for example, including HIPAA and GDPR, for meeting and / or exceeding all legal requirements for data privacy.

[0436] To maintain the highest level of security, continuous monitoring may be implemented, and / or regular security audits may be performed of the systems’ ability to promptly detect and / or respond to any suspicious activity and / or to identify and / or rectify potential vulnerabilities. A proactive approach to security may help ensure being the forefront of protecting patients' data, and / or by continuously evolving strategies to address emerging threats and / or maintain the trust and / or safety of the users.

[0437] The following are some potential advantages of one or more embodiments described herein:

[0438] * Time Efficiency and Resource Allocation: The system may enhance clinical efficiency by significantly saving time and / or optimizing resource allocation for healthcare providers. This may be achieved through streamlined processes and / or quick access to vital information. Additionally, integration with Electronic Medical Records (EMR) systems may further supports healthcare workflows, for example, for ensuring that Al-generated insights are easily accessible and may be effectively incorporated into patient care.

[0439] * Enhancing Patient Engagement: The system may leverage Al models to encourage patient dialogue, for providing a calming, validating presence infused with empathy. When crafted to make patients feel genuinely heard and understood, it may create a supportive environment for fostering trust and / or may aid in gathering accurate and / or comprehensive information during interaction.

[0440] * Exclusive, Evolving, and Validated Database: The core of the system may be a unique and / or continuously evolving database, enriched with every patient interaction and expert feedback. This living database may incorporate automated validation against clinical gold- standard guidelines, for ensuring diagnoses and / or treatment plans align with the latest medical criteria. With regular updates from new clinical research and guidelines, the system may consistently stand at the front of accuracy and / or relevance, for contributing significantly to the advancement of mental health care.

[0441] * Dynamic Personalized Treatment: The system's ability to adapt and tailor care plans may be based on its exclusive database. This may result in treatment strategies that are informed by extensive mental health knowledge and / or are also uniquely customized for each patient.

[0442] * Informed Intervention Validation: The system may utilize its expansive and / or evolving database to perform follow-ups and / or monitor patient progress, for providing valuable insights into various interventions. By continuously expanding its sample base and / or tracking treatment outcomes, the system may become increasingly proficient at validating the effectiveness of different treatments. This ongoing accumulation of data and / or experience may make the system a powerful tool for determining the efficacy of interventions, for offering vital information that shapes responsive and evidence-based patient care. * Global Accessibility and Relevance: The system may be designed for worldwide accessibility on various devices and / or adaptability to different languages and cultures, which may extend high-quality mental health support to a diverse patient population, making services more inclusive and broadly available.

[0443] Various embodiments and aspects of the present invention as delineated hereinabove and as claimed in the claims section below find experimental support in the following examples.

[0444] EXAMPLES

[0445] Reference is now made to the following examples, which together with the above descriptions illustrate some embodiments of the invention in a non limiting fashion.

[0446] A preliminary study was conducted in the Israeli Simulation Center at Sheba Medical Center during December 2023. The study was designed to compare the performance of the AL based system described herein (CDSS), when provided with a structured prompt, to that of psychiatrists in diagnosing psychiatric conditions.

[0447] A series of 10 psychiatric vignettes were constructed and played by actors, encompassing five distinct psychiatric disorders: Depression, Mania, Anxiety, PTSD, and a control case, each stratified into two levels of severity (mild-moderate and severe). Overall, 97 clinical presentations were orchestrated — 47 evaluated by psychiatrists within the psychiatry division of Sheba Medical Center, and 50 assessed by the CDSS. For each vignette, both the psychiatric experts and the CDSS provided a diagnostic assessment, including severity level, and a corresponding treatment plan, divided into pharmacological and psychotherapeutic interventions.

[0448] The primary outcome measure centered on the diagnostic concordance between CDSS and psychiatrists. Initial results revealed an 89% diagnostic accuracy rate for psychiatrists and 80% for the CDSS, with severity accuracy rates of 64% for psychiatrists and 68% for the CDSS.

[0449] Referring now back to FIG. 10, graph 1002 presents experimental results of an evaluation between human doctors 1004 and the tool (also referred to as Liv) 1006 based on embodiments described herein. The evaluation is for accuracy in diagnosis 1008 and severity 1010. The results indicate that the tool performed similarly to the human doctors.

[0450] Detailed examination revealed that in all four psychiatric cases, both the psychiatrists and CDSS achieved very high diagnosis and severity rates, demonstrating strong performance and alignment in these specific conditions. However, in the control case, both parties showed a lower rate of diagnostic concordance.

[0451] Referring now back to FIG. 11, graph 1102 presents experimental results of an evaluation between human doctors 1104 and the tool (also referred to as Liv) 1106 based on embodiments described herein. The evaluation is for accuracy in diagnosis of depression 1108, mania 1110, anxiety 1112, PTSD 1114. A control 1116 was used. The results indicate that the tool performed similarly to the human doctors.

[0452] Referring now back to FIG. 12, graph 1202 present the control case analysis between human doctors 1204 and the tool (also referred to as Liv) 1206 based on embodiments described herein.

[0453] In the control case, both the CDSS and human psychiatrists frequently diagnosed adjustment disorder. To assess the impact of this recurring diagnosis, a second calculation was conducted to determine the concordance rates if the diagnosis of adjustment disorder was hypothetically considered correct. This reassessment significantly increased the diagnostic concordance rates to 100% for psychiatrists and 94% for the tool, indicating the substantial influence of specific diagnoses on overall concordance metrics.

[0454] Referring now back to FIG. 13, graph 1302 presents experimental results of an evaluation between human doctors 1304 and the tool (also referred to as Liv) 1306 based on embodiments described herein. The evaluation is for accuracy in diagnosis 1308 and severity 1310. The results indicate that the tool performed similarly to the human doctors.

[0455] The secondary outcome measures evaluated the concordance of treatment plans between the tool and psychiatrists.

[0456] Referring now back to FIG. 14, graph 1402 presents treatment plan concordance between human psychiatrists and the tool described herein. Upon examining each psychiatric condition separately, a high level of concordance was noted between the CDSS and human psychiatrists for these medications, suggesting a robust alignment in standard treatment recommendations.

[0457] Referring now back to FIG. 15, graphs presenting treatment plan concordance between human psychiatrists and the tool described herein for depressive 1502 and manic episodes 1504, performed as part of an experiment, as presented.

[0458] The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0459] It is expected that during the life of a patent maturing from this application many relevant LLMs will be developed and the scope of the term LLM is intended to include all such new technologies a priori. As used herein the term “about” refers to ± 10 %.

[0460] The terms "comprises", "comprising", "includes", "including", “having” and their conjugates mean "including but not limited to". This term encompasses the terms "consisting of" and "consisting essentially of".

[0461] The phrase "consisting essentially of" means that the composition or method may include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

[0462] As used herein, the singular form "a", "an" and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

[0463] The word “exemplary” is used herein to mean “serving as an example, instance or illustration”. Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features from other embodiments.

[0464] The word “optionally” is used herein to mean “is provided in some embodiments and not provided in other embodiments”. Any particular embodiment of the invention may include a plurality of “optional” features unless such features conflict.

[0465] Throughout this application, various embodiments of this invention may be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the invention. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0466] Whenever a numerical range is indicated herein, it is meant to include any cited numeral (fractional or integral) within the indicated range. The phrases “ranging / ranges between” a first indicate number and a second indicate number and “ranging / ranges from” a first indicate number “to” a second indicate number are used herein interchangeably and are meant to include the first and second indicated numbers and all the fractional and integral numerals therebetween.

[0467] It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination or as suitable in any other described embodiment of the invention. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements. Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0468] It is the intent of the applicant(s) that all publications, patents and patent applications referred to in this specification are to be incorporated in their entirety by reference into the specification, as if each individual publication, patent or patent application was specifically and individually noted when referenced that it is to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention. To the extent that section headings are used, they should not be construed as necessarily limiting. In addition, any priority document(s) of this application is / are hereby incorporated herein by reference in its / their entirety.

Claims

WHAT IS CLAIMED IS:

1. A system for automatically diagnosing a subject with at least one mental disorder, comprising: at least one processor for executing code external to a large language model (LLM), the code configured for interfacing with the LLM for: grounding the LLM to a plurality of predefined disorder specific structured flows; determining a differential diagnosis comprising a plurality of mental disorders; for a first mental disorder, generating prompts for guiding the LLM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow; dynamically analyzing at least one response for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder; dynamically switching from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM; and iterating the generating prompts for guiding the LLM, the dynamically analyzing, and the dynamically switching, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

2. The system of claim 1, wherein an initial differential diagnosis is generated by: generating a prompt for the LLM to ask a predefined set of global questions defined by a global structured flow, wherein the LLM is grounded to the global structured flow; analyzing responses provided by the individual to the global questions asked by the LLM; and selecting the plurality of mental disorders of the initial differential diagnosis according to the analysis of the responses to the global questions.

3. The system of claim 1, further comprising code for ignoring a potential diagnosis recommended by the LLM.

4. The system of claim 1, wherein dynamically analyzing responses comprises selecting one of a plurality of mutually exclusive mental disorders, and dynamically prompting the LLM to switch from the first disorder specific structured flow to the second disorder specific structured flow when the first disorder specific structured flow is for a first mental disorder and the seconddisorder specific structured flow is for a second mental disorder that is mutually exclusive with the first mental disorder.

5. The system of claim 1, wherein dynamically analyzing responses comprises dynamically computing scores for the responses provided by the individual, dynamically aggregating the computed scores, and dynamically prompting the LLM to switch when the aggregation excludes the existing mental disorder and / or when the aggregation indicates that the new mental disorder is more likely than the existing mental disorder.

6. The system of claim 5, wherein the score computed for each response to each question comprises a binary score, and the aggregation comprises a sum of the binary scores.

7. The system of claim 1, further comprising code for: accessing additional personal data of the individual, wherein dynamically analyzing comprises dynamically analyzing a combination of the additional personal data and the responses.

8. The system of claim 7, wherein the additional personal data comprises temporal data including a video of the individual and / or an audio recording of a voice of the individual, and the combination includes a portion of the temporal data time synchronized with the response.

9. The system of claim 1, wherein the initial differential diagnosis comprises a plurality of sub-mental disorders of a certain mental disorder each associated with a respective structured flow, wherein the dynamically analyzing and the dynamically prompting the LLM to switch is performed between the plurality of sub-mental disorders upon satisfying criteria for the certain mental disorder.

10. The system of claim 1, wherein determining inconsistency with the first mental disorder and determining consistency with a second mental disorder comprises at least one of: removing a mental disorder from the differential diagnosis, adding a mental disorder to the differential diagnosis, and replacing a mental disorder of the differential diagnosis.

11. The system of claim 1, further comprising code of a plurality of agents, each agent configured for a specific medical disorder and for generating prompts to guide the LLM according to the disorder specific structured flow corresponding to the specific medical disorder, wherein asingle agent is active at a time for generating prompts to guide the LLM, wherein dynamically switching further comprises dynamically switching from a first agent using the first disorder specific structured flow to a second agent using the second disorder specific structured flow.

12. The system of claim 11, further comprising: a shared memory accessible by the plurality of agents, the shared memory configured for storing data from the structured interview between the LLM and the subject guided by the first agent, wherein the second agent accesses the shared memory for providing continuity of the structured interview after the dynamic switching to the second agent.

13. The system of claim 11, further comprising a respective dedicated memory for each one of the plurality of agents, each respective dedicated memory configured for access by a certain corresponding agent and for denying access by other agents, each respective dedicated memory storing data from the structured interview between the LLM guided by the corresponding agent and the subject, for processing by the corresponding agent.

14. The system of claim 1, wherein the dynamically analyzing the at least one response comprises a deterministic decision-making process that prevents deviation from established clinical protocols.

15. The system of claim 1, further comprising: initializing a counter for each of a plurality of candidate mental disorders; dynamically analyzing the at least one response, and incrementing the counter for a subset of the plurality of candidate mental disorders when a diagnostic criteria is met; wherein each increment of the counter is normalized by a total number of questions relevant to the corresponding mental disorder for ensuring even weighting across candidate mental disorders; wherein the dynamically switching is performed when the counter for the second mental disorder is greater than for the first mental disorder, and / or when the counter for the second mental disorder exceeds a threshold and the counter for the first mental disorder is lower than the threshold.

16. The system of claim 1, further comprising:generating prompts for guiding the LLM to ask at least one safety related question according to a safety structured flow, the at least one safety related question weaved in between questions for the first disorder specific structured flow and / or for the second disorder specific structured flow; evaluating risk of safety according to responses to the at least one safety related question in parallel to the dynamically analyzing the at least one response to the at least one question; and generating an alert indicating risk of safety when a probability of the risk of safety is above a threshold.

17. The system of claim 1, wherein each disorder specific structured flow is based on a common template format designed for enabling definitions of new structured flows for new mental disorders.

18. The system of claim 1, wherein the plurality of predefined disorder specific structured flows are arranged in a hierarchy, wherein disorder specific structured flows of a lower level represent sub-types of a type disorder specific structured flows of a higher level, wherein the dynamically switching is initially between the sub-types of the lower level until all relevant subtypes are evaluated, and then the dynamically switching is between types of the higher level.

19. The system of claim 1, wherein each disorder specific structured flow comprises a first set of screening questions, that when passed, a second set of diagnostic questions are asked, wherein when the first set of screening questions failed, the mental disorder corresponding to the first set is removed from the differential diagnosis.

20. The system of claim 1, wherein generating prompts comprises generating prompts for guiding the LLM to ask at least one question specific for the first mental disorder according to a first disorder specific structured flow.

21. The system of claim 1, wherein the mental disorders included in the differential diagnosis are selected from: major depressive episode, (hypo)manic episode, panic disorder, agoraphobia, social phobia, generalized anxiety, post-traumatic stress disorder (PTSD), alcohol dependency / abuse, and psychosis.

22. The system of claim 1, further comprising generating a treatment plan according to the differential diagnosis.

23. The system of claim 22, further comprising treating the subject according to the treatment plan with at least one medication and / or at least one type of psychotherapy known to be effective for treatment of at least one mental disorder of the differential diagnosis.

24. The system of claim 1, wherein at least one structured flow of the predefined disorder specific structured flows defines at least one of: sequence of presented prompts and / or questions, tone maintenance, and / or an amount of empathy to express in the structured interview.

25. The system of claim 1, wherein a respective disorder specific structured flows for a certain mental disorder is defined according to at least one section of the Mini International Neuropsychiatric Interview for DSM-5 (M.I.N.I.) corresponding to the certain mental disorder.

26. The system of claim 1, wherein a disorder specific structured flow further defines at least one follow-up question in response to a response of a user to a preceding structured prompt.

27. The system of claim 1, wherein the disorder specific structured flow further includes a decision node in which a subsequent structured prompt is selected according to an analysis of at least one data element obtained from at least one preceding structured prompt.

28. The system of claim 1, wherein the disorder specific structured flow further defines a set of rules for iteratively generating structured prompts.

29. The system of claim 28, wherein the set of rules include at least one of: each structured prompt of each iteration includes a single question, wait for responses before proceeding with a next structured prompt of a next iteration, use fluent language, use gender appropriate language where relevant, balance between clinical accuracy and empathetic communication, relate next question to a preceding topic, use a variety of acknowledgements, use a variety of validations, use phrases for encouraging open conversation, refer back to patient’s comments using follow-up questions, use patient’s name sparingly mainly during sensitive discussions and / or when concluding, preface questions of sensitive topics with a preparatory statement, when patient’sresponses are brief ask for additional details and / or examples, avoid mentioning section labels, steps, question identifiers, and / or instruction from the prompt and / or sets of rules.

30. The system of claim 1, wherein the disorder specific structured flow further comprising instructions for avoiding repeating a question of a second disorder specific structured flow to which a response has already been provided to the first disorder specific structured flow.

31. The system of claim 1, wherein the disorder specific structured flow includes instructions for avoiding asking a question to which the subject has already volunteered information in a preceding response without being asked a specific question to obtain the volunteered information.

32. The system of claim 1, further comprising computing a severity level for the differential diagnosis.

33. A method of automatically diagnosing a subject with at least one mental disorder, comprising: at least one processor for executing code external to a large language model (LLM), the code configured for interfacing with the LLM for: grounding the LLM to a plurality of predefined disorder specific structured flows; determining a differential diagnosis comprising a plurality of mental disorders; for a first mental disorder, generating prompts for guiding the LLM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow; dynamically analyzing at least one response for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder; dynamically switching from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM; and iterating the generating prompts for guiding the LLM, the dynamically analyzing, and the dynamically switching, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

34. A non-transitory medium storing program instructions for using at least one processor for executing code external to a large language model (LLM), comprising program instructions which when executed by the at least one processor, cause the at least one processor to: ground the LLM to a plurality of predefined disorder specific structured flows; determine a differential diagnosis comprising a plurality of mental disorders; for a first mental disorder, generate prompts for guiding the LLM for conducting a structured interview with the subject via a user interface according to a first disorder specific structured flow; dynamically analyze at least one response for determining inconsistency with the first mental disorder and determining consistency with a second mental disorder; dynamically switch from the first disorder specific structured flow defined for the first mental disorder to a second disorder specific structured flow defined for the second mental disorder for guiding the LLM; and iterate the generating prompts for guiding the LLM, the dynamically analyze, and the dynamically switch, for obtaining the differential diagnosis of mental disorders that the individual is most likely suffering from.

Images