A system for assessing a patient's emotional state

Abstract

A system for assessing a patient’s emotional state, for example anxiety, stress or distress levels, comprises: an imaging means which is operative to obtain at least one image of a gesture of the patient during a consultation with a clinician and to output gesture data indicative of the at least one gesture. An input terminal receives clinician input data from a clinician and patient status data is supplied which may be indicative of patient trauma. An artificial neural network determines a qualitative assessment of a patient’s emotional state based on a data matrix of the clinician input data and patient status data and a comparison of gesture data with a database of gesture data. A processor determines a patient emotional state and when a threshold in the patient emotional state is exceeded, and causes an alert to be transmitted. The gesture may be a facial gesture.

Description

Many individuals in modem life struggle with balancing workloads and achieving an ideal, so-called ‘work-life’ balance. The reasons for this are myriad. They include factors such as workload and stress caused by working long hours, as well as personal or family matters, which can occur for example when relationships break down or when there is a bereavement of a family member or close friend; monetary pressures; and more recently anxiety caused by prolonged access to or exposure to social media. Those suffering from the aforementioned pressures include people of all ages as well as, in particular, many professionals, where an individual is exposed to prolonged hours at very high levels of stress, a need to deliver results to deadlines, or perform important tasks. Such long-term stress can lead to substance abuse or alcohol dependency as a form of stress release or relaxation. In addition, some academic studies have indicated a link between social media usage and an increased risk of depression, loneliness, anxiety, and suicidal thoughts. The risk of these consequences occurring can be increased where individuals are subject to negative experiences or have a low selfworth or feelings of inadequacy. Where such individuals are identified, for example by family, friends or employers, remedial help may be sought and often, provided that adequate support and counselling is available, self-harm or suicide may be averted and a full recovery promoted. However, there is an increasing pressure on professional counsellors, psychotherapists and psychologists to consult efficiently with people experiencing these forms of stress and often it is not possible to provide continual support to a large number of patients, and therefore there is a risk of patients relapsing in between support or counselling sessions. There already exists a wide range of mental health and wellness forms of application specific software (APPs) that can be accessed via mobile and wearable devices that are available to clinical practitioners. There is also a growing market in medical practice management software aimed at mental health and wellness practitioners. Many of these comply with health insurance portability and accountability legislation (HIPPA) and local data protection laws such as general data protection regulations (GDPR) in UK and Europe. The invention arose as an aid to professional counsellors, psychotherapists and psychologists to assist in counselling and therapy sessions and to help assess a patient’s emotional state, and so provide a risk indicator. Summary of the Invention According to a first aspect of the invention, there is provided a system for assessing a patient’s emotional state comprising: an imaging means which is operative to obtain at least one image of a gesture of the patient during a consultation with a clinician and to output gesture data indicative of the at least one gesture; an input terminal which receives clinician input data from a clinician; a trauma indicator for supplying patient status data; an artificial neural network which determines a qualitative assessment of a patient’s emotional state based on a data matrix of input data, patient status data and a comparison of gesture data with a database of gesture data; and a processor which determines a patient emotional state and when a threshold in the patient emotional state is exceeded, causes an alert to be transmitted. The system is thus able to receive a number of inputs from a number of sources, both in real time, for example during a consultation session, as well as between consultation sessions, for example at night when a patient would normally be sleeping, for example from a device that is worn by a patient. Importantly the invention provides an insight to the patients emotional, physiological and behavioural states in conjunction with the therapeutic dialogue. This enables the clinician or provider of the therapeutic service to understand how the patient is reacting to the therapy and to match future sessions with behaviour and a narrative with and between therapy sessions. The invention also enables a service provider and the patient to adjust therapies based on more detailed information than is currently available to a clinician during therapy, therefore improving the clinician’s and the patient’s understanding of conditions and therapeutic outcomes. Consequently, by combining data for example from an imaging means, (such a video camera) and data input from a clinician during a counselling or therapy session and data supplied by automatic devices, such as devices used or worn by a patient, an indication of a patient’s emotional state is derived from devices, conversations and from the patient supplying patient data concerning how they feel. All these different data types are in turn may be interpreted by a professional as a risk to a patient’s mental health or well-being. In some embodiments a user operable device, which is operated or worn by a patient, supplies a signal indicative of a patient status. Examples of a user operable device include: a mobile communication device, such as a mobile telephone, a tablet, a personal computer (PCC), a palm pilot, a laptop and a wearable electronic device. Provided local data usage and personal data privacy laws are complied with, the user operable devices may be configured to supply patient status data automatically and optionally in real time to the artificial neural network. Examples of such patient status data, include times that are specified as rest periods during when a user (patient) has agreed to time based rules specifying non-use period of, for example, their mobile telephone, personal computer or laptop. Such ‘rest periods’ may typically be defined as between midnight and 6AM or when patients would normally be expected to sleep. Thus, for example when a user (patient) is supposed to be asleep if the patient is instead accessing emails or accessing social media sites or reading messages, patient status data is automatically transmitted to the system and this patient status data is appropriately flagged or labelled as warning data. Another form of user operable device includes a wearable device, such as, for example, a smartwatch, a fitness tracker, a virtual reality (VR) headset, an augmented reality (AR) headset and a so-called ‘smart sensor’ patch. These wearable devices tend to operate without user input and so are effectively autonomous and operate to provide physiological signs indicative of, for example, sleep and rest periods, heartbeat and blood pressure, blood sugar / insulin levels and cortisol levels. Data that is supplied or output from these devices is hereinafter referred to as device data and this may also be included in the data matrix of input data which is subsequently analysed by the artificial neural network to determine the qualitative assessment of a patient’s emotional state. Another example of a device that supplies device data automatically is a voice transcription device which may be used to convert spoken words into text. Examples of such devices are often electronic devices running dedicated software, such as application specific software (APPs), for example Siri (RTM) voice recognition software. These physiological gestures and physical changes may be measured or detected during a counselling or therapy session and actively used to determine mood or anxiety levels during a session or they may be stored and compared with similar data during one or more subsequent sessions. The imaging means, which is ideally a digital camera, enables physiological gestures or traits, such as twitching or closing of eyes, facial expressions, a patient averting their eyes repeatedly from the camera or looking downwards, crying or shaking, to be imaged and stored. Image or gesture recognition software may be used to detect specific flagged gestures and processing means is operative to transform these into gesture data for inputting into the artificial neural network. The imaging means therefore captures these physiological gestures, which may also include erratic gestures or sudden movements, so that the gesture data can be configured and processed, with the device data and input data from the clinician and the patient status data to determine the level of stress or anxiety. An input terminal receives input data from a clinician. The input data is received by the artificial neural network and used to compile the data matrix. The input data that is input by a clinician includes observations of mood, vocalisation of answers to questions or comments, level of engagement and may include a score or assessment of other factors provided by the clinician. This data is referred to collectively as clinician input data and is included in the data matrix. Other clinician input data can include the tone and language used by a patient, narrative volume tone, answers to certain questions, words or phrases such as for example, what drinking triggers have been encountered and active memory of specific triggering events. In addition to live imaged video data a patient may submit a selfrecorded video message for review by a psychotherapist or clinician and this may be input for digital analysis by an automatic gesture recognition system. A semantic recognition module is optionally configured to perform natural language processing (NLP) or semantic recognition of words or phrases. A neural network model may be used to determine a qualitative assessment of a patient’s emotional state based on a data matrix of input data and patient status data and a comparison of gesture data with a database of gesture data. According to a second aspect of the invention there is provided a method for monitoring a patient’s anxiety level comprising the steps of: receiving at least one gesture image of the patient, wherein at least one of the gesture images includes an image of a face of the patient and for the at least one facial image applying an image recognition software to obtain gesture data; transmitting the gesture data to an artificial-intelligence model, supplying clinician input data from a clinician; supplying patient status data indicative of patient trauma; and operating an artificial neural network to determine a qualitative assessment of a patient’s emotional state based on a data matrix of input data and patient status data and a comparison of gesture data with a database of gesture data; and when a threshold in the patient emotional state is exceeded, causing an alert to be transmitted. It is understood that aspect of the system may be included in the methodology. Optionally the clinician input data is derived from a menu which is accessible, for example via a personal computer, by the clinician and is generated using advisory notes, for example during a consultation when expressions such as physical changes, narrative volume tone, or other trauma indicators are presented which infer behaviour such as self-harm or excessive drinking. A preferred embodiment of the invention will now be described, by way of example only, and with reference to the drawings in which: Brief Description of the Drawings Figure 1 is an overall diagrammatical view showing key stages of interaction between a clinician and a patient leading to assessment and evaluation of a patient’s condition; Figure 2 is an overall diagrammatical view showing key stages of acquisition of different data types; Figure 3 is an overall diagrammatical view showing key stages of interaction between a patient, their clinician and a manager or supervisor; Figure 4 illustrates one example of an automated ‘mood tracker’ and shows diagrammatically how different data indicating a patient’s state of mind and well-being can vary throughout a year; Figure 5A to 5E show in diagrammatical form key steps of active session for one embodiment of the invention; and Figure 6 is a functional block diagram of an example of an artificial neural network operating in accordance with the invention. Detailed Description of Preferred Embodiment of the Invention Referring to Figure 1 there is shown an overall diagrammatical view of key stages of interaction between a clinician and a patient leading to assessment and evaluation. Use of an artificial intelligent neural network and generative artificial intelligence models to generate an output indicative of a patient’s emotional state, from a plurality of data inputs, is shown in diagrammatic form in Figure 6. Optionally when a threshold in the patient’s emotional state is exceeded, an alert is transmitted, for example to the patient, their employer, a trusted friend or family member or to the clinician. The system shown in Figure 6 can be adapted quickly for a wide range of downstream tasks without needing task-specific training. Qualitative assessment of a patient’s emotional state based on a data matrix of input data, patient status data and a comparison of gesture data with a database of gesture data is performed by a processor as described below. Zero-shot learning, which is a machine learning scenario in which an artificial intelligence (Al) model is trained to recognise and categorise objects or concepts without previously having seen any examples of those categories or concepts. Zeroshot learning therefore enables the system to deploy a general understanding of the relationship between different concepts to make predictions and does not use any specific examples. In-context learning builds on this capability, whereby a model can be prompted to generate novel responses on subjects or topics that the system has not been ‘taught’ during training sessions. In-context learning techniques include one-shot learning, which is a technique where the system is primed to make predictions with a single example. In few-shot learning, the system is primed with a small number of examples and is then able to generate responses in an unseen domain. In a preferred embodiment the invention is deployed as part of a ‘cloud-based’ platform via a secure client portal end-to-end encryption for audio and video calls. Referring to the Figures generally, and to Figure 6 in particular, an embodiment of the invention will now be described as a set of 12 steps below. Figure 6 is an overall diagrammatic view of a system 100 for assessing a patient’s 102 emotional state. The system 100 comprises an imaging means 104 which is operative to obtain at least one image of a gesture of the patient 102 during a consultation with a clinician 106. The imaging means 104 outputs gesture data 115 indicative of the at least one gesture to an input terminal 103 which relays the gesture data to a memory 108. The clinician also inputs data from the clinician’s interview notes or from conversations with the patient 102. This is referred to as clinician data 110 and is also stored in memory 108. Additionally, the patient 102 provides patient status data 112 such as the patient’s general mood or responses to specific questions, as explained below. An artificial neural network 114 determines a qualitative assessment of a patient’s emotional state based on a data matrix 116 of data including, patient status data 112 and a comparison of gesture data 115 with a database 125 of gesture data; and a processor 120 which determines a patient emotional state and supplies this to a clinician or to another appointed person. The system 100 may also be configured to cause an alert to be transmitted, when a threshold in the patient emotional state is exceeded. Examples of some steps, in the operation of the aforementioned system, are now set out below with reference to the Figures and the Tables. Steps 1 to 3 are shown diagrammatically in Figures 1 and 2. Examples of psychological trauma indicators, such as sudden surges of anxiety, shutting down from communication or angry mood swings, as well as replies given by the patient during counselling such as ‘YES’ or ‘NO’ answers to questions such as “Have you been drinking?” Other behaviours and responses are set out in Table 1 below. Data from a patient may be supplied automatically from a mobile communication device, such as a mobile telephone, a tablet, a personal computer (PCC), a palm pilot and a laptop. The different types of data 110, 112 and 115, obtained is also transferred to a database 130. Data stored in the database 130 is transmitted to an artificial intelligence (Al) engine 130 where it is processed. Steps 5 and 6 show how data is obtained, summarised and transcribed into key points or specific language which may then be automatically converted into digital format as code or data, for example by an automatic speech converter (not shown). Captured images which may be shared (such as whiteboard drawings that are completed during sessions may also be digested and stored. The system depicted in Figure 6 shows how different types of data gathered using a variety of applications, including natural language processors and user worn devices 140, can be used to build a digital patient profile. Wearable electronic devices, such as a smart watch, a fitness tracker, a virtual reality (VR) headset, an augmented reality (AR) headset and a ‘smart sensor’ patch may be configured to supply device data 160 which may be obtained automatically and passively, that is without conscious effort of from the patient. Trauma-specific symptoms and behaviours As per Dr Janina Fisher’s teaching on trauma Triggered reactions As per Dr Janina Fisher’s teaching on trauma Common trauma-based cognitions / schemas As per Dr Janina Fisher’s teaching on trauma • Nervous system dysregulation • Decreased concentration • Decreased interest • Irritability • Depression • Emotional numbing • Insomnia • Hopelessness • Nightmares Flashbacks • Shame Self-loathing • Hypervigilance • Mistrust • Social anxiety • Panic attacks • Chronic pain • Eating disorders • Suicidality and self-harm • Addictions • Triggered reactions = sudden, intense, and hard-to-shift anxiety, fear • Increased heart rate • Pit, tightness, clenching in stomach • Shallow breathing, hyperventilation, or holding the breath • Obsessive thinking • Response disproportional to event (major change in the previous state) • 0-to-60 reactions • “I’m doing something 1 shouldn’t / didn’t want to do” • Emotional deprivation: “No one cares about me”, No-one will help / be there for me. • Abandonment: “1 will always be alone”, people are unreliable”. • Mistrust / Abuse: “1 cannot trust others”, “People are out to get me”, people are selfish and harmful”. • Defectiveness: “1 am flawed”, “1 am damaged goods, 1 am unlovable”. • Social Isolation: “1 am alone, alienated, on the outside”. • Vulnerability: “The world is a dangerous place, “catastrophic events that 1 cannot cope with are ahead”. • Dependence / lncompetence: “1 cannot trust my own judgement”. • Enmeshment / Undeveloped Self: “1 am not an individual”, “My identity is not separate from my significant others” • Failure: “1 will fail, “1 cannot perform well enough” • Muscle tension (either whole body or specific areas) • Twitches, tics • Jumping to conclusions • Jumping to “worst-case scenario” • Feeling that ‘the sky is falling’. • Sense of not belonging, being on the outside looking in • Fear of abandonment or aloneness, feeling small • Subjugation: “I must submit to the control of others, or else punishment or rejection is forthcoming”. • Self-Sacrifice: “I must sacrifice my own needs for the sake of others”. • Approval-Seeking / Recognition-Seeking: “Approval, attention and recognition matters more than genuine self-expression”. • Emotional Inhibition: “I must control my self-expression or others will reject me” • Negativity / Pessimism: “I have negative expectations for the future”. • Unrelenting Standards: “I must be the best”. • Punitiveness: “People should be harshly punished for their mistakes” • Entitlement / Grandiosity: “I am more important than others”, I do not have to follow the rules”. • Insufficient Self-Control / Self- Discipline: “I cannot accomplish goals, especially if the process contains boring, repetitive, or frustrating aspects”. Table 1 Hard signs of dissociation As per Dr Janina Fisher’s teaching on trauma Soft signs of dissociation As per Dr Janina Fisher’s teaching on trauma • Overwhelming emotions: desperation, despair, shame and self-loathing, hopelessness and helplessness, rage • Chronic expectation of danger: hypervigilance and mistrust, fear and terror, “post-traumatic paranoia” • Body sensations: numbing, dizziness, tightness in the chest and jaw, nausea, constriction, sinking, quaking • Movements and impulses: restlessness, ‘hang-dog’ posture, frozen states, impulses to “get out,” violence turned against the body, “sex, drugs &rock ‘n roll,” huddling or hunkering down • Difficulties with memory: unable to give coherent accounts, coming late or forgetting appointments, forgetting conversations, having different accounts • Somatic signs: headaches, chronic pain, paradoxical / non-response to meds • Childlike speech, affect or cognition: out of character with level of functioning; perceptions or reactions consistent with younger developmental levels: “Are you mad? Are you going to hurt me? Do you like me?” • Chronic condition of being “stuck” in life development, unable to grow professionally or relationally; each step forward followed by a step back • “Terminal ambivalence” about even minor decisions: “undoing” decisions, sabotaging them, failing to take the steps to carry them out • A history of rocky or failed or prolonged therapeutic treatments without much improvement or diagnostic clarity. • Therapist disempowerment: the therapist feels incompetent, confused, overwhelmed, and helpless. It seems as if “nothing works” with this client. • Signs of identity confusion or conflict: remaining in therapy while devaluing it or the therapist, feeling meek and submissive but perceived by others as enraged or difficult, being both overresponsible and self-destructive. • Chronic self-destructive, self-harming or addictive behaviour: despite patient’s hard work and good treatment, the self-destructiveness or addictive behaviour is either unremitting or constantly re-surfacing. Table 2 Clinicians use the information in Table 2 to inform clinical response and onward referral, for example to a specialist, such as a home treatment teams, eating disorder services or a chronic pain practitioner. Individualised trackers may be generated for each patient. Examples of individualised trackers are list Table 3. Risk Behaviours Attendance • History of trauma • Hospitalisation • Relapse • Self-disclosed use of social media • Eating disorders • Self-harming behaviours • Regular attendance • Missed appointments. • Physical health problems • Comorbidity • Problems with finances • Social isolation • Relationship problems • Moving home • Suicidality (thoughts, research, planning with intent, testing, attempts on life) • Drugs and alcohol • Gambling • Sexual addictions • Overspending • Withdrawal from therapy with an explanation • Withdrawal from therapy without explanation • Requesting additional sessions • Signs of dependence (resistance to stop but with nothing further to work on) Table 3 Table 4 shows some examples of Education / Tools most used / referred to Psychiatry / Medication Recovery • Psychoeducation • CBT tools • DBT tools • CFT tools • ACT tools • Trauma specific tools • Diary writing • Exposure exercises • Coaching exercises • Sensorimotor psychotherapy • Schema Therapy • Psychiatric referral • Antidepressants / anti-anxiety - SSRIs, SNRIs, NDRIs. • Benzodiazepines • Stimulants • Antipsychotics • Sleeping pills • Tranquillisers • Mood stabilisers • Period well / reviews • Relapse prevention plan • Protective Behaviours • Social support / stability • Future goals / intentions • Purpose and meaning defined Table 4 The system is capable by comparing trackers, for example risk trackers (an overall view of which is shown in Figure4) , over time with activity trackers. Accordingly, the system is able to determine whether the patient’s activities (drinking alcohol, exercise, working long hours) have an impact on risk of relapse, for example. Information that is managed and manipulated by the neural network 114, shown in Figure 6, is automatically separated into sections relevant to each person involved with the therapeutic process (patient, clinician, manager) and may be sent to a respective recipient’s mobile telephone, smartphone and / or computer. Examples of information which is managed and manipulated by the neural network 114 for each group includes: Client Clinician Manager • Summary of the last session • Key concerns / problems across therapy timeline • Goals agreed • Education and skills / tools learned. • Agreed actions (therapist and client) • Capture physiological information from the video content • Summary of the last session • Key concerns / problems across therapy timeline • Goals agreed • Education and skills / tools learned • Agreed actions (therapist and client) • Identify the client’s individual interests, values and drivers • Integrate with a number of other clinical management and diary systems. • Provide a risk register of high to low-risk clients. • Provide action prompts to the clinic management team based on the client’s risk register. • Provide action prompts to the clinician based • Physiological data • Physiological data of on the clinician’s self- about client from the both client and reported stress levels. sessions from health therapist from the • Summarise key client watches / wristbands video content information for letter- • Summarising areas of • Physiological data of writing purposes. the session where the both client and • Flag clients who client displayed the therapist from the regularly cancel or strongest physiological sessions from a range change their responses through of wearables such as appointments or who language, body health may require additional posture, facial watches / wristbands. management time to expression and • The areas where the coordinate their case - physiological data from clinician displayed the for example, those health strongest physiological who have a watches / wristbands reactions multidisciplinary team • Tracker information (corroborated by self- involved. • Other agencies reporting) • Capture client data as involved • Prompt the clinician to a live feed to provide • Capture and sends seek additional broader research diagrams and support where insights and patterns. drawings shared on physiological signs of the whiteboard during distress have been sessions. identified. • Feedback extracts of • Summarising areas of the therapy in different the session where the formats, i.e. voice client displayed the summaries, visual or strongest physiological written prompts (you responses through have reported feeling language, body most stressed posture, facial when....., assertive expression and statements developed physiological data from in the sessions, affirmations etc.) • Extract specific sections of the voice recording which can be listened to in between sessions. • Prompt the individual to take action, based on their goals. • Integrate with the client’s diary system. • Map out the therapy schedule and prompt before the session. • Provide access to a library of educational content and tools that support the content covered in the sessions, enabling the user to highlight which ones they want to focus on using. health watches / wristbands • Tracker information Table 5 The information is collated and labelled and, where appropriate converted into a suitable digitised format so that it can be managed and manipulated by the neural network 114, stored on a database 125 and transmitted to a remote recipient, for example a family member, manager, the clinician or in some circumstances the patient. In the latter case, this has the advantage that: the patient may in fact be more likely view this information on their smartphone. The clinician tends to use a smartphone or computer and a manager has access as a supervisor to the patient’s computer. Referring briefly to Figure 3 information sent to the patient, clinician and manager at stage 8 is fed back into the session management and preparation stage. Figure 4 illustrates one example of an automated ‘mood tracker’ and shows diagrammatically how different data indicating a patient’s state of mind and well-being can vary throughout a year. It shows an example of what can be tracked. It is not just mood that is tracked. The diagram shows examples of the sorts of information gathered, via various devices and inputs and from data obtained both during and between sessions that is tracked, together with other inputs such as, how mood impacts behavioural patterns, or if the patient shows signs of being in a ‘fight’, ‘flight’ or ‘freeze’, ‘submit’, ‘cry for help’ mentality. Or it the patient state changes during therapy and if so how this impacts on the therapeutic progress and the therapeutic alliance. Examples of types of anonymised data, are listed below. Once obtained anonymised data is then sent to a live service data tracker containing the following information: o Age o Gender o Job role o Exercise o Psychiatric referrals o GP referrals o Healthcare providers involved o Crisis o Physical Health information o Family information o Triggers for distress o Psychological tools most used o Tools used for different problems o Protective factors o Problematic coping strategies o Engagement o Work pressure o Living circumstances o Economic circumstances o Experience of migration o Equality-based data and information o Personal life stressors o Psychometric testing scores o Diagnosis o Risk o Relapse o Referral out o Service use (length of time, consistency, return) o Reported experience of therapy o Follow up Referring to Figures 5A to 5E there is shown in diagrammatical form key steps of an example of an active session for one embodiment of the invention. Managers have the option to access the system and review specific client files and appropriate permissions can be imposed or granted to users, administrators or manager to support the administration of clinical service data. Clinicians are able to log into a ‘cloud based’ clinical system via their computers and their phones to view key details about patients they are due to see. Patients also are given access to log into the system, for example via through their phones to view information about previous sessions, review progress, education and tools, and remind themselves of agreed tasks. Referring to Figure 5B, the activity of all users is logged in a clinical tool and fed into the database. During an active session, as depicted in Figure 5C data is derived from a patient’s computer, telephone, health watches and wristbands. Referring to Figure 5D, sessions are recorded capturing both video and voice. The clinician and the patient are using the Clinical Tool at this stage. Figure 6 shows that when the patient is wearing a smart watches, wristband or even a ring 140 data about their physiological responses is captured and may also be transmitted to the neural network 114 or the database 125. Following a consultation video and voice recordings are downloaded for storage and archive. Automatic voice transcription (not shown) is performed and so speech and conversation may be summarised. Physiological responses and symptoms are tracked alongside the narrative to measure how both patients and clinicians are responding to treatment approaches, relationship dynamics, risk and other factors impacting therapy. Data captured from devices 140 during or between sessions includes: Heath watch / wristband Facial indicators Body indicators Behavioural indicators emotion of During sessions Eyes: Posture types: • Laughing • Resting heart • Blinking • Body and spine in • Giggling rate quickly alignment and • Smiling • High / low • Staring shoulders upright • Staring heart rate without • Rounded • Sighing notifications blinking / shoulders with • Smirking • Heart rhythm Staring hunched • Crying intensely appearance • Blood oxygen • Dilated pupils • Head pushed • Uncontrollable saturation • Closing eyes forward sobbing • Blood • Looking away • Leaning away • Frowning pressure • Squinting from computer • Spiting • Sweating • Slamming the • Cortisol Eyebrows: Hands and arms: desk with levels • Raised and • Arms crossed hand / objects • Body arched • Arms behind • Slamming doors temperature • Lowered and head / other objects pulled • Arms raised • Kicking objects together above the head • Swearing • Drawn up in • Head tilted • Rapid breathing the inner upwards • Hyperventilating corners • Hands touching • Passing out the face hands Mouth: hovering near the Voice / speech: • A dropped jaw head • Raising speech • Open mouth • Hands on hips volume • One side of and elbows out • Shouting the mouth • Head facing • Changes in voice raised at the forward tone and quality corner • Hands covering - for example, • Both sides of face suddenly the mouth • Head down switching to a raised at the • Moving arms and little girl corners hands around • Quiet / timid • Corners that frenetically speech are drawn • Pressured down Other movement: speech (fast • Lip biting • Lying down speaking) • Pursed lips across desk • Slurring speech • Standing up • Delayed • Walking around response time • Covering the • Leaving the room • Thought mouth with • Remaining very disordered hand still speech • Appearing • Appearing to be Muscle changes: physically frozen speaking to • Clenching • Moving out of someone else jaw sight of camera - • Visible for example lying tension in on the floor neck • Putting head muscles between knees / on lap • Shaking • Flushed appearance • Loss of colour in face • Feet on desk • Lying on / in a bed / sofa • Walking and talking • Fidgeting with or without an object • Flinching / startled reaction • Jumping • Suddenly beathing inwards Referring to Figure 5E, in-between sessions stage, with client permission, data from client's phones, health watches / wristbands and Health Apps is tracked and recorded on a database. This data is fed into the Al process of summarising information for stages 8, 9, 10 and 11. Information tracked includes: Heath watch / wristband / Phone Health APPS During sessions • Resting heart rate • High / low heart rate notifications • Heart rhythm • Blood oxygen saturation • Blood pressure • Sweating • Cortisol levels • Body temperature • Sleep time and quality (Frequent waking, Awake periods, Asleep periods) • Sitting periods • Steps walked • Time Online • Time on social media • Sleep / wake patterns • Mental health • Healthy lifestyle • Addictions • Fitness and exercise • Pregnancy • Menstruation • Menopause • Men’s health • Telemedicine • Doctor Management • Appointment management • Chronic disease / condition management • Medication tracking • Creative pursuits • Gaming • Meditation • Reminders • Diary • Activity monitoring Table 7 Clinicians have access to this information as it is summarised and made available on the clinical tool. Figure 5F shows what occurs at an evaluation stage: Managers and clinicians will use the data contained within the clinical tool to evaluate the effects of clinical interventions to guide future service developments. The information contained within a service data tracker informs the service evaluation stage. BENEFITS TO STAKEHOLDERS CLIENTS CLINICIANS SERVICE ADM IN / M AN AGERS ■ Risk trackers provide clients with key information about what triggers risky behaviours in them, leading to a more informed and empowered client who can reduce risk at earlier stages. ■ Clients learn to recognise early warning signs / triggers for distress enabling them to utilise clinical tools shared to reduce symptoms at the earliest ■ Clinicians is able to map responses to specific treatments and alter the treatment approaches based on this information. For example, they is able to track trauma-based interventions with physiological / behavioural responses in order to assess how the client is responding to that specific intervention. ■ Risk trackers enable clinicians to work with ■ Risk trackers enable admin / managers to detect early signs of risk and work closely with clinicians and other health professionals to manage and minimise client / service risk. ■ Managers have insights to detailed data that will enable them to allocate resources to more time-consuming stages. This increases patient recovery and safety. ■ Clients have access to easily accessible, summarised information about their therapy, increasing engagement between sessions. Clinical Tool aids and encourages client engagement equalling improved outcomes. ■ Clients are encouraged to take responsibility for planning for their sessions equalling increased motivation. ■ Clients have access to easily readable reminders of the last session details and agreed actions improving client engagement and motivation. ■ Supports client / clinician relationship. ■ Increased feeling of ease and confidence in clinical service. ■ Time-saving for clients. clients and service admin / managers to detect early signs of risk to increase client / service safety. ■ Clinicians have access to clients’ logged activities and responses to clinical interventions. This leads to improved clinical intervention and therefore outcomes. ■ Clinicians have easily readable reminders of the last session details and agreed actions. This leads to: o Reduced clinician stress. o Reduced planning time for clinicians. o Increased clinician ease and confidence. o Support strong clientclinician relationship. ■ clients in an intelligent way. ■ Managers understand what clients are most triggering / stress-inducing to clinicians and input support systems to protect them. ■ Managers and clinicians can view shared data to make better joint decisions improving internal communications between team members. ■ ■ Clients have concise, lasting data about their recovery containing relapse prevention plans and skills that they have specifically responded to. This provides a lifelong resource that prevents relapse and reduce the need to return to therapy. Table 8 FAMILY / NOK OTHER HEALTHCARE PROVIDERS ■ Family members (with agreement) have access to controlled information enabling them to be better informed about client care and how to support them. This will broaden access to informed / educated support in the client’s social network. ■ Other health services and local authorities benefit from improved planning and communication for patients / clients who are open to their services. ■ As the clients is provided with a life-long resource that will prevent relapse and reduce the need to return to therapy, this should reduce pressure on the demand for statutory mental health services. ■ Data tracking physiological and behavioural indicators may be used to better understand or predict the onset of physical healthcare problems. This data can be used in partnership with physical healthcare providers to inform their research / development of clinical interventions. Table 9 WIDER STAKEHOLDERS: - Other Health Services (private and statutory) - Local Authorities - Government systems - Research departments - Educational providers - Coaching services - NGO’s / Charities - Al specific research centres and businesses ■ Service evaluations is generated into informative reports / best practice guidance that will inform better working practices for all of the above wider stakeholders. ■ Through evaluation, key areas for service integration are identified and developed in partnership with other stakeholders improving joint working, patient safety and clinical standards. ■ Service data from live trackers and evaluations will enable the ability to map large, detailed data pools which will inform research leading to broader improvements in patient safety, clinical standards, integrative working, and innovations. ■ Data will inform the development of the Clinical Tool trailed / adapted for use in other service areas including other health services, education providers, coaching services and NGO’s. ■ Service data is used to inform government strategy and policy to improve population health outcomes and innovate government systems. ■ Data that specifically maps narrative / voice alongside physiological and behavioural indicators of distress is used to teach / inform Facial Emotion Recognition (FER) leading to the safer and more responsible use of Al in mental health fields. ■ Data that specifically maps narrative / voice alongside physiological and behavioural indicators of distress will specifically inform research into trauma-based treatments and their efficacy, leading to the development of new treatments and innovations within this field. Table 10 Figure 6 is a functional block diagram of an example of an artificial neural network operating in accordance with the invention. Referring again to to Figure 6, it is understood therefore that the artificial neural network 114 determines the qualitative assessment all forms of data as well the gesture data 115 obtained during a consultation or from a live video or from a recorded video. Where images may be obtained using a video camera which are configured to image blood flow and / or pulse. A video camera may include infra-red (IR) imagers which are sensitive to very small variations in skin temperature which data may be included as physiological data and may be indicative of increased stress in a patient or used as an indicator of a patient lying or attempting to conceal a true response to a question for example. The invention has been described by way of examples only and it is appreciated that variation may be made to the aforementioned embodiments without departing form the scope of protection as defined by the claims appended hereto. It is also understood that the invention may be used as part of a range of tools by a clinician in order to help to assess risk and as part of a package of therapy and patient monitoring.

Claims

1 A system for assessing a patient’s emotional state comprises:an imaging means which is operative to obtain at least one image of a gesture of the patient during a consultation with a clinician and to output gesture data indicative of the at least one gesture;an input terminal receives clinician input data from a clinician;a trauma indicator supplies patient status data;an artificial neural network determines a qualitative assessment of a patient’s emotional state based on a data matrix of input data and patient status data and a comparison of gesture data with a database of gesture data; anda processor determines a patient emotional state and when a threshold in the patient emotional state is exceeded, causes an alert to be transmitted.2 A system according to claim 1 wherein a user operable device supplies data indicative of a patient status.3 A system according to claim 2 wherein the user operable device is from the set comprising: a mobile communication device, such as a mobile telephone, a tablet, a personal computer (PCC), a palm pilot and a laptop.4 A system according to any preceding claim wherein device data is provided by at least one of: a wearable electronic device, such as a smart watch, a fitness tracker, a virtual reality (VR) headset, an augmented reality (AR) headset, and a ‘smart sensor’ patch.5 A system according to claim 4 wherein the wearable device is from the set comprising: a smart watch, a fitness trackers, a smart ring, a virtual reality (VR) headset, an augmented reality (AR) headset and a sensor patch.6 A system according to any preceding claim wherein a signal indicative of a patient status is provided by an automatic voice transcription device.7 A system according to any preceding claim wherein the clinician input data is derived from a menu which is accessible, for example via a personal computer, by the clinician.8 A system according to any preceding claim wherein the clinician input data is generated using advisory notes.9 A system according to any preceding claim wherein the artificial neural network determines the qualitative assessment of the gesture data during a consultation.10 A system according to any of claims 1 to 6 wherein the artificial neural network determines the qualitative assessment of the gesture data between at least two11 A system according to claim 10 wherein the gesture includes facial gesture.12 A system according to any preceding claim wherein the input terminal includes a display which is operative to present a menu and the clinician uses a mouse to select input data.13 A system according to claim 12 wherein the menu is an interactive menu, and the clinician uses a mouse to select input data contemporaneously to observing image data during a consultation.14 A system according to any preceding claim receives at least one patient input which is supplied automatically by a patient’s electronic device when configured to provide authorised data indicative of patient activity.15 A system according to any preceding claim includes a voice recognition device that detects words spoken by a patient and a natural language processor determines a textual feature to derive an emotional index, wherein the emotional index associates the textual feature with a patient’s emotional status.16 A system according to claim 15 includes a neural linguistic processor which assesses a textual feature, compares actual spoken words with the assessed textualfeature and determines an emotional state one or more interactions based on an aggregation of emotional features.17 A system according to any preceding claim for monitoring a patent’s state of well-being.18 A system according to claim 17 which is configured to transmit an alert when a patient well-being threshold value is reached.19 A method for monitoring a patients level of distress comprising the steps of: receiving at least one gesture image of the patient, wherein at least one of the gesture images includes an image of a face of the patient and for the at least one facial image applying an image recognition software to obtain gesture data; transmitting the gesture data to an artificial-intelligence model, supplying clinician input data from a clinician; supplying patient status data indicative of patient trauma; and operating an artificial neural network to determine a qualitative assessment of a patient’s emotional state based on a data matrix of input data and patient status data and a comparison of gesture data with a database of gesture data; and when a threshold in the patient emotional state is exceeded, causing an alert to be transmitted.

Images