System for communicating with a patient based on determined sedation state

Abstract

The present disclosure provides a solution for monitoring the sedation / cognitive state of a patient, such as an Intensive Care Unit (ICU) patient, who may be determined to be in multiple sedation / cognitive states ranging from unresponsive to fully responsive, and providing selected interactions appropriate to the determined state, in order to optimize interactions with the patient and, optionally, improve the patient's sedation state. The determination of the patient's sedation state may be made by one or more sensing devices or input utilities, such as eye tracking sensors, EEG sensors, or any applicable user interface.

Description

[Technical field]

[0001] The present disclosure is in the field of automated systems for monitoring the cognitive status of and communicating with a patient. [Background technology]

[0002] References believed to be relevant background to the presently disclosed subject matter are listed below. -WO2021 / 024257 -WO2016 / 142933 -WO2019 / 111257

[0003] The admission of the above references herein should not be inferred as meaning that they are in any way relevant to the patentability of the subject matter of the present disclosure.

[0004] General Description The first part of the disclosure provides a system and method for interacting with a patient who may be judged to be in multiple sedation / cognitive states ranging from unresponsive to fully responsive. The terms sedation state and cognitive state are interchangeable, both referring to the patient's state of consciousness and the patient's ability to communicate with the environment. The system and method provide a solution to adapt the type of communication provided to the patient according to the patient's sedation state. This is done to ensure that the communication is effective and that there is a reasonable chance that the patient will be able to receive, process and sometimes respond to the communication. Such adapted communication is much more likely to improve the patient's sedation state.

[0005] Accordingly, aspects of the present disclosure provide a system for interacting with a patient, the system comprising a communication module configured to operate in at least three communication modes, including (i) a one-way mode in which the communication module outputs a communication to the patient that does not require a patient response. This communication is associated with a responsive mode in which the communication module outputs a communication that requires a patient response when the patient is in a sedated state in which the patient does not have the ability to respond. This may include a question to the patient or a request from the patient. This communication is associated with (iii) an open communication mode in which the communication module allows the patient to proactively initiate communication with the system when the patient has the ability to respond to the communication but lacks the ability to initiate the communication. This communication may be in various forms, for example, the patient may request from the system to play music, communicate with a clinician, caregiver, family member, watch a video, etc.

[0006] The system further comprises a processing circuit having an input module configured to receive input data indicative of a sedation state of the patient, the processing circuit being configured to: (1) determine a sedation state of the patient based on the input data, and (2) in response to the determined sedation state, trigger the communication module to operate in a selected communication mode and output a selected communication scheme based thereon.

[0007] Below are various embodiments of the system. It is understood that, unless specifically stated otherwise, certain features of the presently disclosed subject matter that are, for clarity, described in the context of separate embodiments, may also be combined and provided in a single embodiment.

[0008] In some embodiments of the system, the communication module may provide the patient with: (i) audible communication, i.e., outputting an audible communication protocol audibly to the patient and also being able to receive voice commands; (ii) video-based communication, i.e., outputting a video to the patient on a screen; (iii) eye-based communication, i.e., a camera that records the patient's eyes and the system classifies the patient's eye movements into differentiated gestures that enable communication with the system; (iv) touchscreen-based communication, i.e., the patient can communicate via a touchscreen; (v) tactile communication; and (vi) EEG-based communication, i.e., recording the patient's EEG signal, e.g., the patient's (vii) EOG-based communication, i.e., communication based on detection of signals received by an EEG unit coupled to the head, (viii) automatic lip reading communication, i.e., a camera that records images of the patient's lips and a processing circuit configured to determine from these lip images words spoken by the patient and record same as the patient's communication, (ix) head gesture-based communication, i.e., a camera that records images of the patient's head and a processing circuit configured to determine from these head images head gestures that are classified as specific communications performed by the patient, (x) or any combination thereof, which communications are enabled as part of the output communication to the patient in response to a determined sedated state.

[0009] In some embodiments of the system, determining the patient's sedation state based on the input data includes classifying the patient's sedation state into at least three sedation levels, each level triggering one or more level-specific communication modes.

[0010] In some embodiments of the system, the classifying includes scoring the patient's sedation state, the score defining the level of sedation. There are at least three score ranges, each range defining a different level. Note that two levels can have overlapping scores.

[0011] In some embodiments of the system, the scoring is a Richmond Agitation-Sedation Scale (RASS) score. For example, a score below -2 and above +3 triggers a unidirectional mode, a score between -1 and +2 inclusive triggers a responsive mode, and a score between 0 and +1 inclusive that overlaps with a second range score triggers an open communication mode. The open communication mode can operate alone or in combination with the responsive mode in a particular communication protocol profile.

[0012] In some embodiments of the system, the processing circuitry comprises: - triggering a unidirectional mode upon determining a non-responsive sedated state of the patient; - triggering a responsive mode and / or a unidirectional mode upon determining a low level of responsive sedation of the patient; - configured to apply at least one or any combination of: triggering an open communication mode and / or a response mode and / or a unidirectional mode upon determining a high level of responsive sedation of the patient.

[0013] In some embodiments of the system, the selection of the communication mode is aimed at improving the patient's sedation state.

[0014] In some embodiments of the system, the patient's sedated state is indicative of the patient's delirium state or is an actual delirium state.

[0015] In some embodiments of the system, the input data includes the patient's recorded communications with the communication module.

[0016] In some embodiments of the system, the input data includes eye image data indicative of a recorded image of the patient's eye.

[0017] In some embodiments, the system comprises a camera unit configured to record an image of the patient's eye and generate eye image data based thereon, and the input data comprises the image data.

[0018] In some embodiments of the system, the input data includes EEG data indicative of a recorded EEG signal of the patient.

[0019] In some embodiments, the system comprises an EEG unit configured to record an EEG signal of a patient and to generate EEG data based thereon, and the input data comprises the EEG data.

[0020] Yet another aspect of the present disclosure provides a method for interacting with a patient, the method comprising: input data indicative of a sedation state of a patient; determining a sedation state of the patient based on the input data; and outputting the selected communication in a communication mode selected from at least one of three communication modes including a one-way mode, a reply mode, and an open communication mode in response to the determined sedation state of the patient, the three modes being defined above with respect to the system aspects and these definitions also apply here.

[0021] In some embodiments of the method, the selected communication is any one of audible communication, video-based communication, eye-based communication, touchscreen-based communication, tactile communication, EEG-based communication, EOG-based communication, automatic lip reading communication, head gesture-based communication, or any combination thereof.

[0022] In some embodiments of the method, determining the patient's sedation state based on the input data includes classifying the patient's sedation state into at least three sedation levels, each level triggering one or more level-specific communication modes.

[0023] In some embodiments of the method, the classifying includes scoring the patient's sedation state, the score defining the level of sedation. There are at least three ranges of scores, each range defining a different level. Note that two levels can have overlapping scores.

[0024] In some embodiments of the method, the scoring is a Richmond Agitation-Sedation Scale (RASS) score. For example, a score below -2 and above +3 triggers a unidirectional mode, a score between and including -1 and +2 triggers a responsive mode, and a score between and including 0 and +1 that overlaps with a second range score triggers an open communication mode. The open communication mode can operate alone or in combination with the responsive mode in a particular communication protocol profile.

[0025] In some embodiments of the method, the outputting comprises: selecting a unidirectional mode upon determining that the patient is in an unresponsive sedated state; - upon determining a low level of responsive sedation of the patient, selecting a responsive mode and / or a unidirectional mode; - Upon determining a high level responsive sedation state of the patient, at least one or any combination of selecting an open communication mode and / or a response mode and / or a one-way mode.

[0026] In some embodiments of the method, the selection of the communication mode is aimed at improving the patient's sedation state.

[0027] In some embodiments of the method, the sedated state of the patient is indicative of a delirious state of the patient or is an actual delirious state.

[0028] In some embodiments of the method, the input data includes a patient's recorded communication with a communication module.

[0029] In some embodiments of the method, the input data includes eye image data indicative of a recorded image of the patient's eye.

[0030] In some embodiments of the method, the input data comprises EEG data indicative of a recorded EEG signal of the patient.

[0031] Another portion of the present disclosure provides a system for monitoring a patient's sedation / cognitive state by continuously monitoring the patient's ocular activity and generating a date of ocular images based thereon. The monitor is further configured to provide selected outputs, such as questionnaires, audible and / or visual outputs, to the patient to increase patient awareness and reduce the risk or condition of delirium.

[0032] Optionally, the system is configured to receive EEG data indicative of a recorded EEG signal of the patient time-correlated with the patient's recorded eye activity, and the patient's sedation state is determined based on either the EEG data, the eye image data, or a combination thereof. Different sedation states of the patient can be determined by applying different weighting factor profiles of the two data sets.

[0033] Upon determination of the patient's sedation state, the processing circuit, i.e., the processor / controller of the system, is configured to operate the communication module to trigger a selected output of an engagement communication to the patient. The output may be interactive, i.e., requiring a response from the patient, or passive, which needs to be received only by one of the patient's senses without any required response therefrom. The output communication is intended to stimulate the patient's cognitive activity, thereby improving the patient's cognitive state.

[0034] Accordingly, an aspect of the present disclosure provides a system for monitoring a patient's sedation level. The system includes: (1) a camera unit configured to record and generate eye image data based on an image of the patient's eye; (2) a communication module operable to output a desired communication protocol; and (3) a processing circuit. The processing circuit includes an input module configured to receive EEG data indicative of the patient's EEG signal and is in data communication with the camera. The processing circuit is configured to (i) receive and process the eye image data and the EEG data; (ii) determine a sedation or cognitive state of the user based on at least one of the eye image data and the EEG data; and (iii) trigger the communication module to output a selected communication protocol in response to the determined sedation state. The communication protocol can be a questionnaire, playing music, outputting a recorded voice of a family member or friend, etc.

[0035] Yet another aspect of the present disclosure provides a system for monitoring a patient's sedation level, the system comprising: (1) a camera unit configured to record an image of a patient's eye and generate ocular image data thereon; (2) a communication module operable to output a desired communication protocol; and (3) a processing circuit. The processing circuit is in data communication with the camera and operable to (i) receive and process the ocular image data, (ii) determine a sedation or cognitive state of a user based on the ocular image data, and (iii) trigger the communication module to output a selected communication protocol in response to the determined sedation state. The communication protocol can be a questionnaire, playing music, outputting the voice of a family member or friend, etc.

[0036] The following are optional embodiments of any of the above aspects: It is understood that, unless specifically stated otherwise, certain features of the presently disclosed subject matter that are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment.

[0037] In some embodiments the system further comprises an EEG unit configured to record an EEG signal of the patient and to generate the EEG data based thereon.

[0038] It should be noted that any combination of the described embodiments is applicable for any aspect of the present disclosure, in other words, any aspect of the present disclosure can be defined by any combination of the described embodiments.

[0039] In some embodiments of the system, the processing circuitry is configured to calculate a patient's sedation score, such as a Richmond Agitation and Sedation Scale (RASS) score, and categorize the score into two or more score ranges, with each range triggering a different communication protocol.

[0040] In some embodiments of the system, in at least one score range, the determination of the sedation state and / or communication protocol is triggered based solely on the EEG data. Thus, in a score indicating that the patient is sedated and there is no eye activity that can be monitored by the camera unit, the patient's sedation state is determined solely based on the EEG data.

[0041] In some embodiments of the system, in at least one score range, the determination of the sedation state and / or communication protocol is triggered based on a combination of the eye image data and the EEG data. That is, in scores indicating that the patient is alerted at some level and there is eye activity that can be monitored by the camera unit, the determination of the patient's sedation state is determined based on a combination of certain levels of the two data sets. Depending on the recorded activity of the patient's eyes and brain, the influence of each data for determining sedation is determined by the processing unit. Typically, when the patient is somewhat responsive and there is eye activity, the eye image data is more important for the determination of the sedation state.

[0042] In some embodiments of the system, the processing circuitry is configured to determine the user's sedation state by continuously classifying the patient's recorded eye activity into defined gestures, the temporal profile of the eye gestures defining the patient's sedation state.

[0043] In some embodiments of the system, the processing circuitry is configured to apply different and varying weighting factors to the EEG data and the eye image data based on the amount of information or the assumed sedation state of the patient, i.e., during a particular time period, the processing circuitry is configured to apply weighting factors to the data sets acquired by the EEG or image sensor based on the most recent determined sedation state and / or based on the changing amount of information received by the EEG or image sensor during the particular time period.

[0044] In some embodiments of the system, the processing circuitry is configured to apply weighting factors to the EEG data and the eye image data based on a recently determined sedation state, and to update the weighting factors when a change in sedation state is identified. Thus, when the patient is in a sedated state with negligible eye activity, the weighting factor on the EEG data is much more significant, and in a sedated state with significant eye activity by the user, the weighting factor on the eye image data is significantly increased.

[0045] In some embodiments of the system, the processing circuitry is configured to apply a temporal analysis to the eye image data and the EEG data to determine a correlation between eye movements, brain activity, and sedation levels, which can be performed by applying a machine learning algorithm and training the system by inputting sedation score levels under different scenarios of eye movements and brain activity.

[0046] In some embodiments of the system, the processing circuitry is configured to analyze a selected time window of the eye image data and / or EEG data following output of the communication protocol to identify a patient response to the communication protocol and determine an updated sedation state of the patient based on the response.

[0047] In some embodiments of the system, the processing circuitry is further configured to transmit a signal carrying sedation state data indicative of the sedation state of the patient to the remote unit, which may be performed via a transmission unit of the processing circuitry.

[0048] In some embodiments of the system, the processing circuitry is further configured to identify specific eye gestures in the eye image data, which either trigger a communication protocol or affect a selected communication protocol, i.e., the identified eye gestures can be used for execution of commands in the system and / or used to analyze the patient's sedation state in order to adapt an associated communication protocol.

[0049] Throughout this specification, an eye gesture should be interpreted as an eye gesture that is identified from many possible eye gestures. For example, an eye gesture can be an iris movement in a particular direction (up, down, right, or left), a blink, a steady gaze direction, a circular movement of the iris, a sequence of specific eye gestures, etc.

[0050] In some embodiments of the system, the processing circuitry is configured to classify the eye gestures into a patient's response to a questionnaire. For example, the questionnaire can be a Confusion Assessment for Intensive Care Units (CAM-ICU) questionnaire that is audibly output to the patient, and the patient responds to each question in the questionnaire with a particular eye gesture that indicates the patient's particular response to the question.

[0051] In some embodiments of the system, the processing circuitry is configured to classify the eye gesture as a command to play an audible output, the audio output being selected from specific music and voice recordings of relatives, such as greetings from friends and family.

[0052] In some embodiments of the system, the processing circuitry is configured to analyze the eye images and EEG data and identify a signature, i.e., a particular temporal pattern of eye activity, brain activity, or a combination thereof, that is indicative of the patient's clinical condition.

[0053] In some embodiments of the system, the processing circuitry is configured to correlate the temporal profile of the eye image data and / or EEG data with predefined temporal profiles corresponding to a plurality of signatures indicative of a plurality of clinical conditions, and identify a correlation that satisfies a particular condition, such as a best match or a particular threshold of match. The predefined temporal profiles are stored in a predefined database, and the processing circuitry is in data communication with the predefined database. In some embodiments, the system further includes the predefined database.

[0054] In some embodiments of the system, the processing circuitry is configured to update the database or store in its memory to generate a personalized signature for the patient upon identifying it. Identification of the personalized signature can be from clinical indicators entered into the system or by manual indicators entered by a user that identify specific temporal patterns in the patient's clinical state.

[0055] In some embodiments of the system, the clinical condition is selected from at least one of pain, thirst, hunger, and delirium.

[0056] Yet another aspect of the present disclosure provides a method for monitoring a sedation state of a patient, the method comprising: (i) receiving and processing ocular image data indicative of a recorded image of the patient's eye; and (ii) EEG data indicative of the patient's EEG signal; determining a sedated state of the user based on at least one of the eye image data and the EEG data; and outputting a selected communication protocol in response to the determined sedation state.

[0057] In some embodiments, the method includes calculating a sedation score for a patient and categorizing the score into two or more score ranges, each range triggering a different output of a communication protocol.

[0058] In some embodiments of the method, in at least one score range, a determination of sedation state and / or communication protocol is triggered based solely on EEG data.

[0059] In some embodiments of the method, in at least one score range, a determination of sedation state and / or communication protocol is triggered based on a combination of eye image data and EEG data.

[0060] In some embodiments, the method includes determining a sedated state of the user by continuously classifying the patient's recorded eye activity into defined gestures, the temporal profile of the eye gestures defining the patient's sedated state.

[0061] In some embodiments, the method further comprises applying a temporal analysis to the eye image data and the EEG data, and determining correlations between eye movements, brain activity, and sedation states.

[0062] In some embodiments, the method includes applying different and varying weighting factors to the EEG data and eye image data based on the amount of information or assumed sedation state of the patient, i.e., during a particular time period, the method includes applying weighting factors to the data sets acquired by the EEG or image sensor based on the most recent determined sedation state and / or based on the varying amount of information received by the EEG or image sensor during the particular time period.

[0063] In some embodiments, the method includes applying weighting factors to the EEG data and the eye image data based on a recently determined sedation state, and updating the weighting factors when a change in sedation state is identified. Thus, when the patient is in a sedated state where there is negligible eye activity, the weighting factor on the EEG data is much more significant, and in a sedated state where there is significant eye activity by the user, the weighting factor on the eye image data is significantly increased.

[0064] In some embodiments, the method includes analyzing a selected time window of the eye image data and / or EEG data following output of the communication protocol to identify a patient response to the communication protocol and determining an updated sedation state of the patient based on the response.

[0065] In some embodiments, the method includes transmitting a signal to the remote unit carrying sedation state data indicative of a sedation state of the patient.

[0066] In some embodiments, the method further includes identifying eye gestures in the eye image data and either triggering a communication protocol or influencing a selected communication protocol based on the eye gestures, i.e., the identified eye gestures can be used for execution of commands in the system and / or used to analyze the sedation state of the patient to adapt an associated communication protocol.

[0067] In some embodiments of the method, the eye gestures are used for the patient's response to a questionnaire.

[0068] In some embodiments of the method, the eye gestures are used to play an audible output, the audible output being selected from particular music and voice recordings of relatives.

[0069] In some embodiments, the method includes analyzing the eye images and the EEG data and identifying a signature indicative of a clinical condition of the patient.

[0070] In some embodiments, the method includes correlating a temporal profile of the eye image data and / or EEG data with predefined temporal profiles corresponding to a plurality of signatures stored in a database, and identifying correlations that meet certain conditions.

[0071] In some embodiments of the method, the clinical condition is selected from at least one of pain, thirst, hunger, delirium.

[0072] In some embodiments, the method includes generating a personalized signature for the patient and updating the database or storing the personalized signature in a memory.

[0073] Embodiment The following are optional embodiments and combinations thereof according to aspects of the present disclosure. 1. A system for interacting with a patient, comprising: a communication module configured to operate in at least three communication modes, including a one-way mode, a response mode, and an open communication mode; a processing circuit comprising an input module configured to receive input data indicative of a sedation state of the patient, the processing circuit comprising: i. determining a sedation state of the patient based on the input data; ii. in response to the determined sedation state, triggering the communication module to operate in the selected communication mode.

[0074] 2. The system of embodiment 1, wherein the communication module is configured to perform and / or enable audible communication, video-based communication, eye-based communication, touchscreen-based communication, tactile communication, EEG-based communication, EOG-based communication, automatic lip reading communication, head gesture-based communication, or any combination thereof, of the patient.

[0075] 3. A system as described in embodiment 1 or 2, wherein determining the patient's sedation state based on the input data includes classifying the patient's sedation state into at least three sedation levels, each level triggering one or more level-specific communication modes.

[0076] 4. The system of embodiment 3, wherein the classifying comprises scoring the patient's sedation state, the score defining the sedation level.

[0077] 5. The system of embodiment 4, wherein the scoring is the Richmond Agitation-Sedation Scale (RASS) score.

[0078] 6. The processing circuit - triggering a unidirectional mode upon determining a non-responsive sedated state of the patient; - triggering a responsive mode and / or a unidirectional mode upon determining a low level of responsive sedation of the patient; - A system described in any one of embodiments 1 to 5, configured to trigger an open communication mode and / or a response mode and / or a unidirectional mode upon determining a high level of responsive sedation of the patient.

[0079] 7. A system described in any one of embodiments 1 to 6, wherein the selection of the communication mode is intended to improve the patient's sedation state.

[0080] 8. A system described in any one of embodiments 1 to 7, wherein the patient's sedation state is the patient's delirium state.

[0081] 9. A system as described in any one of embodiments 1 to 8, wherein the input data includes the patient's recorded communications with a communication module.

[0082] 10. A system described in any one of embodiments 1 to 9, wherein the input data includes eye image data indicative of a recorded image of a patient's eye.

[0083] 11. A system described in any one of embodiments 1 to 10, comprising a camera unit configured to record an image of a patient's eye and generate eye image data therefrom, and the input data includes the image data.

[0084] 12. A system described in any one of embodiments 1 to 11, wherein the input data includes EEG data indicative of a patient's recorded EEG signal.

[0085] 13. A system described in any one of embodiments 1 to 12, comprising an EEG unit configured to record an EEG signal of a patient and generate EEG data based thereon, and the input data includes the EEG data.

[0086] 14. A method for interacting with a patient, comprising: receiving and processing input data indicative of a sedation state of a patient; determining a sedation state of the patient based on the input data; and outputting the selected communication in a communication mode selected from at least one of three communication modes including a one-way mode, a response mode, and an open communication mode in response to the patient's determined sedation state.

[0087] 15. The method of embodiment 14, wherein the selected communication is any one of audible communication, video-based communication, eye-based communication, touchscreen-based communication, tactile communication, EEG-based communication, EOG-based communication, automatic lip-reading communication, head gesture-based communication, or any combination thereof.

[0088] 16. The method of embodiment 14 or 15, wherein determining the patient's sedation state based on the input data includes classifying the patient's sedation state into at least three sedation levels, each level triggering one or more level-specific communication modes.

[0089] 17. The method of embodiment 16, wherein said classifying comprises scoring the patient's sedation state, the score defining the level of sedation.

[0090] 18. The method of embodiment 17, wherein the scoring is the Richmond Agitation-Sedation Scale (RASS) score.

[0091] 19. The output is selecting a unidirectional mode upon determining that the patient is in an unresponsive sedated state; - upon determining a low level of responsive sedation of the patient, selecting a responsive mode and / or a unidirectional mode; - The method according to any one of embodiments 15 to 18, comprising selecting an open communication mode and / or a response mode and / or a unidirectional mode upon determining a high level of responsive sedation of the patient.

[0092] 20. The method of any one of embodiments 15 to 19, wherein the selection of the communication mode is aimed at improving the patient's sedation state.

[0093] 21. The method of any one of embodiments 15-20, wherein the sedated state of the patient is a delirious state of the patient.

[0094] 22. The method of any one of embodiments 15 to 21, wherein the input data includes the patient's recorded communications with the communication module.

[0095] 23. The method of any one of embodiments 15 to 22, wherein the input data includes eye image data indicative of a recorded image of the patient's eye.

[0096] 24. A method according to any one of embodiments 15 to 23, wherein the input data comprises EEG data indicative of a recorded EEG signal of a patient.

[0097] 25. A system for monitoring a patient's sedation state, the system comprising: a camera unit configured to record an image of the patient's eye and to generate eye image data thereon; a communication module operable to output a desired communication protocol; a processing circuit comprising an input module configured to receive EEG data indicative of an EEG signal of a patient, the processing circuit being in data communication with the camera; i. receiving and processing said ocular image data and EEG data; ii. determining a sedation state of the patient based on at least one of the ocular image data and the EEG data; iii. the system is operable to: trigger a communication module to output a selected communication protocol in response to the determined sedation state.

[0098] 26. The system of embodiment 25, comprising an EEG unit configured to record an EEG signal of a patient and generate the EEG data based thereon.

[0099] 27. The system of embodiment 25 or 26, wherein the processing circuit is configured to calculate a patient's sedation score and classify the score into two or more score ranges, each range triggering a different communication protocol.

[0100] 28. The system of embodiment 27, wherein in at least one score range, a determination of sedation state and / or communication protocol is triggered based solely on EEG data.

[0101] 29. A system as described in embodiment 27 or 28, wherein in at least one score range, a determination of sedation state and / or communication protocol is triggered based on a combination of eye image data and EEG data.

[0102] 30. A system described in any one of embodiments 25 to 29, wherein the processing circuitry is configured to apply temporal analysis to the eye image data and EEG data to determine correlations between eye movements, brain activity, and sedation state.

[0103] 31. A system described in any one of embodiments 25 to 30, wherein the processing circuitry is configured to analyze a selected time window of the eye image data and / or EEG data following output of a communication protocol to identify a patient's response to the communication protocol and determine an updated sedation state of the patient based on the response.

[0104] 32. A system described in any one of embodiments 25 to 31, wherein the processing circuitry is further configured to transmit a signal carrying sedation status data indicative of the patient's sedation status to the remote unit.

[0105] 33. A system described in any one of embodiments 25 to 32, wherein the processing circuitry is further configured to identify eye gestures in the eye image data, the eye gestures triggering a communication protocol or affecting a selected communication protocol.

[0106] 34. The system of embodiment 33, wherein the eye gestures are used for the patient's response to a questionnaire.

[0107] 35. A system as described in embodiment 33 or 34, wherein the eye gestures are used to play an audible output, the audible output being selected from specific music and voice recordings of relatives.

[0108] 36. A system described in any one of embodiments 25 to 35, wherein the processing circuitry is configured to analyze the eye images and EEG data and identify a signature indicative of the patient's clinical condition.

[0109] 37. The system of embodiment 36, wherein the processing circuitry is configured to correlate the temporal profile of the eye image data and / or EEG data with predetermined temporal profiles corresponding to a plurality of signatures stored in a database and identify correlations that meet certain conditions.

[0110] 38. The system of embodiment 36 or 37, wherein the clinical condition is selected from at least one of pain, thirst, hunger, and delirium.

[0111] 39. A system described in any one of embodiments 36 to 38, wherein the processing circuit is configured to generate a personalized signature for the patient upon identifying it and to update the database or store the personalized signature for the patient in its memory.

[0112] 40. A system described in any one of embodiments 25 to 39, wherein the processing circuitry is configured to apply different and varying weighting factors to the EEG data and eye image data based on the patient's information content or assumed sedation state.

[0113] 41. A system described in any one of embodiments 25 to 40, wherein the processing circuitry is configured to apply weighting factors to the EEG data and eye image data based on a recently determined sedation state, and to update the weighting factors when a change in sedation state is identified.

[0114] 42. A system as described in any one of embodiments 25 to 41, wherein the processing circuitry is configured to determine the user's sedation state by continuously classifying the patient's recorded eye activity into defined gestures, and a temporal profile of the eye gestures defines the patient's sedation state.

[0115] 43. A system for monitoring a patient's sedation state, the system comprising: a camera unit configured to record an image of the patient's eye and generate eye image data thereon; an EEG unit configured to record an EEG signal of a patient and to generate EEG data based thereon; A communication module; a processing circuit in data communication with the camera and the EEG unit, the processing circuit comprising: i. receiving and processing said ocular image data and EEG data; ii. determining a sedated state of the user based on at least one of the eye image data and the EEG data; iii. the system is operable to: trigger a communication module to output a selected communication protocol in response to the determined sedation state.

[0116] 44. A system for monitoring a patient's sedation state, the system comprising: a camera unit configured to record an image of the patient's eye and to generate eye image data thereon; a communication module operable to output a desired communication protocol; a processing circuit in data communication with the camera, the processing circuit comprising: i. receiving and processing said ocular image data; ii. determining a sedation state of the user based on the ocular image data; iii. the system is operable to: trigger a communication module to output a selected communication protocol in response to the determined sedation state.

[0117] 45. A method for monitoring the sedation state of a patient, comprising: (i) receiving and processing ocular image data indicative of a recorded image of the patient's eye; and (ii) EEG data indicative of the patient's EEG signal; determining a sedated state of the user based on at least one of the eye image data and the EEG data; and outputting a selected communication protocol in response to the determined sedation state.

[0118] 46. ​​The method of embodiment 45, comprising calculating a patient's sedation score and categorizing the score into two or more score ranges, each range triggering a different output of the communication protocol.

[0119] 47. The method of embodiment 46, wherein in at least one score range, a determination of sedation state and / or communication protocol is triggered based solely on EEG data.

[0120] 48. The method of embodiment 47, wherein in at least one score range, a determination of sedation state and / or communication protocol is triggered based on a combination of eye image data and EEG data.

[0121] 49. A method according to any one of embodiments 45 to 48, comprising applying a temporal analysis to the eye image data and the EEG data, and determining correlations between eye movements, brain activity, and sedation state.

[0122] 50. A method as described in any one of embodiments 45 to 49, comprising analyzing a selected time window of eye image data and / or EEG data following output of a communication protocol to identify a patient's response to the communication protocol, and determining an updated sedation state of the patient based on the response.

[0123] 51. The method of any one of embodiments 45-50, comprising transmitting a signal to the remote unit carrying sedation status data indicative of the sedation status of the patient.

[0124] 52. A method as described in any one of embodiments 45 to 51, further comprising identifying an eye gesture in the eye image data and either triggering a communication protocol or influencing a selected communication protocol based on the eye gesture.

[0125] 53. The method of embodiment 52, wherein the eye gestures are used for the patient's response to a questionnaire.

[0126] 54. The method of embodiment 52 or 53, wherein the eye gestures are used to play an audible output, the audible output being selected from particular music and voice recordings of relatives.

[0127] 55. A method according to any one of embodiments 45 to 54, comprising analyzing the eye images and EEG data and identifying a signature indicative of the patient's clinical condition.

[0128] 56. A method as described in embodiment 55, comprising correlating a temporal profile of the eye image data and / or EEG data with predetermined temporal profiles corresponding to a plurality of signatures stored in a database, and identifying correlations that meet certain conditions.

[0129] 57. The method of embodiment 55 or 56, wherein the clinical condition is selected from at least one of pain, thirst, hunger, delirium.

[0130] 58. A method according to any one of embodiments 55 to 57, comprising generating a personalized signature of the patient and updating the database or storing the personalized signature in memory.

[0131] 59. A method as described in any one of embodiments 45 to 58, wherein the determining includes applying different and varying weighting factors to the EEG data and eye image data based on the patient's information content or assumed sedation state.

[0132] 60. A method as described in any one of embodiments 45 to 59, wherein the determining step includes applying weighting factors to the EEG data and eye image data based on a recently determined sedation state, and updating the weighting factors when a change in sedation state is identified.

[0133] 61. A method according to any one of embodiments 45 to 60, wherein the determining step includes determining the user's sedation state by successively classifying the patient's recorded eye activity into defined gestures, a temporal profile of the eye gestures defining the patient's sedation state.

[0134] In order to better understand the subject matter disclosed herein and to illustrate how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0135] [Figure 1A] 1A-1C are block diagrams of different non-limiting embodiments of a system according to aspects of the present disclosure. [Figure 1B] 1A-1C are block diagrams of different non-limiting embodiments of a system according to aspects of the present disclosure. [Figure 1C] 1A-1C are block diagrams of different non-limiting embodiments of a system according to aspects of the present disclosure. [Figure 1D] 1A-1C are block diagrams of different non-limiting embodiments of a system according to aspects of the present disclosure. [Figure 2A] FIG. 2 is a block diagram of another non-limiting embodiment of a system according to another aspect of the disclosure. [Figure 2B] FIG. 2 is a block diagram of another non-limiting embodiment of a system according to another aspect of the disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0136] The following figures are provided to illustrate embodiments and implementations of the invention of the present disclosure.

[0137] 1A-1D, which are non-limiting examples of different embodiments of a system for monitoring a patient's sedation level and engaging the patient with a selected engagement protocol based on the monitored sedation level. It should be noted that the term "sedation level" is interchangeable with the terms "cognitive level," "sedation state," or "cognitive state," all of which refer to a particular sedation / cognitive scale, such as, but not limited to, the Richmond Agitation Sedation Scale (RASS) score. The RASS score indicates what the patient's state is on a scale between "comatose," where the patient is unresponsive to vocal or physical stimuli, and "combative," where the patient is fully aware and openly combative or violent.

[0138] FIG. 1A illustrates a system 100 including a camera unit 102 configured to continuously image and record a patient's eye and generate eye image data EID indicative of the patient's recorded eye images. By monitoring the patient's eye activity, the patient's sedation state can be inferred. This is performed by continuously classifying the patient's recorded eye activity into defined gestures. The temporal profile of the eye gestures defines the patient's sedation state.

[0139] The system further includes a communication module 104 configured to output a selected communication protocol CP to the patient in response to the patient's determined sedation state by the monitoring system 100. The communication module 104 includes a plurality of predefined communication protocols CP, a particular protocol being selected in response to the determined sedation state based on a best match criterion. That is, for each particular sedation state, there is a particular communication protocol. The communication protocols can be tailored to the patient, that is, personalized content is output to the patient in the communication protocol CP. The communication protocols can include various types of communication, some of which are interactive communications, i.e. communications that require a patient response, and some of the communication protocols CP are constituted by a mere output of the communication module that does not require a patient response.

[0140] The processing circuit 106 is configured to receive the eye image data EID from the camera unit 102 and process it to determine a temporal profile of eye gestures made by the patient. Based on the identification of the signature in the temporal profile of the eye gestures, the patient's sedation level is determined. Once the sedation level is determined, the processing circuit is configured to operate the communication module 104 to output a selected communication protocol CP to the patient based on the patient's determined sedation level. Once the communication protocol CP is output, the camera unit 102 continues to record the patient's eye activity and generate eye image data. This new eye image data EID is processed by the processing circuit 106 to determine an updated sedation state of the patient by analyzing the temporal profile of eye gestures made by the patient in a time window following the output of the communication protocol CP and to identify whether the communication protocol CP is affecting the patient's sedation state. By analyzing the patient's response to the communication protocol over time, the processing circuit can learn how to better match the best communication protocol to the patient in order to achieve the patient's best progress on the sedation scale.

[0141] FIG. 1B is another example of a system, which differs from that of FIG. 1A by including an input module 108 in the processing circuit 106. The input module 108 is configured to receive eye image data EID generated by the camera unit 102, and further to receive EEG data EEGD indicative of a patient's EEG signal indicating that the patient's eye activity is monitored by the system 100. In this example, the EEG data EEGD is generated by an EEG unit that is external to and not part of the system 100. The processing circuit 106 is configured to determine the patient's sedation state based on the EEG data EEGD and the eye image data EID. That is, the processing circuit is configured to assign different weighting factors to each data set (EEG data EEGD and eye image data EID) based on the amount of information or the assumed sedation state of the patient. If the patient has a low sedation score and the patient's eyes are unresponsive, the processing circuit assigns a large weighting factor to the EEG data, and when the patient's sedation score increases, the weighting factor of the eye image data EID increases relatively. Therefore, the EEG data EEGD is very important when the sedation state cannot be determined by analyzing the patient's eye activity, i.e., below a certain sedation score. It is noted that the processing circuit is configured to continuously update the weighting factors for each patient to generate personalized weighting factors. In other words, there are default weighting factors at the start of monitoring each new patient, but the processing circuit is configured to update the weighting factors to be adjusted for each patient. The processing circuit may apply machine learning algorithms to calculate and update new weighting factors.

[0142] Reference is now made to Fig. 1C, which is another example of a monitoring system of the present disclosure. This example differs from the example of Fig. 1B by including an EEG unit 110 that generates EEG data EEGD indicative of the EEG signal of the monitored patient. The EEG unit 110 is configured to continuously record the brainwave activity of the patient and generate EEG data EEGD based thereon, which is transmitted to the input module 108 of the processing circuit 106 to assist in determining the patient's sedation level.

[0143] FIG. 1D is another example of a monitoring system of the present disclosure. The system of this example differs from that of FIG. 1C by including a database 112 that stores a plurality of predefined signatures of EEG patterns and eye image data, each of which correlates with a particular medical condition of the patient. The predefined signatures are collected from various patients or are synthetically generated, each of which is assigned a medical condition such that the processing circuit 106 can apply a correlation or matching algorithm to the collected eye image data and EEG data to find a matching signature MS in the database and determine the medical condition of the patient based thereon. The processing circuit 106 can record new signatures NS collected from the monitored patients in the database 112. It should be noted that the database 112 may be physically connected to the processing circuit 106 or may be located on a remote cloud server and communicate with the processing circuit 106 via standard network protocols.

[0144] Reference is now made to Figures 2A-2B, which are block diagrams of different embodiments of non-limiting examples of systems for interacting with a patient according to aspects of the present disclosure. Figure 2A illustrates a system 250 comprising a communication module 252 and a processing circuit 254. The communication module is configured to output a selected communication protocol CP in one or more types of communication modes selected from: (i) a one-way mode, which is a communication output to the patient without expecting a patient response; (ii) a responsive mode, which is a communication output to the patient and which requires a patient response, such that the communication module is designed to receive a communication response from the patient (e.g., through a touch screen, reading eye movements, processing voice communication, etc.); and an open communication mode, which is a communication initiated by the patient and recorded by the communication module, which can trigger an action such as playing music, calling a caregiver, communicating with a family member, etc.

[0145] The processing circuitry 252 is configured to receive input data ID indicative of a patient's sedation state and process it to determine the subject's sedation state. Based on the determination, the processing circuitry sends execution data ED to the communication module 252 to trigger a selected communication protocol CP of the selected communication type. In response to receiving the execution data ED, the communication protocol outputs the required communication protocol CP to the patient.

[0146] FIG. 2B differs from FIG. 2A by showing that the system further comprises a sensing unit 256 which records sensed data of the patient (which may be eye movement recordings, EEG signals, EOG signals, head gesture recordings, a microphone, etc.), generates sensed input data SID indicative of the patient's sedation state, and transmits it to a processing circuit 254 which processes the data and determines the patient's sedation state.

Claims

1. A system for interacting with a patient, comprising a communication module configured to operate in at least three communication modes, wherein the at least three communication modes are: (1) a unidirectional mode in which the communication module outputs to the patient a communication that does not require a response from the patient; (2) a response mode in which the communication module outputs to the patient a communication that requires a response from the patient, the response mode including a question to the patient or a request from the patient; and (3) an open communication mode in which the communication module enables the patient to actively initiate communication with the system, a processing circuit comprising an input module configured to receive input data indicative of the patient's sedation state, the processing circuit being configured to: i. determine the patient's sedation state based on the input data; ii. trigger the communication module to operate in a selected communication mode in response to the determined sedation state; wherein determining the patient's sedation state based on the input data includes classifying the patient's sedation state into one of at least three sedation levels, each level triggering one or more level-specific communication modes; wherein the classifying includes scoring the patient's sedation state, the score defining the sedation level, three score ranges defining the sedation level, each range defining a different level; the processing circuit being configured to: - trigger the unidirectional mode when determining a non-responsive sedation state of the patient; - trigger the response mode and / or the unidirectional mode when determining a low-level responsive sedation state of the patient; - trigger the open communication mode and / or the response mode and / or the unidirectional mode when determining a high-level responsive sedation state of the patient. A system as claimed in claim 1.

2. The system according to claim 1, wherein the communication module is configured to implement and / or enable audible communication, video-based communication, eye-based communication, touch-screen-based communication, tactile communication, EEG-based communication, EOG-based communication, automatic lip-reading communication, head gesture-based communication, or any combination thereof of the patient.

3. The system according to claim 1, wherein the scoring is a Richmond Agitation-Sedation Scale (RASS) score.

4. The system according to any one of claims 1 to 3, wherein the sedated state of the patient is the delirious state of the patient.

5. The system according to any one of claims 1 to 3, wherein the input data includes the recorded communication of the patient with the communication module.

6. The system according to any one of claims 1 to 3, wherein the input data includes eye image data showing a recorded image of the patient's eyes.

7. The system according to any one of claims 1 to 3, comprising a camera unit configured to record an image of the patient's eyes and generate eye image data based thereon, wherein the input data includes the image data.

8. The system according to any one of claims 1 to 3, wherein the input data includes EEG data showing the recorded EEG signal of the patient.

9. The system according to any one of claims 1 to 3, comprising an EEG unit configured to record the EEG signal of the patient and generate EEG data based thereon, wherein the input data includes the EEG data.

10. A method for interacting with a patient, comprising: receiving and processing input data indicating the sedated state of the patient; determining the sedated state of the patient based on the input data; In response to the determined sedation state of the patient, a communication module outputs communication selected in a communication mode selected from at least one of three communication modes, including: The three communication modes are: (1) a unidirectional mode in which the communication module outputs communication to the patient that does not require a response from the patient; (2) a response mode in which the communication module outputs communication that requires a response from the patient, and the response mode includes a question to the patient or a request from the patient; and (3) an open communication mode in which the communication module enables the patient to actively initiate communication with the system. Determining the sedation state of the patient based on the input data includes classifying the sedation state of the patient into one of at least three sedation levels, each level triggering one or more level-specific communication modes. The classifying includes scoring the sedation state of the patient, the score defining the sedation level, three score ranges defining the sedation level, and each range defining a different level. The outputting is - When determining a non-responsive sedation state of the patient, selecting the unidirectional mode. - When determining a low-level responsive sedation state of the patient, selecting the response mode and / or the unidirectional mode. - When determining a high-level responsive sedation state of the patient, selecting the open communication mode and / or the response mode and / or the unidirectional mode. A method including this.

11. The method according to claim 10, wherein the selected communication is any one of audible communication, video-based communication, eye-based communication, touch-screen-based communication, tactile communication, EEG-based communication, EOG-based communication, automatic lip-reading communication, head gesture-based communication, or any combination thereof.

12. The method according to claim 10, wherein the scoring is a Richmond Agitation-Sedation Scale (RASS) score. **Claim 13** The method according to any one of claims 10 to 12, wherein the sedation state of the patient is the delirium state of the patient. **Claim 14** The method according to any one of claims 10 to 12, wherein the input data includes the recorded communication of the patient with the communication module. **Claim 15** The method according to any one of claims 10 to 12, wherein the image data includes eye image data showing a recorded image of the patient's eyes. **Claim 16** The method according to any one of claims 10 to 12, wherein the input data includes EEG data showing the recorded EEG signal of the patient.

Images