Systems and methods for nonverbal communication

Abstract

A computer-executable method for non-verbal communication is performed by mapping at least one non-verbal action of a user by uniquely associating it with a corresponding communication parameter, then providing textual content including at least one selectable text element to the user via a graphical interface, while continuously acquiring the user's non-verbal actions, and processing the at least one text element according to the communication parameter uniquely associated with the acquired non-verbal action.

Description

Technical Field

[0001] The present invention relates to the technical field of communication systems and methods.

[0002] In particular, the present invention relates to systems and methods for non-verbal communication that are particularly advantageous in the field of medical / care for patients who cannot communicate their thoughts verbally, such as intubated patients or tracheostomized patients.

Background Art

[0003] The ability to communicate effectively with others is a basic need for all individuals.

[0004] However, impairments can occur in a wide range of situations, making such communication processes difficult if not impossible, thus causing considerable inconvenience.

[0005] As an example, the medical field is particularly mentioned, in which a person who is hospitalized or undergoing a specific medical procedure and / or treatment may be unable to communicate their thoughts verbally with the people around them, especially medical staff.

[0006] Consider a patient in the intensive care unit who is intubated or tracheostomized and thus unable to speak.

[0007] In this situation, there is a further clear need for a method and system that enables the patient not only to express basic needs but, more importantly, to express specific problems that may occur that cannot be fully understood / interpreted by the simple activation of an alarm, in order to inform the medical staff.

[0008] For this purpose, there are devices that receive non-verbal input, convert it into a message, and through which a patient can communicate with others.

[0009] However, known devices still have drawbacks and inefficiencies that make implementation unsatisfactory and user-unfriendly.

[0010] In particular, known devices are configured to uniquely identify a single specific nonverbal action, and since a single specific nonverbal action is uniquely associated with a single type of message, they are extremely inflexible in their operation and therefore cannot be adapted to the specific needs of individual patients.

[0011] Therefore, if a patient is unable to perform or has difficulty performing a specific and unique nonverbal action that can be identified by the device, the device is not actually useful and cannot be used. [Overview of the Initiative]

[0012] In this regard, the fundamental technical problem of the present invention is to propose a system and related methods for nonverbal communication that overcome at least some of the aforementioned drawbacks of the prior art.

[0013] In particular, the object of the present invention is to provide a system and method that enables people who are unable to express their thoughts verbally, or who have difficulty doing so, especially intubated or tracheostomy patients, to communicate nonverbally with a second person simultaneously, accurately and efficiently, in a particularly versatile manner.

[0014] The identified technical problems and objectives are substantially achieved by systems and methods comprising the technical features described in one or more of the claims of the attached patent claims.

[0015] According to the present invention, methods for nonverbal communication, particularly computer-executable methods, are described herein.

[0016] This method involves mapping at least one nonverbal action of a user to a corresponding communication parameter by uniquely associating it with that parameter.

[0017] Once the mapping step is complete, actions and each parameter are uniquely associated, and text content is provided to the user.

[0018] In particular, text content is provided through a graphical interface.

[0019] Furthermore, the text content includes at least one selectable text element.

[0020] Such text elements may include, for example, one or more alphanumeric characters, symbols, words, phrases, or graphic elements.

[0021] During the delivery of text content, user nonverbal actions are continuously captured, and the text content is processed according to communication parameters uniquely associated with the progressively captured nonverbal actions.

[0022] Advantageously, the method proposed herein is applicable to a wide range of situations because it is possible to match communication parameters to actions that users can perform more easily and readily, and is not limited to detecting pre-specified nonverbal actions, but can be easily adapted to each user thanks to the mapping procedure.

[0023] Furthermore, an object of the present invention is to provide a system for carrying out the above-described method.

[0024] In particular, this system includes processing units designed to control the execution of the steps identified above (mapping, text content provision, and acquisition of non-verbal actions).

[0025] The system further comprises a graphical interface capable of displaying at least text content provided to the user.

[0026] There is also at least one sensor that can be associated with at least a part of the user's body to acquire the user's non-verbal actions.

[0027] The dependent claims incorporated herein by reference correspond to various embodiments of the present invention.

[0028] Further features and advantages of the present invention will become more apparent from the following description of preferred but non-exclusive embodiments of the method and system for non-verbal communication.

Mode for Carrying Out the Invention

[0029] The term non-verbal communication is intended to mean a type of communication that includes any aspect of communication exchange in which a message or information content can be conveyed without at least one of the individuals involved having to speak or make a sound.

[0030] Specifically, the method is executable by a computer, that is, the execution of the steps constituting the method as a whole is controlled and managed by one or more computer resources capable of storing and executing the instructions necessary to implement the method by controlling one or more peripheral devices.

[0031] In terms of operation, the execution of the method includes an initial mapping step in which at least one non-verbal action of the user is uniquely associated with communication parameters.

[0032] In other words, a unique association is generated between the non-verbal actions executable by the user and the respective different communication parameters.

[0033] Generally, the term "nonverbal action" is intended to refer to any action that a user can perform and that can be detected by sensors without the user having to speak or make any sounds.

[0034] Advantageously, the mapping process allows users to associate actions they can more easily perform with specific parameters through which they can communicate.

[0035] In other words, the operation of this method can be adjusted according to the user's specific requirements and abilities, and even if these requirements and abilities should change due to, for example, the development of clinical symptoms, it is sufficient to simply remap the communication parameters and quickly adapt them to the user's new state in order to ensure the accurate and efficient execution of this method.

[0036] Furthermore, the mapping procedure allows users to gradually learn how the method works and to personally adjust its behavior.

[0037] Once the mapping is performed, that is, once the correspondence between nonverbal actions and specific communication parameters is clearly and precisely defined, the user is provided with text content containing at least one selectable text element.

[0038] The user is then provided with text content, including user-selectable portions (at least one text element), for example, by displaying it on a graphical interface, to constitute a message that may be received by a communication partner, such as a member of the medical staff caring for the user, or a message that may be sent, for example, by email.

[0039] More specifically, the message structure is obtained by continuously capturing nonverbal actions performed by the user while the text content is being delivered.

[0040] The text content, specifically at least one text element, is then processed according to communication parameters uniquely associated with the acquired nonverbal action.

[0041] In other words, performing nonverbal actions allows users to interact with text content by dynamically selecting individual text elements for the composition of a message.

[0042] These text elements may include at least one of alphanumeric characters, symbols, words, phrases, or graphic elements, thereby providing a wide range of content for the user to choose from and enabling the user to communicate in a clearly expressed, accurate, and complete manner.

[0043] More specifically, the mapping process is performed by a training procedure selected from several possible, distinct procedures.

[0044] In other words, this method involves the implementation of separate and different training procedures, and the mapping step involves the execution of a specific training procedure selected from the available training procedures.

[0045] Specifically, each training procedure makes it possible to uniquely associate a distinct communication parameter with a specific state of immobility and / or movement of at least a part of the user's body.

[0046] Therefore, mapping can be performed by selecting the most appropriate and suitable training procedure from among various possibilities, based on the user's specific needs and athletic abilities / potential.

[0047] Regarding movement, the selectable training procedures include monitoring body parts that the user can move more easily (hands, fingers, feet, shoulders) and associating the movement / immobility of such specific parts with their respective communication parameters.

[0048] The movement of body parts can be captured at a level of detail corresponding to the specific training procedure being performed.

[0049] In fact, as will be explained in detail below, some training procedures aim at communication where users can express simple binary decisions of selecting or not selecting text elements, while other training procedures allow for the creation of messages with more complex mechanisms.

[0050] Therefore, nonverbal actions, specifically actions that involve moving at least a part of the body, can be measured and characterized not only as a simple function of execution / non-execution of the action itself, but also as a function of multiple parameters, including, for example, the range, direction, and velocity of the movement.

[0051] Therefore, advantageously, the training procedures can also be uniquely selected according to the user's specific communication needs and motor skills.

[0052] In fact, if communication can be resolved by simply creating affirmative or negative responses, simpler and more immediate training procedures can be implemented. Conversely, if it is necessary to provide users with a greater degree of freedom in communication, more complex training procedures can be implemented.

[0053] In this case as well, if user needs change over time, it is sufficient to simply repeat the training procedure by selecting the most appropriate option each time.

[0054] Possible training procedures are identified as “Training Procedure 1” for the purposes of this specification.

[0055] Such a training procedure is performed by associating a first operational state of at least a part of the body with a communication parameter corresponding to a main command, and associating a second operational state of at least a part of the body (preferably, but not necessarily, the same body part), which is different from the first operational state, with a communication parameter corresponding to a secondary command.

[0056] Advantageously, the first operational state can be one in which at least one body part is immobile, thus reducing the effort the user must exert for training and subsequent communication steps.

[0057] Such mapping can be adjusted by instructing the user to keep a body part still or use it in the most convenient manner while monitoring it with the appropriate sensor (described later), and by associating signals separately detected by the sensors with main and secondary commands.

[0058] In the scenario defined by such training procedures, the main command may be a text element selection command, the secondary command may be a text element deselection command, and the provision of text content is performed by displaying a series of individual text elements.

[0059] For each element, the user's nonverbal actions (i.e., the movement or non-movement of a body part uniquely associated with the main or secondary command during training) are captured at predetermined time intervals, and a decision is made accordingly whether to select the corresponding text element.

[0060] Therefore, this method involves processing text elements to constitute a text message by arranging and arranging text elements that are progressively selected by the user in an orderly manner.

[0061] In other words, a series of text elements are displayed to the user, each of which can be selected by the user by a first operational state corresponding to a main selection command, or discarded by the user by a second operational state corresponding to a secondary deselection command, until the creation of the message is complete.

[0062] Regarding its operation, the method is then performed by generating a series of text elements and measuring an input, which in turn selects the element of interest.

[0063] In this regard, the text element may preferably include characters that are displayed sequentially to the user until a word or phrase is formed, and which can be individually selected or discarded by the user.

[0064] The provision of text elements can be performed in a precise and predefined order, or it can be managed dynamically according to previously selected text elements.

[0065] In other words, the order in which text elements are provided is also defined and / or modified according to the text elements that are progressively arranged to form a text message.

[0066] Therefore, when a user selects a specific string, that string is taken into consideration when determining the next character to be displayed and subsequently selected.

[0067] Advantageously, the aforementioned training modes can also be used to offer various communication modes / methods.

[0068] In other words, the training procedures described above make it possible to generally define a binary selection mode for text element selection that can be used in any communication situation where a patient's message can be constructed, defined, or determined as a function in relation to a binary interaction with text elements.

[0069] Such binary processes can, conveniently, be associated not only with the concepts of choice / non-choice, but also with the concepts of affirmative / negative responses.

[0070] In other words, the first training procedure can be performed by associating the main command with a communication parameter corresponding to a negative / positive response, and the secondary command with a communication parameter corresponding to a positive / negative response.

[0071] Therefore, the first training procedure may be performed, either alternatively or additionally, by associating the main command with a positive response.

[0072] At the same time, secondary commands are instead associated with negative responses.

[0073] In the situation defined above, the provision of text content can be accomplished by providing a series of queries, particularly a series of select-response queries, i.e., a series of queries that can be responded to with a binary yes or no answer.

[0074] The user's nonverbal actions are then captured for each query, and the response to this query is determined as either affirmative or negative depending on the specific action performed.

[0075] Preferably, a series of queries define a decision tree, and the order in which the queries are presented is determined and / or modified according to the inputs received.

[0076] In other words, this method is implemented by providing an interconnected tree or network of queries in which the order in which queries are presented (and therefore the content of the queries provided to the user) is determined according to the responses given by the user through nonverbal communication.

[0077] This procedure is particularly efficient because it allows for the rapid definition of user needs or the message the user wants to convey, by starting with general questions and gradually narrowing them down until the message the user wants to communicate or the need the user wants to express is identified.

[0078] Conveniently, once the end of the series of queries is reached, i.e., once the message to be conveyed is identified, the method may include a verification step (a generally applicable but particularly useful embodiment in this situation) in which the user is asked whether the conclusions drawn from the decision tree are correct.

[0079] Further training procedures are identified as “Second Training Procedure” for the purposes of this specification.

[0080] Such training procedures are carried out by displaying a series of markers at predefined locations on a graphical interface.

[0081] A pointer is displayed along with a series of markers, and the operating state of at least one body part is associated with a communication parameter corresponding to the pointer's movement.

[0082] In other words, the movement of the pointer is coordinated by uniquely associating it with the movement of at least one body part.

[0083] Specifically, such mapping can be adjusted by instructing the user to move a body part while monitoring that body part with a sensor, and then associating the detected movement with the movement of a pointer.

[0084] In this regard, the acquisition of nonverbal actions is not limited to the identification of movement / immobility of at least one body part, but rather to the detailed identification of a specific movement by, for example, identifying one or more of the above parameters, paying particular attention to the direction and range of the movement.

[0085] In the context defined by such training procedures, the provision of text content can, advantageously, be carried out by displaying multiple text elements on a graphical interface.

[0086] Preferably, multiple graphic elements containing alphanumeric characters are displayed, arranged to form a keyboard.

[0087] Advantageously, one or more words and / or phrases can also be displayed, preferably next to the keyboard, for example, above it.

[0088] These words and / or phrases can also be displayed and suggested to the user according to alphanumeric characters and / or words previously selected by the user, or according to a predefined list.

[0089] In terms of operation, a text element is selected when a pointer remains on that text element for a predetermined period of time.

[0090] In other words, non-verbal actions are captured when the pointer is moved, and a particular text element is selected when the pointer persists over that particular text element.

[0091] Therefore, this method involves processing text elements and arranging the text elements progressively selected by the user in an orderly manner to construct a text message.

[0092] Therefore, this method is implemented by allowing the cursor to be moved to progressively identify and select text elements that are assembled to constitute the user's message.

[0093] In general, the above training procedures can be used for other purposes as well.

[0094] In fact, such a procedure allows a user to associate specific actions they can perform with corresponding pointer movements, thereby enabling the user to control the operation of any system that can be manipulated by such pointer movements, not necessarily solely for the purpose of constructing messages, but for example, to control one or more functions of the computer on which the method described herein is being performed.

[0095] Advantageously, the method may also include an initiation step that acquires a set of user-related information elements that can be used to control the selection of a particular training procedure and therefore the communication mode, which is also more efficient when considering the user's clinical picture.

[0096] In particular, the startup step acquires at least one data element relating to the user, preferably including at least one of one or more sensors configured or configurable to acquire age, gender, language, health status, position of the graphical interface relative to the user, and / or nonverbal actions.

[0097] In other words, information about the user's health status and information about the system performing this method, paying particular attention to the system's position relative to the user, are obtained.

[0098] Advantageously, the present invention achieves its proposed objectives by overcoming the shortcomings reported in the prior art by providing users with a method for nonverbal communication that can be easily and efficiently adapted to their specific needs.

[0099] The present invention also relates to a nonverbal communication system specifically configured to perform a method having one or more of the above-described technical features.

[0100] Structurally, such a system comprises a computer configured to perform one or more steps of a nonverbal communication method, or in any case, processing and arithmetic units that define or contribute to defining computer resources.

[0101] Generally, processing and arithmetic units are configured to control, instruct, and manage the operation of additional components of the system in order to operate them in accordance with this method.

[0102] The system also includes a graphical interface configured to display at least text content and connected to processing and arithmetic units.

[0103] Advantageously, graphical interfaces can also display additional information and graphical content.

[0104] More specifically, the graphical interface is defined by a monitor or mobile device equipped with a screen, such as a touchscreen, e.g., a smartphone, tablet, or computer.

[0105] The system also includes at least one sensor that can be associated with at least a part of the user's body in order to acquire the user's nonverbal actions.

[0106] As a non-limiting example, such a sensor may be an accelerometer, a gyroscope, or a magnetic sensor, or may include an accelerometer, a gyroscope, or a magnetic sensor, or the system may include multiple sensors of the same or different types.

[0107] Advantageously, this system can interface with multiple peripheral devices, enabling further optimization of user communication capabilities.

[0108] For example, the system may include audible and / or optical indicators, or may be connected to a processing and arithmetic units, and the audible and / or optical indicators may be activated by a user using, for example, a selection procedure equivalent to that outlined above for selecting text elements.

[0109] In other words, a text element may contain one or more text elements, and the selection of each will enable a different indicator.

[0110] In this way, even users who cannot communicate verbally can activate an alarm to alert others (such as healthcare workers) of an emergency.

[0111] This system can also interface with remote terminals / databases (e.g., computing clouds), enabling users to remotely send configured messages and establish remote communication.

[0112] This aspect also allows for the collection and storage of data / information that can be analyzed later.

[0113] This system can also interface with home automation devices, and the text element may consist of one or more text elements, the selection of which causes the operation of one or more of these devices, or in any case, control of the operation of one or more devices.

Claims

1. A computer-executable method for nonverbal communication, - A step of mapping at least one nonverbal action of the user to a corresponding communication parameter by uniquely associating it with the parameter, - A step of supplying text content, which includes at least one selectable text element, to the user via a graphical interface, - The process includes the steps of continuously obtaining the user's nonverbal actions while the text content is being supplied, and processing at least one text element according to the communication parameters uniquely associated with the obtained nonverbal actions, The mapping step is performed using a training procedure selected from a plurality of training procedures, wherein the immobility and / or movement state of at least one body part of the user is associated with the respective communication parameter.

2. The method according to claim 1, wherein the text element includes at least one of alphanumeric characters, symbols, words, phrases, and graphic elements.

3. The aforementioned multiple training procedures include a first training procedure, and the first training procedure is - A step of associating a first operational state of at least one body part with a communication parameter corresponding to a main command, The method according to claim 1 or 2, further comprising the step of associating a second operating state of the at least one body part, which is different from the first operating state, with a communication parameter corresponding to a secondary command.

4. The method according to claim 3, wherein the first operating state is an immobile state of at least one body part.

5. The main command corresponds to the command to select the text element, and the secondary command corresponds to the command to deselect the text element. - The provision of the text content is carried out by providing a series of individual text elements. The method according to claim 3, wherein the step of processing the at least one text element is performed by constructing a text message by arranging and ordering the text elements progressively selected by the user.

6. The method according to claim 5, wherein the order in which the text elements are provided is also determined and / or modified according to the text elements arranged side by side to form the text message.

7. The method according to claim 3, wherein the main command corresponds to an affirmative response, the secondary command corresponds to a negative response, and the text content supply is performed by supplying a series of queries.

8. The method according to claim 7, wherein the series of queries define a decision tree, and the order in which the series of queries are provided is defined and / or modified according to the input obtained.

9. The aforementioned multiple training procedures include a second training procedure, and the aforementioned second training procedure is - The steps of displaying a series of markers at predetermined positions on the graphical interface, - The step of displaying a pointer on the graphical interface, The method according to claim 1 or 2, further comprising the step of associating the operating state of the at least one body part with a communication parameter corresponding to the movement of a pointer.

10. The method according to claim 9, wherein the provision of the text content is performed by displaying a plurality of text elements on the graphical interface, the text element is selected by the pointer being on the text element for a predetermined period of time, and the step of processing the at least one text element is performed by arranging the progressively selected text elements in an orderly manner to form a text message.

11. The method according to claim 1 or 2, comprising a startup step in which at least one data element relating to the user is obtained.

12. The method according to claim 11, wherein the at least one data element includes at least one of one or more sensors configured or configurable to acquire age, gender, language, the user's health status, the location of the graphical interface relative to the user, and / or nonverbal actions.

13. It is a system for nonverbal communication, - A processing and arithmetic unit configured to perform the nonverbal communication method described in claim 1 or 2, - A graphical interface configured to display at least the aforementioned text content, - A system comprising at least one sensor that can be associated with at least a part of the user's body in order to acquire the user's nonverbal actions.