Systems and methods for non-verbal communication - Patents.com

Abstract

A computer-executable method for non-verbal communication is performed by mapping at least one non-verbal action of a user by uniquely associating it with a corresponding communication parameter, then providing textual content including at least one selectable text element to the user via a graphical interface, while continuously acquiring the user's non-verbal actions, and processing the at least one text element according to the communication parameter uniquely associated with the acquired non-verbal action.

Description

[Technical field]

[0001] The present invention relates to the technical field of communication systems and methods.

[0002] In particular, the present invention relates to a system and method for non-verbal communication that is particularly advantageous in the medical / care field for patients who are unable to communicate their thoughts verbally, such as intubated or tracheotomized patients. [Background technology]

[0003] The ability to communicate effectively with other people is a fundamental need for all individuals.

[0004] However, obstacles can arise in a wide range of situations that make such communication processes difficult, if not impossible, and thus cause considerable inconvenience.

[0005] By way of example, particular reference is made to the medical field, where individuals who are hospitalized or undergoing certain medical procedures and / or treatments may find it impossible to verbally communicate their thoughts to those around them, in particular the medical staff.

[0006] Think of a patient in an intensive care unit who is intubated or tracheotomized and therefore cannot speak.

[0007] In this context, the need for methods and systems that allow patients to not only express their basic needs, but more importantly, to inform medical staff that a specific problem may occur that cannot be adequately understood / interpreted by the activation of a simple alarm, becomes even more evident.

[0008] To this end, there are devices that receive non-verbal input and convert it into messages through which the patient can communicate with other people.

[0009] However, known devices still suffer from drawbacks and inefficiencies that make them unsatisfactory to implement and less user friendly.

[0010] In particular, because known devices are configured to uniquely identify a single specific non-verbal action, which is uniquely associated with a single type of message, they are highly inflexible in their operation and therefore unable to adapt to the specific needs of an individual patient.

[0011] Thus, if a patient is unable or has difficulty performing a specific and unique non-verbal action identifiable by the device, the device is effectively useless and cannot be used. Summary of the Invention

[0012] In this context, the technical problem underlying the present invention is to propose a system and associated method for non-verbal communication that overcomes at least some of the above-mentioned drawbacks of the prior art.

[0013] In particular, it is an object of the present invention to provide a system and method that can enable persons who are unable or have difficulty in expressing themselves verbally, in particular intubated or tracheotomized patients, to communicate non-verbally with a second person in a particularly versatile manner and at the same time accurately and efficiently.

[0014] The identified technical problems and objects are substantially achieved by a system and method comprising the technical features recited in one or more of the appended claims.

[0015] In accordance with the present invention, presented herein is a method, particularly a computer-implemented method, for non-verbal communication.

[0016] The method includes mapping at least one non-verbal action of a user by uniquely associating it with a corresponding communication parameter.

[0017] Once the mapping step is complete, an action and each parameter are uniquely associated and text content is provided to the user.

[0018] In particular, the textual content is presented through a graphical interface.

[0019] Additionally, the textual content includes at least one selectable text element.

[0020] Such text elements may include, for example, one or more alphanumeric characters, symbols, words, phrases, and graphic elements.

[0021] During the presentation of the textual content, the user's non-verbal actions are continually captured, and the textual content is processed according to communication parameters uniquely associated with the incrementally captured non-verbal actions.

[0022] Advantageously, the method proposed herein is applicable to a wide range of situations since it is possible to match communication parameters to actions that the user can more easily and readily perform, and therefore can be easily adapted to each user thanks to a mapping procedure, rather than being tied to the detection of pre-specified non-verbal actions.

[0023] It is also an object of the present invention to provide a system for carrying out the above-mentioned method.

[0024] In particular, the system comprises a processing unit designed to control the execution of the steps specified above (mapping, providing textual content, obtaining non-verbal actions).

[0025] The system further comprises a graphical interface capable of displaying at least the textual content to be provided to the user.

[0026] There is also at least one sensor that can be associated with at least a portion of the user's body to capture non-verbal actions of the user.

[0027] The dependent claims, which are incorporated herein by reference, correspond to different embodiments of the invention.

[0028] Further features and advantages of the present invention will become more apparent from the following description of preferred, but non-exclusive, embodiments of a method and system for non-verbal communication. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The term nonverbal communication is intended to mean a type of communication that includes all aspects of a communicative exchange in which a message or information content can be conveyed without the need for at least one of the individuals involved to speak or utter a sound.

[0030] In particular, the method is computer executable, i.e. the execution of the steps which together make up the method is controlled and managed by one or more computer resources capable of storing and executing the instructions necessary to perform the method by controlling one or more peripheral devices.

[0031] In operation, execution of the method includes an initial mapping step in which at least one non-verbal action of a user is uniquely associated with a communication parameter.

[0032] In other words, a unique association is created between each non-verbal action that can be performed by the user and each different communication parameter.

[0033] In general, the term "non-verbal action" is intended to refer to any action that can be performed by a user and that can be detected by a sensor without the user having to speak or make a sound.

[0034] Advantageously, the mapping operation allows actions that a user can more easily perform to be associated with specific parameters through which the user can communicate.

[0035] In other words, the operation of the method can be adjusted according to the specific requirements and capabilities of the user, and even if these requirements and capabilities should change, e.g. due to the evolution of a clinical condition, it is sufficient to remap the communication parameters and quickly adapt them to the user's new condition to ensure a correct and efficient execution of the method.

[0036] Moreover, the mapping procedure allows the user to gradually learn how the method works and to personally tailor its operation.

[0037] Once the mapping has been performed, i.e. the correspondence between the non-verbal actions and the particular communication parameters has been clearly and precisely defined, the textual content, including at least one selectable text element, is provided to the user.

[0038] The user is then provided with text content comprising user-selectable parts (at least one text element), for example by displaying it on a graphical interface, in order to compose a message that can be received by a communication partner, for example a member of the medical staff caring for the user, or a message to be sent, for example by email.

[0039] More specifically, the composition of the message is obtained by continuously capturing non-verbal actions performed by the user while the text content is being provided.

[0040] The textual content, specifically the at least one textual element, is then processed according to communication parameters uniquely associated with the captured non-verbal action.

[0041] In other words, performance of non-verbal actions allows a user to interact with the textual content by operatively selecting individual textual elements for composition of a message.

[0042] These text elements may include alphanumeric characters, symbols, words, phrases, and / or graphic elements, thereby providing a wide range of content for users to choose from and enabling users to communicate in a clearly expressed, accurate and complete manner.

[0043] More specifically, the mapping task is performed by a training procedure selected from a number of possible distinct procedures.

[0044] In other words, the method involves the implementation of separate and distinct training procedures, and the mapping step involves the execution of a particular training procedure selected from the available training procedures.

[0045] In particular, each training procedure allows a separate and specific communication parameter to be uniquely associated with a particular state of immobility and / or movement of at least a part of the user's body.

[0046] Thus, the mapping may be performed by selecting, from among various possibilities, the training procedure that is most relevant and appropriate to the user's specific needs and athletic capabilities / potential.

[0047] With respect to movement, selectable training procedures include monitoring which body parts the user can move more easily (hands, fingers, feet, shoulders) and associating the movement / immobility of such particular parts with respective communication parameters.

[0048] Body part movements may be captured at a level of detail that depends on the particular training routine being performed.

[0049] Indeed, as we will explain in more detail below, some training procedures are aimed at communications in which users can express simple binary choices of selecting / not selecting a text element, while other training procedures offer the possibility of creating messages with more complex mechanisms.

[0050] Thus, non-verbal actions, specifically actions that move at least a part of the body, can be measured and characterized not only as a function of the simple execution / non-execution of the action itself, but also as a function of multiple parameters including, by way of example, the range, direction and speed of the movement.

[0051] Thus, advantageously, the training routine may also be uniquely selected according to the user's particular communication needs and the user's athletic ability.

[0052] Indeed, simpler and more immediate training procedures can be implemented where communication can be resolved by simply making a positive or negative response, or more complex training procedures can be implemented where greater freedom in communication needs to be provided to the user.

[0053] Again, if the user's needs change over time, it is sufficient to repeat the training procedure by selecting each time the one that best suits the situation.

[0054] The possible training procedure is identified for purposes of this specification as the "first training procedure."

[0055] Such a training procedure is performed by associating a first motion state of at least a part of the body with communication parameters corresponding to a main command, and associating a second motion state of at least a part of the body (preferably, but not necessarily, the same body part), different from the first motion state, with communication parameters corresponding to a secondary command.

[0056] Advantageously, the first motion state may be an immobility state of at least one body part, thus reducing the effort the user has to exert for the training and subsequent communication steps.

[0057] Such mapping can be refined by instructing the user to not move the body part or to use the body part in the most convenient manner while the body part is being monitored with appropriate sensors (described below) and by associating signals detected separately by the sensors with primary and secondary commands.

[0058] In a situation defined by such a training procedure, the main command may be a text element selection command, the secondary command may be a text element deselection command, and the presentation of the text content is performed by displaying a series of individual text elements.

[0059] For each element, the user's non-verbal action (i.e., movement or non-movement of a body part that is uniquely associated during training with a main or secondary command) is obtained at predetermined time intervals, and a decision is made accordingly as to whether to select the corresponding text element.

[0060] The method thus involves processing the text elements to compose a text message by arranging the text elements progressively selected by the user side by side.

[0061] In other words, a series of text elements are displayed to the user, each of which can be selected by the user by a first operating state corresponding to a main selection command, or discarded by the user by a second operating state corresponding to a secondary non-selection command, until composition of the message is completed.

[0062] In operation, the method is then carried out by generating a sequence of text elements and measuring the input according to which an element of interest is selected.

[0063] In this regard, the text elements preferably include letters which are sequentially displayed to the user and individually selected or discarded by the user until a word or phrase is formed.

[0064] The presentation of the text elements can be performed in a precise and predefined order, or can be dynamically managed according to previously selected text elements.

[0065] In other words, the order of presentation of the text elements is also defined and / or changed according to the text elements being progressively sequenced to form the text message.

[0066] Thus, when a user selects a particular character string, such character string is taken into consideration to determine the next character to be displayed and then subjected to the selection process.

[0067] Advantageously, the training modes described above can also be used to provide various communication modes / methods.

[0068] In other words, the training procedure described thus far makes it possible to define in general terms a binary choice mode of selection of text elements that can be used in any communication situation in which the patient's message can be constructed, defined, or determined as a function of dyadic interaction with text elements.

[0069] Such a binary process can be advantageously linked not only to the notion of choice / non-choice, but also to the notion of a positive / negative response.

[0070] In other words, the first training procedure can be performed by associating a main command with a communication parameter corresponding to a negative / positive response and a secondary command with a communication parameter corresponding to a positive / negative response, respectively.

[0071] Thus, the first training procedure may alternatively or additionally be performed by associating a main command with a positive response.

[0072] At the same time, the secondary command is instead associated with a negative response.

[0073] In the context just defined, the provision of textual content can be performed by providing a series of queries, in particular a series of choice-answer type queries, i.e. a series of queries that can be answered with a binary yes or no response.

[0074] The user's non-verbal actions are then captured for each query, and the response to this query is determined as positive or negative, depending on the particular action performed.

[0075] Preferably, the sequence of queries defines a decision tree, and the order in which the queries are presented is defined and / or varied according to the obtained input.

[0076] In other words, the method is performed by providing a tree or network of interconnected queries in which the presentation order of the queries (and therefore the content of the queries provided to the user) is determined according to responses given by the user through non-verbal communication.

[0077] Such a procedure is particularly efficient in that it allows the user's needs or the message he or she wishes to convey to be rapidly defined by starting with general questions and gradually narrowing and specifying the questions until the message he or she wishes to convey or the need he or she wishes to express is identified.

[0078] Advantageously, once the end of the series of queries has been reached, i.e. once the message to be conveyed has been identified, the method may include (a generally applicable aspect but which is particularly useful in this context) a verification step in which the user is asked whether the conclusion reached from the decision tree is correct or not.

[0079] The further training procedure is identified for purposes of this specification as a "second training procedure."

[0080] Such a training procedure is performed by displaying a series of markers at predefined locations on a graphical interface.

[0081] A pointer is also displayed along with the set of markers, and a motion state of at least one body part is associated with a communication parameter corresponding to movement of the pointer.

[0082] In other words, the movement of the pointer is coordinated by uniquely associating it with the motion of at least one body part.

[0083] In particular, such mapping can be refined by instructing the user to move a body part while the body part is being monitored by a sensor, and correlating the detected movements with pointer movements.

[0084] In this regard, the acquisition of non-verbal actions is not limited to identifying the movement / immobility of at least one body part, but also specifically identifies specific movements, with particular attention to the direction and range of the movement, e.g. by identifying one or more of the parameters mentioned above.

[0085] In a situation defined by such a training procedure, the provision of textual content may advantageously be performed by displaying a number of textual elements on a graphical interface.

[0086] Preferably, a plurality of graphical elements are displayed, including alphanumeric characters arranged to form a keyboard.

[0087] Advantageously, one or more words and / or phrases may also be displayed, preferably adjacent to, for example above, the keyboard.

[0088] These words and / or phrases may also be displayed and thus suggested to the user according to the user's own previously selected alphanumeric characters and / or words, or according to a predefined list.

[0089] In operation, a text element is selected by having the pointer over the text element for a predetermined period of time.

[0090] In other words, non-verbal actions are captured when moving the pointer and holding the pointer over a particular text element causes that particular text element to be selected.

[0091] The method thus involves processing the text elements and composing a text message by arranging the text elements progressively selected by the user side by side.

[0092] Thus, the method is carried out by enabling a cursor to be moved to progressively identify and select text elements that are assembled to compose the user's message.

[0093] In general, the above training procedures can be used for other purposes as well.

[0094] Indeed, such a procedure allows a user to associate specific actions that the user can perform with corresponding pointer movements, thereby enabling the user to control the operation of any system that can be manipulated by such pointer movements, not necessarily for the purpose of composing messages, but also, for example, to control one or more functions of a computer on which the methods described herein are being performed.

[0095] Advantageously, the method may also comprise a start-up step of obtaining a set of user-related information elements that can be used to control the selection of a particular training procedure and thus of a communication mode, which is also more efficient when taking into account the clinical profile of the user.

[0096] In particular, the activation step acquires at least one data element regarding the user, preferably such data element includes at least one of age, gender, language, health condition, position of the graphical interface relative to the user, and / or one or more sensors configured or configurable to acquire non-verbal actions.

[0097] In other words, information is obtained about the health status of the user and about the system performing the method, paying particular attention to the position of the system relative to the user.

[0098] Advantageously, the present invention overcomes the shortcomings reported in the prior art and achieves the proposed objectives by providing the user with a method for non-verbal communication that can be easily and efficiently adapted to the user's specific needs.

[0099] The present invention also relates to a non-verbal communication system specifically adapted to carry out a method having one or more of the technical features described above.

[0100] Structurally, such a system comprises a computer configured to carry out one or more steps of the non-verbal communication method, or in any case processing and computing units defining or contributing to the definition of computer resources.

[0101] In general, the processing and computing units are configured to control, command and manage the operation of the additional components of the system to cause them to operate in accordance with the method.

[0102] The system also includes a graphical interface configured to display at least the textual content and coupled to the processing and computing unit.

[0103] Advantageously, the graphical interface can also display further information and graphical content.

[0104] In particular, the graphical interface is defined by a monitor or a mobile terminal, such as a smartphone, tablet, computer, equipped with a screen, such as a touch screen.

[0105] The system also includes at least one sensor capable of being associated with at least a portion of the user's body to capture non-verbal actions of the user.

[0106] By way of non-limiting example, such a sensor may be or may include an acceleration sensor, a gyro sensor, or a magnetic sensor, or the system may include multiple sensors of the same or different types.

[0107] Advantageously, the system can interface with multiple peripherals making it possible to further optimize the user's communication possibilities.

[0108] For example, the system may be equipped with or connected via a processing and computing unit to audible and / or optical indicators, which can be activated by a user, for example using a selection procedure equivalent to that outlined above for selecting a text element.

[0109] In other words, the text element may contain one or more text elements, the selection of which activates a respective indicator.

[0110] In this way, users who cannot communicate verbally can still activate an alarm to alert others (such as medical personnel) to an emergency.

[0111] The system can also interface with remote terminals / databases (eg, computing clouds), allowing users to send configured messages remotely and establish remote communications.

[0112] This aspect also allows for the collection and storage of data / information that can be analyzed at a later time.

[0113] The system may also interface with home automation devices and the text elements may comprise one or more text elements the selection of which causes the actuation of one or more of these devices, or in either case the control of the operation of one or more of the devices.

Claims

1. 1. A computer-implemented method for non-verbal communication, comprising: - mapping at least one non-verbal action of a user by uniquely associating it with a corresponding communication parameter; - providing said user with textual content, including at least one selectable text element, via a graphical interface; - continuously acquiring non-verbal actions of the user during the provision of the textual content, and processing at least one text element according to the communication parameters uniquely associated with the acquired non-verbal actions.

2. The method of claim 1 , wherein the text elements include at least one of alphanumeric characters, symbols, words, phrases, and graphic elements.

3. 3. The method according to claim 1 or 2, wherein the mapping step is performed using a training procedure selected from a plurality of training procedures, and wherein a state of immobility and / or movement of at least one body part of the user is associated with a respective communication parameter.

4. The plurality of training procedures includes a first training procedure, the first training procedure comprising: - associating a first motion state of at least one body part with a communication parameter corresponding to a main command; - associating a second motion state of the at least one body part, different from the first motion state, with a communication parameter corresponding to a secondary command.

5. The method of claim 4 , wherein the first motion state is an immobility state of the at least one body part.

6. the main command corresponds to a selection command for the text element and the secondary command corresponds to a deselection command for the text element; - the provision of said textual content is performed by providing a series of individual text elements, The method according to claim 4, wherein said step of processing said at least one text element is performed by composing a text message by arranging and ordering said text elements progressively selected by said user.

7. The method of claim 6 , wherein the order of presentation of the text elements is also defined and / or varied according to the text elements arranged in sequence to form the text message.

8. The method of claim 4 , wherein the main command corresponds to an affirmative response and the secondary command corresponds to a negative response, and the providing of the textual content is performed by providing a series of queries.

9. The method of claim 8 , wherein the sequence of queries defines a decision tree, and the order in which the sequence of queries is provided is defined and / or varied according to the obtained input.

10. The plurality of training procedures includes a second training procedure, the second training procedure comprising: - displaying a series of markers at predetermined positions on said graphical interface; - displaying a pointer on said graphical interface; - associating a motion state of the at least one body part with a communication parameter corresponding to a pointer movement.

11. 11. The method of claim 10, wherein providing the text content is performed by displaying a plurality of text elements on the graphical interface, a text element being selected by the presence of the pointer on a text element for a predetermined period of time, and wherein processing the at least one text element is performed by composing a text message by arranging the text elements progressively selected by the user in an orderly fashion.

12. 3. The method according to claim 1 or 2, comprising a start-up step in which at least one data element relating to the user is acquired, preferably comprising at least one of the following: age, gender, language, health condition of the user, position of the graphical interface relative to the user, and / or one or more sensors configured or configurable to acquire non-verbal actions.

13. 1. A system for non-verbal communication, comprising: - a processing and computing unit adapted to carry out the non-verbal communication method according to claim 1 or 2; a graphical interface adapted to display at least said textual content; at least one sensor that can be associated with said at least a part of the user's body to obtain non-verbal actions of said user.