Monitoring system for monitoring a patient and procedures for operating the monitoring system

Abstract

Monitoring system (15) for monitoring a patient, in particular during a medical diagnostic or therapeutic procedure, comprising • a voice cloning device (10) with a voice generator (11) which is designed to replace a person's natural voice with a cloned foreign voice different from the person's voice, wherein at least two foreign voices are selectable, • a measuring unit (13) which is designed to record at least one physiological parameter of the patient, • an evaluation unit (14) which is designed to evaluate at least one measured physiological parameter of the patient, and • a control unit (12) which is configured to control the voice cloning device (10) in such a way that a foreign voice is selected or rejected depending on the result of the evaluation by the evaluation unit, in particular for the further operation of the voice cloning device (10).

Description

[0001] The invention relates to a monitoring system for monitoring a patient, in particular during a medical diagnostic or therapeutic procedure, according to claim 1, and to a method according to claim 9.

[0002] When treating patients in doctors' offices, hospitals, or other medical institutions, communication with patients is a crucial factor, alongside optimal medical care. Communication with elderly, confused, or demented patients, as well as with children, is particularly challenging. A breakdown in communication can be medically problematic, for example, in situations such as medical examinations, interventions, or surgeries without general anesthesia. Often, a trusted person or caregiver is present during preparatory consultations before examinations or procedures to facilitate communication. However, this is no longer the case during the examinations, interventions, or surgeries themselves. Patients frequently have to communicate directly with the doctor, which can be especially problematic for the aforementioned patient groups.

[0003] German patent DE 10 2017 220 500 A1 discloses a medical system for supporting medical procedures with a human-machine interface for communication with a patient, where the interface uses, for example, an artificially created avatar. German patent DE 10 2016 219 157 A1 discloses an assistance device for communication with patients in care facilities, featuring a face displayed on a screen, which records the patient's vital parameters. German patent DE 196 37 383 A1 discloses a wearable data acquisition and evaluation device for monitoring the condition of a person following their daily routine.

[0004] The object of the present invention is to provide a device which enables improved communication with patients, in particular young, old, confused or demented patients, even during medical procedures; furthermore, it is an object of the invention to provide a method for operating the device.

[0005] The object is achieved according to the invention by a monitoring system for monitoring a patient, in particular during a medical diagnostic or therapeutic procedure, according to claim 1 and by a method for operating the monitoring system according to claim 9. Advantageous embodiments of the invention are the subject of the respective dependent claims.

[0006] The invention utilizes voice cloning devices trained on individual speakers in a surprising way. Such devices are generally known, for example from the online article https: / / www.vice.com / de / article / 3k7mgn / diese-kibraucht-nur-eine-37-sekunden-lange-aufnahme-dann-klont-sieeure-stimme or from the article by Arik, Chen et al., “Neural Voice Cloning with a Few Samples”, NIPS'18 Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 10040-10050, Montréal, Canada, 2018.

[0007] The monitoring system according to the invention for monitoring a patient, in particular during a medical diagnostic or therapeutic procedure, comprises a voice cloning device with a voice generator which is designed to reproduce the natural voice of a person (in particular medical personnel, such as...)to replace the voice of a physician) with a cloned foreign voice different from the person's own voice, wherein at least two foreign voices are selectable, a measuring unit which is trained to record at least one physiological parameter of the patient, an evaluation unit which is trained to evaluate the at least one measured physiological parameter of the patient, and a control unit which is trained to control the voice cloning device in such a way that a foreign voice is selected or discarded depending on the result of the evaluation by the evaluation unit (for further operation of the voice cloning device).The inventive method for operating a monitoring system comprises the following steps: activating the voice cloning device, wherein a first foreign voice is selected, recording at least one physiological parameter of the patient during operation of the voice cloning device with the first foreign voice, evaluating the at least one physiological parameter of the patient, and automatically controlling the voice cloning device depending on the result of the evaluation. In particular, the first foreign voice is selected or rejected for further operation of the voice cloning device depending on the result of the evaluation by the evaluation unit.

[0008] When such a device or procedure is used in a medical setting, particularly during examinations, procedures, and operations, with patients, especially very young, elderly, confused, or demented patients, it can significantly improve communication. For example, medical personnel, such as doctors performing a procedure, can speak to the patient using a familiar voice, such as that of a relative or acquaintance, thereby achieving better cooperation and / or having a calming effect on the patient. This can also prevent risks that might arise from an anxious patient. The device ensures a calming effect by measuring and evaluating the patient's physiological parameters and selecting or rejecting the appropriate voice accordingly.

[0009] According to one embodiment of the invention, the cloned voices were generated using a pre-trained machine learning algorithm. AI-based voice cloning algorithms are known, such as the "Deep Voice" algorithm presented by Baidu. These can clone human voices very realistically. According to another embodiment of the invention, the algorithm is pre-trained using voice samples from people known to the patient, particularly those close to them. These people known to the patient could be, for example, relatives (parents, children, siblings, spouse, etc.) or acquaintances (caregivers, friends) whom the patient trusts. The pre-training is carried out using the voice of the person to be "learned," for example, live or using voice samples, in order to learn the characteristics of the voice.After the learning phase, the voice generator of the voice cloning device is able to reproduce sentences spoken by any person (doctor, etc.) in the (foreign) voice of the person the patient knows. For example, for "Deep Voice," only a 3.7-second audio clip is needed for training, although longer or more examples naturally yield better results. If sufficient audio material exists, it is also irrelevant whether the person to be voice-cloned is still alive. The age, gender, or accent of the speaker are also immaterial. Thus, known voice-cloning algorithms can replace a female voice with a male voice or a British accent with an American one.

[0010] According to a further embodiment of the invention, the evaluation unit is configured to compare at least one measured physiological parameter of the patient with at least one threshold value. This is carried out as part of the evaluation to easily obtain a clear result on the basis of which the voice can be selected or rejected. In this context, the control unit is configured to control the voice cloning device in such a way that a voice is selected which does not cause the threshold value to be exceeded, or a voice is rejected which causes the threshold value to be exceeded. Thus, for example, if a physiological parameter of the patient rises significantly above a threshold value during communication with a particular voice, the voice is rejected, as it obviously does not have a calming effect on the patient.Conversely, the external voice continues to be used if the parameter remains consistently below the threshold, meaning the patient appears calm. For some parameters, the voice cloning device can also be controlled in such a way that an external voice is selected that causes the threshold to be exceeded, or an external voice is discarded that causes the threshold to be fallen below.

[0011] According to a further embodiment of the invention, the measuring unit is configured to measure a physiological parameter that can be used as an indicator of the patient's emotional state, and the evaluation unit is configured to evaluate the physiological parameter with regard to the patient's emotional state. The patient's emotional state is understood to mean the patient's overall acute mental, psychological, and emotional situation. The physiological parameters used can be derived, for example, from one of the following measurements: blood pressure, pulse, EEG. Other parameters or a multitude of different parameters can also be used. The measuring unit can include sensors, cameras, or other measuring devices.

[0012] According to a further embodiment of the invention, the control unit is configured to control the voice cloning device in such a way that a foreign voice is selected which induces a positive emotional state, or a foreign voice which induces a negative emotional state is rejected. A positive emotional state is understood here to be a state in which the patient is calm and / or cooperative and / or feels comfortable. A negative emotional state is understood to be a state in which the patient is, for example, stressed, restless, or anxious.

[0013] According to further embodiments of the invention, the evaluation of at least one physiological parameter of the patient includes a comparison with at least one threshold value and / or an evaluation with regard to the patient's emotional state.

[0014] Advantageously, the control process is such that the first external voice is selected if the evaluation of at least one physiological parameter shows that a threshold value for the parameter is not exceeded, and that the first external voice is discarded if the evaluation of at least one physiological parameter shows that a threshold value for the parameter is exceeded. Furthermore, the control process is advantageously such that the first external voice is selected if it induces a positive emotional state in the patient, and that the first external voice is discarded if it induces a negative emotional state.

[0015] In a further embodiment of the invention, at least one physiological parameter is measured and evaluated for at least one second voice (in addition to the first). Depending on the evaluation of the physiological parameter, one voice is selected from the first and at least two voices for further operation of the voice cloning device. Thus, at least two voices are tested with regard to their (e.g., calming) effect on the patient. Advantageously, the voice that produces a more positive mood in the patient or contributes most effectively to calming the patient is then selected.

[0016] The invention and further advantageous embodiments according to the features of the dependent claims are explained in more detail below with reference to schematically illustrated exemplary embodiments in the drawing, without thereby limiting the invention to these exemplary embodiments. The drawing shows: Fig. 1. A sequence of the basic steps of the method according to the invention; Fig. 2 a view of a device according to the invention; and Fig. 3. A further development of the procedure with additional steps.

[0017] The invention aims to improve communication, especially with elderly, confused or demented patients, as well as with children, particularly in situations such as medical examinations, interventions or operations without general anesthesia, using voice cloning systems. These systems allow doctors to communicate with patients using the voices of caregivers (foreign voices) in order to achieve better cooperation and / or to have a calming effect.

[0018] In the Fig. Figure 1 shows basic steps of the method according to the invention. These are monitored by means of a monitoring system 15 as shown in the Fig. The procedure is shown in Figure 2. It is used, for example, in communication between a doctor A and a patient P. In a first step 1, a voice cloning device 10 of the monitoring system 15 is activated, setting an initial external voice F1. This initial external voice F1 was cloned beforehand—as will be described later—for example, using a machine learning algorithm, and represents the voice of a person known or familiar to the patient P. The external voice F1 could, for example, be the voice of the patient P's spouse, a child of the patient P, a friend or relative of the patient P, a parent of the patient P, or a caregiver of the patient P.If the voice cloning device 10 is activated, the integrated voice generator 11 ensures that the original voice of Doctor A is replaced by the cloned voice F1, meaning that Doctor A's spoken words are simultaneously replaced by the voice F1. The voice cloning device 10 has at least two cloned voices to choose from, for example, the first voice F1 and a second voice F2. The selection and settings can be made automatically or manually.

[0019] In a second step 2, during operation of the voice cloning device with the first recorded voice, at least one physiological parameter of the patient is recorded or measured. This is done using a measuring unit 13 assigned to the monitoring system 15, which, for example, has or is comprised of one or more sensors, cameras, or other measuring devices. The at least one physiological parameter can, for example, be used as an indicator of the patient's emotional state. The at least one physiological parameter can, for example, be derived from one of the following measurements: blood pressure, pulse, EEG, where, for example, pulse sensors, EEG electrodes, or blood pressure sensors can be used. Other physiological parameters that can provide information about the emotional state or well-being of the patient P can also be measured.The physiological parameter(s) can be measured once or over a pre-defined period. In a third step, at least one of the patient's physiological parameters is evaluated using an evaluation unit 14 assigned to the monitoring system 15. The evaluation can, for example, simply involve a comparison with one or more threshold values ​​or a more complex evaluation regarding the patient's positive or negative emotional state.

[0020] Following the evaluation, in a fourth step 4, the voice cloning device 10 is automatically controlled by a control unit 12 assigned to the monitoring system 15, depending on the result of the evaluation. For example, the first voice recording F1 is retained or selected if the evaluation shows a positive effect of the first voice recording F1 on the patient P, i.e., if their mood is or becomes positive, they are or become calmer, or they cooperate better than before. In the simplest case, the first voice recording F1 is retained or selected if the parameter does not exceed a threshold value (such as pulse, blood pressure, etc.). Otherwise, the first voice recording F1 is discarded or selected.The process is terminated if the evaluation reveals a negative effect of the first external voice (F1) on the patient (P), for example, if their mood is or becomes negative, they become agitated, anxious, or disturbed, they show symptoms of stress, or their willingness to cooperate decreases. In the simplest case, the first external voice (F1) is discarded or terminated if the evaluation reveals that a threshold value is exceeded.

[0021] In the Fig. Figure 3 shows an extended embodiment of the invention. After the first voice input F1 is set in step 1, at least one physiological parameter is measured in step 2 and evaluated in step 3, a second voice input F2 is activated in step 5, so that, for example, the doctor A now speaks with the second voice input F2. Subsequently, in step 6, at least one physiological parameter of the patient is recorded or measured using the measuring unit 13, in particular the same physiological parameter(s) as with the first voice input F1, to ensure comparability. Then, in step 7, the measured parameter(s) are evaluated using the evaluation unit. The evaluation is preferably carried out in a manner comparable to the evaluation when using the first voice input F1, e.g.,simply by comparison with one or more threshold values, or more complex with regard to a positive or negative emotional state of the patient.

[0022] Subsequently, in an eighth step (8), the external voice that has the most positive effect on patient P's emotional state is automatically selected from the two available external voices, based on the evaluations. Of course, more than two external voices, e.g., three or four, can also be used.

[0023] In a preliminary step, the foreign voices can be cloned using machine learning. This can be done, for example, using audio recordings of the voices to be cloned. The voices of living people or even those of deceased people can be cloned; all that is needed are audio recordings (video, audio recordings, answering machine messages, etc.) of sufficient length. In the Fig.In step 9.1, the first foreign voice was cloned, and in step 9.2, the second foreign voice was cloned prior to the actual procedure. This can be done immediately before or even some time before (days, weeks, etc.) the actual procedure. Methods for creating cloned voices are generally known; see, for example, the section described above.

[0024] The general procedure is primarily used in medical settings, for example, during examinations, interventions, and surgeries without general anesthesia, but can also be used in other settings, such as nursing homes. The procedure can be performed during the described situations or beforehand to prepare for the specific situation. The selected voice can then be used by the monitoring system 15 in the medical setting.

[0025] It may also be possible to select different voices for different situations. For example, (objective) results, such as whether the patient follows instructions, can be included in the evaluations and factored into the selection or rejection of a voice. In this context, individual situations are tested with each voice, such as following instructions (e.g., breathing commands) on the one hand, and remaining calm on the other. It may happen that a different voice is more suitable for each situation; for example, the first voice might be suitable for calming the patient, and the second for following instructions. The monitoring system 15 can learn this (e.g., using machine learning) so that, based on the evaluation, it can then select the most appropriate voice for each situation using the control unit.

[0026] The invention can be summarized as follows: For particularly good communication, even with difficult patients, a monitoring system for monitoring a patient, especially during a medical diagnostic or therapeutic procedure, is required, comprising a voice cloning device with a voice generator configured to replace a person's natural voice with a cloned foreign voice different from that person's voice, wherein at least two foreign voices are selectable, a measuring unit configured to record at least one physiological parameter of the patient, an evaluation unit configured to evaluate the at least one measured physiological parameter of the patient, and a control unit configured to control the voice cloning device in such a way as to...that a foreign voice is selected or rejected depending on the result of the evaluation by the evaluation unit (for the further operation of the voice cloning device).

Claims

[1] Monitoring system (15) for monitoring a patient, in particular during a medical diagnostic or therapeutic procedure, comprising • a voice cloning device (10) with a voice generator (11) which is designed to replace a person's natural voice with a cloned foreign voice different from the person's voice, wherein at least two foreign voices are selectable, • a measuring unit (13) which is designed to record at least one physiological parameter of the patient, • an evaluation unit (14) which is designed to evaluate at least one measured physiological parameter of the patient, and • a control unit (12) which is configured to control the voice cloning device (10) in such a way that a foreign voice is selected or rejected depending on the result of the evaluation by the evaluation unit, in particular for the further operation of the voice cloning device (10). [2] Monitoring system according to claim 1, wherein the evaluation unit (14) is configured to compare the at least one measured physiological parameter of the patient with at least one threshold value. [3] Surveillance system according to any of the preceding claims, wherein the cloned foreign voices were generated by using a pre-trained machine learning algorithm. [4] Monitoring system according to claim 3, wherein the pre-training of the algorithm is carried out by means of speech samples from persons known to the patient, in particular persons close to him. [5] Monitoring system according to claim 2, wherein the control unit (12) is configured to control the voice cloning device (10) in such a way that a foreign voice is selected which causes the threshold not to be exceeded or a foreign voice which causes the threshold to be exceeded is rejected. [6] Monitoring system according to one of the preceding claims, wherein the measuring unit (13) is configured to measure a physiological parameter which can be used as an indicator of the patient's emotional state, and the evaluation unit (14) is configured to evaluate the physiological parameter with regard to the patient's emotional state. [7] Monitoring system according to one of the preceding claims, wherein the at least one physiological parameter is formed from one of the following measured values: blood pressure, pulse, EEG. [8] Monitoring system according to one of the preceding claims, wherein the control unit (12) is configured to control the voice cloning device (10) in such a way as to select a foreign voice which causes a positive emotional state or to reject a foreign voice which causes a negative emotional state. [9] Method for operating a monitoring system (15) according to any one of claims 1 to 8 comprising the following steps: • Activating the voice cloning device (10), whereby a first foreign voice is set (1), • Recording at least one physiological parameter of the patient during operation of the voice cloning device (10) with the first foreign voice (2), • Evaluating at least one of the patient's physiological parameters (3), and • Automatic activation of the voice cloning device (10) depending on the result of the evaluation. [10] Method according to claim 9, wherein the first foreign voice is selected or rejected for further operation of the voice cloning device (10) depending on the result of the evaluation by the evaluation unit (14) (4). [11] Method according to claim 9 or 10, wherein the evaluation of at least one physiological parameter of the patient includes a comparison with at least one threshold value. [12] Method according to any of the preceding claims, wherein the evaluation of the at least one physiological parameter includes an evaluation with regard to a patient's emotional state. [13] Method according to claim 11, wherein the control is carried out such that the first foreign voice is selected if the evaluation of the at least one physiological parameter shows that a threshold value for the parameter is not exceeded and that the first foreign voice is rejected if the evaluation of the at least one physiological parameter shows that a threshold value for the parameter is exceeded. [14] Method according to claim 12, wherein the control is carried out such that the first external voice is selected if it causes a positive emotional state of the patient and that the first external voice is rejected if it causes a negative emotional state. [15] Method according to one of claims 9 to 14, wherein at least one physiological parameter is measured and evaluated for at least one second foreign voice (5,6,7) and depending on the evaluation of the physiological parameter, a foreign voice is selected from the selection of the first foreign voice and the at least second foreign voice for the further operation of the voice cloning device (10) (8). [16] Method according to claims 12 and 15, wherein the external voice is selected which produces the most positive emotional state of the patient. [17] Method according to any one of claims 9 to 16, wherein parameters are recorded in different situations and evaluated depending on the situation and a situation-dependent control is carried out.

Images