Multimodal sensor arrangement for close-to-the-body application

Abstract

A sensor arrangement (41) for a close-to-the-body application comprises a support structure (112) and one or more conductor structures (122) that are mounted on the support structure (112) and designed to transmit or detect electromagnetic waves of a radar measurement. The sensor arrangement moreover comprises at least one sensor (351) that is mounted on the support structure (112) and designed to detect at least one measured variable. This allows for example the vocal tract of the person to be characterized, for example in order to determine an intended vocal utterance.

Description

[0001] DESCRIPTION

[0002] MULTI-MODAL SENSOR ARRANGEMENT FOR CLOSE-TO-BODY APPLICATION

[0003] TECHNICAL FIELD

[0004] Various examples relate to a sensor arrangement that combines an antenna for radar measurement with another sensor, such as an acceleration sensor or strain sensor. Such sensor arrangements can be used to acquire sensor data, based on which, for example, a vocal tract can be characterized for the purpose of determining an intended speech utterance.

[0005] BACKGROUND

[0006] Vocal tract characterization can be performed for the purpose of determining the person's intended speech utterance. See, for example, Wagner, Christoph, et al., "Silent Speech Command Word Recognition Using Stepped Frequency Continuous Wave Radar" (2021).

[0007] Technically, the human voice is often described using the so-called source-filter model. The lungs, trachea, and larynx together form the source. The air is compressed in the lungs and flows up through the trachea to the larynx. In the larynx, the vocal folds—colloquially known as the “vocal cords”—form the glottis. The laryngeal muscles keep the vocal folds tense by exerting force via the arytenoid cartilage. During voiced speech, the pressure in the trachea and the tension of the vocal folds cause them to open and close periodically, creating an acoustic vibration, a sound wave. This sound wave is acoustically filtered by the time-varying shape of the vocal tract, consisting of the pharynx, oral cavity, and nasal cavity, before it exits the mouth and nostrils.The production of speech from corresponding speech utterances consists of the process of phonation, technically speaking, the stimulation of an acoustic vibration by the vocal folds, and articulation, i.e., the filtering of the sound spectrum through the temporally changing shape of the vocal tract. The formation of the vocal tract occurs with the soft palate, which opens or closes the nasal cavity, the tongue, the upper and lower rows of teeth, and the upper and lower lips.

[0008] To characterize the vocal tract, the changes in electromagnetic waves from a radar measurement through reflection or transmission across a person's vocal tract are exploited. The radar signals can then be processed, for example, using an artificial neural network to determine an intended speech utterance.

[0009] SHORT SUMMARY

[0010] There is a need for sensor arrays and measurement techniques to characterize the vocal tract, particularly to enable the determination of an intended speech utterance.

[0011] This problem is solved by the features of the independent patent claims. The features of the dependent patent claims define embodiments.

[0012] Several examples are based on the realization that an accurate determination of an intended speech utterance can often benefit from the use of several different measurement techniques to measure a person's vocal tract, especially vocal tract movements.

[0013] A sensor arrangement for a body-worn application comprises a support structure and one or more conductor structures. The one or more conductor structures are attached to the support structure. The one or more conductor structures are configured to emit and / or detect electromagnetic waves of a radar measurement. Furthermore, the sensor arrangement comprises at least one sensor attached to the support structure, wherein the at least one sensor is configured to detect at least one measured variable. Examples would be strain and acceleration. Conductivity could also be measured.

[0014] The sensor arrangement can thus provide for the integration of an antenna formed by the one or more conductor structures (as a "radar sensor") and the at least one additional sensor on the common support structure. Integration into a common sensor arrangement can ensure that measurement signals from the different measurement methods are related to one another in the aforementioned relationship. The sensor and the radar sensor can involve different measurement modalities. The sensor and the radar sensor can measure different physical observables.

[0015] Various techniques disclosed herein make it possible to determine a characterization of the vocal tract, or in particular, an intended speech utterance, based on multiple input data for a processing algorithm. In other words, it is possible to use sensor fusion of several different sensors.

[0016] In particular, techniques are described in which a radar measurement is combined with one or more other measurement methods, such as the measurement of an acceleration in the area of ​​the vocal tract and / or a deformation or tension of the skin surface in the area of ​​the vocal tract.

[0017] A radar measurement can, for example, involve determining the impulse response of the channel. A radar measurement can therefore involve the interaction of electromagnetic waves with the tissue and the acquisition of information about this interaction.

[0018] There are different ways to implement the support structure of the antenna array.

[0019] For example, it would be conceivable for the support structure to be two-dimensional. The support structure can be formed as a thin layer. The support structure can be elastic. In this way, the sensor arrangement could be configured to be applied to the underside of the sensor arrangement on the surface of the person's skin.

[0020] In another example, it would be conceivable for the support structure and the antenna to be formed as foils. A polyimide foil could be used. In this case, the support structure and the antenna can be implemented as thin, elastic layers based on a foil. The antenna foil and the support structure foil can be glued together. The support and antenna foils can also be the same foil (i.e., combined).

[0021] For example, it would be conceivable for the support structure to be designed as a textile layer. This means that the support structure can consist of or comprise a woven material. In such an example, the one or more conductor structures of the antenna could be incorporated into the textile layer, for example, as threads or ribbons.

[0022] In particular, if the support structure is implemented as a two-dimensionally extended layer, the sensor arrangement can be designed for direct wear on the skin surface. For example, it would be conceivable for the sensor arrangement to also have an adhesive layer that extends along the support structure and has a recess. The sensor arrangement can be adhered to the skin by means of the adhesive layer. The at least one sensor could then be arranged within the recess. A sensitive area of ​​the at least one sensor could be in direct contact with the skin surface. This enables, in particular, contact-based measuring methods, for example measuring the stretching / tension of the skin surface or moisture of the skin surface.

[0023] For example, it would be conceivable for the processing algorithm to be implemented by an artificial neural network. This could process, for example in different channels, input data obtained from the radar measurement and further input data obtained based on sensor data from the at least one sensor. This means that sensor fusion can be implemented by the artificial neural network. In such an example, it is particularly helpful if the antenna for the radar measurement and the at least one sensor are arranged in a specific relative arrangement to one another and to the person or the skin surface. This makes it possible to create the same measurement conditions with regard to such a relative arrangement during the training of the artificial neural network as during the measurement itself.By integrating the antenna and at least one sensor into the common sensor array, such reproducible measurement conditions can be ensured. This can increase the accuracy of determining intended speech utterances.

[0024] It would also be conceivable for the at least one sensor to be used to verify the correct fit of the sensor array on the person's skin surface. For example, it would be conceivable for the conductivity to be measured and thus verified whether the one or more conductive structures are resting on the skin surface or extend along the skin surface.

[0025] Some examples for at least one sensor are described below:

[0026] For example, a strain gauge could be used to measure the stretch of the skin surface.

[0027] A microelectromechanical sensor with an oscillating mass could be used to detect acceleration.

[0028] An ultrasound sensor could be used to measure movement of the person's vocal tract.

[0029] An electromyography sensor could be used to determine the electrical muscle activity of a muscle in the vocal tract, such as the tongue.

[0030] It would be conceivable to measure the conductivity of the skin surface. A conductivity sensor could be used to measure the resistance between the sensors, which are attached, for example, to the cheeks or additionally under the chin, before and during speech. This makes it possible to obtain information about the fit and position of the antennas and their changes during speech, skin resistance (perspiration), etc. The sensor arrangement can also include a communication interface configured to multiplex or demultiplex first signals and second signals. The communication interface can be implemented in hardware and optionally in software.

[0031] By means of the communication interface, the first signals and the second signals can be transmitted via a common cable. The first signals can be associated with the one or more conductor structures; while the second signals are associated with the at least one sensor. For example, the first signals could be received by a radio frequency (RF) front end and coupled into the one or more conductor structures; in this way, electromagnetic waves can be emitted. The first signals could be RF currents that implement radar chirps. Currents in the one or more conductor structures caused by detected electromagnetic waves can also be forwarded in the form of the first signals to an RF receiving circuit, which for example comprises a signal amplifier and an analog-to-digital converter, via the cable.

[0032] Accordingly, the second signals may, for example, comprise a supply voltage for the at least one sensor. The second signals could comprise sensor signals of the at least one sensor. Depending on the implementation of the at least one sensor, it would be conceivable for the first signals and the second signals to have inherently different frequencies; and thus, the multiplexing relates to a simple combination of the first and second signals. For example, a supply voltage for the at least one sensor could be separated from RF currents for the one or more conductor structures using low-pass filtering.

[0033] In principle, however, more complicated multiplexing methods would also be conceivable, for example a time slot method or a modulation of carrier frequencies.

[0034] Using techniques such as those described above in connection with the communication interface, it is possible to use a single cable to contact the sensor arrangement. This simplifies use and reduces the risk of incorrect contact by the user. A system comprises a sensor arrangement as described above. The system also comprises an RF front end which is electrically coupled to the one or more conductor structures. The RF front end is configured to perform radar measurements based on electromagnetic waves. Furthermore, the system comprises a sensor driver circuit which is electrically coupled to the at least one sensor. The sensor driver circuit is configured to perform a measurement of the at least one measurand. Finally, the system also comprises a processor.The processor is configured to determine an intended speech utterance of the person based on the radar measurement and on the measurement of at least one measured variable.

[0035] The RF front end and the sensor driver circuit can be connected to the sensor array via a common cable. The corresponding signals can be transmitted via the communication interface over the common cable, as described above.

[0036] A method comprises characterizing a person's vocal tract using a radar measurement and a measurement of at least one measurand. A sensor arrangement as described above is used. Based on the characterization of the vocal tract, an intended speech utterance of the person can then be determined.

[0037] The features set forth above and features described below may be used not only in the corresponding explicitly set forth combinations, but also in further combinations or in isolation, without departing from the scope of the present invention.

[0038] SHORT DESCRIPTION OF THE CHARACTERS

[0039] FIG. 1 is a schematic side view of an exemplary sensor arrangement.

[0040] FIG. 2 is a schematic view of a system according to various examples. DETAILED DESCRIPTION

[0041] The above-described properties, features and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more clearly understood in connection with the following description of the embodiments, which are explained in more detail in connection with the drawings.

[0042] The present invention is explained in more detail below using preferred embodiments with reference to the drawings. In the figures, identical reference numerals designate identical or similar elements. The figures are schematic representations of various embodiments of the invention. Elements shown in the figures are not necessarily drawn to scale. Rather, the various elements shown in the figures are depicted in such a way that their function and general purpose will be understood by those skilled in the art. Connections and couplings between functional units and elements shown in the figures can also be implemented as an indirect connection or coupling. A connection or coupling can be implemented wired or wirelessly. Functional units can be implemented as hardware, software, or a combination of hardware and software.

[0043] The following discloses techniques that enable radar measurements in close-to-body applications. For example, medical applications could be enabled. Examples include the determination of intended speech utterances when a person has no or only limited ability to produce sounds.

[0044] A multimodal sensor arrangement is disclosed that combines an antenna for radar measurement with one or more additional sensors that measure one or more parameters. This allows the vocal tract of a person to be characterized. For example, silent speech recognition can be realized.

[0045] For example, one or more additional sensors could be used to check whether the antenna is correctly positioned in relation to the skin surface.

[0046] It would be possible to determine a person's intended speech utterance. It would be conceivable to determine a text that describes a textual representation of the speech corresponding to the intended speech utterance. An audio output (i.e., an audio signal) could be determined that corresponds to the speech utterance. The term "determining a person's synthetic speech" is used below for all such variants.

[0047] Synthetic speech can also be determined, especially when the person is unable to produce any sounds. This means that "silent speech" is enabled. It is known that three to nine percent of the population in industrialized countries are affected by voice disorders. These manifest themselves in a range of symptoms summarized under the term dysphonia: from hoarseness and weak or distorted speech to a complete loss of voice, known as aphonia. Voice disorders can be functional or organic in origin. Organic voice disorders can be further divided into structural or neurogenic disorders. For individuals with such voice disorders, "silent speech" can be enabled.

[0048] Synthetic speech can also be generated if the person intentionally whispers or articulates silently. This could, for example, enable a private, telephone-based "conversation."

[0049] FIG. 1 is a schematic side view of an exemplary sensor arrangement 41. The sensor arrangement 41 is applied on its underside 45 to a skin surface 196 of the person. The sensor arrangement 41 has a support structure 112, implemented here as a carrier layer or carrier film. The support structure can be elastically configured to adapt to different surface shapes of the skin surface 196.

[0050] A metal layer 113 is applied to the carrier structure 112; this could, for example, be implemented as a foil-based circuit. The layer 113 comprises a plurality of metallic conductor structures 122 (shown with black contrast in FIG. 1). These conductor structures 122 are configured to emit or detect electromagnetic waves. For this purpose, RF currents can be fed into the conductor structures 122 or detected. An adhesive film 114 is provided beneath the conductor structures 122, although this is optional. The sensor arrangement 41 can be fixed to the skin surface using the adhesive film 114 without an air gap.

[0051] A sensor 351 is also provided. The sensor is attached with a sensitive area 352 directly to the skin surface 196. This is made possible by a corresponding recess 359 that extends through layers 112-114. The sensor 351 is partially arranged in the recess 359. In general, the recess would not have to penetrate all layers 112, 113, and 114; for example, it would be conceivable for only the adhesive film 114 to have a recess if the sensor 351 were implemented particularly flatly (such as a MEMS acceleration sensor).

[0052] For example, the sensor 351 could measure a stretch of the skin surface 196.

[0053] Another possibility would be for sensor 351 to measure acceleration. In such a case, the sensor would not have to rest directly on the skin surface 196.

[0054] FIG. 2 shows aspects related to an exemplary system 80. The system 80 includes the sensor arrangement 41, which—as described in connection with FIG. 1—has the conductor structures 122 and the sensor 351.

[0055] In addition, a communication interface 361 is provided, which is connected to a corresponding communication interface 362 via a cable 369. The communication interface 361 can, for example, be configured to multiplex or demultiplex signals from or for the conductor structures 122 with signals from or for the sensor 351. In a simple variant, the communication interface 361 could feed the signals of both types into the cable 369 in a superimposed manner. Active multiplexing techniques could also be used, either in the frequency domain or in the time domain. The corresponding communication interface 362 can also be configured according to the implementation of the communication interface 361.

[0056] In principle, it would also be conceivable for the radar signals to be transmitted wired via cable 369, but for signals from or for sensor 351 to be transmitted wirelessly. For example, radar signals could be transmitted via cable 369 along with a DC supply voltage for sensor 351, while measurement data from sensor 351 are transmitted wirelessly.

[0057] Also shown in FIG. 2 is an RF front end 81. This is configured to feed RF currents into the conductor structures 122 via the cable 369 and / or to detect RF currents induced by electromagnetic waves in the conductor structures 122 and convert them into digital signals. The RF front end 81 may include, for example, a frequency mixer, oscillator, RF amplifier, and filter.

[0058] Also shown is a sensor driver circuit 363. This is configured to implement a measurement of a measurand using sensor 351. The sensor driver circuit can supply sensor 351 with an operating voltage. The sensor driver circuit could read sensor 351.

[0059] The system 80 also includes a processor 82. The processor 82 is configured to determine an intended speech utterance of the person based on the radar measurement by means of the conductor structures 122 and based on the measurement of the measurand by means of the sensor 351, for example based on program code that is loaded from the memory 83 and executed.

[0060] In FIG. 1 and FIG. 2, the sensor arrangement 41 comprises only a single sensor 351. In principle, it would be conceivable for the sensor arrangement 41 to comprise multiple sensors (in addition to the antenna for radar measurement). These multiple sensors can measure different variables, for example, acceleration and strain / tension.

[0061] The radar measurement can generally be carried out using reflections; then, one and the same conductor structures 122 can be used for transmitting and detecting the electromagnetic waves. In some examples, however, a radar measurement using transmission would also be conceivable. Then, a further antenna can be provided to transmit the electromagnetic waves, which are subsequently detected by the conductor structures 122; or a further antenna can be provided to detect the transmitted electromagnetic waves. Of course, the features of the previously described embodiments and aspects of the invention can be combined with one another. In particular, the features can be used not only in the described combinations, but also in other combinations or on their own, without departing from the scope of the invention.

Claims

PATENT CLAIMS 1. Sensor arrangement (41) for a close-contact application, wherein the sensor arrangement (41) comprises: - a support structure (112), - one or more conductor structures (122) attached to the support structure (112) and configured to emit or detect electromagnetic waves of a radar measurement, and - at least one sensor (351) attached to the support structure (112) and configured to detect at least one measurement quantity.

2. Sensor arrangement (41) according to claim 1, wherein the at least one measured quantity comprises a strain.

3. Sensor arrangement (41) according to claim 1 or 2, wherein the at least one measured quantity comprises an acceleration.

4. Sensor arrangement (41) according to one of the preceding claims, wherein the at least one measured quantity comprises a conductivity.

5. Sensor arrangement (41) according to one of the preceding claims, further comprising: - an adhesive layer (114) extending along the support structure (112) and comprising a recess (359), wherein the at least one sensor (251) is arranged at least partially within the recess (359).

6. Sensor arrangement according to one of the preceding claims, further comprising: - a communication interface (361) which is set up to multiplex and / or demultiplex first signals associated with the one or more conductor structures and second signals associated with the at least one sensor for transmission via a cable (369).

7. System (80), which includes: - the sensor arrangement (41) according to one of the preceding claims, - an RF front end (81) coupled to the one or more conductor structures (122) and configured to perform radar measurement based on electromagnetic waves, - a sensor driver circuit (363) coupled to the at least one sensor (351) and configured to perform a measurement of the at least one measured quantity, and - a processor (82) configured to determine an intended speech utterance based on radar measurement and based on the measurement of at least one measured quantity.

8. Procedure, which includes: - Characterizing the vocal tract of a person by means of a radar measurement and a further measurement of at least one measured quantity using the sensor arrangement (41) according to one of claims 1 to 6, and - Determining the intended speech expression of the person based on characterizing the vocal tract.

9. Procedure, which includes: - Verification of contact between the sensor arrangement according to one of claims 1 to 6 and a skin surface based on a measurement of the at least one measured quantity.

Images