Vehicle with a beamforming system for improving speech recognition in the vehicle's interior, and methods for improving speech recognition in a vehicle's interior.

Abstract

A vehicle with a beamforming system for improving speech capture in an interior (1) of the vehicle, wherein the beamforming system comprises a microphone array (2), the microphone array (2) comprising a plurality of microphones (3) arranged at known positions in the interior (1) for capturing acoustic signals, the beamforming system comprising a camera system, the camera system comprising at least one camera (4) configured to optically capture the interior (1), an evaluation unit (5) of the camera system being configured to determine the position of a speaking person (6) in the interior (1), and the beamforming system being configured to calculate, based on the determined position of the speaking person (6), expected values ​​for time-of-arrival differences (7) of a speech signal (8) emitted by the speaking person (6) for the plurality of microphones (3).wherein the beamforming system is configured to determine the speech signal (8) emitted by the speaker (6) from the acoustic signals captured by the microphones (3) based on the expected values ​​for the time-of-arrival differences (7), wherein the beamforming system is configured to determine the positions of several speakers (6) in the interior space (1), wherein the beamforming system is configured to calculate expected values ​​for time-of-arrival differences (7) of the speech signal (8) emitted by the respective speaker (6) for the plurality of microphones (3) based on the determined position of the respective speaker (6), wherein the beamforming system is configured to determine the speech signal (8) emitted by the respective speaker (6) from the acoustic signals captured by the microphones (3) based on the expected values ​​for the time-of-arrival differences (7),wherein at least some of the microphones (3) of the microphone array (2) are arranged in a headliner of the vehicle interior (1) .,

Description

[0001] The present invention relates to a vehicle with a beamforming system for improving speech recognition in the interior of the vehicle and to a method for improving speech recognition in the interior of a vehicle.

[0002] Noisy environments, such as those found in the interior of a vehicle, make it difficult to correctly interpret a speaker's voice. For example, ambient noise can make it hard for the listener to understand the speaker during a phone call. Similarly, when operating functions via voice control, ambient noise can cause a voice command to be missed or incorrectly recognized.

[0003] Noise-cancelling systems or sound-cancelling systems are known from the prior art, which generate noise suppression in a local area through destructive interference.

[0004] It is generally known to use a microphone array to separate useful signals, such as speech, emitted from a specific area of ​​space from interference signals. Such a system is disclosed, for example, in US 11,676,570 B2. DE 10 2014 201 228 B4, US 9,595,251 B2, and US 10 650,798 B2 disclose further prior art.

[0005] German patent DE 10 2004 000 043 A1 discloses a method, a device, and a control program for the selective recording of an acoustic signal. In this method for the selective recording of an acoustic signal, images of persons at least partially within range of a directive microphone are recorded by a recording medium in a first step. In a second step, an image analysis algorithm detects at least one position of a person using a predefinable recognition feature. Finally, in a third step, the directive microphone is adapted to the at least one person using the detected position. Advantageously, the method optimizes the focusing of directive microphones using visual information. German patent DE 10 2014 203 914 A1 discloses a method and a system for receiving desired tones. US patent 2017 / 0 150 256 A1 discloses further prior art.

[0006] The object of the present invention is to provide a vehicle that enables improved speech recognition in the vehicle's interior. Furthermore, it is an object of the invention to provide an improved method for speech recognition in a vehicle's interior.

[0007] These tasks are solved by the subject matter of the independent claims. The dependent claims concern advantageous further developments.

[0008] The vehicle features a beamforming system for improving speech capture in the vehicle's interior. The beamforming system includes a microphone array, the microphone array comprising a plurality of microphones arranged at known positions within the interior for capturing acoustic signals. The beamforming system also includes a camera system, the camera system comprising at least one camera configured to optically capture the interior. An evaluation unit of the camera system is configured to determine the position of a speaking person within the interior. Based on the determined position of the speaking person, the beamforming system is configured to calculate expected values ​​for the time-of-arrival differences of a speech signal emitted by the speaking person for the plurality of microphones.Based on the expected values ​​for the time-of-flight differences, the speech signal emitted by the speaker is determined from the acoustic signals captured by the microphones.

[0009] The vehicle according to the invention, thanks to its camera system, makes it possible to optically detect whether a person in the vehicle is speaking and to determine their position. Based on this detection and the position of the speaking person in the vehicle, expected values ​​for the time-of-arrival differences of this person's speech signal at the multiple microphones can be calculated. This facilitates the separation, amplification, or filtering of the speech signal from ambient noise by the beamforming system, since the expected values ​​for the time-of-arrival differences provide at least an indication of the temporal correlation between the speech signal contained in the detected acoustic signal from one of the microphones and the same speech signal contained in the detected acoustic signal from another of the microphones.To determine the speech signal, the acoustic signals captured by the microphones, which contain the speech signal and background noise, can be corrected and superimposed with the corresponding time differences, resulting in an amplification of the speech signal, i.e., the desired signal, relative to the background noise. A so-called delay-and-sum beamforming method can be used for this purpose, for example.

[0010] The advantage of the vehicle according to the invention lies particularly in the fact that the position in the interior, which forms the source of the useful signals, namely the position of the person, can be dynamically detected, and thus the expected values ​​for the time-of-arrival differences can be dynamically adjusted. This makes it possible, for example, to track the position of a person moving and speaking in the interior by evaluating the camera images, to dynamically determine their position relative to the multiple microphones, and thereby to dynamically adjust the expected values ​​for the time-of-arrival differences, so that the focus of the useful signal detection is always on the speaking person, thus improving speech recognition.

[0011] It is considered particularly advantageous if the evaluation unit of the camera system is configured to calculate expected values ​​for level differences of the speech signal emitted by the speaker for the majority of microphones, based on the determined position of the speaker. The beamforming system is then configured to determine the speech signal emitted by the speaker from the acoustic signals captured by the microphones, based on the expected values ​​for the time-of-arrival differences and the expected values ​​for the level differences. By using both the expected values ​​for the time-of-arrival differences and the expected values ​​for the level differences, a particularly accurate detection or filtering of the speaker's speech signal can be achieved.

[0012] The beamforming system is designed to determine the positions of multiple speakers within the interior space. Based on each speaker's position, the system calculates expected time-of-arrival (TAR) differences for the speech signal emitted by that speaker to the multiple microphones. Using these TAR values, the system then uses these TAR values ​​to separately identify the speech signal emitted by each speaker from the acoustic signals captured by the microphones. The identified speech signals can then be processed separately, for example, if the individual speaker is on a phone call.

[0013] It is considered particularly advantageous if the vehicle has a voice-controlled operating device, wherein the beamforming system is connected to the voice-controlled operating device via signal technology and is configured to transmit the detected voice signal to the voice-controlled operating device.

[0014] It is planned that at least some of the microphones of the microphone array will be located in the headliner of the vehicle interior. Locating at least some microphones in the headliner has the advantage that the speech signals reach the microphones in the headliner with minimal attenuation or scattering, which is beneficial for determining the speech signal to be captured or determined using the expected values ​​for the time-of-arrival differences and / or level differences.

[0015] In a particularly preferred embodiment, the vehicle is an aircraft.

[0016] In a preferred embodiment, it is provided that the calculation of the expected values ​​for runtime differences is computer-aided and / or that the determination of the speech signal is computer-aided.

[0017] The method according to the invention is a method for improving speech recognition in the interior of a vehicle. The vehicle in question is the vehicle according to the invention, and the method is carried out using the vehicle's beamforming system.

[0018] The above statements regarding the vehicle according to the invention apply accordingly to the method according to the invention and vice versa.

[0019] The following figures explain the invention in more detail using one exemplary embodiment, without being limited to this embodiment. They show: Fig. 1 A vehicle with a beamforming system to improve speech recognition in a schematic representation.

[0020] The Fig.Figure 1 shows a schematic representation of the interior 1 of a vehicle, for example, an aircraft. Several people 6, in this case several passengers, can be located in the interior 1. The vehicle has a beamforming system for improving speech recognition in the interior 1. The beamforming system includes a microphone array 2, wherein the microphone array 2 comprises a plurality of microphones 3 arranged at known positions in the interior 1. In this case, these microphones 3 are arranged in the headliner of the interior 1. The microphones 3 are configured to detect acoustic signals in the interior 1. The beamforming system also includes a camera system, wherein the camera system comprises at least one camera 4. The camera 4 is configured to optically detect the interior 1.An evaluation unit 5 of the camera system is configured to determine the current position of a speaking person 6 in the interior space 1, based on the images captured by the camera 4. To determine whether a person 6 is currently speaking, the evaluation unit 5 can be configured to detect and evaluate the lip movements of the person 6.

[0021] The speech signals 8 emitted by the speaking person 6 reach the majority of microphones 3 with different arrival times. The beamforming system, in this case the evaluation unit 5 of the camera system, is configured to determine the position of the speaking person 6 in the interior space 1 and, based on the determined position of the speaking person 6 and the known positions of the microphones 3, to calculate expected values ​​for the arrival time differences 7 of the speech signal 8 emitted by the speaking person 6 for the majority of microphones 3.

[0022] The beamforming system includes a detection device 11, wherein the expected values ​​for the time-of-arrival differences 7 and the output signals 9 of the microphones 3 form input signals for the detection device 11. The output signals 9 of the microphones 3 can be electrical signals representing the acoustic signals detected by the microphones 3. The detection device 11 is configured to determine an electrical signal 8' representing the speech signal 8 based on the expected values ​​for the time-of-arrival differences 7 and the electrical output signals 9 of the microphones 3. For this purpose, the detection device 11 can be configured to apply a delay-and-sum beamforming method.

[0023] The detected electrical signal 8' is then transmitted by the detection device 11 to a voice-controlled operating device 12 as an input signal.

Claims

[1] Vehicle with a beamforming system for improving speech capture in an interior (1) of the vehicle, wherein the beamforming system comprises a microphone array (2), the microphone array (2) comprising a plurality of microphones (3) arranged at known positions in the interior (1) for capturing acoustic signals, the beamforming system comprising a camera system, the camera system comprising at least one camera (4) configured to optically capture the interior (1), an evaluation unit (5) of the camera system being configured to determine the position of a speaking person (6) in the interior (1), the beamforming system being configured to calculate, based on the determined position of the speaking person (6), expected values ​​for time-of-arrival differences (7) of a speech signal (8) emitted by the speaking person (6) for the plurality of microphones (3),wherein the beamforming system is configured to determine the speech signal (8) emitted by the speaker (6) from the acoustic signals captured by the microphones (3) based on the expected values ​​for the time-of-arrival differences (7), wherein the beamforming system is configured to determine the positions of several speakers (6) in the interior space (1), wherein the beamforming system is configured to calculate expected values ​​for time-of-arrival differences (7) of the speech signal (8) emitted by the respective speaker (6) for the plurality of microphones (3) based on the determined position of the respective speaker (6), wherein the beamforming system is configured to determine the speech signal (8) emitted by the respective speaker (6) from the acoustic signals captured by the microphones (3) based on the expected values ​​for the time-of-arrival differences (7),wherein at least some of the microphones (3) of the microphone array (2) are arranged in a headliner of the vehicle interior (1) ., [2] Vehicle according to claim 1, wherein the evaluation unit (5) of the camera system is configured to calculate expected values ​​for level differences of the speech signal (8) emitted by the speaking person (6) for the plurality of microphones (3) based on the determined position of the speaking person (6), wherein the beamforming system is configured to determine the speech signal (8) emitted by the speaking person (6) from the acoustic signals captured by the microphones (3) based on the expected values ​​for the time differences (7) and the expected values ​​for the level differences. [3] Vehicle according to one of claims 1 to 2, wherein the vehicle has a voice-controlled operating device (12), wherein the beamforming system is connected to the voice-controlled operating device (12) via a signal connection and is configured to transmit the detected voice signal (8) to the voice-controlled operating device (12). [4] Vehicle according to any one of claims 1 to 3, wherein the vehicle is an aircraft. [5] Vehicle according to one of claims 1 to 4, wherein the calculation of the expected values ​​for time-of-flight differences (7) is computer-aided and / or wherein the determination of the speech signal (8) is computer-aided. [6] Method for improving speech recognition in an interior (1) of a vehicle according to any one of claims 1 to 5 using the vehicle's beamforming system.

Images