Audio Ducking with Spatial Speech Separation for Vehicle Audio Systems

Abstract

To provide a system for voice ducking with spatial speech separation for a vehicle audio system.SOLUTION: A vehicle loudspeaker system includes at least two microphones forming a microphone array, at least one loudspeaker configured to emit non-human sound, and a processor programmed to receive incoming audio signals from the microphone array. The processor applies beamforming to the incoming voice signal to determine if human-generated voice is detected in the voice signal, and is programmed to instruct the loudspeaker to adjust a non-human sound in response to human generated sound being detected.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 62 / 954,991, filed December 30, 2019, the disclosure of which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates to a system for audio ducking with spatial speech separation for vehicle audio systems. [Background technology]

[0003] Current vehicle cabin acoustics dictates that any sound generated within the vehicle cabin is generally perceived as a noise stimulus. Common examples of interference sources include traffic noise, wind noise, passenger speech, and multimedia content. The presence of these noise sources complicates speech perception by reducing speech intelligibility, signal-to-noise ratio, and subjective speech quality. Audio ducking can be used to increase in-vehicle quality and driver satisfaction. However, the algorithms used to determine whether an audio signal contains human speech are often cumbersome and inaccurate. Summary of the Invention [Means for solving the problem]

[0004] The vehicle loudspeaker system includes at least two microphones forming a microphone array, at least one loudspeaker configured to emit a non-human sound, and a processor programmed to receive an incoming sound signal from the microphone array, the processor further programmed to apply beamforming to the incoming sound signal, determine whether a human generated sound is detected in the sound signal, and instruct the loudspeaker to adjust the non-human sound in response to the human generated sound being detected.

[0005] A non-transitory computer-readable medium including instructions for a loudspeaker system for generating sound in a vehicle includes receiving an incoming sound signal from a vehicle microphone array, applying beamforming to the incoming sound signal, determining whether human generated sounds are detected in the sound signal, and instructing the loudspeaker to adjust non-human sounds in response to the human generated sounds being detected.

[0006] A method for producing audio via a loudspeaker system in a vehicle includes receiving an incoming audio signal from a vehicle microphone array, applying beamforming to the incoming audio signal, determining whether human-generated audio is detected in the audio signal, and instructing a loudspeaker to adjust non-human audio in response to the human-generated audio being detected. For example, the present application provides the following: (Item 1) 1. A vehicle loudspeaker system comprising: at least two microphones forming a microphone array; at least one loudspeaker configured to emit a non-human sound; receiving an incoming voice signal from the microphone array; Applying beamforming to the incoming voice signal; determining whether human generated speech is detected within said speech signal; In response to detecting human-produced speech, instructing the loudspeaker to adjust the non-human speech. A processor programmed to The vehicle loudspeaker system comprises: (Item 2) The system of claim 1, wherein the processor is programmed to determine whether the human generated voice is detected within the voice signal based at least in part on an acoustic input of at least one of the incoming voice signals exceeding a threshold. (Item 3) 13. The system of claim 12, wherein the acoustic input is associated with a particular area within the vehicle. (Item 4) 2. The system of claim 1, wherein the acoustic input is associated with the particular region via the beamforming. (Item 5) 2. The system of claim 1, wherein the microphone array is positioned adjacent to a head restraint for a passenger in the vehicle. (Item 6) 20. The system of claim 19, wherein the processor is programmed to reduce a volume of the non-human voice in response to human speech being detected in at least one outgoing audio signal. (Item 7) 2. The system of claim 1, wherein the processor is programmed to apply at least one of noise suppression or echo cancellation to at least one incoming voice signal by the beamforming. (Item 8) 2. The system of claim 1, wherein the processor is programmed to detect human generated speech with the microphone array in response to the microphone array detecting an acoustic signal. (Item 9) 2. The system of claim 1, wherein the microphone array includes four microphones. (Item 10) 1. A non-transitory computer readable medium comprising instructions for a loudspeaker system for generating sound in a vehicle, the non-transitory computer readable medium comprising: receiving an incoming audio signal from a vehicle microphone array; applying beamforming to the incoming voice signal; determining whether human generated speech is detected within the speech signal; instructing the loudspeaker to adjust the non-human audio in response to detecting a human generated audio; The non-transitory computer-readable medium. (Item 11) The media described in the preceding item, wherein the determining whether human generated speech is detected within the audio signal is based at least in part on acoustic input within the audio signal exceeding a threshold. (Item 12) 2. The medium of claim 1, wherein the acoustic input is associated with a particular area within the vehicle. (Item 13) The medium of any one of the preceding claims, wherein the acoustic input is associated with the specific region via the beamforming. (Item 14) 2. The media of claim 1, wherein the microphone array is positioned adjacent to a head restraint for an occupant in the vehicle. (Item 15) The media of any one of the preceding claims, further comprising, in response to human speech being detected in at least one audio signal, reducing a volume of the non-human sounds. (Item 16) The medium of any one of the preceding items, further comprising applying at least one of noise suppression or echo cancellation to at least one audio signal by the beamforming. (Item 17) The medium of any one of the preceding claims, further comprising detecting human generated speech with the microphone array in response to the microphone array detecting an acoustic signal. (Item 18) 1. A method for producing sound through a loudspeaker system in a vehicle, comprising: receiving an incoming audio signal from a vehicle microphone array; applying beamforming to the incoming voice signal; determining whether human generated speech is detected within the speech signal; instructing the loudspeaker to adjust the non-human audio in response to detecting a human generated audio; The above method. (Item 19) 11. The method of claim 10, wherein determining whether human generated speech is detected in at least one audio signal is based at least in part on acoustic input in the audio signal exceeding a threshold. (Item 20) 13. The method of claim 1, wherein the acoustic input is associated with a particular area within the vehicle. (Summary) The vehicle loudspeaker system includes at least two microphones forming a microphone array, at least one loudspeaker configured to emit non-human sounds, and a processor programmed to receive an incoming audio signal from the microphone array, the processor further programmed to apply beamforming to the incoming audio signal, determine whether a human generated sound is detected in the audio signal, and in response to the human generated sound being detected, direct the loudspeaker to adjust the non-human sound. [Brief description of the drawings]

[0007] [Figure 1] 1 illustrates a telecommunications network for facilitating telecommunications between a near-end participant within a vehicle and a remote far-end participant located outside the vehicle, in accordance with one or more embodiments of the present disclosure. [Diagram 2] FIG. 1 is a block diagram of an in-vehicle noise cancellation system for far-end telecommunications in accordance with one or more embodiments of the present disclosure. [Diagram 3] 3 is a simplified exemplary flow diagram illustrating a method 300 for noise cancellation for far-end telecommunications, in accordance with one or more embodiments of the present disclosure. [Figure 4] 1 illustrates an exemplary microphone arrangement in accordance with one or more embodiments of the present disclosure. [Diagram 5] 1 illustrates an example setup of a headrest-based telecommunications system for a vehicle in accordance with one or more embodiments of the present disclosure. [Figure 6]1 illustrates another example setup of a headrest-based telecommunications system for a vehicle in accordance with one or more embodiments of the present disclosure. [Figure 7] FIG. 1 is a top view of a vehicle including at least one headrest microphone array for use in an in-vehicle noise cancellation system in accordance with one or more embodiments of the present disclosure. [Figure 8] FIG. 2 is another plan view of a vehicle including at least one headrest microphone array for use in an in-vehicle noise cancellation system in accordance with one or more embodiments of the present disclosure. [Figure 9] FIG. 2 is yet another plan view of a vehicle including at least one head rest microphone array and a rearview mirror assembly microphone array for use in an in-vehicle noise cancellation system in accordance with one or more embodiments of the present disclosure. [Figure 10] FIG. 2 is yet another plan view of a vehicle including a plurality of different headrest microphone arrays for use in an in-vehicle noise cancellation system in accordance with one or more embodiments of the present disclosure. [Figure 11] FIG. 1 shows a block diagram of an audio ducking system. [Figure 12] 12 illustrates an example process of the system of FIG. 11. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Where necessary, detailed embodiments of the present invention are disclosed herein, but it will be understood that the disclosed embodiments are merely examples of the invention that may be embodied in various and alternative forms. The figures are not necessarily to scale, and some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to variously use the present invention.

[0009] Described herein is a beamforming approach to audio ducking. An audio ducking system can help automatically adjust the volume of distracting media signals when a user is attempting to converse. Once speech is completed, the media may automatically adjust to resume a louder volume. An array of microphones and a beamforming process may be used to receive and identify speech signals from a small spatial region of interest around the user's head and mouth. This allows any and all sounds from that region of interest to be instantly sent to the audio ducking system.

[0010] Voice ducking is a common method to promote clear communication. When a voice signal is present, the volume of the medium is seamlessly "ducked," or attenuated, to facilitate communication. Many existing systems use algorithms to determine when an incoming speech signal at a microphone contains human speech. These algorithms can be cumbersome and large, and require a given latency before the incoming signal can be confirmed as a speech signal. That is, the beginning of a user's utterance may not be sent to the ducking engine. Furthermore, non-speech utterances such as "oh," "hmm," or "shhh," may not be classified as speech signals and therefore are not sent to the ducking engine.

[0011] Because the concepts disclosed herein rely on a spatial region of interest, any and all audio from that region of interest is automatically and instantly sent to the ducking engine, including non-speech utterances, and does not require any delay since the region of interest is already defined and localized via beamforming, thus avoiding directional microphones and cumbersome signal processing.

[0012] The present disclosure also describes an in-vehicle noise cancellation system for optimizing the far-end user experience. The noise cancellation system may improve the intelligibility of near-end speech at the far-end of a communication exchange, including a telecommunication exchange or a virtual personal assistant interaction. The noise cancellation system may incorporate real-time acoustic input from the vehicle, not just from the microphone of the telecommunication device. Furthermore, audio signals from small embedded microphones mounted in the car can be processed and mixed into the outgoing telecommunication signal to effectively remove acoustic energy from one or more unwanted sound sources within the vehicle. In addition to the unwanted noises captured by the embedded microphones (e.g., children screaming and background conversations), audio playing from known audio streams in the vehicle's infotainment system (e.g., music, sound effects, and movie dialogue) can be used as direct inputs to the noise cancellation system. Thus, as direct inputs, these streams can be removed from the outgoing telecommunication signal, thus providing a much higher signal-to-noise ratio, speech quality, and speech intelligibility to the user's far-end communication partner.

[0013] FIG. 1 illustrates a telecommunications network 100 for facilitating telecommunications between a near-end participant 102 in a vehicle 104 and a remote far-end participant 106 located outside the vehicle via a cellular base station 108. The vehicle 104 may include a telecommunications system 110 for processing incoming and outgoing telecommunications signals, collectively shown in FIG. 1 as telecommunications signals 112. The telecommunications system 110 may include a digital signal processor (DSP) 114 for processing voice telecommunications signals, as described in more detail below. According to another embodiment, the DSP 114 may be a separate module from the telecommunications system 110. A vehicle infotainment system 116 may be connected to the telecommunications system 110. A first transducer 118 or speaker may transmit the incoming telecommunications signal to a near-end participant of a telecommunications exchange inside the vehicle cabin 120. Thus, the first transducer 118 may be located proximate to the near-end participant or may generate a localized sound field at a particular seat location occupied by the near-end participant. The second transducer 122 may transmit audio from the vehicle's infotainment system 116 (eg, music, sound effects, and dialogue from a movie audio).

[0014] A first microphone array 124 may be located within the vehicle cabin 120 to receive speech of a near-end participant of the telecommunications (i.e., the driver or another occupant of the source vehicle). A second microphone array 126 may be located within the vehicle cabin 120 to detect unwanted sound sources (e.g., traffic noise, wind noise, background speech, and multimedia content), collectively referred to as noise. Collectively, the telecommunications system 110, the DSP 114, the infotainment system 116, the transducers 118, 122, and the microphone arrays 124, 126 may form an in-vehicle noise cancellation system 128 for far-end telecommunications.

[0015] FIG. 2 is a block diagram of the noise cancellation system 128 shown in FIG. 1. As shown in FIG. 2, an incoming telecommunication signal 112a from a far-end participant (not shown) may be received by the DSP 114. The DSP 114 may be a hardware-based device such as a combination of specialized microphone processors and / or integrated circuits optimized for the operational needs of digital signal processing, which may be specific to the voice application disclosed herein. The incoming telecommunication signal 112a may be subjected to automatic gain control in an automatic gain control device (AGC) 202. The AGC 202 may provide a controlled signal amplitude at its output despite variations in the amplitude of the input signal. The average or peak output signal level is used to dynamically adjust the input and output gain to an appropriate value, allowing the circuit to function well with a wider range of input signal levels. The output from the AGC 202 may then be received by a loss control device 204 to be subjected to loss control, which is then passed to an equalizer 206 to equalize the incoming telecommunication signal 112a. Equalization is the process of adjusting the balance between frequency components within an electronic signal. An equalizer enhances (boosts) or attenuates (cuts) the energy in certain frequency bands or "frequency ranges."

[0016] The output of the equalizer 206 may be received by a limiter 208. A limiter is a circuit that allows signals below a specified input power or level to pass unaffected while attenuating stronger signal peaks above this threshold. Limiting is a type of dynamic range compression. It is any process that prevents a specified characteristic (usually amplitude) of the output of a device from exceeding a predetermined value. Limiters are common as safety devices in live voice and broadcast applications to prevent sudden volume peaks from occurring. The digitally processed incoming telecommunication signal 112a' may then be received by the first transducer 118 for audible transmission to a near-end participant of the telecommunication exchange.

[0017] As also shown in FIG. 2, the noise cancellation system 128 may include a first microphone array 124 and a second microphone array 126. The first microphone array 124 may include multiple small embedded microphones strategically placed in the vehicle cabin to receive speech from a near-end participant of a telecommunications exchange (i.e., the driver or another occupant of the source vehicle). The first microphone array 124 may be positioned as close as possible to the near-end participant while being as far away as possible from reflective surfaces. For example, the first microphone array 124 may be embedded in a headrest or headliner, as shown in FIG. 4. The second microphone array 126 may include multiple small embedded microphones strategically placed in the vehicle cabin to detect unwanted sound sources (e.g., traffic noise, wind noise, background speech, and multimedia content), collectively referred to as noise.

[0018] Both inputs to the first and second microphone arrays, near-end speech and noise, respectively, may be processed using the DSP 114. The first set of audio signals 209 (i.e., indicative of near-end speech) from the first microphone array 124 may be sent to a first beamformer 210 for beamforming, while the second set of audio signals 211 (i.e., indicative of noise) may be sent to a second beamformer 212. Beamforming or spatial filtering is a signal processing technique used in sensor arrays for directional signal transmission or reception. This is achieved by combining elements in the array such that signals at certain angles experience constructive interference, while other signals experience destructive interference. Beamforming can be used on both the transmit and receive sides to achieve spatial selectivity. The improvement compared to omnidirectional receive / transmit is known as the directivity of the array. To change the directivity of the array on transmit, the beamformer controls the phase and relative amplitude of the signal at each transmitter to create patterns of constructive and destructive interference in the wavefront. On receive, the information from the different sensors is combined so that expected radiation patterns are preferentially observed.

[0019] The first beamformer 210 may output a near-end speech signal 213 indicative of near-end speech detected by the first microphone array 124. Alternatively, the near-end speech signal 213 may be received by the DSP 114 directly from the first microphone array 124 or from individual microphones in the first microphone array. The second beamformer 212 may output a noise signal 218 indicative of unpredictable background noise detected by the second microphone array 126. Alternatively, the noise signal 218 may be received by the DSP 114 directly from the second microphone array 126 or from individual microphones in the second microphone array.

[0020] The near-end speech signal 213 may be received by the echo canceller 214 along with the digitally processed incoming telecommunication signal 112a' from the far-end participant 106. Echo cancellation is a method in telephony technology to improve voice quality by removing echo after it is already present. In addition to improving subjective quality, this process increases the capacity achieved through silence suppression by preventing echo from traveling across the network. There are various types of echo and causes of echo with unique characteristics including acoustic echo (speech from a loudspeaker reflected and recorded by a microphone, which may vary considerably over time) and line echo (electrical impulses caused by, for example, coupling between transmitting and receiving wires, impedance mismatch, electrical reflections, etc., which vary much less than acoustic echo). However, in practice, the same techniques are used to handle all types of echo, so an audio echo canceller can remove line echo as well as audio echo. Echo cancellation requires first identifying the originally transmitted signal, which reappears with some delay in the transmitted or received signal. Once the echo is recognized, it can be removed by subtracting it from the transmitted or received signal. This technique is typically implemented digitally using a digital signal processor or software, although the technique can also be implemented with analog circuitry.

[0021] The output of the echo canceller 214 may be mixed in the noise suppressor 216 with a noise signal 218 (i.e., unpredictable noise) from the second beamformer 212 and an infotainment audio signal 220 (i.e., predictable noise) from the infotainment system 116. By mixing the near-end speech signal 213 with the noise signal 218 and / or the infotainment audio signal 220 in the noise suppressor 216, acoustic energy from one or more unwanted sound sources in the vehicle 104 may be effectively removed. Sounds playing from known audio streams in the vehicle's infotainment system 116 (e.g., music, sound effects, and dialogue from a movie audio) may be considered predictable noise and may be used as a direct input to the noise cancellation system 128 and removed or suppressed from the near-end speech signal 213. Additionally, additional unwanted and unpredictable noises captured by the embedded microphones (e.g., children screaming and background conversations) may also be used as a direct input to the noise cancellation system 128. Unwanted noise may be removed or suppressed from the near-end speech signal 213 by a noise suppressor 216 based on the noise signal 218 and the infotainment audio signal 220 before being communicated to the far-end participant as an outgoing telecommunication signal 112b. Noise suppression is a speech pre-processor that removes background noise from a captured signal.

[0022] The noise-suppressed near-end speech signal 213' may be output from the noise suppressor 216 and mixed with the processed incoming telecommunication signal 112a' from the far-end participant at the echo suppressor 222. Echo suppression, like echo cancellation, is a telephony technique for improving voice quality by preventing echo from being created or removing echo once it is already present. Echo suppressors work by detecting the voice signal traveling in one direction on a circuit and then inserting a large loss in the other direction. Typically, an echo suppressor at the far end of a circuit adds this loss when the echo suppressor detects incoming voice from the near end of the circuit. This added loss prevents the talker from hearing his or her own voice.

[0023] The output from the echo suppressor 222 may then undergo automatic gain control in an automatic gain control device (AGC) 224. The AGC 224 may provide a controlled signal amplitude at its output despite variations in the amplitude of the input signal. The average or peak output signal level is used to dynamically adjust the input and output gain to appropriate values, allowing the circuit to function well with a wider range of input signal levels. The output from the AGC 224 may then be received by an equalizer 226 to equalize the near-end speech signal. Equalization is the process of adjusting the balance between frequency components within an electronic signal. An equalizer enhances (boosts) or attenuates (cuts) the energy in a particular frequency band or "frequency range."

[0024] The output from the equalizer 226 may be sent to a loss control device 228 for loss control. The output may then pass through a comfort noise generator (CNG) 230. The CNG 230 is a module that inserts comfort noise during periods when there is no signal received. The CNG may be used in conjunction with discontinuous transmission (DTX). DTX means that the transmitter is switched off during silent periods. Thus, the background acoustic noise suddenly disappears at the receiving end (e.g., the far-end). This may be very annoying for the receiver (e.g., the far-end participant). If the silent periods are long enough, the receiver may even think that the line has been cut. To overcome these problems, a "comfort noise" may be generated at the receiving end (i.e., the far-end) every time the transmission is switched off. The comfort noise is generated by the CNG. If the comfort noise is well matched to the comfort noise of the background acoustic noise transmitted during the speech periods, the gaps between the speech periods can be filled so that the receiver does not notice the switch during the conversation. Since the noise is constantly changing, the comfort noise generator 230 may be updated periodically.

[0025] The output from the CNG 230 may then be transmitted by the telecommunications system to a far-end participant in a telecommunications exchange as the outgoing telecommunications signal 112b. By removing the noise input directly from the outgoing telecommunications signal, the user's far-end counterpart may be afforded a much higher signal-to-noise ratio, speech quality, and speech intelligibility.

[0026] Although the noise cancellation system 128 has been shown and described as improving near-end speech intelligibility at a far-end participant of a telecommunications exchange, it may be used to improve near-end speech intelligibility at the far-end of any communication exchange. For example, the noise cancellation system 128 may be used in conjunction with a virtual personal assistance (VPA) application to optimize speech recognition at the far-end (i.e., a virtual personal assistant). Thus, background (unwanted) noise may be similarly suppressed or removed from near-end speech of a communication exchange with a VPA.

[0027] FIG. 3 is a simplified exemplary flow diagram illustrating a noise cancellation method 300 for far-end telecommunications. At step 305, near-end speech may be received at the noise cancellation system 128 by a microphone array, such as the first microphone array 124. Meanwhile, the noise cancellation system 128 may receive an audio input stream from an unwanted source, such as unpredictable noise from the second microphone array 126 and / or predictable noise from the infotainment system 116, as shown at step 310. The near-end speech may be processed into an outgoing telecommunications signal 112b for reception by a far-end participant of the telecommunications exchange. Thus, at step 315, the near-end speech signal may undergo an echo cancellation operation to improve the voice quality by removing the echo once it is already present. As mentioned above, echo cancellation requires first recognizing the originally transmitted signal, which reappears with some delay in the transmitted or received signal. Once the echo is recognized, it can be removed by subtracting it from the transmitted or received signal.

[0028] The near-end speech signal may be received at a noise suppressor (step 320) along with the noise input received in step 310 and the incoming telecommunication signal for the far-end participant. During noise cancellation, noise may be removed or suppressed from the near-end speech signal, as shown in step 325. In step 330, speech intelligibility in the near-end speech signal may be restored by reducing or removing the effect of masking by external sounds. The near-end speech signal may then undergo echo suppression, as shown in step 335, using the incoming telecommunication signal. As mentioned above, echo suppression, like echo cancellation, is a telephony technology method for improving voice quality by preventing echo from being created or removing echo once it is already present. The near-end speech signal may undergo additional audio filtering in step 340 before it is transmitted as an outgoing telecommunication signal over the telecommunication network to the far-end participant (step 345). Meanwhile, the incoming telecommunication signal may be played in the vehicle cabin through a speaker (step 350).

[0029] FIG. 4 illustrates an exemplary microphone arrangement within the cabin 120 of the vehicle 104, according to one or more embodiments of the present disclosure. For example, a first microphone 124a from a first microphone array 124 for capturing near-end speech may be embedded in one or more headrests 410. Also, a second microphone 126a from a second microphone array 126 for capturing noise may be embedded in one or more headrests 410, a headliner (not shown), or the like. As illustrated, a microphone positioned toward the interior of the passenger relative to the cabin 120, such as as close as possible to the user's mouth, may minimize the reflected energy of the signal as compared to a microphone positioned toward the exterior of the passenger relative to the cabin. This is because a microphone positioned toward the exterior of the passenger relative to the cabin may receive more reflected energy from reflective surfaces 412 surrounding the cabin 120, such as glass. Minimizing the reflected energy of the near-end speech signal may increase speech intelligibility at the far end of the telecommunication. The microphone arrangement and / or location illustrated in FIG. 4 is merely exemplary. The exact location of the microphone array depends on the perimeter and interior coverage area of ​​the vehicle.

[0030] FIG. 5 illustrates an exemplary setup of a headrest-based telecommunications system for a vehicle. A first forward-facing microphone array 502 may be positioned near the front 504 of a passenger headrest 506 to receive near-end speech of the telecommunications exchange. A second rearward-facing microphone array 508 may be positioned near the rear 510 of the passenger headrest 506 to receive noise, including background speech. FIG. 6 illustrates another exemplary setup of a headrest-based telecommunications system for a vehicle. A first forward-facing microphone array 602 may be positioned near the front 604 of a passenger headrest 606 to receive near-end speech of the telecommunications exchange. A second forward-facing microphone array 608 may be positioned near the front 610 of a rear passenger headrest 612 to receive noise, including background speech. As with FIG. 4, the exact locations of the microphone arrays illustrated in FIG. 5 and FIG. 6 depend on the boundaries and the coverage area within the vehicle.

[0031] 7-10 show various plan views of sample microphone configurations for a noise cancellation system 128 (not shown) in a vehicle cabin 120, such as the vehicle 104. Similar to the microphones and microphone arrays described in connection with FIGS. 1 and 2, the various microphone arrays and / or individual microphones shown in FIGS. 7-10 may communicate with a digital signal processor 114 to function in connection with a vehicle communication system, such as an in-vehicle communication system or telecommunication system 110. For example, FIG. 7 is a plan view of a vehicle 104 showing a first sample microphone configuration, according to one or more embodiments of the present disclosure. As shown, the noise cancellation system 128 (not shown) may include at least one microphone array 710 including at least two microphones—a first microphone 710a and a second microphone 710b. The first and second microphones may be attached to an exterior surface 712 of a first headrest 714 at spaced apart locations. The first headrest 714 may be a driver's headrest.

[0032] The exterior surface 712 of the first headrest 714 may include an inwardly facing side 716 and an outwardly facing side 718. The inwardly facing side 716 is closer to the center of the vehicle cabin 120 than the outwardly facing side 718, which is closer to the side of the vehicle 104 that includes the reflective surface 412 (see FIG. 4). As shown in FIG. 7, the first and second microphones 710a, 710b may be positioned flush with the inwardly facing side 716 of the first headrest 714. The first and second microphones 710a, 710b may be spaced apart at least longitudinally with respect to the vehicle 104. Thus, the distance separating the first and second microphones may include at least a longitudinal distance X to create at least one longitudinally oriented first listening zone 720 and second listening zone 722. The longitudinal distance X between the two microphones of the microphone array 710 may indicate the direction of the incoming sound, generally a front or rear indication. Thus, the first listening zone 720 may encompass a forward area of ​​the vehicle cabin 120, such as an area including the front seat row, while the listening zone 722 may encompass an area oriented toward the rear of the first listening zone 720, such as an area including the rear passenger seats. In one embodiment, the longitudinal distance X between the first and second microphones 710a, 710b may be approximately 1 inch, although other distances between the microphones may also be used to indicate the direction of the incoming sound, generally a front or rear indication.

[0033] The digital signal processor 114 may be programmed to receive microphone signals indicative of speech from the microphone array 710 as shown in FIG. 2 and, based on the microphone signals, identify whether the speech is received from the direction of the first listening zone 720 or the direction of the second listening zone 722. For example, the digital signal processor 114 may compare the microphone signals from the first and second microphones 710a, 710b and localize the direction of the speech from either the first listening zone or the second listening zone based on the time difference of the reception of the microphone signals at each of the two microphones. Furthermore, the digital signal processor 114 may suppress or remove microphone signals indicative of speech from (the direction of) the second listening zone 722, which may be equated to unwanted or noisy background noise. On the other hand, the digital signal processor 114 may transmit microphone signals indicative of sound from (the direction of) the first listening zone 720, which may be equated to desired near-end speech, to a far-end participant of the communication exchange.

[0034] According to one embodiment, the first and second microphones 710a, 710b may be omnidirectional microphones. According to another embodiment, the first and second microphones 710a, 710b may be directional microphones with a direction in the direction of the corresponding listening zone. Thus, incoming audio may be attenuated based on the microphone's directionality such that audio from the first listening zone 720 may be transmitted to the far-end participant, while audio from the second listening zone 722 may be suppressed.

[0035] 8 is a plan view of a vehicle 104 illustrating another sample microphone configuration, according to one or more embodiments of the present disclosure. As shown, a noise cancellation system 128 (not shown) may include at least one first microphone array 810 including at least two microphones—a first microphone 810a and a second microphone 810b—attached to a bottom surface 811 of an outer surface 812 of a first headrest 814. Similar to FIG. 7, the first and second microphones 810a, 810b may be spaced apart longitudinally relative to the vehicle 104. Thus, the distance separating the first and second microphones 810a, 810b may include at least a longitudinal distance X to create at least one longitudinally oriented first listening zone 820 and second listening zone 822. As described with respect to Figure 7, the digital signal processor 114 may be programmed to receive microphone signals indicative of speech from the microphone array 810, as shown in Figure 2, and to identify, based on the microphone signals, whether speech is received from the direction of the first listening zone 820 or the direction of the second listening zone 822. Additionally, the digital signal processor 114 may suppress or remove microphone signals indicative of speech from the (direction of) the second listening zone 822, which may be equated to unwanted or noisy background noise. On the other hand, the digital signal processor 114 may transmit microphone signals indicative of speech from the (direction of) the first listening zone 820, which may be equated to desired near-end speech, to a far-end participant of the communication exchange.

[0036] 8, the first and second microphones 810a, 810b may be spaced apart laterally with respect to the vehicle 104. Thus, the distance separating the first and second microphones 810a, 810b may further include a lateral distance Y such that the first listening zone 820 includes two listening subzones oriented laterally with respect to the vehicle 104. For example, the first listening subzone 820a may encompass an area surrounding the driver's seat 824, while the second listening subzone 820b may encompass an area surrounding the passenger seat 826. The lateral distance Y between the two microphones 810a, 810b of the first microphone array 810 may indicate the direction of the incoming voice, generally left or right, so that the digital signal processor 114 may further identify, based on the microphone signals, whether the voice is received from the direction of the first listening subzone 820a or the direction of the second listening subzone 820b. Furthermore, the digital signal processor 114 may be programmed to suppress or remove microphone signals indicative of voice from (the direction of) the second listening subzone 820b, which may be equated with unwanted or noisy background noise. On the other hand, the digital signal processor 114 may transmit microphone signals indicative of voice from (the direction of) the first listening subzone 820a, which may be equated with desired near-end speech, to the far-end participants of the communication exchange.

[0037] As further shown in FIG. 8, the noise cancellation system may include a second microphone array 828 including at least two microphones—a first microphone 828a and a second microphone 828b—attached to a bottom surface 830 of a second headrest 832 laterally adjacent to the first headrest 814. The configuration of the second microphone array may closely resemble that of the first microphone array. Thus, the first and second microphones 828a, 828b of the second microphone array 828 may also be spaced apart both longitudinally and laterally to provide an additional indication of the direction of the incoming sound, generally left or right, so that the digital signal processor 114 may further identify, based on the microphone signals, whether the sound is received from the direction of the first listening subzone 820a or the second listening subzone 820b. The microphones of the first and / or second microphone array may be either omnidirectional or directional microphones.

[0038] FIG. 9 illustrates yet another sample microphone configuration similar to the three-zone configuration illustrated in FIG. 8. As illustrated, a first microphone array 910 may be attached to an inward-facing side 916 of a headrest 914, such as the microphone array illustrated in FIG. 7. Similar to FIG. 7, the first microphone array 910 may include a first microphone 910a and a second microphone 910b positioned on the inward-facing side 916 at spaced apart locations separated by a longitudinal distance to provide an indication of the direction, forward or rearward, of an incoming sound. Thus, as discussed above, the longitudinal separation of the first and second microphones 910a, 910b may create a longitudinally oriented first listening zone 920 and a second listening zone 922. A second microphone array 934 including first and second microphones 934a, 934b may be disposed on the rearview mirror assembly 936 rather than by the second headrest (as in FIG. 8) to indicate the direction of the incoming sound, left or right, so that the digital signal processor 114 may further identify whether the sound is received from the direction of the first listening subzone 920a or the direction of the second listening subzone 920b based on the microphone signals. The first and second microphones 910a, 910b of the first microphone array 910 may be omnidirectional microphones. Furthermore, the first and second microphones 934a, 934b of the second microphone array 934 may be directional microphones.

[0039] FIG. 10 is a plan view of a vehicle 1004 illustrating yet another sample microphone configuration, according to one or more embodiments of the present disclosure. As shown, the vehicle 1004 may include three rows of seats. The microphone configuration illustrated in FIG. 10 may use a combination of the various configurations described above with respect to FIGS. 7-9. For example, a first row 1040 of seats may include a first microphone array 1010 in a first headrest 1014 and a second microphone array 1028 in a second headrest 1030, as shown in FIG. 8. Thus, the microphones in each of the first and second microphone arrays 1010, 1028 may be attached to the bottom surface 1011 of the respective headrest and spaced apart both longitudinally and laterally. The lateral spacing may create a first listening zone 1020 including a first listening subzone 1020a and a second listening subzone 1020b having a lateral orientation, as described above. Additionally, longitudinal spacing may create a second listening zone 1022 behind the first listening zone 1020.

[0040] At least one headrest 1042 in the second row 1044 of seats may include a third microphone array 1046 similar to the microphone array 710 shown in FIG. 7. Thus, the microphones of the third microphone array 1046 may be attached to the inwardly facing side 1016 of the headrest 1042 and spaced apart at least longitudinally to create a third listening zone 1050 rearward of the second listening zone 1022 encompassing the third row 1052 of seats. The vehicle 1004 may include additional microphone arrays 1054 positioned generally along the centerline of the vehicle and in the ceiling or headliner (not shown) of the vehicle. These additional microphone arrays 1054 may include three or four microphones (as shown) that may be omnidirectional. All the various microphone arrays shown in FIG. 10 may form part of the noise cancellation system 128 and may cooperate with the digital signal processor 114 in a similar manner as described in connection with FIGS. 7-9. Additionally, one or more of the headrests shown in Figure 10 may further include at least one speaker 1056. The headrest mounted speaker 1056 may be used to transmit audio from a far-end participant in a communications exchange.

[0041] FIG. 11 illustrates an example ducking system 800. The ducking system 800 may include a microphone array 802 configured to capture and receive an incoming voice signal 804. The array 802 may be similar to the arrays described above and may include a plurality of small embedded microphones strategically positioned within the vehicle cabin to receive speech from a near-end participant of a telecommunications exchange (i.e., the driver or another occupant of the source vehicle). The arrangement and location of the array 802 may be arranged as described herein with respect to FIGS. 5-10, for example. The array 802 is shown in FIG. 11 as including four microphones, but may include more or fewer microphones. The incoming voice signal 804 may include human speech as well as other noise signals including music, background noise, traffic noise, and the like. Although the array 802 is shown as including two microphones, the array 802 may include more than two microphones, including a four-microphone array. The microphones may be directional microphones.

[0042] The system 800 may include a real-time speech cleaning algorithm or a beamformer 806 similar to the beamformers described above, such as the first beamformer 210 and the second beamformer 212. Although the beamformer 806 is described in this manner, this block may include other processing, such as noise suppression, echo cancellation, etc. Beamforming or spatial filtering is a signal processing technique used in sensor arrays for directional signal transmission or reception. This is achieved by combining elements in the array so that signals at certain angles experience constructive interference, while other signals experience destructive interference. Beamforming may be used on both the transmit and receive sides to achieve spatial selectivity. The improvement compared to omnidirectional receive / transmit is known as the directivity of the array. To change the directivity of the array on transmit, the beamformer controls the phase and relative amplitude of the signals at each transmitter to create patterns of constructive and destructive interference in the wavefront. On receive, information from the different sensors is combined so that the expected pattern of radiation is preferentially observed.

[0043] The beamformer 806 may output a speech signal 810. The speech signal 810 may include a human voice signal that has been removed from the incoming audio signal 804. That is, noise may be removed, leaving only human speech. This may be useful in recognizing whether the incoming audio signal includes human speech. Also, the speech signal 810 may include any type of human-generated sound, such as humming, finger snapping, clapping, etc., and is not limited to speech. Conventional methods for determining whether human speech is included often require large memory and high processing power. Furthermore, due to the need for extensive processing to determine whether human speech is present, existing systems require a latency period before the incoming signal can be confirmed as a speech signal. This may result in the beginning of a conversation not being sent to the ducking station. Furthermore, human noise, which is not speech, such as "oh," "hmm," or "shhh," may not be classified as a speech signal and may not be sent properly to the ducking station.

[0044] The beamformer relies on a spatial region of interest so that any and all audio from that region is sent immediately to the ducking station. This includes non-speech utterances, and because beamforming is being used to detect speech, the region of interest is already defined and localized, thus avoiding latency issues.

[0045] Non-human generated audio emanating from vehicle speakers, such as music and other media, may be played during vehicle use. However, during conversations, occupants may wish to lower the non-speech audio to allow easier hearing and listening. Audio ducking may enable this automatically without requiring human interaction with a volume control. In FIG. 11, these non-speech audio 812 may be manually controlled in block 814 or may be automatically ducked in block 816. The ducked speech signal may be sent to the vehicle audio system.

[0046] Figure 12 illustrates an example process 1200 for the system 800 of Figure 11. The process 1200 begins at block 1205 where the processor 114 may receive an incoming voice signal from the array 802.

[0047] At block 1210, the processor 114 may apply the beamformer 806. The beamformer 806 may spatially filter out human speech sounds.

[0048] In block 1215, the processor 114 may determine from block 1210 whether a spatial region of interest is detected, such as whether an acoustic signal from an occupant is detected; i.e., any acoustic input from the region of interest that exceeds a threshold. If so, the threshold is used to adjust the non-speech media levels. This can be visualized as a spotlight shining toward the head / mouth area of ​​each occupant. If so, the process 1200 proceeds to block 1220; if not, the process returns to block 1205.

[0049] At block 1220, the processor 114 may adjust the volume of the non-speech or non-human audio being played by the vehicle audio system. The volume may be adjusted down to a predetermined level; i.e., the volume may be reduced but still audible to the occupants; the volume may be muted completely; the playback may be paused completely; additionally or alternatively, the volume may be faded down to mute or to a predetermined level; the volume level may be predefined by the user, user selection, etc.

[0050] At block 1225, the processor 114 may monitor the beamformed audio signal to determine whether human speech is still present. If human speech is absent for a predetermined amount of time, then the conversation may be over. In this case, non-speech audio may resume playing at its normal volume level. If speech is no longer detected, the process 1200 proceeds to block 1230.

[0051] In block 1230, the processor 114 may direct playback of the non-speech audio to be adjusted. This may include directing playback of the non-speech audio to resume at a volume level previously established before ducking. Fading, muting, etc. may be a form of adjustment to playback volume, as well as adjustments during ducking.

[0052] The process 1200 may then end.

[0053] Thus, a system is disclosed for detecting human generated speech and ducking non-human speech emanating from vehicle speakers. Localized microphone and / or beamforming techniques are used so that any acoustic speech above a threshold may be deemed human generated. Thus, cumbersome signal processing to determine whether a speech is human generated or not can be avoided. Furthermore, directional microphones are not required. The beamforming process described herein can be used in conjunction with noise suppression, echo cancellation, and other signal processing to increase the viability of human generated speech.

[0054] Any one or more of the controllers or devices described herein include computer executable instructions that may be compiled or interpreted from computer programs created using a variety of programming languages ​​and / or technologies. In general, a processor (such as a microphone processor) receives instructions from, for example, a memory, a computer readable medium, etc., and executes the instructions. The processing unit includes a non-transitory computer readable storage medium capable of executing instructions of a software program. The computer readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0055] Although exemplary embodiments have been described above, it is not intended that these embodiments describe every possible form of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Moreover, features of various implementation embodiments may be combined to form further embodiments of the invention.

Claims

1. 1. A loudspeaker system in a vehicle, comprising: at least two microphones forming a microphone array; at least one loudspeaker configured to emit sounds, including non-human sounds; A processor, the processor comprising: receiving an incoming voice signal from the microphone array; applying beamforming to the incoming voice signal; determining whether human generated speech is detected within the incoming voice signal based at least in part on a volume of at least one acoustic input of the incoming voice signal exceeding a threshold, the acoustic input being associated with a particular region within the vehicle via beamforming; in response to detecting human generated speech, reducing a volume of the non-human speech produced by the loudspeaker system; a processor programmed to A loudspeaker system comprising:

2. 10. The system of claim 1, wherein the processor is programmed to determine whether the human generated voice is detected within the incoming voice signals based at least in part on a volume of an acoustic input of at least one of the incoming voice signals exceeding a threshold.

3. The system of claim 2 , wherein the acoustic input is associated with a particular area within the vehicle.

4. The system of claim 3 , wherein the acoustic input is associated with the particular region via the beamforming.

5. The system of claim 1 , wherein the microphone array is positioned adjacent to a head of an occupant within the vehicle.

6. The system of claim 1 , wherein the processor is programmed to apply at least one of noise suppression or echo cancellation to at least one incoming voice signal by the beamforming.

7. The system of claim 1 , wherein the microphone array includes four microphones.

8. 1. A non-transitory computer readable medium containing a program for a loudspeaker system for generating sound in a vehicle, the program comprising: receiving an incoming audio signal from a vehicle microphone array; applying beamforming to the incoming voice signal; determining whether human generated speech is detected within the incoming voice signal based at least in part on a volume of an acoustic input within the incoming voice signal exceeding a threshold, the acoustic input being associated with a particular region within the vehicle via the beamforming; in response to detecting human generated speech, reducing a volume of the non-human speech produced by the loudspeaker system; A non-transitory computer readable medium for executing the method of the present invention.

9. 9. The medium of claim 8, wherein determining whether human generated speech is detected within the incoming speech signal is based at least in part on a volume of an acoustic input within the incoming speech signal exceeding a threshold.

10. The medium of claim 9 , wherein the acoustic input is associated with a particular area within the vehicle.

11. The medium of claim 8 , wherein the acoustic input is associated with the particular region via the beamforming.

12. The medium of claim 8 , wherein the microphone array is positioned adjacent to a head of an occupant within the vehicle.

13. The program further comprises: The medium of claim 8 , wherein the beamforming performs at least one of noise suppression or echo cancellation on at least one audio signal.

14. 1. A method for producing sound through a loudspeaker system in a vehicle, comprising: receiving an incoming audio signal from a vehicle microphone array; applying beamforming to the incoming voice signal; determining whether human generated speech is detected within the incoming voice signal based at least in part on a volume of an acoustic input within the incoming voice signal exceeding a threshold, the acoustic input being associated with a particular region within the vehicle via the beamforming; in response to detecting human generated speech, reducing a volume of the non-human speech produced by the loudspeaker system; A method comprising:

Images