Audio Scaling

By applying spatial filtering technology to the input signal of the headphone microphone, the amplitude and phase of the enhanced audio signal are adjusted, solving the problem of the user's interest in the sound being masked by interference, and achieving a better audio listening experience.

CN117242788BActive Publication Date: 2024-11-12QUALCOMM INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280032954.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-05-10
Filing Date
2022-05-09
Publication Date
2024-11-12
Estimated Expiration
2042-05-09

AI Technical Summary

Technical Problem

When using headphones with existing portable computing devices, the audio signals captured by the external microphone that the user is interested in are easily masked by interference, making them difficult to hear.

Method used

By applying spatial filtering technology to the microphone input signal of the headphones, the amplitude and phase of the enhanced audio signal are adjusted to generate an enhanced audio signal, and the phase difference and amplitude difference are maintained when the signal is played in the earpiece, thereby maintaining the binaural sensation.

Benefits of technology

It effectively enhances the sounds that users are interested in, maintains the directionality and relative distance information of the audio source, and provides a better audio listening experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117242788B_ABST
    Figure CN117242788B_ABST
Patent Text Reader

Abstract

The device includes one or more processors configured to execute instructions to determine a first phase based on a first audio signal in the first audio signal and to determine a second phase based on a second audio signal in the second audio signal. The one or more processors are further configured to execute instructions to apply spatial filtering to a selected audio signal in the first audio signal and the second audio signal to generate an enhanced audio signal. The one or more processors are further configured to execute instructions to generate a first output signal, the first output signal comprising combining the amplitude of the enhanced audio signal with the first phase; and to generate a second output signal, the second output signal comprising combining the amplitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to an audio scaling signal.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority from commonly owned U.S. non-provisional patent application No. 17 / 316,529, filed on May 10, 2021, the contents of which are expressly incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure generally relates to performing audio scaling. Background Art

[0004] Advances in technology have led to smaller and more powerful computing devices. For example, there are currently various portable personal computing devices, including wireless phones (e.g., mobile phones and smart phones), small, lightweight and easily carried tablet computers and laptop computers. These devices can transmit voice and data packets through wireless networks. In addition, many such devices incorporate additional functions, such as digital cameras, digital video cameras, digital recorders and audio file players. In addition, such devices can process executable instructions, including software applications, such as web browser applications, which can be used to access the Internet. As such, these devices can include significant computing capabilities.

[0005] Such computing devices often incorporate the functionality of receiving audio signals from one or more microphones. For example, the audio signal may represent a user's voice captured by the microphone, external sounds captured by the microphone, or a combination thereof. The captured sounds may be played back to the user of such a device. However, some of the captured sounds that the user may be interested in listening to may be difficult to hear due to other interfering sounds. Summary of the invention

[0006] According to one implementation of the present disclosure, a device includes a memory and one or more processors. The memory is configured to store instructions. One or more processors are configured to execute instructions to determine a first phase based on a first audio signal in a first audio signal, and to determine a second phase based on a second audio signal in a second audio signal. One or more processors are also configured to execute instructions to apply spatial filtering to a selected audio signal in the first audio signal and the second audio signal to generate an enhanced audio signal. One or more processors are further configured to execute instructions to generate a first output signal, including combining the amplitude of the enhanced audio signal with the first phase. One or more processors are also configured to execute instructions to generate a second output signal, including combining the amplitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to an audio scaling signal. According to another implementation of the present disclosure, a method includes determining a first phase based on a first audio signal in a first audio signal at a device. The method also includes determining a second phase based on a second audio signal in a second audio signal at the device. The method also includes applying spatial filtering to a selected audio signal in the first audio signal and the second audio signal at the device to generate an enhanced audio signal. The method also includes generating a first output signal at the device, including combining the amplitude of the enhanced audio signal with the first phase. The method also includes generating a second output signal at the device, including combining the amplitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to audio scaling signals.

[0007] According to another implementation of the present disclosure, a non-transitory computer-readable medium includes instructions, which, when executed by one or more processors, cause the one or more processors to determine a first phase based on a first audio signal in a first audio signal, and to determine a second phase based on a second audio signal in a second audio signal. When the instructions are executed by the one or more processors, the one or more processors also apply spatial filtering to a selected audio signal in the first audio signal and the second audio signal to generate an enhanced audio signal. When the instructions are executed by the one or more processors, the one or more processors further generate a first output signal, including combining the amplitude of the enhanced audio signal with the first phase. When the instructions are executed by the one or more processors, the one or more processors also generate a second output signal, including combining the amplitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to an audio scaling signal. According to another implementation of the present disclosure, a device includes a unit for determining a first phase based on a first audio signal in a first audio signal. The device also includes a unit for determining a second phase based on a second audio signal in a second audio signal. The device also includes a unit for applying spatial filtering to a selected audio signal in the first audio signal and the second audio signal to generate an enhanced audio signal. The apparatus further comprises means for generating a first output signal comprising combining the amplitude of the enhanced audio signal with the first phase. The apparatus further comprises means for generating a second output signal comprising combining the amplitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to audio scaled signals.

[0008] Other aspects, advantages, and features of the present disclosure will become apparent after reading the entire application, including the following sections: Brief Description of the Drawings, Detailed Description, and Claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram of certain illustrative aspects of a system operable to perform audio scaling according to some examples of the present disclosure.

[0010] Figure 2 According to some examples of the present disclosure Figure 1 Schematic diagram of illustrative aspects of a signal selector and spatial filter of an illustrative system.

[0011] Figure 3 According to some examples of the present disclosure Figure 1 A schematic diagram of a specific implementation of a pair selection method that may be performed by a pair selector of an illustrative system.

[0012] Figure 4 According to some examples of the present disclosure Figure 1 A schematic diagram of illustrative aspects of the operation of a system.

[0013] Figure 5 According to some examples of the present disclosure Figure 1 A schematic diagram of illustrative aspects of an implementation of components of a system.

[0014] Figure 6 According to some examples of the present disclosure Figure 1 A schematic diagram of illustrative aspects of another implementation of components of a system.

[0015] Figure 7 According to some examples of the present disclosure Figure 1 A schematic diagram of illustrative aspects of another implementation of components of a system.

[0016] Figure 8 is a schematic diagram of an example of a vehicle operable to perform audio zooming according to some examples of the present disclosure.

[0017] Fig. 9 An example of an integrated circuit operable to perform audio scaling according to some examples of the present disclosure is shown.

[0018] Fig.10 is a schematic diagram of a first example of headphones operable to perform audio scaling according to some examples of the present disclosure.

[0019] Fig.11 is a schematic diagram of a second example of a headset, such as a virtual reality or augmented reality headset, operable to perform audio scaling in accordance with some examples of the present disclosure.

[0020] Fig.12 Some examples according to the present disclosure may be Figure 1 A schematic diagram of a specific implementation of a method for performing audio scaling performed by a system.

[0021] Fig.13 is a block diagram of a specific illustrative example of a device operable to perform audio scaling according to some examples of the present disclosure. DETAILED DESCRIPTION

[0022] An external microphone on a device such as a headset can capture external sounds that are transmitted to the user wearing the headset. Because the external microphone can also capture other interfering sounds, some of the captured sounds that the user is interested in may be difficult to hear. Therefore, the user's experience of listening to the sounds of interest may be negatively affected by the presence of interfering sounds.

[0023] Systems and methods for performing audio scaling are disclosed. In one illustrative example, an audio enhancer receives a left input signal from a microphone externally mounted to a left earpiece of a headset, and receives a right input signal from a microphone externally mounted to a right earpiece of the headset. The audio enhancer receives a user input indicating a scaling target. The audio enhancer selects an input signal from the left input signal and the right input signal based at least in part on the scaling target. The audio enhancer performs spatial filtering on the selected input signal based at least in part on the scaling target to generate an enhanced audio signal (e.g., an audio scaling signal). For example, the enhanced audio signal corresponds to an amplification (e.g., a higher gain) applied to an input signal associated with an audio source corresponding to the scaling target, an attenuation (e.g., a lower gain) applied to an input signal associated with the remaining audio sources, or both.

[0024] In some implementations, the audio enhancer modifies the enhanced audio signal for playback at each earpiece in the earpiece by adjusting the amplitude and phase of the enhanced audio signal based on the input signal from the microphone at the corresponding earpiece. In an illustrative example, the audio enhancer determines a left normalization factor and a right normalization factor corresponding to the relative difference between the amplitude of a representative left input signal in the left input signal and the amplitude of a representative right input signal of the right input signal. The audio enhancer generates a left output signal by combining the left normalized amplitude of the enhanced audio signal with the phase of one of the left input signals. The audio enhancer also generates a right output signal by combining the right normalized amplitude of the enhanced audio signal with the phase of the representative right input signal. The audio enhancer provides the left output signal to the speaker of the left earpiece and provides the right output signal to the speaker of the right earpiece.

[0025] A normalization factor is used to maintain the relative difference in amplitude of the left output signal and the right output signal similar to the relative difference between the amplitude of the representative left input signal and the amplitude of the right input signal. The same phase as the representative left input signal and the representative right input signal is used for the left output signal and the right output signal, and the phase difference between the left output signal and the right output signal is maintained. The phase difference and the amplitude difference are maintained to maintain the overall binaural feeling when the user listens to the output signal. For example, the phase difference and the amplitude difference are maintained to maintain the directionality and relative distance of the scaled audio source. If the audio source is on the right side of the user, the sound from the audio source arrives at the right microphone earlier than the left microphone (as indicated by the phase difference); and if the audio source is closer to the right ear than the left ear, the sound from the audio source captured by the right microphone is larger than the sound captured by the left microphone (as indicated by the amplitude difference).

[0026] In audio scaling techniques that do not maintain phase and amplitude differences, the original spatial auditory scene will be lost and a mono-like or stereo-like user experience will be provided. For example, audio scaling techniques that use amplification to scale an audio source may enable a user to perceive the audio source as louder, but without maintaining phase and amplitude differences, the directional information and relative distance of the audio source will be lost. For example, if a visually impaired pedestrian is using headphones to perform audio scaling on an audible "walk / don't walk" traffic signal at a noisy intersection, the pedestrian relies on the directional information and relative distance to tell whether the street ahead or the street to the left is signaled as safe to cross. In another example, if the headphone audio is scaled to the sound of an ambulance, the user relies on the directional information and relative distance to determine the direction and proximity of the ambulance.

[0027] Certain aspects of the present disclosure are described below with reference to the accompanying drawings. In the specification, common features are designated by common reference numerals. As used herein, various terms are used only for the purpose of describing a particular implementation and are not intended to limit the implementation. For example, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, some features described herein are singular in some implementations and plural in other implementations. For example, Figure 1 Depicts a system including one or more processors ( Figure 1 190 ), indicating that in some implementations, the device 102 includes a single processor 190 , while in other implementations, the device 102 includes multiple processors 190 .

[0028] As used herein, the terms "comprise," "comprises," and "comprising" may be used interchangeably with "include," "includes," or "including." Additionally, the term "wherein" may be used interchangeably with "wherein." As used herein, "exemplary" indicates examples, implementations, and / or aspects and should not be interpreted as limiting or indicating a preferred or preferred implementation. As used herein, ordinal terms (e.g., "first," "second," "third," etc.) used to modify elements (e.g., structures, components, operations, etc.) do not, by themselves, indicate any priority or order of the element relative to another element, but merely distinguish the element from another element having the same name (but using an ordinal term). As used herein, the term "set" refers to one or more specific elements, while the term "plurality" refers to multiple (e.g., two or more) specific elements.

[0029] Unless otherwise specified, "coupling" as used herein may include "communicative coupling", "electrical coupling" or "physical coupling", and may also (or alternatively) include any combination thereof. Two devices (or components) may be directly or indirectly coupled (e.g., communicatively coupled, electrically coupled or physically coupled) via one or more other devices, components, lines, buses, networks (e.g., wired networks, wireless networks or combinations thereof), etc. As an illustrative and non-limiting example, two electrically coupled devices (or components) may be included in the same device or different devices, and may be connected via electronic devices, one or more connectors or inductive coupling. In some implementations, two devices (or components) that are communicatively coupled (such as in electronic communication) may directly or indirectly (via one or more lines, buses, networks, etc.) send and receive signals (e.g., digital signals or analog signals). As used herein, "directly coupled" may include two devices that are coupled (e.g., communicatively coupled, electrically coupled or physically coupled) without an intermediate component. Unless otherwise specified, two "coupled" devices (or components) may be directly and / or indirectly coupled.

[0030] In the present disclosure, terms such as "determine", "calculate", "estimate", "shift", "adjust", etc. may be used to describe how to perform one or more operations. It should be noted that such terms should not be interpreted as limiting, and other techniques may be used to perform similar operations. In addition, as mentioned herein, "generate", "calculate", "estimate", "use", "select", "access", and "determine" may be used interchangeably. For example, "generating", "calculating", "estimating" or "determining" a parameter (or signal) may refer to actively generating, estimating, calculating or determining the parameter (or signal), or may refer to using, selecting or accessing a parameter (or signal) that has already been generated (e.g., by another component or device).

[0031] refer to Figure 1 , particular illustrative aspects of a system configured to perform audio scaling are disclosed and generally designated 100. System 100 includes a device 102. In one particular aspect, device 102 is configured to be coupled to headphones 104.

[0032] The headset 104 includes an earpiece 110 (e.g., a right earpiece), an earpiece 112 (e.g., a left earpiece), or both. In a specific example, the earpiece 110 is configured to at least partially cover one ear of the wearer of the headset 104, and the earpiece 112 is configured to at least partially cover the other ear of the wearer of the headset 104. In a specific example, the earpiece 110 is configured to be at least partially placed in one ear of the wearer of the headset 104, and the earpiece 112 is configured to be at least partially placed in the other ear of the wearer of the headset 104.

[0033] The earpiece 110 includes one or more microphones (mic) 120, such as microphone 120A, one or more additional microphones, microphone 120N or a combination thereof. One or more microphones 120 mounted on the earpiece 110 in a linear configuration are provided as an illustrative example. In other examples, one or more microphones 120 can be mounted on the earpiece 110 in any configuration (e.g., linear, partially linear, rectangular, T-shaped, S-shaped, circular, non-linear or a combination thereof). The earpiece 110 includes one or more speakers 124, such as speaker 124A. An earpiece 110 including a speaker is provided as an illustrative example. In other examples, the earpiece 110 may include more than one speaker. In a particular aspect, one or more microphones 120 are mounted on the outside of the earpiece 110, and the speaker 124A is mounted on the inside of the earpiece 110, or both. For example, the speaker 124A is mounted on the surface of the earpiece 110, and the surface is configured to be at least partially located in the ear of the wearer of the earphone 104, facing the ear of the wearer of the earphone 104, or both. In one specific example, one or more microphones 120 are mounted on a surface of the earpiece 110 that is configured to face away from the ear of the wearer of the headset 104. By way of example, one or more microphones 120 are configured to capture external sounds that can be used for noise reduction or delivered to the wearer of the headset 104. For example, one or more microphones 120 are configured to capture sounds from one or more audio sources 184. As illustrative, non-limiting examples, one or more audio sources 184 include people, animals, speakers, devices, waves, wind, leaves, vehicles, robots, machines, musical instruments, or combinations thereof. Speaker 124A is configured to output audio to the wearer of the headset 104.

[0034] Likewise, earpiece 112 comprises one or more microphones (mic) 122, such as microphone 122A, one or more additional microphones, microphone 122N or its combination.One or more microphones 122 installed on earpiece 112 with linear configuration are provided as illustrative examples.In other examples, one or more microphones 122 can be installed on earpiece 112 with any configuration.Earpiece 112 comprises one or more loudspeakers, such as, loudspeaker 124B.In a particular aspect, one or more microphones 122 are installed on the outside of earpiece 112, and loudspeaker 124B is installed on the inside of earpiece 112, or both.Loudspeaker 124A and loudspeaker 124B are shown with dotted lines, to indicate the generally invisible internal components of earphone 104 outsides.

[0035] As an illustrative example, device 102 is depicted as being external to earphone 104. In other implementations, one or more (or all) components of device 102 are integrated into earphone 104. Device 102 is configured to perform audio zooming using audio enhancer 140. Device 102 includes one or more processors 190, which include zoom target analyzer 130, audio enhancer 140, or both. In one particular aspect, one or more processors 190 are coupled to a depth sensor 132 (e.g., an ultrasonic sensor, a stereo camera, a time of flight sensor, an antenna, a position sensor, or a combination thereof). In a particular example, depth sensor 132 is integrated into device 102. In some implementations, depth sensor 132 is integrated into earphone 104 or another device external to device 102.

[0036] The zoom target analyzer 130 is configured to receive user input 171 indicating a zoom target 192 and determine a zoom direction 137, a zoom depth 139, or both of the zoom target 192 relative to the headset 104. Example 150 indicates a zoom direction 137 (e.g., "30 degrees"), a zoom depth 139 (e.g., "8 feet"), or both of the zoom target 192 starting from the center of the headset 104 in a horizontal plane. In a particular aspect, performing an audio zoom simulation moves the headset 104 from the location of the user 101 to the location of the zoom target 192. In a particular aspect, audio source 184A and audio source 184B (e.g., people sitting at a table across the room) are closer to the zoom target 192 than the user 101, while audio source 184C (e.g., another person sitting at an adjacent table) is closer to the user 101 than the zoom target 192. In one particular aspect, the simulated movement of earphones 104 from the location of user 101 to the location of zoom target 192 is perceived by user 101 as zooming closer to audio sources 184A and 184B (e.g., people sitting at a table across the room), zooming away from audio source 184C (e.g., people sitting at an adjacent table), or both.

[0037] In one particular example, audio source 184D is equidistant from user 101 and zoom target 192. In one particular aspect, the simulated movement of earphone 104 from the location of user 101 to the location of zoom target 192 is perceived by user 101 as no zoom being applied to audio source 184D.

[0038] In one particular aspect, the audio zoom corresponds to a focus applied to the zoom target 192. For example, sound from any audio source (e.g., audio sources 184B-D) outside a threshold distance 193 of the zoom target 192 is muted. In this example, the simulated movement of the headset 104 from the location of the user 101 to the location of the zoom target 192 is perceived by the user as zooming toward the audio source 184A and zooming away from the audio source 184B-D. In one particular example, the zoom direction 137 is based on a first direction in the horizontal plane, a second direction in the vertical plane, or both, of the zoom target 192 from the center of the headset 104. Similarly, in one particular example, the zoom distance 139 is based on a first distance in the horizontal plane, a second distance in the vertical plane, or both, of the zoom target 192 from the center of the headset 104.

[0039] The audio enhancer 140 includes a subband analyzer 142, which is coupled to an amplitude extractor 146 via a signal selector and spatial filter 144. The subband analyzer 142 is also coupled to a plurality of phase extractors 148 (e.g., such as phase extractor 148A and phase extractor 148B) and a plurality of amplitude extractors 158 (e.g., amplitude extractor 158A and amplitude extractor 158B). Each of the plurality of amplitude extractors 158 is coupled to a normalizer 164. For example, amplitude extractor 158A is coupled to a normalizer (norm) 164A, and amplitude extractor 158B is coupled to a normalizer 164B. Each of the amplitude extractor 146, the normalizer 164A, and the phase extractor 148A is coupled to a subband synthesizer 170 via a combiner 166A. Each of the amplitude extractor 146, the phase extractor 148B, and the normalizer 164B are coupled to the sub-band synthesizer 170 via the combiner 166B.

[0040] The subband analyzer 142 is configured to receive the input signal 125 from the headset 104 via one or more interfaces. The subband analyzer 142 is configured to generate the audio signal 155 by transforming the input signal 125 from the time domain to the frequency domain. For example, the subband analyzer 142 is configured to apply a transform (e.g., a fast Fourier transform (FFT)) to each of the input signals 125 to generate a corresponding one of the audio signals 155.

[0041] The signal selector and spatial filter 144 is configured to perform spatial filtering on the selected audio signal 155 to generate spatially filtered audio signals and output one of the spatially filtered audio signals as the audio signal 145, as shown in FIG. Figure 2 In one particular aspect, the audio signal 145 corresponds to an enhanced audio signal (e.g., a scaled audio signal in which some audio sources are amplified, other audio sources are attenuated, or both, such as described above with respect to example 150). The audio signal 145 is received by an amplitude extractor 146, which is configured to determine an amplitude 147 of the audio signal 145.

[0042] One of the audio signals 151 associated with one or more microphones 120 (e.g., the right microphone) is provided to each of the phase extractor 148A and the amplitude extractor 158A to generate a phase 161A and an amplitude 159A, respectively. In one particular aspect, the phase 161A and the amplitude 159A correspond to a representative phase and a representative amplitude of the sound received by one of the microphones 120 (e.g., the selected one of the right microphones).

[0043] One of the audio signals 153 associated with one or more microphones 122 (e.g., the left microphone) is provided to each of the phase extractor 148B and the amplitude extractor 158B to generate a phase 161B and an amplitude 159B, respectively. In a particular aspect, the phase 161B and the amplitude 159B correspond to a representative phase and a representative amplitude of the sound received by one of the microphones 122 (e.g., the selected one of the left microphones).

[0044] Normalizer 164A is configured to generate a normalization factor 165A based on amplitude 159A and amplitude 159B (e.g., normalization factor 165A=amplitude 159A / (maximum (amplitude 159A, amplitude 159B))). Normalizer 164B is configured to generate a normalization factor 165B based on amplitude 159A and amplitude 159B (e.g., normalization factor 165B=amplitude 159B / (maximum (amplitude 159A, amplitude 159B))). Combiner 166A is configured to generate an audio signal 167A based on normalization factor 165A, amplitude 147, and phase 161A. Combiner 166B is configured to generate an audio signal 167B based on normalization factor 165B, amplitude 147, and phase 161B.

[0045] Generating audio signal 167 using normalization factor 165 achieves maintaining the difference in amplitude of audio signal 167. For example, the difference (e.g., ratio) between the amplitude of audio signal 167A and the amplitude of audio signal 167B is the same as the difference (e.g., ratio) between amplitude 159A (e.g., representing sound captured by one or more microphones 120) and amplitude 159B (e.g., representing sound captured by one or more microphones 122).

[0046] Generating audio signal 167 using phase 161 achieves maintaining the difference in phase of audio signal 167. For example, audio signal 167A has phase 161A (e.g., representing sound captured by one or more microphones 120), and audio signal 167B has phase 161B (e.g., representing sound captured by one or more microphones 122).

[0047] The subband synthesizer 170 is configured to generate the output signal 135 by transforming the audio signal 167 from the frequency domain to the time domain. For example, the subband analyzer 142 is configured to apply a transform (e.g., an inverse FFT) to each of the audio signals 167 to generate a corresponding one of the output signals 135. In one particular aspect, the audio enhancer 140 is configured to provide the output signal 135 to one or more speakers 124 of the headset 104.

[0048] In some implementations, the device 102 corresponds to or is included in one or more types of devices. In one illustrative example, the one or more processors 190 are integrated into the headset 104, such as further referenced Fig.10 In other examples, such as reference Fig.11 As described, one or more processors 190 are integrated into a virtual reality headset or an augmented reality headset. In another illustrative example, one or more processors 190 are integrated into a vehicle that also includes one or more microphones 120, one or more microphones 122, or a combination thereof, such as reference 1 Figure 8 Further described.

[0049] During operation, the user 101 wears the headset 104. The one or more microphones 120 and the one or more microphones 122 of the headset 104 capture sounds from the one or more audio sources 184. The zoom target analyzer 130 receives a user input 171 from the user 101. The user input 171 includes information indicating how to perform audio zooming. In various implementations, the user input 171 may include or indicate a selection of a particular target (e.g., an audio source 184, a location, or both), a selection to adjust the audio in a manner that simulates moving the headset 104, or a combination thereof. For example, the user input 171 may include a user selection of a particular target and a zoom depth 139 that indicates how close the headset 104 should be perceived to be located to the particular target (e.g., 2 feet).

[0050] In one particular aspect, the user input 171 includes (or indicates) an audio input, an option selection, a graphical user interface (GUI) input, a button activation / deactivation, a swipe input, a touch screen input, a user click detected via a touch sensor of the headset 104, a movement of the headset 104 detected by a motion sensor of the headset 104, a keyboard input, a mouse input, a touchpad input, a camera input, a user gesture input, or a combination thereof. For example, the user input 171 indicates an audio source (e.g., zoom to "samida", "guitar", or "bird"), a zoom depth 139 (e.g., zoom "10 feet"), a zoom direction 137 (e.g., zoom "forward", "inward", "outward", "right"), a location of a zoom target 192 (e.g., a specific area in the sound field), or a combination thereof. As an illustrative example, the user input 171 includes a user click detected via a touch sensor of the headset 104 and corresponds to a zoom depth 139 (e.g., "zoom in 2 feet") and a zoom direction 137 (e.g., "forward"). In another example, the GUI depicts a sound field, and the user input 171 includes a GUI input indicating a selection of a particular area of ​​the sound field.

[0051] In some implementations, the zoom target analyzer 130 detects the specific target by performing image analysis on the camera input, sound analysis on the audio input, position detection of a device associated with the specific target, or a combination thereof in response to determining that the user input 171 indicates the specific target (e.g., "samida", "guitar", or "stage"). In one specific implementation, the zoom target analyzer 130 designates the specific target or a position relative to the specific target (e.g., "closer" or "halfway") as the zoom target 192.

[0052] In one particular aspect, the zoom target analyzer 130 uses one or more position analysis techniques (e.g., image analysis, audio analysis, device position analysis, or a combination thereof) to determine the zoom direction 137, the zoom depth 139, or both from the headset 104 to the zoom target 192. In one example, the zoom target analyzer 130 receives sensor data 141 from a depth sensor 132 (e.g., an ultrasonic sensor, a stereo camera, an image sensor, a time-of-flight sensor, an antenna, a position sensor, or a combination thereof) and determines the zoom direction 137, the zoom depth 139, or both of the zoom target 192 based on the sensor data 141. By way of example, in one particular example, the depth sensor 132 corresponds to an image sensor, the sensor data 141 corresponds to image data, and the zoom target analyzer 130 performs image recognition on the sensor data 141 to determine the zoom direction 137, the zoom depth 139, or both to the zoom target 192. In another particular example, the depth sensor 132 corresponds to a position sensor, and the sensor data 141 includes position data indicating the position of the zoom target 192. Zoom target analyzer 130 determines zoom direction 137, zoom depth 139, or both based on the position of zoom target 192. For example, zoom target analyzer 130 determines zoom direction 137, zoom depth 139, or both based on a comparison of the position of zoom target 192 with the position, orientation, or both of headset 104.

[0053] In one particular aspect, user input 171 indicates zoom direction 137, zoom depth 139, or both. Zoom target analyzer 130, in response to determining that user input 171 (e.g., "zoom in") indicates zoom direction 137 (e.g., "forward in the direction the headset is facing" or "0 degrees"), zoom depth 139 (e.g., a default value such as 2 feet), or both, specifies zoom target 192 as corresponding to zoom direction 137, zoom depth 139, or both.

[0054] The audio enhancer 140 receives the zoom direction 137, the zoom depth 139, or both from the zoom target analyzer 130. The audio enhancer 140 also receives an input signal 121 from the earpiece 110, an input signal 123 from the earpiece 112, or a combination thereof.

[0055] The subband analyzer 142 generates the audio signal 151 by applying a transform (e.g., FFT) to the input signal 121. For example, the subband analyzer 142 generates the audio signal 151A by applying a transform to the input signal 121A received from the microphone 120A. By way of example, the input signal 121A corresponds to a time domain signal, which is converted to a frequency domain signal to produce the audio signal 151A. As another example, the subband analyzer 142 generates the audio signal 151N by applying a transform to the input signal 121N received from the microphone 120N. Similarly, the subband analyzer 142 generates the audio signal 153 by applying a transform (e.g., FFT) to the input signal 123. In one particular aspect, each of the audio signals 155 includes frequency subband information.

[0056] The subband analyzer 142 provides the audio signal 155 to the signal selector and spatial filter 144. The signal selector and spatial filter 144 processes the audio signal 155 (e.g., performs spatial filtering and signal selection on the audio signal 155) based at least in part on the zoom direction 137, the zoom depth 139, the position information of the one or more audio sources 184, the configuration of the one or more microphones 120, the configuration of the one or more microphones 122, or a combination thereof to output the audio signal 145, as shown in FIG. Figure 2 In one particular aspect, the audio signal 145 corresponds to an enhanced audio signal (e.g., a scaled audio signal). The signal selector and spatial filter 144 provides the audio signal 145 to an amplitude extractor 146. The amplitude extractor 146 outputs an amplitude 147 of the audio signal 145 (e.g., the scaled audio signal) to each of the combiner 166A and the combiner 166B.

[0057] In one particular aspect, the audio signal is represented in the frequency domain by X(jω)=|X(jω)|e^(j∠X(jω)), where X(jω) corresponds to the frequency response, |X(jω)| corresponds to the signal amplitude, and ∠X(jω) corresponds to the signal phase. In one particular aspect, each audio signal in the audio signals 155 includes amplitude and phase information for each frequency subband in the plurality of frequency subbands.

[0058] In addition, the subband analyzer 142 provides one of the audio signals 151 corresponding to the earpiece 110 to each of the phase extractor 148A and the amplitude extractor 158A, and provides one of the audio signals 153 corresponding to the earpiece 112 to each of the phase extractor 148B and the amplitude extractor 158B. For example, the subband analyzer 142 provides the audio signal 151A to each of the phase extractor 148A and the amplitude extractor 158A, and provides the audio signal 153A to each of the phase extractor 148B and the amplitude extractor 158B. In other examples, the subband analyzer 142 may alternatively provide another audio signal 151 corresponding to another microphone 120 to each of the phase extractor 148A and the amplitude extractor 158A, and provide another audio signal 153 corresponding to another microphone 122 to each of the phase extractor 148B and the amplitude extractor 158B.

[0059] The phase extractor 148A determines the phase 161A of the audio signal 151A (or another representative audio signal 151) and provides the phase 161A to the combiner 166A. The phase extractor 148B determines the phase 161B of the audio signal 153A (or another representative audio signal 153) and provides the phase 161B to the combiner 166B. In one particular aspect, the phase 161A is indicated by a first phase value, and each of the first phase values ​​indicates the phase of a corresponding frequency subband of the audio signal 151A (e.g., the representative audio signal 151). In one particular aspect, the phase 161B is indicated by a second phase value, and each of the second phase values ​​indicates the phase of a corresponding frequency subband of the audio signal 153A (e.g., the representative audio signal 153).

[0060] The amplitude extractor 158A determines the amplitude 159A of the audio signal 151A (e.g., the representative audio signal 151) and provides the amplitude 159A to each of the normalizer 164A and the normalizer 164B. The amplitude extractor 158B determines the amplitude 159B of the audio signal 153A (e.g., the representative audio signal 153) and provides the amplitude 159B to each of the normalizer 164A and the normalizer 164B.

[0061] Normalizer 164A generates a normalization factor 165A based on amplitude 159A and amplitude 159B (e.g., normalization factor 165A=amplitude 159A / (maximum value (amplitude 159A, amplitude 159B))), and provides normalization factor 165A to combiner 166A. Normalizer 164B is configured to generate a normalization factor 165B based on amplitude 159A and amplitude 159B (e.g., normalization factor 165B=amplitude 159B / (maximum value (amplitude 159A, amplitude 159B))), and provides normalization factor 165B to combiner 166B.

[0062] In one particular aspect, the amplitude 159A is indicated by first amplitude values, and each of the first amplitude values ​​indicates an amplitude of a corresponding frequency subband of the audio signal 151A (e.g., the representative audio signal 151). In this regard, the normalization factor 165A is indicated by first normalization factor values, and each of the first normalization factor values ​​indicates a normalization factor for a corresponding frequency subband of the audio signal 151A (e.g., the representative audio signal 151).

[0063] Similarly, in one particular aspect, the amplitude 159B is indicated by second amplitude values, and each of the second amplitude values ​​indicates an amplitude of a corresponding frequency subband of the audio signal 153A (e.g., the representative audio signal 153). In this regard, the normalization factor 165B is indicated by second normalization factor values, and each of the second normalization factor values ​​indicates a normalization factor of a corresponding frequency subband of the audio signal 153A (e.g., the representative audio signal 153).

[0064] Combiner 166A generates audio signal 167A based on normalization factor 165A, amplitude 147, and phase 161A. For example, the amplitude of audio signal 167A is represented by amplitude values, each of which indicates the amplitude of a corresponding frequency subband of audio signal 167A. By way of example, each of the first normalization factor value of normalization factor 165A and the first amplitude value of amplitude 147 corresponds to the same specific frequency subband. Combiner 166A determines the amplitude value corresponding to the specific frequency subband of audio signal 167A by applying the first normalization factor value to the first amplitude value. Similarly, combiner 166B generates audio signal 167B based on normalization factor 165B, amplitude 147, and phase 161B.

[0065] In one particular aspect, normalization factor 165A is applied to amplitude 147, and normalization factor 165B is applied to amplitude 147 to keep the relative amplitude difference between audio signal 167A and audio signal 167B the same (or similar) as the relative amplitude difference between audio signal 151A (representative of audio received by one or more microphones 120) and audio signal 153A (representative of audio received by one or more microphones 122). Phases 161A and 161B are applied to keep the relative phase difference between audio signal 167A and audio signal 167B the same (or similar) as the relative phase difference between audio signal 151A (representative of audio received by one or more microphones 120) and audio signal 153A (representative of audio received by one or more microphones 122), respectively.

[0066] The subband synthesizer 170 generates an output signal 135 based on the audio signal 167A and the audio signal 167B. For example, the subband synthesizer 170 generates the output signal 131 by applying a transform (e.g., an inverse FFT (iFFT)) to the audio signal 167A, and generates the output signal 133 by applying a transform (e.g., an iFFT) to the audio signal 167B. For example, the subband synthesizer 170 converts the audio signal 167A and the audio signal 167B from the frequency domain to the time domain to generate the output signal 131 and the output signal 133, respectively. In one particular aspect, the subband synthesizer 170 outputs the output signal 135 to the earphone 104. For example, the subband synthesizer 170 provides the output signal 131 to the speaker 124A of the earpiece 110, and provides the output signal 133 to the speaker 124B of the earpiece 112. The output signal 135 corresponds to an audio scaled signal (e.g., a binaural audio scaled signal).

[0067] The system 100 achieves providing audio zooming while preserving an overall binaural perception for the user 101 listening to the output signal 135. For example, the overall binaural perception is preserved by preserving the phase difference and amplitude difference between the output signal 131 output by the speaker 124A and the output signal 133 output by the speaker 124B. The phase difference is preserved by generating the output signal 131 based on the phase 161A of the audio signal 151A (e.g., a representative right input signal) and generating the output signal 133 based on the phase 161B of the audio signal 153A (e.g., a representative left input signal). The amplitude difference is preserved by generating the output signal 131 based on the normalization factor 165A and by generating the output signal 133 based on the normalization factor 165B. Thus, directional information and relative distance are preserved. For example, if a visually impaired pedestrian is performing audio zooming on an audible "walk / don't walk" traffic signal using headphones at a noisy intersection, the pedestrian can perceive the directionality and relative distance to distinguish whether the street ahead or the street to the left is signaled as safe to cross. In another example, if the headphone audio is zoomed to the sound of an ambulance, the user can sense the directionality and relative distance to determine the direction and proximity of the ambulance. In one particular aspect, extracting the phase and amplitude of the selected signal and applying the phase and amplitude to preserve the directionality and relative distance is less computationally expensive than applying a head-related impulse response (HRIR) or a head-related transfer function (HRTF), thereby enabling the processor 190 to generate binaural signals more efficiently, compared to using conventional techniques that require more processing resources, higher power consumption, higher latency, or a combination thereof.

[0068] Although one or more microphones 120, one or more microphones 122, speakers 124A, and speakers 124B are shown as being coupled to the headset 104, in other implementations, one or more microphones 120, one or more microphones 122, speakers 124A, speakers 124B, or a combination thereof may be independent of the headset. In some implementations, the input signal 125 corresponds to a playback file. For example, the audio enhancer 140 decodes the audio data of the playback file to generate the input signal 125 (e.g., input signal 121 and input signal 123). In some implementations, the input signal 125 corresponds to received streaming data. For example, a modem coupled to the one or more processors 190 provides audio data to the one or more processors 190 based on the received streaming data, and the one or more processors 190 decode the audio data to generate the input signal 125.

[0069] In one particular example, the audio data includes location information indicating the location of a source (e.g., one or more audio sources 184) of each of the input signals 125. In one particular aspect, the audio data includes a multi-channel audio representation corresponding to the ambient sound data. For example, the multi-channel audio representation indicates configuration information of microphones (e.g., actual microphones or simulated microphones) that are perceived to have captured the input signals 125. As shown in FIG. Figure 2 As described, the signal selector and spatial filter 144 generates the audio signal 145 based on the zoom direction 137, the zoom depth 139, the position information, the configuration information, or a combination thereof.

[0070] See also Figure 2 , shows a schematic diagram 200 of illustrative aspects of the signal selector and spatial filter 144 and a pair selection example 250. The signal selector and spatial filter 144 includes a pair selector 202 coupled to a signal selector 206 via one or more spatial filters 204 (e.g., one or more adaptive beamformers).

[0071] The pair selector 202 is configured to select a pair of audio signals 155 for a corresponding spatial filter 204 based on the zoom direction 137, the zoom depth 139, the position information 207 of the one or more audio sources 184, the microphone configuration 203 of the one or more microphones 120 and the one or more microphones 122, or a combination thereof. The position information 207 indicates a position (e.g., location) of each of the one or more audio sources 184. For example, the position information 207 indicates that the audio source 184A has a first position (e.g., a first direction and a first distance) relative to the position of the earphone 104, and the audio source 184B has a second position (e.g., a second direction and a second distance) relative to the position of the earphone 104. The microphone configuration 203 indicates a first configuration of the one or more microphones 120 (e.g., a front-to-back linear arrangement of the right earpiece) and a second configuration of the one or more microphones 122 (e.g., a front-to-back linear arrangement of the left earpiece).

[0072] In a particular implementation, the pair selector 202 may access selection mapping data that maps the zoom direction 137, the zoom depth 139, the position information 207, the microphone configuration 203, or a combination thereof to a particular microphone pair. In the pair selection example 250, the selection mapping data indicates that the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof is mapped to the microphone pair 220 and the microphone pair 222. In a particular aspect, the microphone pair 220 includes the microphone 120A (e.g., the front-most microphone) of the one or more microphones 120 and the microphone 122A (e.g., the front-most microphone) of the one or more microphones 122. In a particular aspect, the microphone pair 222 includes the microphone 122A (e.g., the front-most microphone) of the one or more microphones 122 and the microphone 122N (e.g., the rear-most microphone) of the one or more microphones 122.

[0073] In one particular aspect, the selection mapping data is based on default data, user input, configuration settings, or a combination thereof. In one particular aspect, the audio enhancer 140 receives the selection mapping data from a second device external to the device 102, retrieves the selection mapping data from a memory of the device 102, or both. The pair selector 202 provides the audio signals 211A and 211B corresponding to the microphone pair 220 to the spatial filter 204A (e.g., an adaptive beamformer), and provides the audio signals 213A and 213B corresponding to the microphone pair 222 to the spatial filter 204B (e.g., an adaptive beamformer).

[0074] In pair selection example 250, microphone pair 220 includes microphone 120A and microphone 122A. Pair selector 202 provides audio signal 151A (corresponding to microphone 120A) as audio signal 211A, and audio signal 153A (corresponding to microphone 122A) as audio signal 211B to spatial filter 204A. Similarly, microphone pair 222 includes microphone 122A and microphone 122N. Pair selector 202 provides audio signal 153A (corresponding to microphone 122A) as audio signal 213A, and audio signal 153N (corresponding to microphone 122N) as audio signal 213B to spatial filter 204B.

[0075] The spatial filter 204 applies spatial filtering (e.g., adaptive beamforming) to the selected audio signals (e.g., audio signal 211A, audio signal 211B, audio signal 213A, and audio signal 213B) to generate an enhanced audio signal (e.g., an audio scaling signal). In a specific embodiment, the spatial filter 204A applies a first gain to the audio signal 211A to generate a first gain adjustment signal, and applies a second gain to the audio signal 211B to generate a second gain adjustment signal. The spatial filter 204A combines the first gain adjustment signal and the second gain adjustment signal to generate an audio signal 205A (e.g., an enhanced audio signal). Similarly, the spatial filter 204B applies a third gain to the audio signal 213A to generate a third gain adjustment signal, and applies a fourth gain to the audio signal 213B to generate a fourth gain adjustment signal. The spatial filter 204B combines the third gain adjustment signal and the fourth gain adjustment signal to generate an audio signal 205B (e.g., an enhanced audio signal).

[0076] In a specific implementation, the spatial filter 204 applies spatial filtering with correction for overhead shadow effects. For example, the spatial filter 204A determines the first gain, the second gain, or both based on the size of the user 101 head, the movement of the user 101 head, or both. As another example, the spatial filter 204B determines the third gain, the fourth gain, or both based on the size of the user 101 head, the movement of the user 101 head, or both. In a specific example, a single spatial filter in the spatial filter 204A or the spatial filter 204B applies spatial filtering with correction for overhead shadow effects.

[0077] The spatial filter 204 applies spatial filtering based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. For example, the spatial filter 204A determines the first gain and the second gain based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. For example, the spatial filter 204A identifies one of the audio signal 211A or the audio signal 211B as corresponding to a microphone closer to the zoom target 192 based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. During the generation of the audio signal 205A, the spatial filter 204A applies a higher gain to the identified audio signal, a lower gain to the remaining audio signal, or both. Similarly, the spatial filter 204B determines the third gain and the fourth gain based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. For example, the spatial filter 204B identifies one of the audio signal 213A or the audio signal 213B as corresponding to a microphone closer to the zoom target 192 based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. In generating the audio signal 205B, the spatial filter 204B applies amplification (e.g., higher gain) to the identified audio signal, applies attenuation (e.g., lower gain) to the remaining audio signal, or both.

[0078] In one particular implementation, the signal selector and spatial filter 144 applies spatial filtering based on the zoom direction 137 without relying on receiving the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. For example, the selector 202 and the spatial filter 204 generate audio signals 205 corresponding to the zoom direction 137 and corresponding to various values ​​of the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof, and the signal selector 206 selects one of the audio signals 205 as the audio signal 145. In one particular aspect, selecting various values ​​of the zoom depth 139 corresponds to performing automatic zooming, as described with reference to FIG. Figure 3 Further described.

[0079] In one specific implementation, the signal selector and spatial filter 144 applies spatial filtering based on the zoom depth 139 without relying on receiving the zoom direction 137, the microphone configuration 203, the position information 207, or a combination thereof. For example, the selector 202 and the spatial filter 204 generate audio signals 205 corresponding to the zoom depth 139 and corresponding to various values ​​of the zoom direction 137, the microphone configuration 203, the position information 207, or a combination thereof, and the signal selector 206 selects one of the audio signals 205 as the audio signal 145.

[0080] In one specific implementation, the signal selector and spatial filter 144 applies spatial filtering based on the microphone configuration 203 without relying on receiving the zoom direction 137, the zoom depth 139, the position information 207, or a combination thereof. For example, the selector 202 and the spatial filter 204 generate audio signals 205 corresponding to the microphone configuration 203 and corresponding to various values ​​of the zoom direction 137, the zoom depth 139, the position information 207, or a combination thereof, and the signal selector 206 selects one of the audio signals 205 as the audio signal 145.

[0081] In one specific implementation, the signal selector and spatial filter 144 applies spatial filtering based on the position information 207, without depending on the received zoom direction 137, zoom depth 139, microphone configuration 203, or a combination thereof. For example, the selector 202 and the spatial filter 204 generate audio signals 205 corresponding to the position information 207 and corresponding to various values ​​of the zoom direction 137, zoom depth 139, microphone configuration 203, or a combination thereof, and the signal selector 206 selects one of the audio signals 205 as the audio signal 145.

[0082] In one particular implementation, the signal selector and spatial filter 144 applies spatial filtering independent of the receiving microphone configuration 203 because the selector 202 and spatial filter 204 are configured to generate the audio signal 205 for a single microphone configuration (eg, the default headset microphone configuration 203).

[0083] In the pair selection example 250, the audio signal 211A corresponds to the microphone 120A, and the audio signal 211B corresponds to the microphone 122A. The spatial filter 204A determines that the zoom target 192 is closer to the microphone 122A than the microphone 120A based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, or a combination thereof. In one specific implementation, the spatial filter 204A applies a second gain to the audio signal 211B (corresponding to the microphone 122A) in response to determining that the zoom target 192 is closer to the microphone 122A than the microphone 120A, the second gain being higher than the first gain applied to the audio signal 211A (corresponding to the microphone 120A) to generate the audio signal 205A (e.g., an audio zoom signal).

[0084] Similarly, in the pair selection example 250, the audio signal 213A corresponds to the microphone 122A, and the audio signal 213B corresponds to the microphone 122N. The spatial filter 204B determines that the zoom target 192 is closer to the microphone 122A than the microphone 120N based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, or a combination thereof. In one specific implementation, the spatial filter 204B applies a third gain to the audio signal 213A (corresponding to the microphone 122A) in response to determining that the zoom target 192 is closer to the microphone 122A than the microphone 120N, the third gain being higher than the fourth gain applied to the audio signal 213B (corresponding to the microphone 122N) to generate the audio signal 205B (e.g., an audio zoom signal).

[0085] The signal selector 206 receives the audio signal 205A from the spatial filter 204A and the audio signal 205B from the spatial filter 204B. The signal selector 206 selects one of the audio signal 205A or the audio signal 205B to output as the audio signal 145. In a specific implementation, the signal selector 206 selects one of the audio signal 205A or the audio signal 205B corresponding to the lower energy to output as the audio signal 145. For example, the signal selector 206 determines a first energy of the audio signal 205A and a second energy of the audio signal 205B. In response to determining that the first energy is less than or equal to the second energy, the signal selector 206 outputs the audio signal 205A as the audio signal 145. Alternatively, in response to determining that the first energy is greater than the second energy, the signal selector 206 outputs the audio signal 205B as the audio signal 145. In one particular aspect, the selected one of the audio signal 205A or the audio signal 205B corresponding to the lower energy is less interfered by audio sources other than the zoom target 192 (e.g., the audio source 184B). Therefore, the audio signal 145 corresponds to an enhanced audio signal (e.g., an audio zoom signal) that amplifies the sound of audio sources closer to the zoom target 192, attenuates the sound of audio sources farther from the zoom target 192, or both.

[0086] See also Figure 3 , showing a specific implementation of the selected method 300 and an automatic scaling example 350. In one particular aspect, one or more operations of the method 300 are performed by the spatial filter 204A, the spatial filter 204B, the signal selector and Figure 1 The method is performed by at least one of the spatial filter 144, the audio enhancer 140, the processor 190, the device 102, the system 100, or a combination thereof.

[0087] In the automatic scaling example 350, the signal selector and spatial filter 144 generates the audio signal 145 (e.g., performs audio scaling) independently of the received scaling depth 139. By way of example, the signal selector and spatial filter 144 performs the method 300 to iteratively select microphone pairs corresponding to various scaling depths, perform spatial filtering on the selected microphone pairs to generate audio enhancement signals, and select one of the audio enhancement signals as the audio signal 145.

[0088] The method 300 includes scaling in a scaling direction 137 using a far field assumption at 302. For example, the signal selector and spatial filter 144 selects a scaling depth 339A (eg, an initial scaling depth, a default value, or both) corresponding to the far field assumption.

[0089] The method 300 also includes reducing the zoom depth by changing the direction of arrival (DOA) corresponding to the zoom depth. For example, Figure 2 The signal selector and spatial filter 144 in the zoom depth 339A reduces the zoom depth from the zoom depth 339A to the zoom depth 339B by changing the DOA from the first set of DOAs corresponding to the zoom depth 339A to the second set of DOAs corresponding to the zoom depth 339B. In a specific aspect, the pair selector 202 selects the microphone pair 220 and the microphone pair 222 based at least in part on the zoom depth 339B. Each of the spatial filter 204A and the spatial filter 204B performs spatial filtering (e.g., beamforming) based on the second set of DOAs corresponding to the zoom depth 339B. In an illustrative example, the spatial filter 204A determines that the audio signal 211A corresponds to the first microphone and the audio signal 211B corresponds to the second microphone. In response to determining that the first microphone is closer to the zoom target 192 than the second microphone is to the zoom target 192, the spatial filter 204A performs spatial filtering to increase the gain of the audio signal 211A, reduce the gain of the audio signal 211B, or both, to generate the audio signal 205A. Similarly, the spatial filter 204B performs spatial filtering based on the second set of DOAs to generate an audio signal 205B.

[0090] Method 300 also includes determining whether a suitable depth has been found at 306. In one illustrative example, Figure 2 The signal selector and spatial filter 144 in the embodiment of the present invention determines whether the zoom depth 339B is appropriate based on the comparison of the audio signal 205A and the audio signal 205B. For example, the signal selector and spatial filter 144 determines that the zoom depth 339B is appropriate in response to determining that the difference between the audio signal 205A and the audio signal 205B satisfies (e.g., is greater than) a zoom threshold. Alternatively, the signal selector and spatial filter 144 determines that the zoom depth 339B is inappropriate in response to determining that the difference between the audio signal 205A and the audio signal 205B fails to satisfy (e.g., is less than or equal to) the zoom threshold.

[0091] Method 300 includes updating the steering vector at 310 in response to determining at 306 that a suitable depth has been found. For example, Figure 2 In response to determining that the zoom depth 339B is appropriate, the signal selector and spatial filter 144 in selects the zoom depth 339B as the zoom depth 139 and provides the audio signal 205A and the audio signal 205B to Figure 2 The signal selector 206 in . The method 300 ends at 312. The signal selector and the spatial filter 144, the audio enhancer 140, or both may perform one or more additional operations after the method 300 ends.

[0092] The method 300 includes, in response to determining at 306 that an appropriate depth was not found, determining whether the zoom depth 339B corresponds to a very near field at 308. For example, the signal selector and spatial filter 144 determines that the zoom depth 339B corresponds to a very near field in response to determining that the zoom depth 339B is less than or equal to a depth threshold, and the method 300 ends at 312. Alternatively, the signal selector and spatial filter 144 determines that the zoom depth 339B does not correspond to a very near field in response to determining that the zoom depth 339B is greater than the depth threshold, and the method 300 proceeds to 304 to select another zoom depth for analysis.

[0093] In some implementations, the audio enhancer 140 generates audio signals (e.g., enhanced audio signals) corresponding to various zoom depths, and selects one of the audio signals as the audio signal 145 based on a comparison of the energies of the audio signals. For example, the audio enhancer 140 generates a first version of the audio signal 145 corresponding to the zoom depth 339A as the zoom depth 139, as shown in FIG. Figure 2 For example, the audio enhancer 140 performs spatial filtering based on the first DOA set corresponding to the zoom depth 339A to generate a first version of the audio signal 145. The audio enhancer 140 generates a second version of the audio signal 145 corresponding to the zoom depth 339B as the zoom depth 139, as shown in FIG. Figure 2 For example, the audio enhancer 140 performs spatial filtering based on the second set of DOAs corresponding to the zoom depth 339B to generate a second version of the audio signal 145 .

[0094] The audio enhancer 140 selects the first version of the audio signal 145 as the audio signal 145 and selects the zoom depth 339A as the zoom depth 139 based on determining that the first energy of the first version of the audio signal 145 is less than or equal to the second energy of the second version of the audio signal 145. Alternatively, the audio enhancer 140 selects the second version of the audio signal 145 as the audio signal 145 and selects the zoom depth 339B as the zoom depth 139 based on determining that the first energy is greater than the second energy. In one particular aspect, the various zoom depths are based on default data, configuration settings, user input, or a combination thereof.

[0095] Thus, the method 300 enables the signal selector and spatial filter 144 to perform automatic zooming independent of receiving the zoom depth 139. Alternatively, the zoom depth 139 is based on the sensor data 141 received from the depth sensor 132, and the method 300 can implement fine-tuning of the zoom depth 139. In the audio zoom example 350A, the zoom direction 137 is shown as corresponding to a specific value in the horizontal plane (e.g., "directly ahead" or "0 degrees") and a specific value in the vertical plane (e.g., "directly ahead" or "0 degrees"). In other examples, the zoom direction 137 can correspond to any value in the horizontal plane (e.g., greater than or equal to 0 and less than 360 degrees) and any value in the vertical plane (e.g., greater than or equal to 0 and less than 360 degrees).

[0096] See also Figure 4 , showing Figure 1 4 is a schematic diagram of an illustrative aspect of the operation of the system 100. The user 101 is listening to the audio from the audio source 184A, the audio from the audio source 184B, and the background noise. The user 101 activates the audio zoom of the headset 104.

[0097] In one particular implementation, the zoom target analyzer 130 determines the zoom direction 137, the zoom depth 139, or both based on the user input 171, as shown in FIG. Figure 1 As described. In a particular example, the user input 171 includes a calendar event indicating that the user 101 plans to meet with a first person (e.g., "Bohdan Mustafa") and a second person (e.g., "Joanna Sikke") during a particular time period (e.g., "June 22, 2021, 2-3 PM"). If the user 101 is detected to be looking at the first person (e.g., audio source 184A) or the second person (e.g., audio source 184B) during the particular time period, the audio enhancer 140 designates the person as the zoom target 192. In a particular example, the user input 171 includes movement of the headset 104, and the zoom target analyzer 130 outputs the direction in which the headset 104 is facing (e.g., in a horizontal plane, in a vertical plane, or both) as the zoom direction 137. The signal selector and spatial filter 144 performs automatic zooming based on the zoom direction 137, as described in reference to FIG. Figure 3As described, the zoom direction 137 corresponds to the direction the user 101 is facing. In a specific example, the user input 171 includes clicking a touch sensor, a button, a dial, etc., and the zoom target analyzer 130 outputs a zoom depth 139 corresponding to the user input 171. For example, one click corresponds to a first zoom depth, and two clicks correspond to a second zoom depth. When audio zoom is activated, the user 101 looks at the audio source 184A (e.g., "Bohdan Mustafa") during the time range 402, and looks at the audio source 184B (e.g., "Joanna Sikke") during the time range 404. During the time range 402, the audio source 184A (e.g., "Bohdan Mustafa") corresponds to the zoom target 192, and the audio enhancer 140 generates the output signal 135 based on the zoom target 192. During the time range 404 , the audio source 184B (eg, “Joanna Sikke”) corresponds to the zoom target 192 , and the audio enhancer 140 generates the output signal 135 based on the zoom target 192 .

[0098] Graph 450 shows an example of relative signal strength, energy, or perceived prevalence of various audio sources (e.g., audio source 184A, audio source 184B, and one or more additional audio sources) in the combined audio signal 125. The horizontal axis represents time, and the vertical axis indicates the portion of the signal energy that can be contributed by each of the plurality of audio sources, wherein a first diagonal fill pattern corresponds to audio source 184A, a second diagonal fill pattern corresponds to audio source 184B, and a horizontal fill pattern corresponds to background noise from one or more additional audio sources. Graph 452 shows an example of relative signal energy of various audio sources in the combined output signal 135.

[0099] As shown, each of the audio source 184A, the audio source 184B, and the background noise spans the vertical range of the graph 450, indicating that none of the audio source 184A, the audio source 184B, or the background noise is preferentially enhanced or attenuated when received by the one or more microphones 120 and the one or more microphones 122 and input to the audio enhancer 140. In contrast, the graph 452 shows that in the time range 402, the audio source 184A spans the entire vertical range, but the span of the audio source 184B and the background noise is reduced to a relatively small portion of the vertical range; and in the time range 404, the audio source 184B spans the entire vertical range, while the span of the audio source 184A and the background noise is reduced to a relatively small portion of the vertical range. Therefore, when the user 101 looks in the direction of the audio source, when the user 101 selects the audio source for audio zoom, or both, the audio source becomes more perceptible to the user 101.

[0100] refer to Figure 5 , showing Figure 1 500 is a schematic diagram of illustrative aspects of an implementation of components of system 100, wherein Figure 1 At least a portion of the audio scaling processing performed by the device 102 in the headset 104 is performed in the headset 104 instead. As shown in the schematic diagram 500, one or more components of the audio enhancer 140 are integrated in the headset 104. For example, the signal selector and spatial filter 144 are distributed across the earpiece 110 and the earpiece 112. For example, the earpiece 110 includes the spatial filter 204A and the signal selector 206 in the signal selector and spatial filter 144, and the earpiece 112 includes the spatial filter 204B. In another implementation, the signal selector 206 is integrated in the earpiece 112 instead of the earpiece 110. The earpiece 110 includes a subband analyzer 542A coupled to the spatial filter 204A. The earpiece 112 includes a subband analyzer 542B coupled to the spatial filter 204B.

[0101] In one particular implementation, the headset 104 is configured to perform signal selection and spatial filtering on the audio signals from the microphones 120 and 122 and provide the resulting audio signal 145 to the device 102. In one example, Figure 1 The device 102 includes a phase extractor 148, an amplitude extractor 158, a normalizer 164, a combiner 166, an amplitude extractor 146, and a sub-band synthesizer 170. In this example, the signal selector 206 is configured to provide the audio signal 145 from the earpiece 110 to the amplitude extractor 146 of the device 102. In other examples, additional functions such as phase extraction, amplitude extraction, amplitude normalization, combining, sub-band synthesis, or any combination thereof may be performed at the earpiece 104 instead of the device 102.

[0102] In a specific example, two microphones are mounted on each of the earpieces. For example, microphone 120A and microphone 120B are mounted on earpiece 110, and microphone 122A and microphone 122B are mounted on earpiece 112. Subband analyzer 542A receives input signal 121 from microphone 120. For example, subband analyzer 542A receives input signal 121A from microphone 120A and input signal 121B from microphone 120B. Subband analyzer 542A applies a transform (e.g., FFT) to input signal 121A to generate audio signal 151A, and applies a transform (e.g., FFT) to input signal 121B to generate audio signal 151B.

[0103] Similarly, subband analyzer 542B receives input signal 123 from microphone 122. For example, subband analyzer 542B receives input signal 123A from microphone 122A and input signal 123B from microphone 122B. Subband analyzer 542B applies a transform (e.g., FFT) to input signal 123A to generate audio signal 153A, and applies a transform (e.g., FFT) to input signal 123B to generate audio signal 153B.

[0104] The spatial filter 204A applies spatial filtering to the audio signal 151A and the audio signal 151B based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof to generate an audio signal 205A, as shown in FIG. Figure 2 Similarly, the spatial filter 204B applies spatial filtering to the audio signal 153A and the audio signal 153B based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof to generate an audio signal 205B, as shown in FIG. Figure 2 described.

[0105] Spatial filter 204B is connected to a communication link such as (a registered trademark of BluetoothSig, Inc., Kirkland, Washington) communication link) provides the audio signal 205B from the earpiece 112 to the signal selector 206 of the earpiece 110. In one particular aspect, the earpiece 112 compresses the audio signal 205B before transmitting it to the earpiece 110 to reduce the amount of data transmitted. Figure 2 As described above, the signal selector 206 generates the audio signal 145 based on the audio signal 205A and the audio signal 205B.

[0106] Performing sub-band analysis, spatial filtering, and signal selection at earpiece 104 enables a reduced amount of wireless data transmission between earpiece 104 and device 102 (e.g., sending audio signals 151A, 153A, and 145 as opposed to sending all of input signal 125 to device 102). Distributing sub-band analysis and spatial filtering between earpieces 110 and 120 enables earpiece 104 to perform the functions using reduced processing resources, as compared to performing the functions at a single earpiece, and thereby reduces component cost and power consumption of each earpiece.

[0107] Reference Figure 6 , Figure 1 600 is a schematic diagram of another illustrative aspect of another implementation of components of the system 100, wherein one or more components of the audio enhancer 140 are integrated into the earphone 104. For example, with respect to the embodiment of the present invention, wherein the signal selector and the spatial filter 144 are distributed between the earpiece 110 and the earpiece 112. Figure 5Compared to schematic diagram 500 of FIG. 1 , signal selector and spatial filter 144 are integrated in earpiece 110 .

[0108] The subband analyzer 542B of the earpiece 112 provides a single signal (e.g., audio signal 153A) in the audio signal 153 to the signal selector and spatial filter 144. The signal selector and spatial filter 144 include a spatial filter 204A and a spatial filter 604B. The spatial filter 604B performs spatial filtering on the audio signal 151A corresponding to the microphone 120A and the audio signal 153A corresponding to the microphone 122A to generate an audio signal 605B (e.g., an enhanced audio signal, such as an audio zoom signal). The spatial filter 604B performs spatial filtering based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. In a specific aspect, the spatial filter 604B performs spatial filtering with correction for the head shadow effect. In a specific aspect, the operation described with reference to the schematic diagram 600 supports a first value of the zoom direction 137 (e.g., from 225 degrees to 315 degrees or to the right side of the user 101).

[0109] The signal selector 206 selects one of the audio signal 205A and the audio signal 605B as the audio signal 145 for output, as shown in FIG. Figure 2 In one particular aspect, the signal selector 206 outputs the audio signal 145 based on a comparison of a first frequency range (e.g., less than 1.5 kHz) of the audio signal 205A and a first frequency range of the audio signal 605B. For example, the signal selector 206 selects an audio signal having a first frequency range corresponding to a lower energy in the audio signal 205A or the audio signal 605B. In a particular implementation, the signal selector 206 outputs the selected audio signal in the audio signal 205A or the audio signal 605B as the audio signal 145. In an alternative implementation, the signal selector 206 extracts a first frequency portion corresponding to the first frequency range of the selected audio signal in the audio signal 205A or the audio signal 605B. The signal selector 206 extracts a second frequency portion corresponding to the second frequency range (e.g., greater than or equal to 1.5 kHz) of an audio signal in the audio signal 205A or the audio signal 605B. The signal selector 206 generates the audio signal 145 by combining the first frequency portion and the second frequency portion. Thus, the audio signal 145 may include a second frequency portion from the same audio signal or a different audio signal as the first frequency portion. The signal selector and spatial filter 144 integrated in the earpiece 110 is provided as an illustrative example. In another example, the signal selector and spatial filter 144 are integrated in the earpiece 112.

[0110] refer to Figure 7, showing Figure 1 FIG. 7 is a schematic diagram 700 of illustrative aspects of an implementation of components of the system 100. One or more components of the audio enhancer 140 are integrated into the earphone 104. For example, the signal selector and the spatial filter 144 are integrated into the earpiece 110.

[0111] The subband analyzer 542A provides a single signal (e.g., audio signal 151A) in the audio signal 151 to the signal selector and spatial filter 144. The subband analyzer 542B provides the audio signal 153A and the audio signal 153B to the signal selector and spatial filter 144. The signal selector and spatial filter 144 includes a spatial filter 704A and a spatial filter 204B. The spatial filter 704A performs spatial filtering on the audio signal 151A corresponding to the microphone 120A and the audio signal 153A corresponding to the microphone 122A to generate an audio signal 705B (e.g., an enhanced audio signal, such as an audio zoom signal). The spatial filter 704A performs spatial filtering based on the zoom direction 137, the zoom depth 139, the microphone configuration 203, the position information 207, or a combination thereof. In one particular aspect, the spatial filter 704A performs spatial filtering with head shadow effect correction. As shown in reference Figure 2 As described, spatial filter 204B performs spatial filtering on audio signal 153A and audio signal 153B to generate audio signal 205B. In one particular aspect, the operations described with reference to schematic diagram 700 support a second value of zoom direction 137 (e.g., from 45 degrees to 135 degrees or to the left of user 101). In one particular aspect, earpiece 110 and earpiece 112 are as described with reference to Figure 6 The first value of the zoom direction 137 (e.g., from 225 degrees to 315 degrees) described in the schematic diagram 600 of the reference diagram is operated, and the second value of the zoom direction 137 (e.g., from 45 degrees to 135 degrees) described in the reference diagram 700 is operated, as described in the reference diagram Figure 5 The operation is performed for a third value of the zoom direction 137 (eg, from 0-45 degrees, 135-225 degrees, and 315-359 degrees) as described in the schematic diagram 500 .

[0112] The signal selector 206 selects one of the audio signal 705A and the audio signal 205B as the audio signal 145 for output, as shown in FIG. Figure 2As described. The signal selector and spatial filter 144 integrated in the earpiece 110 are provided as an illustrative example. In another example, the signal selector and spatial filter 144 are integrated in the earpiece 112. In a particular aspect, the signal selector 206 outputs an audio signal 145 based on a comparison of a first frequency range (e.g., less than 1.5 kHz) of an audio signal 705A and a first frequency range of an audio signal 205B. For example, the signal selector 206 selects an audio signal with a first frequency range corresponding to lower energy in the audio signal 705A or the audio signal 205B. In a specific implementation, the signal selector 206 outputs an audio signal selected in the audio signal 705A or the audio signal 205B as the audio signal 145. In an alternative implementation, the signal selector 206 extracts a first frequency portion corresponding to the first frequency range of a selected audio signal in the audio signal 705A or the audio signal 205B. The signal selector 206 extracts a second frequency portion corresponding to a second frequency range (e.g., greater than or equal to 1.5 kHz) of one of the audio signals 705A or 205B. The signal selector 206 generates the audio signal 145 by combining the first frequency portion and the second frequency portion. Thus, the audio signal 145 may include the second frequency portion from the same audio signal or a different audio signal as the first frequency portion.

[0113] Figure 8 An implementation 800 is depicted in which the device 102 corresponds to or is integrated within a vehicle 812 (shown as a car). The vehicle 812 includes a processor 190 that includes a zoom target analyzer 130, an audio enhancer 140, or both. The vehicle 812 also includes one or more microphones 120, one or more microphones 122, or a combination thereof. The one or more microphones 120 and the one or more microphones 122 are positioned to capture speech of an operator, one or more passengers, or a combination thereof of the vehicle 812.

[0114] User voice activity detection can be performed based on audio signals received from one or more microphones 120 and one or more microphones 122 of the vehicle 812. In some implementations, user voice activity detection can be performed based on audio signals received from internal microphones (e.g., one or more microphones 120 and one or more microphones 122), for example, for voice commands from authorized passengers. For example, user voice activity detection can be used to detect voice commands from the operator of the vehicle 812 (e.g., a voice command from a parent to set the volume to 5 or set the destination of the autonomous vehicle), and ignore the voice of another passenger (e.g., a voice command from a child to set the volume to 10 or other passengers to discuss another location). In some implementations, user voice activity detection can be performed based on audio signals received from external microphones (e.g., one or more microphones 120 and one or more microphones 122) (e.g., an authorized user of the vehicle). In one particular implementation, in response to receiving a spoken command recognized as user language via operation of zoom target analyzer 130 and audio enhancer 140, the voice activation system initiates one or more operations of vehicle 812 based on one or more keywords detected in output signal 135 (e.g., "unlock," "start engine," "play music," "show weather forecast," or another voice command), such as by providing feedback or information via a display or one or more speakers (e.g., speaker 124A, speaker 124B, or both).

[0115] In one particular aspect, one or more microphones 120 and one or more microphones 122 are mounted on a movable mounting structure (e.g., rearview mirror 802) of vehicle 812. In one particular aspect, speakers 124A and speakers 124B are integrated or mounted on a seat (e.g., headrest) of vehicle 812.

[0116] In one particular aspect, the zoom target analyzer 130 receives user input 171 (e.g., "zoom to left rear passenger" or "zoom to Sarah") from a user 101 (e.g., a second occupant of the vehicle 812) indicating a zoom target 192 (e.g., a first occupant of the vehicle 812). For example, the user input 171 indicates an audio source 184A (e.g., "Sarah"), a first position of the audio source 184A (e.g., the first occupant) (e.g., "left rear"), a zoom direction 137, a zoom depth 139, or a combination thereof. In one particular aspect, the zoom target analyzer 130 determines the zoom direction 137, the zoom depth 139, or both based on the first position of the audio source 184A (e.g., the first passenger), the second position of the user 101 (e.g., the second passenger) (e.g., the driver's seat), or both.

[0117] In one particular aspect, the zoom direction 137, the zoom depth 139, or both are based on a first position of a first occupant (e.g., audio source 184A). For example, the zoom direction 137 is based on a direction of the zoom target 192 (e.g., audio source 184A) relative to the rearview mirror 802. In one particular aspect, the zoom depth 139 is based on a distance of the zoom target 192 (e.g., audio source 184A) from the rearview mirror 802. In one particular aspect, the zoom target analyzer 130 adjusts the zoom direction 137, the zoom depth 139, or both based on a difference between the position of the rearview mirror 802 and the position of the user 101 (e.g., the position of the speaker 124). In one particular aspect, the zoom target analyzer 130 adjusts the zoom direction 137, the zoom depth 139, or both based on a head orientation of the user 101.

[0118] In one particular implementation, the audio enhancer 140 positions the rearview mirror 802 based on the location of the zoom target 192, the location of the audio source 184A (e.g., the first occupant), the zoom direction 137, the zoom depth 139, or a combination thereof. The audio enhancer 140 receives input signals 121 and 123 from one or more microphones 120 and one or more microphones 122 mounted on the rearview mirror 802, respectively.

[0119] The audio enhancer 140 applies spatial filtering to the audio signal 151 (corresponding to the input signal 121) and the audio signal 153 (corresponding to the input signal 123) to generate the audio signal 205A and the audio signal 205B, as shown in FIG. Figure 2 In one particular aspect, the audio enhancer 140 applies spatial filtering based on a first position (e.g., "left rear passenger seat") of a first occupant of the vehicle 812 (e.g., audio source 184A), a zoom direction 137, a zoom depth 139, a microphone configuration 203 of one or more microphones 120 and one or more microphones 122, a head orientation of the user 101 (e.g., a second occupant), a second position of the user 101, or a combination thereof.

[0120] In one particular implementation, the signal selector and spatial filter 144 of the audio enhancer 140 applies spatial filtering based on one of the first position of the first occupant of the vehicle 812 (e.g., the audio source 184A), the zoom direction 137, the zoom depth 139, the microphone configuration 203, the head orientation of the user 101, or the second position of the user 101, and independently of receiving the remaining terms of the first position, the zoom direction 137, the zoom depth 139, the microphone configuration 203, the head orientation of the user 101, and the second position. For example, the signal selector and spatial filter 144 generates an audio signal 205 corresponding to one of the first position of the first occupant of the vehicle 812 (e.g., the audio source 184A), the zoom direction 137, the zoom depth 139, the microphone configuration 203, the head orientation of the user 101, or the second position of the user 101, and various values ​​of the remaining terms of the first position, the zoom direction 137, the zoom depth 139, the microphone configuration 203, the head orientation of the user 101, and the second position. The signal selector 206 selects one of the audio signals 205 as the audio signal 145 .

[0121] In one particular implementation, the signal selector and spatial filter 144 of the audio enhancer 140 applies spatial filtering independently of receiving one or more of the first position of the first occupant of the vehicle 812 (e.g., the audio source 184A), the zoom direction 137, the zoom depth 139, the microphone configuration 203, the head orientation of the user 101, or the second position of the user 101. In one particular example, the signal selector and spatial filter 144 determine the zoom direction 137 based on the first position and the default position of the rearview mirror 802. In one particular example, the signal selector and spatial filter 144 use various values ​​of the zoom depth 139, such as reference Figure 3 In a specific example, the signal selector and spatial filter 144 determines the zoom depth 139 based on the first position and the default position of the rearview mirror 802. In a specific example, the signal selector and spatial filter 144 generates the audio signals 205 using various values ​​of the zoom direction 137, and the signal selector 206 selects one of the audio signals 205 as the audio signal 145.

[0122] In one specific example, the signal selector and spatial filter 144 is configured to generate an audio signal 205 corresponding to a single default second position (e.g., the driver's seat) of the user 101. In one specific example, the signal selector and spatial filter 144 is configured to generate an audio signal 205 corresponding to a single default head orientation (e.g., facing forward) of the user 101. In one specific example, the signal selector and spatial filter 144 is configured to generate an audio signal 205 corresponding to a single default microphone configuration of the vehicle 812.

[0123] In one particular implementation, the signal selector and spatial filter 144 is configured to generate an audio signal 205 corresponding to a single location of the zoom target 192 of the vehicle 812. For example, the vehicle 812 includes a copy of the audio enhancer 140 for each of the seats of the vehicle 812. By way of example, the vehicle 812 includes a first audio enhancer 140, a second audio enhancer 140, and a third audio enhancer 140 configured to perform audio zooming for the left rear seat, the middle rear seat, and the right rear seat, respectively. The user 101 (e.g., an operator of the vehicle 812) can activate the first audio enhancer 140, the second audio enhancer 140, or the third audio enhancer 140 using a first input (e.g., a first button on a steering wheel), a second input (e.g., a second button), or a third input (e.g., a third button), respectively.

[0124] The audio enhancer 140 selects one of the audio signal 205A and the audio signal 205B as the audio signal 145 (see reference 145). Figure 2 ), and generating an output signal 135 based on the audio signal 145 (as shown in reference Figure 1 The audio enhancer 140 provides the output signal 131 and the output signal 133 to the speaker 124A and the speaker 124B, respectively, to play the audio scaling signal to the user 101 (e.g., the second occupant) of the vehicle 812. In a specific aspect, the output signal 135 corresponds to a higher gain applied to the sound received from the audio source 184A, a lower gain applied to the sound received from the audio source 184B, or both. In a specific aspect, the output signal 135 has the same phase difference and the same relative amplitude difference as the representative input signal in the input signal 121 and the representative input signal in the input signal 123 received by the rearview mirror 802.

[0125] Fig. 9An implementation 900 of the device 102 as an integrated circuit 902 including one or more processors 190 is depicted. The integrated circuit 902 also includes an audio input 904 (e.g., one or more bus interfaces) to enable receiving input signals 125 for processing. The integrated circuit 902 also includes a signal output 906 (e.g., a bus interface) to enable sending output signals, such as output signal 135. The integrated circuit 902 implements an implementation of audio scaling as a component in a system including a microphone, such as, Fig.10 The headphones depicted, Fig.11 A virtual reality headset or augmented reality headset as depicted, or Figure 8 The vehicle depicted.

[0126] Fig.10 An implementation 1000 is depicted in which the device 102 includes a headset 104. For example, one or more of the components of the device 102 are integrated into the headset 104. The headset 104 includes an earpiece 110 and an earpiece 112. In a particular aspect, one or more microphones 120 and one or more microphones 122 are mounted on the exterior of the earpiece 110 and the earpiece 112, respectively. In a particular aspect, a speaker 124A and a speaker 124B are mounted on the interior of the earpiece 110 and the earpiece 112, respectively. Components of the processor 190 (including the scaling target analyzer 130, the audio enhancer 140, or both) are integrated into the headset 104. In a particular example, the audio enhancer 140 operates to detect user voice activity (which can cause the headset 104 to perform one or more operations at the headset 104), to send audio data corresponding to the user voice activity to a second device (not shown) for further processing, or a combination thereof. In a particular aspect, the audio enhancer 140 operates to perform audio scaling of external sounds while maintaining binaural perception of the wearer of the headset 104.

[0127] Fig.11An implementation 1100 is depicted in which the device 102 includes a portable electronic device corresponding to a virtual reality, augmented reality, or mixed reality headset 1102. A zoom target analyzer 130, an audio enhancer 140, one or more microphones 120, one or more microphones 122, a speaker 124A, a speaker 124B, or a combination thereof, are integrated into the headset 1102. In a particular aspect, the headset 1102 includes one or more microphones 120 and one or more microphones 122, primarily for capturing ambient sound. User voice activity detection can be performed based on audio signals received from the one or more microphones 120 and one or more microphones 122 of the headset 1102. A visual interface device is positioned in front of the user's eyes to enable an augmented reality or virtual reality image or scene to be displayed to the user while wearing the headset 1102. In a particular example, the visual interface device is configured to display a notification indicating a user voice detected in the audio signal.

[0128] refer to Fig.12 , shows a specific implementation of the audio scaling method 1200. In one particular aspect, one or more operations of the method 1200 are performed by the phase extractor 148A, the phase extractor 148B, the signal selector and spatial filter 144, the combiner 166A, the combiner 166B, the spatial filter 204A, the spatial filter 204B, the audio enhancer 140, the processor 190, the device 102, Figure 1 The system 100 or at least one of a combination thereof is performed.

[0129] The method 1200 includes determining a first phase based on a first audio signal in a first audio signal at 1202. For example, as shown in FIG. Figure 1 As stated, Figure 1 The phase extractor 148A in determines the phase 161A based on the input signal 121A in the input signal 121.

[0130] The method 1200 also includes determining a second phase based on the second audio signal in the second audio signal at 1204. For example, as shown in FIG. Figure 1 As stated, Figure 1 The phase extractor 148B in determines the phase 161B based on the input signal 123A in the input signal 123.

[0131] The method 1200 also includes applying spatial filtering to the selected audio signal of the first audio signal and the second audio signal at 1206 to generate an enhanced audio signal. Figure 2 As stated, Figure 2The pair selector 202 in the audio signal 155 selects the audio signal 211A and the audio signal 211B, and selects the audio signal 213A and the audio signal 213B. The spatial filter 204A applies spatial filtering to the audio signal 211A and the audio signal 211B to generate the audio signal 205A (e.g., the first enhanced audio signal). The spatial filter 204B applies spatial filtering to the audio signal 213A and the audio signal 213B to generate the audio signal 205B (e.g., the second enhanced audio signal). As shown in FIG. Figure 2 As described above, the signal selector 206 outputs one of the audio signal 205A or the audio signal 205B as the audio signal 145 (eg, an enhanced audio signal).

[0132] The method 1200 also includes generating a first output signal including combining the amplitude of the enhanced audio signal with the first phase at 1208. For example, as shown in FIG. Figure 1 As stated, Figure 1 The combiner 166A in FIG. 1 generates an audio signal 167A by combining the amplitude 147 and the phase 161A of the audio signal 145 based on the normalization factor 165A. Figure 1 As described, sub-band synthesizer 170 generates output signal 131 by applying a transform to audio signal 167A.

[0133] The method 1200 also includes generating a second output signal at 1210 including combining the amplitude of the enhanced audio signal with the second phase. Figure 1 As stated, Figure 1 The combiner 166B in FIG. 1 generates an audio signal 167B by combining the amplitude 147 and the phase 161B of the audio signal 145 based on the normalization factor 165B. Figure 1 As described, the sub-band synthesizer 170 generates the output signal 133 by applying the transformation to the audio signal 167B. The output signal 131 and the output signal 133 correspond to the audio scaled signals.

[0134] The method 1200 preserves an overall binaural perception for the user 101 listening to the output signal 135 while providing audio scaling. For example, the overall binaural perception is preserved by maintaining a phase difference and an amplitude difference between the output signal 131 output by the speaker 124A and the output signal 133 output by the speaker 124B. The phase difference is preserved by generating the output signal 131 based on the phase 161A of the audio signal 151A (e.g., a representative right input signal) and generating the output signal 133 based on the phase 161B of the audio signal 153A (e.g., a representative left input signal). The amplitude difference is preserved by generating the output signal 131 based on the normalization factor 165A and the amplitude 147 and by generating the output signal 133 based on the normalization factor 165B and the amplitude 147.

[0135] Fig.12 The method 1200 may be implemented by a field programmable gate array (FPGA) device, an application specific integrated circuit (ASIC), a processing unit such as a central processing unit (CPU), a DSP, a controller, another hardware device, a firmware device, or any combination thereof. For example, Fig.12 The method 1200 may be performed by a processor executing instructions, for example, referring to Fig.13 as described.

[0136] refer to Fig.13 , a block diagram of a particular illustrative implementation of a device is depicted and generally designated 1300. In various implementations, device 1300 may have a plurality of Fig.13 In an illustrative implementation, device 1300 may correspond to device 102. In an illustrative implementation, device 1300 may execute the reference Figure 1-12 One or more operations described.

[0137] In certain implementations, the device 1300 includes a processor 1306 (e.g., a central processing unit (CPU)). The device 1300 may include one or more additional processors 1310 (e.g., one or more DSPs). In certain aspects, Figure 1 The processor 190 may correspond to the processor 1306, the processor 1310, or a combination thereof. The processor 1310 may include a speech and music codec (CODEC) 1308, which includes a speech decoder ("vocoder") encoder 1336, a vocoder decoder 1338, a scaling target analyzer 130, an audio enhancer 140, or a combination thereof.

[0138] The device 1300 may include a memory 1386 and a CODEC 1334. The memory 1386 may include instructions 1356 that may be executed by one or more additional processors 1310 (or processor 1306) to implement the functionality described with reference to the scaling target analyzer 130, the audio enhancer 140, or both. In one particular aspect, as described with reference to Figure 1 As described, the memory 1386 stores the playback file 1358, and the audio enhancer 140 decodes the audio data of the playback file 1358 to generate the input signal 125. The device 1300 may include a modem 1370 coupled to the antenna 1352 via the transceiver 1350.

[0139] The device 1300 may include a display 1328 coupled to a display controller 1326. One or more speakers 124, one or more microphones 120, one or more microphones 122, or a combination thereof may be coupled to a CODEC 1334. The CODEC 1334 may include a digital-to-analog converter (DAC) 1302, an analog-to-digital converter (ADC) 1304, or both. In a particular implementation, the CODEC 1334 may receive analog signals from the one or more microphones 120 and the one or more microphones 122, convert the analog signals to digital signals using the analog-to-digital converter 1304, and provide the digital signals to a speech and music codec 1308. The speech and music codec 1308 may process the digital signals, and the digital signals may be further processed by the audio enhancer 140. In a particular implementation, the speech and music codec 1308 may provide the digital signals to the CODEC 1334. The CODEC 1334 may convert the digital signals to analog signals using the digital-to-analog converter 1302, and may provide the analog signals to the one or more speakers 124.

[0140] In a particular implementation, device 1300 may be included in a system-in-package or system-on-chip device 1322. In a particular implementation, memory 1386, processor 1306, processor 1310, display controller 1326, CODEC 1334, and modem 1370 are included in a system-in-package or system-on-chip device 1322. In a particular implementation, input device 1330 and power supply 1344 are coupled to system-on-chip device 1322. In addition, in a particular implementation, as shown in FIG. Fig.13, the display 1328, input device 1330, one or more speakers 124, one or more microphones 120, one or more microphones 122, antenna 1352, and power supply 1344 are external to the system-on-chip device 1322. In a particular implementation, each of the display 1328, input device 1330, one or more speakers 124, one or more microphones 120, one or more microphones 122, antenna 1352, and power supply 1344 may be coupled to a component of the system-on-chip device 1322, such as an interface or a controller.

[0141] Device 1300 may include a smart speaker, a speaker bar, a mobile communication device, a smart phone, a cellular phone, a laptop, a computer, a tablet device, a personal digital assistant, a display device, a television, a game console, a music player, a radio, a digital video player, a digital video disc (DVD) player, a tuner, a camera, a navigation device, a vehicle, a headset, an augmented reality headset, a virtual reality headset, an aircraft, a home automation system, a voice-controlled device, a wireless speaker and voice-controlled device, a portable electronic device, an automobile, a vehicle, a computing device, a communication device, an Internet of Things (IoT) device, a virtual reality (VR) device, a base station, a mobile device, or any combination thereof.

[0142] In conjunction with the implementation, the apparatus includes a unit for determining a first phase based on the first audio signal in the first audio signal. For example, the unit for determining the first phase may correspond to Figure 1 The phase extractor 148A, the audio enhancer 140, the one or more processors 190, the device 102, Figure 1 The system 100, the processor 1306, the processor 1310, one or more other circuits or components configured to determine the first phase based on the first audio signal, or any combination thereof.

[0143] The apparatus further comprises a unit for determining a second phase based on the second audio signal in the second audio signal. For example, the unit for determining the second phase may correspond to Figure 1 The phase extractor 148B, the audio enhancer 140, the one or more processors 190, the device 102, Figure 1 The system 100, the processor 1306, the processor 1310, one or more other circuits or components configured to determine the second phase based on the second audio signal, or any combination thereof.

[0144] The apparatus further comprises means for applying spatial filtering to the selected audio signal of the first audio signal and the second audio signal to generate an enhanced audio signal. For example, the means for applying spatial filtering may correspond to the signal selector and spatial filter 144, the audio enhancer 140, the one or more processors 190, the device 102, Figure 1 System 100 in Figure 2 The spatial filter 204A, the processor 1306, the processor 1310, one or more other circuits or components configured to apply spatial filtering, or any combination thereof.

[0145] The apparatus also includes means for generating a first output signal (including combining the amplitude of the enhanced audio signal with the first phase). For example, the means for generating a first output signal may correspond to the combiner 166A, the sub-band synthesizer 170, the audio enhancer 140, the one or more processors 190, the device 102, Figure 1 The system 100, the processor 1306, the processor 1310, one or more other circuits or components configured to generate a first output signal, or any combination thereof.

[0146] The apparatus further includes means for generating a second output signal (including combining the amplitude of the enhanced audio signal with the second phase). For example, the means for generating a second output signal may correspond to the combiner 166B, the sub-band synthesizer 170, the audio enhancer 140, the one or more processors 190, the device 102, Figure 1 The system 100, the processor 1306, the processor 1310, one or more other circuits or components configured to generate a second output signal, or any combination thereof. The first output signal and the second output signal correspond to the audio scaling signal.

[0147] In some implementations, a non-transitory computer-readable medium (e.g., a computer-readable storage device, such as memory 1386) includes instructions (e.g., instructions 1356) that, when executed by one or more processors (e.g., one or more processors 190, one or more processors 1310, or processor 1306), cause the one or more processors to determine a first phase (e.g., phase 161A) based on a first audio signal (e.g., input signal 121A) in a first audio signal (e.g., input signal 121), and determine a second phase (e.g., phase 161B) based on a second audio signal (e.g., input signal 123A) in a second audio signal (e.g., input signal 123). The instructions, when executed by the one or more processors, further cause the one or more processors to apply spatial filtering to selected audio signals (e.g., audio signal 211A, audio signal 211B, audio signal 213A, and audio signal 213B) in the first audio signal and the second audio signal to generate an enhanced audio signal (e.g., audio signal 145). The instructions, when executed by the one or more processors, further cause the one or more processors to: generate a first output signal (e.g., output signal 131), including combining the amplitude (e.g., amplitude 147) of the enhanced audio signal with a first phase. The instructions, when executed by the one or more processors, further cause the one or more processors to: generate a second output signal (e.g., output signal 133), including combining the amplitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to audio scaling signals.

[0148] Certain aspects of the present disclosure are described below in related clause groups:

[0149] According to clause 1, the device includes: a memory configured to store instructions; and one or more processors configured to execute the instructions to: determine a first phase based on a first audio signal in a first audio signal; determine a second phase based on a second audio signal in a second audio signal; apply spatial filtering to a selected audio signal of the first audio signal and the second audio signal to generate an enhanced audio signal; generate a first output signal, the first output signal comprising combining the amplitude of the enhanced audio signal with the first phase; and generate a second output signal, the second output signal comprising combining the amplitude of the enhanced audio signal with the second phase, wherein the first output signal and the second output signal correspond to audio scaling signals.

[0150] Item 2 includes the device of Item 1, wherein the one or more processors are further configured to: receive the first audio signal from a first plurality of microphones externally mounted to a first earpiece of a headset; and receive the second audio signal from a second plurality of microphones externally mounted to a second earpiece of the headset.

[0151] Clause 3 includes the apparatus of clause 2, wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the first plurality of microphones and the second plurality of microphones, or a combination thereof.

[0152] Clause 4 includes the device of clause 3, wherein the one or more processors are configured to determine the zoom direction, the zoom depth, or both based on a click detected via a touch sensor of the headset.

[0153] Clause 5 includes the device of clause 3 or clause 4, wherein the one or more processors are configured to determine the zoom direction, the zoom depth, or both based on movement of the headset.

[0154] Clause 6 includes the apparatus of clause 2, wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction.

[0155] Clause 7 includes the device of clause 6, wherein the one or more processors are configured to determine the zoom direction based on a click detected via a touch sensor of the headset.

[0156] Clause 8 includes the device of clause 6 or clause 7, wherein the one or more processors are configured to determine the zoom direction based on movement of the headset.

[0157] Clause 9 includes the apparatus of clause 2, wherein the one or more processors are configured to apply the spatial filtering based on a scaled depth.

[0158] Clause 10 includes the device of clause 9, wherein the one or more processors are configured to determine the zoom depth based on a click detected via a touch sensor of the headset.

[0159] Clause 11 includes the device of clause 9 or clause 10, wherein the one or more processors are configured to determine the zoom depth based on movement of the headset.

[0160] Clause 12 includes the device of clause 2, wherein the one or more processors are configured to apply the spatial filtering based on a configuration of the first plurality of microphones and the second plurality of microphones.

[0161] Clause 13 includes the apparatus of any of clauses 1 to 12, wherein the one or more processors are integrated into a headset.

[0162] Clause 14 includes the apparatus of any one of clauses 1 to 13, wherein the one or more processors are further configured to: provide the first output signal to a first speaker of a first earpiece of a headset; and provide the second output signal to a second speaker of a second earpiece of the headset.

[0163] Clause 15 includes the device of clause 1 or clause 14, wherein the one or more processors are further configured to decode audio data of a playback file to generate the first audio signal and the second audio signal.

[0164] Clause 16 includes the apparatus of clause 15, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, the position information, or a combination thereof.

[0165] Clause 17 includes the apparatus of clause 15, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction.

[0166] Clause 18 includes the apparatus of clause 15, wherein the audio data comprises position information indicating a location of a source of each of the first audio signal and the second audio signal, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom depth.

[0167] Clause 19 includes the apparatus of clause 15, wherein the audio data comprises position information indicating a location of a source of each of the first audio signal and the second audio signal, and wherein the one or more processors are configured to apply the spatial filtering based on the position information.

[0168] Clause 20 includes the device of clause 15, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, the multi-channel audio representation, or a combination thereof.

[0169] Clause 21 includes the device of clause 15, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction.

[0170] Clause 22 includes the apparatus of clause 15, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the one or more processors are configured to apply the spatial filtering based on a scaling depth.

[0171] Clause 23 includes the device of clause 15, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the one or more processors are configured to apply the spatial filtering based on the multi-channel audio representation.

[0172] Clause 24 includes the apparatus of any one of clauses 20 to 23, wherein the multi-channel audio representation corresponds to ambient sound data.

[0173] Clause 25 includes the apparatus of any of clauses 1, 13, or 14, further comprising: a modem coupled to the one or more processors, the modem being configured to provide audio data to the one or more processors based on received streaming data, wherein the one or more processors are configured to decode the audio data to generate the first audio signal and the second audio signal.

[0174] Clause 26 includes the apparatus of clause 1 or any one of clauses 15 to 25, wherein the one or more processors are integrated into a vehicle and wherein the one or more processors are configured to: apply the spatial filtering based on a first position of a first occupant of the vehicle; and provide the first output signal and the second output signal to a first speaker and a second speaker, respectively, to play the audio scaling signal to a second occupant of the vehicle.

[0175] Clause 27 includes the apparatus of clause 26, wherein the one or more processors are configured to: position a movable mounting structure based on the first position of the first occupant; and receive the first audio signal and the second audio signal from a plurality of microphones mounted on the movable mounting structure.

[0176] Clause 28 includes the apparatus of clause 27, wherein the moveable mounting structure comprises a rearview mirror.

[0177] Clause 29 includes the device of clause 27 or clause 28, wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the plurality of microphones, a head orientation of the second occupant, or a combination thereof.

[0178] Clause 30 includes the apparatus of clause 29, wherein the zoom direction, the zoom depth, or both are based on the first position of the first occupant.

[0179] Clause 31 includes the apparatus of clause 27 or clause 28, wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction.

[0180] Clause 32 includes the apparatus of clause 31 , wherein the zoom direction is based on the first position of the first occupant.

[0181] Clause 33 includes the apparatus of clause 27 or clause 28, wherein the one or more processors are configured to apply the spatial filtering based on a scaled depth.

[0182] Clause 34 includes the apparatus of clause 33, wherein the zoom depth is based on the first position of the first occupant.

[0183] Clause 35 includes the device of clause 27 or clause 28, wherein the one or more processors are configured to apply the spatial filtering based on a configuration of the plurality of microphones.

[0184] Clause 36 includes the apparatus of clause 27 or clause 28, wherein the one or more processors are configured to apply the spatial filtering based on a head orientation of the second occupant.

[0185] Clause 37 includes the device of either clause 29 or clause 30, further comprising: an input device coupled to the one or more processors, wherein the one or more processors are configured to receive user input indicating the zoom direction, the zoom depth, the first position of the first occupant, or a combination thereof via the input device.

[0186] Clause 38 includes the device of either clause 29 or clause 30, further comprising: an input device coupled to the one or more processors, wherein the one or more processors are configured to: receive user input indicating the zoom direction via the input device.

[0187] Clause 39 includes the device of either clause 29 or clause 30, further comprising: an input device coupled to the one or more processors, wherein the one or more processors are configured to: receive user input indicating the zoom depth via the input device.

[0188] Clause 40 includes the apparatus of either clause 29 or clause 30, further comprising: an input device coupled to the one or more processors, wherein the one or more processors are configured to receive user input indicating the first position of the first occupant via the input device.

[0189] Clause 41 includes the apparatus of any of clauses 1 to 40, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the first phase.

[0190] Clause 42 includes the apparatus of any of clauses 1 to 41, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the second phase.

[0191] Clause 43 includes the apparatus of any one of clauses 1 to 42, wherein the audio scaled signal comprises a binaural audio scaled signal.

[0192] Clause 44 includes the apparatus of any of clause 1 or clause 43, wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, or both.

[0193] Clause 45 includes the device of clause 44, wherein the one or more processors are configured to receive user input indicating the zoom direction, the zoom depth, or both.

[0194] Item 46 includes the device of item 44, further comprising: a depth sensor coupled to the one or more processors, wherein the one or more processors are configured to: receive user input indicating a zoom target; receive sensor data from the depth sensor; and determine the zoom direction, the zoom depth, or both of the zoom target based on the sensor data.

[0195] Item 47 includes the device of item 46, wherein the depth sensor includes an image sensor, wherein the sensor data includes image data, and wherein the one or more processors are configured to perform image recognition on the image data to determine the zoom direction, the zoom depth, or both of the zoom target.

[0196] Clause 48 includes the device of clause 46, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0197] Item 49 includes the device of item 48, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and wherein the one or more processors are configured to: determine the zoom direction, the zoom depth, or both of the zoom target based on the position of the zoom target.

[0198] Clause 50 includes the apparatus of any one of clauses 44 to 49, wherein the one or more processors are configured to: determine the scaling depth, comprising: applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0199] Clause 51 includes the device of clause 50, wherein applying the spatial filtering based on the zoom direction and the first zoom depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth comprises applying the spatial filtering based on a second set of arrival directions.

[0200] Clause 52 includes the apparatus of any of clauses 44 to 51, wherein the one or more processors are configured to select the selected audio signal based on the zoom direction, the zoom depth, or both.

[0201] Clause 53 includes the apparatus of any of clauses 1 to 43, wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction.

[0202] Clause 54 includes the device of clause 53, wherein the one or more processors are configured to receive user input indicating the zoom direction.

[0203] Item 55 includes the device of item 53, further comprising: a depth sensor coupled to the one or more processors, wherein the one or more processors are configured to: receive user input indicating a zoom target; receive sensor data from the depth sensor; and determine the zoom direction of the zoom target based on the sensor data.

[0204] Clause 56 includes the device of clause 55, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and wherein the one or more processors are configured to perform image recognition on the image data to determine the zoom direction of the zoom target.

[0205] Clause 57 includes the device of clause 55, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0206] Clause 58 includes the device of clause 55, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and wherein the one or more processors are configured to: determine the zoom direction of the zoom target based on the position of the zoom target.

[0207] Clause 59 includes a device of any one of clauses 53 to 58, wherein the one or more processors are configured to: determine a scaling depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and a first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and a second scaling depth to generate a second enhanced signal; and based on determining that a first energy of the first enhanced audio signal is less than or equal to a second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0208] Clause 60 includes the device of clause 59, wherein applying the spatial filtering based on the zoom direction and the first zoom depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth comprises applying the spatial filtering based on a second set of arrival directions.

[0209] Clause 61 includes the apparatus of any of clauses 53 to 60, wherein the one or more processors are configured to select the selected audio signal based on the zoom direction.

[0210] Clause 62 includes the apparatus of any of clauses 1 to 43, wherein the one or more processors are configured to apply the spatial filtering based on a scaled depth.

[0211] Clause 63 includes the device of clause 62, wherein the one or more processors are configured to receive user input indicating the zoom depth.

[0212] Item 64 includes the device of item 62, further comprising: a depth sensor coupled to the one or more processors, wherein the one or more processors are configured to: receive user input indicating a zoom target; receive sensor data from the depth sensor; and determine the zoom depth of the zoom target based on the sensor data.

[0213] Clause 65 includes the device of clause 64, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and wherein the one or more processors are configured to perform image recognition on the image data to determine the zoom depth of the zoom target.

[0214] Clause 66 includes the device of clause 64, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0215] Clause 67 includes the device of clause 64, wherein the depth sensor comprises a position sensor, wherein the sensor data comprises position data indicating the position of the zoom target, and wherein the one or more processors are configured to determine the zoom depth of the zoom target based on the position of the zoom target.

[0216] Clause 68 includes a device of any one of clauses 62 to 67, wherein the one or more processors are configured to: determine the scaling depth, including: applying the spatial filtering to the selected audio signal based on a scaling direction and a first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and a second scaling depth to generate a second enhanced signal; and based on determining that a first energy of the first enhanced audio signal is less than or equal to a second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0217] Clause 69 includes the device of clause 68, wherein applying the spatial filtering based on the zoom direction and the first zoom depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth comprises applying the spatial filtering based on a second set of arrival directions.

[0218] Clause 70 includes the apparatus of any of clauses 62 to 69, wherein the one or more processors are configured to select the selected audio signal based on the zoom depth.

[0219] Clause 71 includes the apparatus of any of clauses 1 to 70, wherein the one or more processors are configured to: apply the spatial filtering to a first subset of the selected audio signals to generate a first enhanced audio signal; apply the spatial filtering to a second subset of the selected audio signals to generate a second enhanced audio signal; and select one of the first enhanced audio signal or the second enhanced audio signal as the enhanced audio signal, the selection being based on determining that a first energy of the enhanced audio signal is less than or equal to a second energy of the other of the first enhanced audio signal or the second enhanced audio signal.

[0220] Clause 72 includes the device of clause 71, wherein the one or more processors are configured to apply the spatial filtering with head shadowing effect correction to one of the first subset or the second subset.

[0221] Clause 73 includes the device of clause 71, wherein the one or more processors are configured to apply the spatial filtering with head shadow effect correction to the first subset.

[0222] Clause 74 includes the device of clause 71, wherein the one or more processors are configured to apply the spatial filtering with head shadow effect correction to the second subset.

[0223] Clause 75 includes the apparatus of any of clauses 1 to 74, wherein the first phase is indicated by first phase values, and wherein each of the first phase values ​​represents a phase of a particular frequency subband of the first audio signal.

[0224] Clause 76 includes the apparatus of any of clauses 1 to 75, wherein the one or more processors are configured to generate each of the first output signal and the second output signal based at least in part on a first amplitude of the first audio signal, wherein the first amplitude is indicated by a first amplitude value, and wherein each of the first amplitude values ​​represents an amplitude of a particular frequency subband of the first audio signal.

[0225] Clause 77 includes the apparatus of any one of clauses 1 to 76, wherein the amplitude of the enhanced audio signal is indicated by third amplitude values, and wherein each of the third amplitude values ​​represents an amplitude of a particular frequency subband of the enhanced audio signal.

[0226] According to clause 78, the method includes: determining a first phase at a device based on a first audio signal in a first audio signal; determining a second phase at a device based on a second audio signal in a second audio signal; applying spatial filtering to a selected audio signal from the first audio signal and the second audio signal at the device to generate an enhanced audio signal; generating a first output signal at the device, including combining the amplitude of the enhanced audio signal with the first phase; and generating a second output signal at the device, including combining the amplitude of the enhanced audio signal with the second phase, wherein the first output signal and the second output signal correspond to audio scaling signals.

[0227] Clause 79 includes the method of clause 78, further comprising: receiving the first audio signal from a first plurality of microphones mounted externally to a first earpiece of a headset; and receiving the second audio signal from a second plurality of microphones mounted externally to a second earpiece of the headset.

[0228] Clause 80 includes the method of clause 79, further comprising: applying the spatial filtering based on a zoom direction, a zoom depth, a configuration of the first plurality of microphones and the second plurality of microphones, or a combination thereof.

[0229] Clause 81 includes the method of clause 80, further comprising: determining the zoom direction, the zoom depth, or both based on a click detected via a touch sensor of the headset.

[0230] Clause 82 includes the method of clause 80 or clause 81, further comprising: determining the zoom direction, the zoom depth, or both based on movement of the headset.

[0231] Clause 83 includes the method of clause 79, further comprising: applying the spatial filtering based on a zoom direction.

[0232] Clause 84 includes the method of clause 83, further comprising: determining the zoom direction based on a click detected via a touch sensor of the headset.

[0233] Clause 85 includes the method of clause 83 or clause 84, further comprising: determining the zoom direction based on movement of the headset.

[0234] Clause 86 includes the method of clause 79, further comprising: applying the spatial filtering based on a scaled depth.

[0235] Clause 87 includes the method of clause 86, further comprising: determining the zoom depth based on a click detected via a touch sensor of the headset.

[0236] Clause 88 includes the method of clause 86 or clause 87, further comprising: determining the zoom depth based on movement of the headset.

[0237] Clause 89 includes the method of clause 79, further comprising: applying the spatial filtering based on a configuration of the first plurality of microphones and the second plurality of microphones.

[0238] Clause 90 includes the method of any one of clauses 78 to 89, wherein the device is integrated into a headset.

[0239] Clause 91 includes the method of any one of clauses 78 to 90, further comprising: providing the first output signal to a first speaker of a first earpiece of a headset; and providing the second output signal to a second speaker of a second earpiece of the headset.

[0240] Clause 92 includes the method of clause 78 or clause 91, further comprising: decoding audio data of a playback file to generate the first audio signal and the second audio signal.

[0241] Clause 93 includes the method of clause 92, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising applying the spatial filtering based on a zoom direction, a zoom depth, the position information, or a combination thereof.

[0242] Clause 94 includes the method of clause 92, wherein the audio data includes position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising applying the spatial filtering based on a zoom direction.

[0243] Clause 95 includes the method of clause 92, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising applying spatial filtering based on the scaling depth.

[0244] Clause 96 includes the method of clause 92, wherein the audio data includes position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising applying the spatial filtering based on the position information.

[0245] Clause 97 includes the method of clause 92, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprises applying the spatial filtering based on a zoom direction, a zoom depth, the multi-channel audio representation, or a combination thereof.

[0246] Clause 98 includes the method of clause 92, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprising applying the spatial filtering based on a zoom direction.

[0247] Clause 99 includes the method of clause 92, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprises applying the spatial filtering based on a scaled depth.

[0248] Clause 100 includes the method of clause 92, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprising applying the spatial filtering based on the multi-channel audio representation.

[0249] Clause 101 includes the method of any one of clauses 97 to 100, wherein the multi-channel audio representation corresponds to ambient sound data.

[0250] Clause 102 includes the method of any of clause 78, clause 90, or clause 91, further comprising: receiving audio data representing streaming data from a modem; and decoding the audio data to generate the first audio signal and the second audio signal.

[0251] Clause 103 includes the method of clause 78 or any one of clauses 92 to clause 102, further comprising: applying the spatial filtering based on a first position of a first occupant of a vehicle; and providing the first output signal and the second output signal to a first speaker and a second speaker, respectively, to play the audio scaling signal to a second occupant of the vehicle.

[0252] Clause 104 includes the method of clause 103, further comprising: positioning a movable mounting structure based on the first position of the first occupant; and receiving the first audio signal and the second audio signal from a plurality of microphones mounted on the movable mounting structure.

[0253] Clause 105 includes the method of clause 104, wherein the moveable mounting structure comprises a rearview mirror.

[0254] Clause 106 includes the method of clause 104 or clause 105, further comprising: applying the spatial filtering based on a zoom direction, a zoom depth, a configuration of the plurality of microphones, a head orientation of the second occupant, or a combination thereof.

[0255] Clause 107 includes the method of clause 106 , wherein the zoom direction, the zoom depth, or both are based on the first position of the first occupant.

[0256] Clause 108 includes the method of clause 104 or clause 105, further comprising: applying the spatial filtering based on a zoom direction.

[0257] Clause 109 includes the method of clause 108 , wherein the zoom direction is based on the first position of the first occupant.

[0258] Clause 110 includes the method of clause 104 or clause 105, further comprising: applying the spatial filtering based on a scaled depth.

[0259] Clause 111 includes the method of clause 110 , wherein the zoom depth is based on the first position of the first occupant.

[0260] Clause 112 includes the method of clause 104 or clause 105, further comprising: applying the spatial filtering based on a configuration of the plurality of microphones.

[0261] Clause 113 includes the method of clause 104 or clause 105, further comprising: applying the spatial filtering based on a head orientation of the second occupant.

[0262] Clause 114 includes the method of either clause 106 or clause 107 , further comprising: receiving a user input via an input device, the user input indicating the zoom direction, the zoom depth, the first position of the first occupant, or a combination thereof.

[0263] Clause 115 includes the method of either clause 106 or clause 107, further comprising: receiving user input indicating the zoom direction via an input device.

[0264] Clause 116 includes the method of either clause 106 or clause 107, further comprising: receiving user input indicating the zoom depth via an input device.

[0265] Clause 117 includes the method of either clause 106 or clause 107 , further comprising: receiving user input via an input device indicating the first position of the first occupant.

[0266] Clause 118 includes the method of any of clauses 78 to 117, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the first phase.

[0267] Clause 119 includes the method of any of clauses 78 to 118, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the second phase.

[0268] Clause 120 includes the method of any one of clauses 78 to 119, wherein the audio scaled signal comprises a binaural audio scaled signal.

[0269] Clause 121 includes the method of any one of clauses 78 to 120, further comprising: applying the spatial filtering based on a zoom direction, a zoom depth, or both.

[0270] Clause 122 includes the method of clause 121 , further comprising: receiving user input indicating the zoom direction, the zoom depth, or both.

[0271] Clause 123 includes the method of clause 121, further comprising: receiving user input indicating a zoom target; receiving sensor data from a depth sensor; and determining the zoom direction, the zoom depth, or both of the zoom target based on the sensor data.

[0272] Clause 124 includes the method of clause 123, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and further comprising: performing image recognition on the image data to determine the zoom direction, the zoom depth, or both of the zoom target.

[0273] Clause 125 includes the method of clause 123, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0274] Clause 126 includes the method of clause 125, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and further includes: determining the zoom direction, the zoom depth, or both of the zoom target based on the position of the zoom target.

[0275] Clause 127 includes the method of any one of clauses 121 to 126, further comprising: determining the scaling depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0276] Clause 128 includes the method of clause 127, wherein applying the spatial filtering based on the zoom direction and the first zoom depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth comprises applying the spatial filtering based on a second set of arrival directions.

[0277] Clause 129 includes the method of any one of clauses 121 to 128, further comprising: selecting the selected audio signal based on the zoom direction, the zoom depth, or both.

[0278] Clause 130 includes the method of any one of clauses 78 to 120, further comprising: applying the spatial filtering based on a zoom direction.

[0279] Clause 131 includes the method of clause 130, further comprising: receiving user input indicating the zoom direction.

[0280] Clause 132 includes the method of clause 130, further comprising: receiving user input indicating a zoom target; receiving sensor data from a depth sensor; and determining the zoom direction of the zoom target based on the sensor data.

[0281] Clause 133 includes the method of clause 132, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and further comprising: performing image recognition on the image data to determine the zoom direction of the zoom target.

[0282] Clause 134 includes the method of clause 132, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0283] Clause 135 includes the method of clause 132, wherein the depth sensor comprises a position sensor, wherein the sensor data comprises position data indicating a position of the zoom target, and further comprising determining the zoom direction of the zoom target based on the position of the zoom target.

[0284] Clause 136 includes the method of any one of clauses 130 to 135, further comprising: determining a scaling depth, comprising: applying the spatial filtering to the selected audio signal based on the scaling direction and a first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and a second scaling depth to generate a second enhanced signal; and based on determining that a first energy of the first enhanced audio signal is less than or equal to a second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0285] Clause 137 includes the method of clause 136, wherein applying the spatial filtering based on the zoom direction and the first zoom depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth comprises applying the spatial filtering based on a second set of arrival directions.

[0286] Clause 138 includes the method of any one of clauses 130 to 137, further comprising: selecting the selected audio signal based on the scaling direction.

[0287] Clause 139 includes the method of any one of clauses 78 to 120, further comprising: applying the spatial filtering based on a scaled depth.

[0288] Clause 140 includes the method of clause 139, further comprising: receiving user input indicating the zoom depth.

[0289] Clause 141 includes the method of clause 139, further comprising: receiving user input indicating a zoom target; receiving sensor data from a depth sensor; and determining the zoom depth of the zoom target based on the sensor data.

[0290] Clause 142 includes the method of clause 141, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and further comprising: performing image recognition on the image data to determine the zoom depth of the zoom target.

[0291] Clause 143 includes the method of clause 141, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0292] Clause 144 includes the method of clause 141, wherein the depth sensor comprises a position sensor, wherein the sensor data comprises position data indicating a position of the zoom target, and further comprising determining the zoom depth of the zoom target based on the position of the zoom target.

[0293] Clause 145 includes the method of any one of clauses 139 to clause 144, further including: determining the scaling depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0294] Clause 146 includes the method of clause 145, wherein applying the spatial filtering based on the zoom direction and the first zoom depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth comprises applying the spatial filtering based on a second set of arrival directions.

[0295] Clause 147 includes the method of any one of clauses 139 to 146, further comprising: selecting the selected audio signal based on the zoom depth.

[0296] Clause 148 includes the method of any one of clauses 78 to 147, further comprising: applying the spatial filtering to a first subset of the selected audio signals to generate a first enhanced audio signal; applying the spatial filtering to a second subset of the selected audio signals to generate a second enhanced audio signal; and selecting one of the first enhanced audio signal or the second enhanced audio signal as the enhanced audio signal, the selection being based on determining that a first energy of the enhanced audio signal is less than or equal to a second energy of the other of the first enhanced audio signal or the second enhanced audio signal.

[0297] Clause 149 includes the method of clause 148, further comprising: applying the spatial filtering with head shadow effect correction to one of the first subset or the second subset.

[0298] Clause 150 includes the method of clause 148, further comprising: applying the spatial filtering with head shadow effect correction to the first subset.

[0299] Clause 151 includes the method of clause 148, further comprising: applying the spatial filtering with head shadow effect correction to the second subset.

[0300] Clause 152 includes the method of any of clauses 78 to 151, wherein the first phase is indicated by first phase values, and wherein each of the first phase values ​​represents a phase of a particular frequency subband of the first audio signal.

[0301] Clause 153 includes the method of any one of clauses 78 to clause 152, further comprising: generating each of the first output signal and the second output signal based at least in part on a first amplitude of the first audio signal, wherein the first amplitude is indicated by a first amplitude value, and wherein each of the first amplitude values ​​represents an amplitude of a specific frequency subband of the first audio signal.

[0302] Clause 154 includes the method of any one of clauses 78 to 153, wherein the amplitude of the enhanced audio signal is indicated by third amplitude values, and wherein each of the third amplitude values ​​represents an amplitude of a particular frequency subband of the enhanced audio signal.

[0303] According to clause 155, a non-transitory computer-readable medium stores instructions that, when executed by the one or more processors, cause the one or more processors to: determine a first phase based on a first audio signal in a first audio signal; determine a second phase based on a second audio signal in a second audio signal; apply spatial filtering to a selected audio signal among the first audio signal and the second audio signal to generate an enhanced audio signal; generate a first output signal, the first output signal comprising combining the amplitude of the enhanced audio signal with the first phase; and generate a second output signal, the second output signal comprising combining the amplitude of the enhanced audio signal with the second phase, wherein the first output signal and the second output signal correspond to audio scaling signals.

[0304] Item 156 includes the non-transitory computer-readable medium of item 155, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive the first audio signal from a first plurality of microphones externally mounted to a first earpiece of a headset; and receive the second audio signal from a second plurality of microphones externally mounted to a second earpiece of the headset.

[0305] Item 157 includes the non-transitory computer-readable medium of item 156, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the first plurality of microphones and the second plurality of microphones, or a combination thereof.

[0306] Item 158 includes the non-transitory computer-readable medium of item 157, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom direction, the zoom depth, or both based on a click detected via a touch sensor of the headset.

[0307] Clause 159 includes the non-transitory computer-readable medium of clause 157 or clause 158, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom direction, the zoom depth, or both based on movement of the headset.

[0308] Clause 160 includes the non-transitory computer-readable medium of clause 156, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction.

[0309] Clause 161 includes the non-transitory computer-readable medium of clause 160, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom direction based on a click detected via a touch sensor of the headset.

[0310] Clause 162 includes the non-transitory computer-readable medium of clause 160 or clause 161, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom direction based on movement of the headset.

[0311] Clause 163 includes the non-transitory computer-readable medium of clause 156, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a scaled depth.

[0312] Clause 164 includes the non-transitory computer-readable medium of clause 163, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom depth based on a click detected via a touch sensor of the headset.

[0313] Clause 165 includes the non-transitory computer-readable medium of clause 163 or clause 164, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom depth based on movement of the headset.

[0314] Clause 166 includes the non-transitory computer-readable medium of clause 156, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: apply the spatial filtering based on a configuration of the first plurality of microphones and the second plurality of microphones.

[0315] Clause 167 includes the non-transitory computer-readable medium of any of clauses 155 to 166, wherein the one or more processors are integrated into a headset.

[0316] Item 168 includes a non-transitory computer-readable medium of any one of items 155 to 167, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: provide the first output signal to a first speaker of a first earpiece of the headset; and provide the second output signal to a second speaker of a second earpiece of the headset.

[0317] Clause 169 includes the non-transitory computer-readable medium of clause 155 or clause 168, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: decode audio data of a playback file to generate the first audio signal and the second audio signal.

[0318] Item 170 includes the non-transitory computer-readable medium of item 169, wherein the audio data includes position information indicating the location of a source of each of the first audio signal and the second audio signal, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction, a zoom depth, the position information, or a combination thereof.

[0319] Item 171 includes the non-transitory computer-readable medium of item 169, wherein the audio data includes position information indicating the location of a source of each of the first audio signal and the second audio signal, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction.

[0320] Item 172 includes the non-transitory computer-readable medium of item 169, wherein the audio data includes position information indicating the location of a source of each of the first audio signal and the second audio signal, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on the scaling depth.

[0321] Item 173 includes the non-transitory computer-readable medium of item 169, wherein the audio data includes position information indicating the location of a source of each of the first audio signal and the second audio signal, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on the position information.

[0322] Item 174 includes the non-transitory computer-readable medium of item 169, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction, a zoom depth, the multi-channel audio representation, or a combination thereof.

[0323] Item 175 includes the non-transitory computer-readable medium of item 169, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction.

[0324] Item 176 includes the non-transitory computer-readable medium of item 169, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on the scaled depth.

[0325] Item 177 includes the non-transitory computer-readable medium of item 169, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on the multi-channel audio representation.

[0326] Clause 178 includes the non-transitory computer-readable medium of any of clauses 174 to 177, wherein the multi-channel audio representation corresponds to ambient sound data.

[0327] Clause 179 includes a non-transitory computer-readable medium of any of clause 155, clause 167, or clause 168, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive audio data representing streaming data from a modem; and decode the audio data to generate the first audio signal and the second audio signal.

[0328] Item 180 includes a non-transitory computer-readable medium of item 155 or any one of items 169 to item 179, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: apply the spatial filtering based on a first position of a first occupant of a vehicle; and provide the first output signal and the second output signal to a first speaker and a second speaker, respectively, to play the audio scaling signal to a second occupant of the vehicle.

[0329] Item 181 includes the non-transitory computer-readable medium of item 180, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: position a movable mounting structure based on the first position of the first occupant; and receive the first audio signal and the second audio signal from a plurality of microphones mounted on the movable mounting structure.

[0330] Clause 182 includes the non-transitory computer-readable medium of clause 181, wherein the moveable mounting structure comprises a rearview mirror.

[0331] Item 183 includes a non-temporary computer-readable medium of item 181 or item 182, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the multiple microphones, a head orientation of the second occupant, or a combination thereof.

[0332] Clause 184 includes the non-transitory computer-readable medium of clause 183, wherein the zoom direction, the zoom depth, or both are based on the first position of the first occupant.

[0333] Clause 185 includes the non-transitory computer-readable medium of clause 181 or clause 182, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction.

[0334] Clause 186 includes the non-transitory computer-readable medium of clause 185 , wherein the zoom direction is based on the first position of the first occupant.

[0335] Clause 187 includes the non-transitory computer-readable medium of clause 181 or clause 182, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on the scaled depth.

[0336] Clause 188 includes the non-transitory computer-readable medium of clause 187, wherein the zoom depth is based on the first position of the first occupant.

[0337] Clause 189 includes the non-transitory computer-readable medium of clause 181 or clause 182, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a configuration of the plurality of microphones.

[0338] Clause 190 includes the non-transitory computer-readable medium of clause 181 or clause 182, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filter based on a head orientation of the second occupant.

[0339] Item 191 includes a non-transitory computer-readable medium of either item 183 or item 184, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input via an input device indicating the zoom direction, the zoom depth, the first position of the first occupant, or a combination thereof.

[0340] Clause 192 includes the non-transitory computer-readable medium of any of clause 183 or clause 184, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive, via an input device, a user input indicating the zoom direction.

[0341] Clause 193 includes the non-transitory computer-readable medium of any of clause 183 or clause 184, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive, via an input device, a user input indicating the zoom depth.

[0342] Clause 194 includes the non-transitory computer-readable medium of either clause 183 or clause 184, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating the first position of the first occupant via an input device.

[0343] Clause 195 includes the non-transitory computer-readable medium of any of clauses 155 to 194, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the first phase.

[0344] Clause 196 includes the non-transitory computer-readable medium of any of clauses 155 to 195, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the second phase.

[0345] Clause 197 includes the non-transitory computer-readable medium of any of clauses 155 to 196, wherein the audio scaled signal comprises a binaural audio scaled signal.

[0346] Clause 198 includes the non-transitory computer-readable medium of any of clauses 155 to 197, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction, a zoom depth, or both.

[0347] Clause 199 includes the non-transitory computer-readable medium of clause 198, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating the zoom direction, the zoom depth, or both.

[0348] Item 200 includes the non-temporary computer-readable medium of item 198, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating a zoom target; receive sensor data from a depth sensor; and based on the sensor data, determine the zoom direction, the zoom depth, or both of the zoom target.

[0349] Item 201 includes the non-temporary computer-readable medium of item 200, wherein the depth sensor includes an image sensor, wherein the sensor data includes image data, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: perform image recognition on the image data to determine the zoom direction, the zoom depth, or both of the zoom target.

[0350] Item 202 includes the non-transitory computer-readable medium of item 200, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0351] Item 203 includes the non-temporary computer-readable medium of item 202, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom direction, the zoom depth, or both of the zoom target based on the position of the zoom target.

[0352] Clause 204 includes a non-transitory computer-readable medium of any one of clauses 198 to clause 203, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the scaling depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and based on determining that a first energy of the first enhanced audio signal is less than or equal to a second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0353] Item 205 includes the non-temporary computer-readable medium of item 204, wherein applying the spatial filtering based on the zoom direction and the first zoom depth includes applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth includes applying the spatial filtering based on a second set of arrival directions.

[0354] Clause 206 includes the non-transitory computer-readable medium of any of clauses 198 to clause 205, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to select the selected audio signal based on the zoom direction, the zoom depth, or both.

[0355] Clause 207 includes the non-transitory computer-readable medium of any of clauses 155 to 197, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a zoom direction.

[0356] Clause 208 includes the non-transitory computer-readable medium of clause 207, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating the zoom direction.

[0357] Item 209 includes the non-temporary computer-readable medium of item 207, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating a zoom target; receive sensor data from a depth sensor; and determine the zoom direction of the zoom target based on the sensor data.

[0358] Item 210 includes the non-temporary computer-readable medium of item 209, wherein the depth sensor includes an image sensor, wherein the sensor data includes image data, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: perform image recognition on the image data to determine the zoom direction of the zoom target.

[0359] Item 211 includes the non-transitory computer-readable medium of item 209, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0360] Item 212 includes the non-temporary computer-readable medium of item 209, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom direction of the zoom target based on the position of the zoom target.

[0361] Clause 213 includes a non-transitory computer-readable medium of any one of clauses 207 to clause 212, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine a scaling depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and based on determining that a first energy of the first enhanced audio signal is less than or equal to a second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0362] Item 214 includes the non-temporary computer-readable medium of item 213, wherein applying the spatial filtering based on the scaling direction and the first scaling depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the scaling direction and the second scaling depth comprises applying the spatial filtering based on a second set of arrival directions.

[0363] Clause 215 includes the non-transitory computer-readable medium of any of clauses 207 to 214, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to select the selected audio signal based on the scaling direction.

[0364] Clause 216 includes the non-transitory computer-readable medium of any of clauses 155 to 197, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to apply the spatial filtering based on a scaled depth.

[0365] Clause 217 includes the non-transitory computer-readable medium of clause 216, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating the zoom depth.

[0366] Item 218 includes the non-temporary computer-readable medium of item 216, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: receive user input indicating a zoom target; receive sensor data from a depth sensor; and determine the zoom depth of the zoom target based on the sensor data.

[0367] Item 219 includes the non-temporary computer-readable medium of item 218, wherein the depth sensor includes an image sensor, wherein the sensor data includes image data, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: perform image recognition on the image data to determine the zoom depth of the zoom target.

[0368] Clause 220 includes the non-transitory computer-readable medium of clause 218, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0369] Item 221 includes the non-temporary computer-readable medium of item 218, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the zoom depth of the zoom target based on the position of the zoom target.

[0370] Clause 222 includes a non-transitory computer-readable medium of any one of clauses 216 to clause 221, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: determine the scaling depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal, and selecting the first scaling depth as the scaling depth.

[0371] Item 223 includes the non-temporary computer-readable medium of item 222, wherein applying the spatial filtering based on the scaling direction and the first scaling depth comprises applying the spatial filtering based on a first set of arrival directions, and wherein applying the spatial filtering based on the scaling direction and the second scaling depth comprises applying the spatial filtering based on a second set of arrival directions.

[0372] Clause 224 includes the non-transitory computer-readable medium of any of clauses 216 to 223, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to select the selected audio signal based on the zoom depth.

[0373] Clause 225 includes a non-transitory computer-readable medium of any one of clauses 155 to 224, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: apply the spatial filtering to a first subset of the selected audio signals to generate a first enhanced audio signal; apply the spatial filtering to a second subset of the selected audio signals to generate a second enhanced audio signal; and select one of the first enhanced audio signal or the second enhanced audio signal as the enhanced audio signal, the selection being based on determining that a first energy of the enhanced audio signal is less than or equal to a second energy of the other of the first enhanced audio signal or the second enhanced audio signal.

[0374] Clause 226 includes the non-transitory computer-readable medium of clause 225, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: apply the spatial filtering with head shadow effect correction to one of the first subset or the second subset.

[0375] Clause 227 includes the non-transitory computer-readable medium of clause 225, wherein the instructions, when executed by one or more processors, further cause the one or more processors to apply spatial filtering with head shadow effect correction to the first subset.

[0376] Clause 228 includes the non-transitory computer-readable medium of clause 225, wherein the instructions, when executed by one or more processors, further cause the one or more processors to apply spatial filtering with head shadow effect correction to the second subset.

[0377] Clause 229 includes the non-transitory computer-readable medium of any of clauses 155 to 228, wherein the first phase is indicated by first phase values, and wherein each of the first phase values ​​represents a phase of a particular frequency subband of the first audio signal.

[0378] Clause 230 includes a non-transitory computer-readable medium of any one of clauses 155 to 229, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: generate each of the first output signal and the second output signal based at least in part on a first amplitude of the first audio signal, wherein the first amplitude is indicated by a first amplitude value, and wherein each of the first amplitude values ​​represents an amplitude of a particular frequency subband of the first audio signal.

[0379] Clause 231 includes the non-transitory computer-readable medium of any one of clauses 155 to 230, wherein the amplitude of the enhanced audio signal is indicated by third amplitude values, and wherein each of the third amplitude values ​​represents an amplitude of a particular frequency subband of the enhanced audio signal.

[0380] According to clause 232, the device includes: a unit for determining a first phase based on a first audio signal in a first audio signal; a unit for determining a second phase based on a second audio signal in a second audio signal; a unit for applying spatial filtering to a selected audio signal from the first audio signal and the second audio signal to generate an enhanced audio signal; a unit for generating a first output signal comprising combining the amplitude of the enhanced audio signal with the first phase; and a unit for generating a second output signal comprising combining the amplitude of the enhanced audio signal with the second phase, wherein the first output signal and the second output signal correspond to audio scaling signals.

[0381] Clause 233 includes the apparatus of clause 232, further comprising: a unit for receiving the first audio signal from a first plurality of microphones externally mounted to a first earpiece of the headset; and a unit for receiving the second audio signal from a second plurality of microphones externally mounted to a second earpiece of the headset.

[0382] Clause 234 includes the apparatus of clause 233, further comprising: means for applying spatial filtering based on a zoom direction, a zoom depth, a configuration of the first plurality of microphones and the second plurality of microphones, or a combination thereof.

[0383] Clause 235 includes the apparatus of clause 234, further comprising: means for determining the zoom direction, the zoom depth, or both based on a click detected via a touch sensor of the headset.

[0384] Clause 236 includes the apparatus of clause 234 or clause 235, further comprising: a unit for determining the zoom direction, the zoom depth, or both based on movement of the headset.

[0385] Clause 237 includes the apparatus of clause 233, further comprising: a unit for applying the spatial filtering based on a scaling direction.

[0386] Clause 238 includes the apparatus of clause 237, further comprising: means for determining the zoom direction based on a click detected via a touch sensor of the headset.

[0387] Clause 239 includes the apparatus of clause 237 or clause 238, further comprising: a unit for determining the zoom direction based on movement of the headset.

[0388] Clause 240 includes the apparatus of clause 233, further comprising: means for applying the spatial filtering based on a scaled depth.

[0389] Clause 241 includes the apparatus of clause 240, further comprising: means for determining the zoom depth based on a click detected via a touch sensor of the headset.

[0390] Clause 242 includes the apparatus of clause 240 or clause 241, further comprising: means for determining the zoom depth based on movement of the headset.

[0391] Clause 243 includes the apparatus of clause 233, further comprising: means for applying the spatial filtering based on a configuration of the first plurality of microphones and the second plurality of microphones.

[0392] Item 244 includes the device of any one of items 232 to 243, wherein the unit for determining the first phase, the unit for determining the second phase, the unit for applying the spatial filtering, the unit for generating the first output signal and the unit for generating the second output signal are integrated into headphones.

[0393] Clause 245 includes the apparatus of any one of clauses 232 to 244, further comprising: a unit for providing the first output signal to a first speaker of a first earpiece of the headset; and a unit for providing the second output signal to a second speaker of a second earpiece of the headset.

[0394] Clause 246 includes the apparatus of clause 232 or clause 245, further comprising: a unit for decoding audio data of a playback file to generate the first audio signal and the second audio signal.

[0395] Clause 247 includes the apparatus of clause 246, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprises a unit for applying the spatial filtering based on a zoom direction, a zoom depth, the position information, or a combination thereof.

[0396] Clause 248 includes the apparatus of clause 246, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising means for applying the spatial filtering based on a zoom direction.

[0397] Clause 249 includes the apparatus of clause 246, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising means for applying spatial filtering based on the scaled depth.

[0398] Clause 250 includes the apparatus of clause 246, wherein the audio data comprises position information indicating a position of a source of each of the first audio signal and the second audio signal, and further comprising means for applying the spatial filtering based on the position information.

[0399] Clause 251 includes the apparatus of clause 246, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprises a unit for applying the spatial filtering based on a zoom direction, a zoom depth, the multi-channel audio representation, or a combination thereof.

[0400] Clause 252 includes the apparatus of clause 246, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprising means for applying the spatial filtering based on a zoom direction.

[0401] Clause 253 includes the apparatus of clause 246, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprising means for applying the spatial filtering based on a scaled depth.

[0402] Clause 254 includes the apparatus of clause 246, wherein the audio data comprises a multi-channel audio representation of one or more audio sources, and further comprising means for applying the spatial filtering based on the multi-channel audio representation.

[0403] Clause 255 includes the apparatus of any of clauses 251 to 254, wherein the multi-channel audio representation corresponds to ambient sound data.

[0404] Clause 256 includes the apparatus of any of clause 232, clause 244, or clause 245, further comprising: means for receiving audio data representing streaming data from a modem; and means for decoding the audio data to generate the first audio signal and the second audio signal.

[0405] Item 257 includes the device of item 232 or any one of items 246 to item 256, and also includes: a unit for applying the spatial filtering based on a first position of a first occupant of a vehicle; and a unit for providing the first output signal and the second output signal to a first speaker and a second speaker, respectively, to play the audio scaling signal to a second occupant of the vehicle.

[0406] Clause 258 includes the apparatus of clause 257, further comprising: a unit for positioning a movable mounting structure based on the first position of the first occupant; and a unit for receiving the first audio signal and the second audio signal from a plurality of microphones mounted on the movable mounting structure.

[0407] Clause 259 includes the apparatus of clause 258, wherein the movable mounting structure comprises a rearview mirror.

[0408] Clause 260 includes the apparatus of clause 258 or clause 259, further comprising: a unit for applying the spatial filtering based on a zoom direction, a zoom depth, a configuration of the plurality of microphones, a head orientation of the second occupant, or a combination thereof.

[0409] Clause 261 includes the apparatus of clause 260, wherein the zoom direction, the zoom depth, or both are based on the first position of the first occupant.

[0410] Clause 262 includes the apparatus of clause 258 or clause 259, further comprising: means for applying the spatial filtering based on a scaling direction.

[0411] Clause 263 includes the apparatus of clause 262, wherein the zoom direction is based on the first position of the first occupant.

[0412] Clause 264 includes the apparatus of clause 258 or clause 259, further comprising: means for applying the spatial filtering based on the scaled depth.

[0413] Clause 265 includes the apparatus of clause 264, wherein the zoom depth is based on the first position of the first occupant.

[0414] Clause 266 includes the apparatus of clause 258 or clause 259, further comprising: means for applying the spatial filtering based on a configuration of the plurality of microphones.

[0415] Clause 267 includes the apparatus of clause 258 or clause 259, further comprising: a unit for applying the spatial filtering based on the head orientation of the second occupant.

[0416] Clause 268 includes the apparatus of either clause 260 or clause 261 , further comprising: means for receiving a user input via an input device, the user input indicating the zoom direction, the zoom depth, the first position of the first occupant, or a combination thereof.

[0417] Clause 269 includes the apparatus of either clause 260 or clause 261, further comprising: means for receiving user input indicating the zoom direction via an input device.

[0418] Clause 270 includes the apparatus of either clause 260 or clause 261 , further comprising means for receiving, via an input device, a user input indicating the zoom depth.

[0419] Clause 271 includes the apparatus of either clause 260 or clause 261 , further comprising: an input device coupled to the one or more processors, and further comprising: means for receiving user input indicating the first position of the first occupant via the input device.

[0420] Clause 272 includes the apparatus of any of clauses 232 to 271, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the first phase.

[0421] Clause 273 includes the apparatus of any of clauses 232 to 272, wherein the amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the second phase.

[0422] Clause 274 includes the apparatus of any one of clauses 232 to 273, wherein the audio scaled signal comprises a binaural audio scaled signal.

[0423] Clause 275 includes the apparatus of any of clauses 232 to 274, further comprising: means for applying the spatial filtering based on a scaling direction, a scaling depth, or both.

[0424] Clause 276 includes the apparatus of clause 275, further comprising: means for receiving user input indicating the zoom direction, the zoom depth, or both.

[0425] Item 277 includes the apparatus of item 275, further comprising: a unit for receiving user input indicating a zoom target; a unit for receiving sensor data from a depth sensor; and a unit for determining the zoom direction, the zoom depth, or both of the zoom target based on the sensor data.

[0426] Clause 278 includes the apparatus of clause 277, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and further comprising: a unit for performing image recognition on the image data to determine the zoom direction, the zoom depth, or both of the zoom target.

[0427] Clause 279 includes the device of clause 277, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0428] Item 280 includes the apparatus of item 279, wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating the position of the zoom target, and further includes: a unit for determining the zoom direction, the zoom depth, or both of the zoom target based on the position of the zoom target.

[0429] Clause 281 includes the apparatus of any one of clauses 275 to clause 280, further comprising: a unit for determining the scaling depth comprising: a unit for applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; a unit for applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and a unit for selecting the first enhanced audio signal as the enhanced audio signal and selecting the first scaling depth as the scaling depth based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal.

[0430] Item 282 includes the device of item 281, wherein the unit for applying the spatial filtering based on the scaling direction and the first scaling depth includes a unit for applying the spatial filtering based on a first set of arrival directions, and wherein the unit for applying the spatial filtering based on the scaling direction and the second scaling depth includes a unit for applying the spatial filtering based on a second set of arrival directions.

[0431] Clause 283 includes the apparatus of any of clauses 275 to 282, further comprising: means for selecting the selected audio signal based on the zoom direction, the zoom depth, or both.

[0432] Clause 284 includes the apparatus of any one of clauses 232 to 274, further comprising: means for applying the spatial filtering based on a zoom direction.

[0433] Clause 285 includes the apparatus of clause 284, further comprising: a unit for receiving user input indicating the zoom direction.

[0434] Clause 286 includes the apparatus of clause 284, further comprising: a unit for receiving user input indicating a zoom target; a unit for receiving sensor data from a depth sensor; and a unit for determining the zoom direction of the zoom target based on the sensor data.

[0435] Clause 287 includes the apparatus of clause 286, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and further comprising: a unit for performing image recognition on the image data to determine the zoom direction of the zoom target.

[0436] Clause 288 includes the device of clause 286, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0437] Clause 289 includes the apparatus of clause 286, wherein the depth sensor comprises a position sensor, wherein the sensor data comprises position data indicating a position of the zoom target, and further comprising: a unit for determining the zoom direction of the zoom target based on the position of the zoom target.

[0438] Clause 290 includes the apparatus of any one of clauses 284 to 289, further comprising: a unit for determining a scaling depth comprising: a unit for applying the spatial filtering to the selected audio signal based on the scaling direction and the first scaling depth to generate a first enhanced signal; a unit for applying the spatial filtering to the selected audio signal based on the scaling direction and the second scaling depth to generate a second enhanced signal; and a unit for selecting the first enhanced audio signal as the enhanced audio signal and selecting the first scaling depth as the scaling depth based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal.

[0439] Clause 291 includes the apparatus of clause 290, wherein the unit for applying the spatial filtering based on the scaling direction and the first scaling depth includes a unit for applying the spatial filtering based on a first set of arrival directions, and wherein the unit for applying the spatial filtering based on the scaling direction and the second scaling depth includes a unit for applying the spatial filtering based on a second set of arrival directions.

[0440] Clause 292 includes the apparatus of any of clauses 284 to clause 291, further comprising: means for selecting the selected audio signal based on the scaling direction.

[0441] Clause 293 includes the apparatus of any of clauses 232 to 274, further comprising: means for applying the spatial filtering based on a scaled depth.

[0442] Clause 294 includes the apparatus of clause 293, further comprising: means for receiving user input indicating the zoom depth.

[0443] Clause 295 includes the apparatus of clause 293, further comprising: a unit for receiving user input indicating a zoom target; a unit for receiving sensor data from a depth sensor; and a unit for determining the zoom depth of the zoom target based on the sensor data.

[0444] Clause 296 includes the apparatus of clause 295, wherein the depth sensor comprises an image sensor, wherein the sensor data comprises image data, and further comprising: means for performing image recognition on the image data to determine the zoom depth of the zoom target.

[0445] Clause 297 includes the device of clause 295, wherein the depth sensor comprises an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

[0446] Clause 298 includes the apparatus of clause 295, wherein the depth sensor comprises a position sensor, wherein the sensor data comprises position data indicating a position of the zoom target, and further comprising: a unit for determining the zoom depth of the zoom target based on the position of the zoom target.

[0447] Clause 299 includes the apparatus of any one of clauses 293 to 298, further comprising: a unit for determining the scaling depth comprising: a unit for applying the spatial filtering to the selected audio signal based on a scaling direction and a first scaling depth to generate a first enhanced signal; a unit for applying the spatial filtering to the selected audio signal based on the scaling direction and a second scaling depth to generate a second enhanced signal; and a unit for selecting the first enhanced audio signal as the enhanced audio signal and selecting the first scaling depth as the scaling depth based on determining that the first energy of the first enhanced audio signal is less than or equal to the second energy of the second enhanced audio signal.

[0448] Item 300 includes the apparatus of item 299, wherein the unit for applying the spatial filtering based on the scaling direction and the first scaling depth includes a unit for applying the spatial filtering based on a first set of arrival directions, and wherein the unit for applying the spatial filtering based on the scaling direction and the second scaling depth includes a unit for applying the spatial filtering based on a second set of arrival directions.

[0449] Clause 301 includes the apparatus of any of clauses 293 to 300, further comprising: means for selecting the selected audio signal based on the zoom depth.

[0450] Clause 302 includes the apparatus of any one of clauses 232 to clause 301, further comprising: a unit for applying the spatial filtering to a first subset of the selected audio signals to generate a first enhanced audio signal; a unit for applying the spatial filtering to a second subset of the selected audio signals to generate a second enhanced audio signal; and a unit for selecting one of the first enhanced audio signal or the second enhanced audio signal as the enhanced audio signal, the selection being based on determining that a first energy of the enhanced audio signal is less than or equal to a second energy of the other of the first enhanced audio signal or the second enhanced audio signal.

[0451] Clause 303 includes the apparatus of clause 302, further comprising: means for applying the spatial filtering with head shadow effect correction to one of the first subset or the second subset.

[0452] Clause 304 includes the apparatus of clause 302, further comprising means for applying the spatial filtering with head shadow effect correction to the first subset.

[0453] Clause 305 includes the apparatus of clause 302, further comprising: means for applying the spatial filtering with head shadow effect correction to the second subset.

[0454] Clause 306 includes the apparatus of any of clauses 232 to 305, wherein the first phase is indicated by first phase values, and wherein each of the first phase values ​​represents a phase of a particular frequency subband of the first audio signal.

[0455] Clause 307 includes the apparatus of any one of clauses 232 to clause 306, further comprising: a unit for generating each of the first output signal and the second output signal based at least in part on a first amplitude of the first audio signal, wherein the first amplitude is indicated by a first amplitude value, and wherein each of the first amplitude values ​​represents an amplitude of a specific frequency subband of the first audio signal.

[0456] Clause 308 includes the apparatus of any of clauses 232 to 307, wherein the amplitude of the enhanced audio signal is indicated by third amplitude values, and wherein each of the third amplitude values ​​represents an amplitude of a particular frequency subband of the enhanced audio signal.

[0457] It will also be appreciated by the skilled person that the various illustrative logic blocks, configurations, modules, circuits, and algorithmic steps described in conjunction with the implementations disclosed herein may be implemented as electronic hardware, computer software executed by a processor, or a combination of the two. Various illustrative components, blocks, configurations, modules, circuits, and steps have been generally described above around their functions. Whether such functions are implemented as hardware or processor executable instructions depends on the specific application and the design constraints imposed on the entire system. A skilled person may implement the described functions in a varying manner for each specific application, and such implementation decisions will not be interpreted as causing a departure from the scope of the present disclosure.

[0458] The steps of the method or algorithm described in conjunction with the implementation disclosed herein can be directly embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software module can reside in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of non-transitory storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative, the storage medium can be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). The ASIC can be located in a computing device or a user terminal. Alternatively, the processor and the storage medium can be located in a computing device or a user terminal as discrete components.

[0459] The previous description of the disclosed aspects is provided to enable those skilled in the art to implement or use the disclosed aspects. Various modifications to these aspects will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but to be given the widest possible scope consistent with the principles and novel features defined by the claims below.

Claims

1. A computing device, comprising: a memory configured to store instructions; as well as One or more processors configured to execute the instructions to perform the following operations: determining a first phase based on a first audio signal of the plurality of first audio signals; determining a second phase based on a second audio signal of the plurality of second audio signals; applying spatial filtering to selected audio signals of the plurality of first audio signals and the plurality of second audio signals, wherein the spatial filtering is applied to a first subset of the selected audio signals to generate a first enhanced audio signal, and wherein the spatial filtering is applied to a second subset of the selected audio signals to generate a second enhanced audio signal; selecting one of the first enhanced audio signal or the second enhanced audio signal as an enhanced audio signal; generating a first output signal comprising combining the amplitude of the enhanced audio signal with the first phase; and A second output signal is generated, the second output signal comprising the amplitude of the enhanced audio signal combined with the second phase.

2. The computing device of claim 1, wherein: The one or more processors are further configured to: receiving the plurality of first audio signals from a first plurality of microphones mounted externally to a first earpiece of the headset; as well as The plurality of second audio signals are received from a second plurality of microphones mounted externally to a second earpiece of the headset.

3. The computing device of claim 2, wherein: The one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the first plurality of microphones and the second plurality of microphones, or a combination thereof.

4. The computing device of claim 3, wherein: The one or more processors are configured to determine the zoom direction, the zoom depth, or both based on a click detected via a touch sensor of the headset.

5. The computing device of claim 3, wherein: The one or more processors are configured to determine the zoom direction, the zoom depth, or both based on movement of the headset.

6. The computing device of claim 1, wherein: The one or more processors are integrated into the headset.

7. The computing device of claim 1, wherein: The one or more processors are further configured to: providing the first output signal to a first speaker of a first earpiece of a headset; and The second output signal is provided to a second speaker of a second earpiece of the headset.

8. The computing device of claim 1, wherein: The one or more processors are further configured to decode audio data of a playback file to generate the plurality of first audio signals and the plurality of second audio signals.

9. The computing device of claim 8, wherein: The audio data comprises position information indicating a position of a source of each of the plurality of first audio signals and the plurality of second audio signals, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, the position information, or a combination thereof.

10. The computing device of claim 8, wherein: The audio data comprises a multi-channel audio representation of one or more audio sources, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, the multi-channel audio representation, or a combination thereof.

11. The computing device of claim 10, wherein: The multi-channel audio representation corresponds to ambient sound data.

12. The computing device of claim 1 , further comprising: A modem is coupled to the one or more processors, the modem being configured to provide audio data to the one or more processors based on the received streaming data, wherein the one or more processors are configured to decode the audio data to generate the plurality of first audio signals and the plurality of second audio signals.

13. The computing device of claim 1, wherein: The one or more processors are integrated into a vehicle, and wherein the one or more processors are configured to: applying the spatial filtering based on a first position of a first occupant of the vehicle; and The first and second output signals are provided to first and second speakers, respectively, to play an audio scaled signal to a second occupant of the vehicle, and the audio scaled signal is generated based on the first and second output signals.

14. The computing device of claim 13, wherein: The one or more processors are configured to: positioning a movable mounting structure based on the first position of the first occupant; and The plurality of first audio signals and the plurality of second audio signals are received from a plurality of microphones mounted on the movable mounting structure.

15. The computing device of claim 14, wherein: The movable mounting structure includes a rearview mirror.

16. The computing device of claim 14, wherein: The one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the plurality of microphones, a head orientation of the second occupant, or a combination thereof.

17. The computing device of claim 16, wherein: The zoom direction, the zoom depth, or both are based on the first position of the first occupant.

18. The computing device of claim 16, further comprising: An input device coupled to the one or more processors, wherein the one or more processors are configured to receive user input via the input device indicating the zoom direction, the zoom depth, the first position of the first occupant, or a combination thereof.

19. The computing device of claim 1, wherein: The amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the first phase.

20. The computing device of claim 1, wherein: The amplitude of the enhanced audio signal is based on a first amplitude of the first audio signal and a second amplitude of the second audio signal combined with the second phase.

21. The computing device of claim 13, wherein: The scaled audio signal comprises a scaled binaural audio signal.

22. The computing device of claim 1, wherein: The one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, or both.

23. The computing device of claim 22, wherein: The one or more processors are configured to receive user input indicating the zoom direction, the zoom depth, or both.

24. The computing device of claim 22, further comprising: a depth sensor coupled to the one or more processors, wherein the one or more processors are configured to: receiving user input indicating a zoom target; receiving sensor data from the depth sensor; and The zoom direction, the zoom depth, or both of the zoom target are determined based on the sensor data.

25. The computing device of claim 24, wherein: The depth sensor comprises an image sensor, wherein the sensor data comprises image data, and wherein the one or more processors are configured to perform image recognition on the image data to determine the zoom direction, the zoom depth, or both of the zoom target.

26. The computing device of claim 24, wherein: The depth sensor includes an ultrasonic sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.

27. The computing device of claim 24, wherein: The depth sensor comprises a position sensor, wherein the sensor data comprises position data indicating a position of the zoom target, and wherein the one or more processors are configured to determine the zoom direction, the zoom depth, or both of the zoom target based on the position of the zoom target.

28. The computing device of claim 22, wherein: The one or more processors are configured to: determine the zoom depth, including: applying the spatial filtering to the selected audio signal based on the scaling direction and a first scaling depth to generate the first enhanced audio signal; applying the spatial filtering to the selected audio signal based on the scaling direction and a second scaling depth to generate the second enhanced audio signal; and Based on determining that a first energy of the first enhancement audio signal is less than or equal to a second energy of the second enhancement audio signal, the first enhancement audio signal is selected as the enhancement audio signal, and the first scaling depth is selected as the scaling depth.

29. The computing device of claim 28, wherein: Applying the spatial filter based on the zoom direction and the first zoom depth comprises applying the spatial filter based on a first set of directions of arrival, and wherein applying the spatial filter based on the zoom direction and the second zoom depth comprises applying the spatial filter based on a second set of directions of arrival.

30. The computing device of claim 22, wherein: The one or more processors are configured to select the selected audio signal based on the zoom direction, the zoom depth, or both.

31. The computing device of claim 1, wherein: The one or more processors are configured to select the enhanced audio signal based on determining that a first energy of the enhanced audio signal is less than or equal to a second energy of the other of the first enhanced audio signal or the second enhanced audio signal.

32. The computing device of claim 1, wherein: The one or more processors are configured to apply the spatial filtering with head shadowing effect correction to one of the first subset or the second subset.

33. The computing device of claim 1, wherein: The first phase is indicated by first phase values, and wherein each of the first phase values ​​represents a phase of a specific frequency sub-band of the first audio signal.

34. The computing device of claim 1, wherein: The one or more processors are configured to generate each of the first output signal and the second output signal based at least in part on a first amplitude of the first audio signal, wherein the first amplitude is indicated by a first amplitude value, and wherein each of the first amplitude values ​​represents an amplitude of a particular frequency subband of the first audio signal.

35. The computing device of claim 1, wherein: The amplitude of the enhanced audio signal is indicated by third amplitude values, and wherein each of the third amplitude values ​​represents an amplitude of a specific frequency sub-band of the enhanced audio signal.