Acoustic crosstalk cancellation based on user location and orientation within environment

Abstract

Various embodiments disclose a computer-implemented method comprising: determining a first location and a first orientation of a user in an environment; identifying a first point based on a first position and a first orientation of the user in a dimension map, the dimension map associating a plurality of transfer functions with a corresponding plurality of points corresponding to positions and orientations in a multi-dimensional space; determining at least one crosstalk cancellation filter based on the plurality of transfer functions; generating a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; and transmitting the plurality of audio signals to the plurality of speakers for output.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to audio reproduction and, more particularly, to acoustic crosstalk cancellation based on a user's location and orientation within an environment. Background Art

[0002] An audio processing system uses one or more speakers to produce sound in a given space. The one or more speakers generate a sound field in which a user in the environment receives the sound included in the sound field. The one or more speakers reproduce sound based on an input signal that typically includes at least two channels, such as a left channel and a right channel. The left channel is intended to be received by the user's left ear, while the right channel is intended to be received by the user's right ear. Binaural rendering algorithms for using the one or more speakers to produce sound rely on crosstalk cancellation algorithms. These crosstalk cancellation algorithms rely on measurements made at a specific location, or they rely on mathematical models that attempt to characterize the audio transmission path from the speaker to the entrance of the user's ear canal.

[0003] At least one disadvantage of conventional audio playback systems that rely on convention is crosstalk between the left channel and the right channel. In other words, the sound generated by the left channel of the one or more speakers in the environment is received by the user's right ear. Similarly, the sound generated by the right channel of the one or more speakers in the environment is received by the user's left ear. Some audio processing and playback systems utilize conventional crosstalk cancellation techniques. Some techniques are highly focused on functioning at a specific point in three-dimensional space and will fail if the user moves or rotates his or her head. Other techniques rely on parametric models to characterize the geometry of the given three-dimensional space in which the user is present. However, these techniques are overturned or provide poor crosstalk cancellation performance. As a result, conventional techniques for reducing crosstalk when playing audio in three-dimensional space cannot adequately cope with the movement of the user.

[0004] As described above, there is a need in the art for more effective techniques to reduce crosstalk when generating sound received by a user in a three-dimensional space within an environment. Summary of the Invention

[0005] Various embodiments disclose a computer-implemented method that includes: determining a first position and a first orientation of a user within an environment; identifying a first point based on the first position and the first orientation of the user in a dimensional map that associates a plurality of transfer functions with corresponding plurality of points corresponding to positions and orientations in a multi-dimensional space; determining at least one crosstalk cancellation filter based on the plurality of transfer functions; generating a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; and transmitting the plurality of audio signals to the plurality of speakers for output.

[0006] Other embodiments particularly provide one or more non - transitory computer - readable media and systems configured to implement the methods set forth above.

[0007] At least one technical advantage of the disclosed technology over the prior art is that, using the disclosed technology, an audio processing system can select a transfer function to be applied to each audio channel, which modifies the audio output of the one or more speakers to improve the performance of crosstalk cancellation. The transfer function modifies the audio input, which is then played by the one or more speakers of the playback system. By improving the performance of crosstalk cancellation, spectral distortion caused by user movement is reduced. Additionally, the audio intended to be received by the user's left and right ears respectively more accurately represents the audio input output by the audio processing and playback system. These technical advantages provide one or more technological advancements over prior art methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to enable a manner of understanding the above - described features of the various embodiments in detail, the inventive concept briefly outlined above may be described in more specific terms by reference to the various embodiments, some of which are illustrated in the drawings. However, it should be noted that the drawings only show typical embodiments of the inventive concept and should not be considered to limit the scope in any way, and there are other equivalent embodiments.

[0009] Figure 1 is a schematic diagram showing an audio processing system according to various embodiments.

[0010] Figure 2 shows an example of how a listener observes crosstalk from an input signal generated by one or more speakers.

[0011] Figure 3 shows an example of a filter that performs crosstalk cancellation based on the observed position and orientation of a listener in three - dimensional space.

[0012] Figure 4 shows a flowchart of method steps for selecting a transfer function for configuring a filter that performs crosstalk cancellation according to one or more embodiments.

[0013] Figure 5 shows a flowchart of method steps for selecting a transfer function for configuring a filter that performs crosstalk cancellation according to one or more embodiments. DETAILED DESCRIPTION

[0014] In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of ordinary skill in the art that the inventive concept may be practiced without one or more of these specific details.

[0015] Figure 1 FIG. is a schematic diagram showing an audio processing system 100 according to various embodiments. As shown, the audio processing system 100 includes, but is not limited to, a computing device 110, an audio source 140, one or more sensors 150, and one or more speakers 160. The computing device 110 includes, but is not limited to, a processing unit 112 and a memory 114. The memory 114 stores (but is not limited to) a crosstalk cancellation application 120, a transfer function 132, a dimension map 134, and one or more filters 138.

[0016] In operation, the audio processing system 100 processes sensor data from one or more sensors 150 to track the position of one or more listeners within a listening environment. The one or more sensors 150 track the position of the listener's head in three-dimensional space, as well as the pitch, yaw, and roll of the listener's head, which are used to locate the relative positions of the user's left and right ears, respectively. Based on the position and / or orientation of the listener's head within the three-dimensional environment, the crosstalk cancellation application 120 selects one or more transfer functions 132 for one or more filters 138, which are used to process the audio source 140 for playback by one or more speakers 160 associated with the audio processing system 100. Additionally, if the position of the listener's head in three-dimensional space changes during the playback of the audio source 140, the crosstalk cancellation application 120 selects different transfer functions 132 and possibly different filters 138, which are used to process the audio source 140 for playback via one or more speakers 160.

[0017] The computing device 110 is a device that drives the speakers 160 to generate a sound field for the listener, in part, by playing the audio source 140. In various embodiments, the computing device 110 is an audio processing unit in a home theater system, a soundbar, a vehicle system, etc. In some embodiments, the computing device 110 is included in one or more devices, such as consumer products ( For example , portable speakers, products for competitions, etc.), vehicles ( For example , the main unit of a car, a truck, a van, etc.), smart home devices ( For example , smart lighting systems, security systems, digital assistants, etc.), communication systems ( For example , conference call systems, video conferencing systems, speaker amplification systems, etc.), etc. In various embodiments, the computing device 110 is located in various environments, including but not limited to indoor environments ( For example , living rooms, conference rooms, convention halls, home offices, etc.) and / or outdoor environments ( For example , patios, rooftops, gardens, etc.).

[0018] The processing unit 112 can be any suitable processor, such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), and / or any other type of processing unit or combination of processing units, such as a CPU configured to operate in conjunction with a GPU. Generally, the processing unit 112 can be any technically feasible hardware unit capable of processing data and / or executing software applications.

[0019] The memory 114 can include random access memory (RAM) modules, flash memory cells, or any other type of memory cells or combinations thereof. The processing unit 112 is configured to read data from the memory 114 and write data to the memory. In various embodiments, the memory 114 includes non-volatile memory, such as optical drives, magnetic drives, flash drives, or other storage devices. In some embodiments, a separate data repository, such as an external data repository (“cloud storage”) included in a network, can supplement the memory 114. The crosstalk cancellation application 120 within the memory 114 can be executed by the processing unit 112 to implement the overall functionality of the computing device 110 and thus overall coordinate the operation of the audio processing device 100. In various embodiments, an interconnect bus (not shown) connects the processing unit 112, the memory 114, the speaker 160, the sensor 150, and any other components of the computing device 110.

[0020] The crosstalk cancellation application 120 determines the position of the listener within the listening environment and selects parameters for one or more filters 138 (such as one or more transfer functions 132) to generate a sound field for the position of the listener. The transfer function 132 is selected to minimize or eliminate crosstalk. The transfer function 132 causes the filter 138 to generate audio in the sound field such that the left ear of the listener perceives the left channel while minimizing crosstalk from the right channel. Similarly, the transfer function 132 causes the filter 138 to generate audio in the sound field such that the right ear of the listener perceives the right channel while minimizing crosstalk from the left channel. In various embodiments, the crosstalk cancellation application utilizes sensor data from the sensors 150 to identify the position of the listener, particularly the position of the listener's head. Based on the position and orientation of the listener, the crosstalk cancellation application 120 selects appropriate filters 138 and transfer functions 132 for processing the audio source 140 for playback. In some embodiments, the crosstalk cancellation application 120 sets the parameters for a plurality of filters 138 corresponding to a plurality of speakers 160. For example, a first transfer function 132 may be used by a first filter 138 for audio played by the first speaker 160, while a second transfer function 132 may be used by a second filter 138 for audio played by the second speaker 160. In other embodiments, a filter network is utilized such that the signals used to drive each speaker 160 travel through a network of a plurality of filters. Additionally or alternatively, the crosstalk cancellation application 120 tracks the positions and orientations of multiple listeners.

[0021] The filter 138 includes one or more filters that modify the input audio source 140. In various embodiments, a given filter 138 modifies the input audio signal by modifying the energy within a particular frequency range, adding directional information, etc. For example, the filter 138 may include filter parameters such as a set of values that modify the operating characteristics of the filter 138 ( For example , center frequency, gain, Q factor, cut-off frequency, etc.). In some embodiments, the filter parameters include one or more digital signal processing (DSP) coefficients that direct the generated sound wave to a particular direction. In such cases, the generated filtered audio signal is used to generate a sound wave in the direction specified in the filtered audio signal. For example, one or more speakers 160 reproduce the audio using one or more filtered audio signals to generate a sound field. In some embodiments, the crosstalk cancellation application 120 sets separate filter parameters, such as selecting different transfer functions 132 for separate filters 138 of different speakers 160. In such instances, one or more speakers 160 use the separate filters 138 to generate a sound field. For example, each filter 138 may generate a filtered audio signal for a single speaker 160 within the listening environment.

[0022] The transfer function 132 includes one or more transfer functions for configuring one or more filters 138 selected by the crosstalk cancellation application 120 to process an input signal (such as a channel of the audio source 140) to produce an output signal for driving the speaker 160. Different transfer functions 132 are utilized depending on the position and orientation of the listener in three-dimensional space.

[0023] In some embodiments, the dimensionality map 134 maps a given position (such as inside a vehicle) within the three-dimensional space to filter parameters for one or more filters 138 (such as one or more finite impulse response (FIR) filters). In various embodiments, the crosstalk cancellation application 120 determines the position and orientation of the listener based on data from the sensors 150 and identifies the transfer function 132 or other filter parameters of the filters 138 corresponding to each speaker 160. Then, when the listener's head moves, the crosstalk cancellation application 120 updates the filter parameters of a particular speaker ( For example , the first filter 138(1) for the first speaker 160(1)). For example, the crosstalk cancellation application 120 may initially generate the filter parameters for a set of filters 138. Upon determining that the listener's head has moved to a new position or orientation, the crosstalk cancellation application 120 then determines whether any of the speakers 160 requires an update to the corresponding filter 138. The crosstalk cancellation application 120 updates the filter parameters of any filter 138 that needs to be updated. In some embodiments, the crosstalk cancellation application 120 generates each of the filters 138 independently. For example, upon determining that the listener has moved, the crosstalk cancellation application 120 may update a single filter 138 ( For example , the filter 138(1) for a particular speaker 160 ( For example , 160(1)). Alternatively, the crosstalk cancellation application 120 updates multiple filters 138.

[0024] The dimensionality map 134 includes a plurality of points representing positions and orientations in the three-dimensional space ( For example, a point in a six - dimensional space identified by x, y, and z position coordinates and three roll, pitch, and yaw orientations). The dimensional map 134 maps positions relative to a reference position in a given environment. The dimensional map 134 further maps orientations relative to a reference orientation in the environment. The dimensional map 134 can be generated by making acoustic measurements on filter parameters (such as transfer function 132) that minimize or eliminate crosstalk in three - dimensional space. Then the dimensional map 134 is stored on the audio processing system 100 and is used to configure the filter 138 utilized by the computing device 110 to minimize or eliminate crosstalk during the playback of the audio source 140. In some embodiments, the dimensional map 134 includes specific coordinates relative to a reference point. For example, the dimensional map 134 can store the potential positions and orientations of the listener's head as distances and angles from a specific reference point. In some embodiments, the dimensional map 134 can include additional orientation information representing the orientation of the listener's head, such as pitch, yaw, and roll. The dimensional map 134 can also include a set of angles relative to the normal orientation of the listener's head ( For example , {µ, φ, ψ}). In such instances, the corresponding positions and orientations defined by the points in the dimensional map 134 are associated with one or more transfer functions 132 for the filter 138. In one example, the dimensional map 134 is structured as a set of points, each point associated with a specific position and orientation in the environment. Each of the points is associated with one or more filters 138 and / or transfer functions 132 that can be used in each of the speakers 160 to reduce or eliminate crosstalk.

[0025] The crosstalk cancellation application 120 selects a transfer function 132 to configure the filter 138, where the transfer function 132 is identified by the dimensional map 134. The transfer function 132 is used to configure the filter 138 that processes the audio source 140. The transfer function 132 is identified based on a mathematical distance (such as the centroid distance) of a set of points representing the position and orientation of the listener's head from one or more points in the set of points in the dimensional map 134. In one example, a given position and orientation of the user are characterized by coordinates in a six - dimensional space. In some embodiments, a graphical search algorithm (such as Delaunay triangulation) is then used to identify the set of nearest points to the coordinates within the dimensional map 134. The centroid distance from each of the nearest points is determined, and the filter 138 is configured using the transfer function 132 associated with the nearest points in the dimensional map 134, which filters the audio signal 140 being played.

[0026] As another example, a simplified method for identifying the transfer function 132 includes reducing the number of dimensions of the user position and orientation considered when identifying the set of transfer functions specified by the dimensionality map 134. As described above, the dimensionality map 134 includes a set of points in a six-dimensional space to account for three parameters representing position and three parameters representing orientation. To reduce the mathematical complexity, a simplified set of parameters representing the user position and orientation can be considered. For example, one or more parameters representing orientation can be removed, and the set of nearest points can be determined based on the mathematical distance from the coordinates of the position and orientation of the listener's head in the table to one or more points in the set of points in the dimensionality map 134. Examples of coordinates that can be removed include yaw angle, pitch angle, and / or roll angle. In one scenario, only the position and yaw angle of the user's head are considered, thus reducing the complexity to considering four dimensions. As another example, only the position of the user's head is considered along with the yaw angle and pitch angle, thus reducing the complexity to five dimensions.

[0027] As another example, an alternative simplified method for identifying the transfer function 132 includes reducing the dimensionality of the dimensionality map 134. As described above, the dimensionality map 134 includes a set of points in a six-dimensional space to account for three parameters representing position and three parameters representing orientation. To reduce the mathematical complexity, a dimensionality map 134 can be generated and utilized that includes a set of points mapped in a three-dimensional, four-dimensional, or five-dimensional space. For example, the dimensionality map 134 can map only the position of the user's head in three-dimensional space and the yaw angle representing orientation, resulting in a four-dimensional map. As another example, the dimensionality map 134 maps only the position of the user's head and two parameters characterizing orientation, thus reducing the complexity of the dimensionality map 134 to five dimensions.

[0028] As another example of a simplified method for reducing the dimensionality of the dimensionality map 134, a plurality of dimensionality maps 134 can be utilized that include three dimensions representing position in three-dimensional space. Each of the three-dimensional maps is associated with a specific orientation parameter or range of orientation parameters. For example, each of the three-dimensional maps is associated with a yaw angle or range of yaw angles. In one scenario, the first three-dimensional map is associated with a yaw angle from zero to ten degrees, the second three-dimensional map is associated with a yaw angle greater than ten to twenty degrees, and so on. In this method, based on the detected yaw angle of the user's head, a three-dimensional map is selected. Then, based on the coordinates of the detected position of the user, a point corresponding to the transfer function 132 within the three-dimensional map is identified, and the filter 138 is configured using the transfer function 132.

[0029] The sensor 150 includes various types of sensors that acquire data about the listening environment. For example, the computing device 110 can include means for receiving several types of sounds ( For example, an auditory sensor for subsonic pulses, ultrasonic waves, voice commands, etc.). In some embodiments, sensor 150 includes other types of sensors. Other types of sensors include optical sensors (such as RGB cameras, time-of-flight cameras, infrared cameras, depth cameras, quick response (QR) code tracking systems), motion sensors (such as accelerometers or inertial measurement units (IMUs)) ( For example , triaxial accelerometers, gyroscopic sensors, and / or magnetometers), pressure sensors, etc. Additionally, in some embodiments, sensor 150 may include wireless sensors (including radio frequency (RF) sensors ( For example , sonar and radar)) and / or wireless communication protocols (including Bluetooth, Bluetooth Low Energy (BLE), cellular protocols, and / or near field communication (NFC)). In various embodiments, the crosstalk cancellation application 120 uses the sensor data acquired by sensor 150 to identify the transfer function 132 for filter 138. For example, the computing device 110 includes one or more transmitters that emit positioning signals, where the computing device 110 includes a detector that generates auditory data including the positioning signals. In some embodiments, the crosstalk cancellation application 120 combines multiple types of sensor data. For example, the crosstalk cancellation application 120 may combine auditory data and optical data ( For example , camera images or infrared data) to determine the position and orientation of the listener at a given time.

[0030] Figure 2 shows an example of how a user observes crosstalk in the input signal generated by one or more speakers 160. When the audio source 140 is played by one or more speakers 160, crosstalk itself appears in the audio, and this crosstalk can be measured at the left ear L and right ear R of the listener 202. Crosstalk naturally occurs when the speakers are far from the listener 202 and there is no crosstalk cancellation. The audio source 140a represents the desired signal at the left ear of the listener 202 or the left channel of the audio source 140. The audio source 140b represents the desired signal at the right ear of the listener 202 or the right channel of the audio source 140. When audio is played in the environment through speakers 160 that are far from the ears of the listener 202, crosstalk occurs. C 1,1 and C 1,2 represent functions that characterize how the environment affects the audio source 140a when the audio processing system 100 plays the audio source 140a. S1 and S2 respectively represent the corresponding portions of the audio source 140a heard by the left ear and right ear of the listener 202. For example, when the audio source 140a is played by the corresponding one or more speakers 160, the environment acts according to C 1,1Alter the audio source 140a such that the audio S1 reaches the left ear of the listener 202. Similarly, the environment alters the audio source 140a according to C1,2 such that the audio S2 reaches the right ear of the listener 202. S2 represents a portion of the audio source 140a that generates crosstalk reaching the right ear of the listener 202. C 2,1 and C 2,2 represent functions that characterize how the audio source 140b is affected when the audio is played. S3 and S4 respectively represent the corresponding portions of the audio source 140b heard by the left and right ears of the listener 202. For example, when the audio source 140b is played by the corresponding one or more speakers 160, the environment alters the audio source 140b according to C 2,2 such that the audio S4 reaches the right ear of the listener 202. Similarly, the environment alters the audio source 140b according to C 2,1 such that the audio S3 reaches the left ear of the listener 202. S3 represents a portion of the audio source 140b that is generated. Accordingly, embodiments of the present disclosure utilize the filter 138 to process the signal, which is then used to drive one or more speakers 160 to reduce or eliminate crosstalk caused by the environment.

[0031] Figure 3 illustrates an example of the filter 138 that performs crosstalk cancellation based on the observed position and orientation of the user in a three-dimensional space according to various embodiments of the present disclosure. As Figure 3 shown, one or more speakers 160 play the audio source 140a corresponding to the left channel of the audio source 140 and the audio source 140b corresponding to the right channel of the audio source 140. As described above in connection with Figure 2 the audio source 140a represents the desired signal at the left ear of the listener 202 or the left channel of the audio source 140. The audio source 140b represents the desired signal at the right ear of the listener 202 or the right channel of the audio source 140. In the unfiltered case, when the audio is played in a three-dimensional environment, such as through speakers 160 that are relatively far from the ears of the listener 202, crosstalk can occur as Figure 2 described.

[0032] The crosstalk cancellation application 120 determines the position and orientation of the listener 202's head based on sensor data from sensors 150 (such as one or more cameras or other devices that detect the position or orientation of the listener 202). The crosstalk cancellation application 120 further determines the distances of the parameters representing the position and orientation of the listener 202's head to one or more points within the dimensional map 134 based on the dimensional map 134. In one example, the crosstalk cancellation application 120 calculates the mathematical distances of the position and orientation of the listener 202's head to the points within the dimensional map 134, such as the centroid distance or the Euclidean distance. Then, the crosstalk cancellation application 120 identifies the transfer function 132 associated with the nearest point based on the calculated centroid or Euclidean distance.

[0033] In Figure 3 the example, the crosstalk cancellation application 120 selects the transfer function for configuring the filter bank that filters portions of the audio sources 140a and 140b played by one or more speakers 160 to reduce or eliminate the crosstalk of the portions of the audio signals Z1, Z2, Z3, and Z4 reaching the left and right ears of the listener 202. As Figure 3 shown, the filters H 1,1 and H 1,2 filter portions of the audio source 140a, and the filters H 2,1 and H 2,2 filter portions of the audio source 140b, such that when the audio source 140 is output in an environment according to C 1,1 、C 1,2 、C 2,1 and C 2,2 that affects the played signal, the crosstalk is reduced or eliminated.

[0034] V1 and V2 respectively represent the corresponding filtered portions of the audio source 140a filtered by the filters H 1,1 and H 1,2 and are output to one or more speakers 160. V3 and V4 respectively represent the corresponding filtered portions of the audio source 140b filtered by the filters H 2,1 and H 2,2 and are output to one or more speakers 160. Thus, when the environment changes the signals output by the filters and played by one or more speakers 160 according to C 1,1 、C 1,2 、C 2,1 and C 2,2 the crosstalk of the signals reaching the ears of the listener 202 has been reduced or eliminated. As Figure 3 shown, H 1,1 and H 1,2Filter the audio source 140a to produce V1 and V2 played by one or more speakers 160 such that when passing through C 1,1 and C 2,1 are affected by the environment, the resulting signals Z1 and Z3 reaching the left ear of the listener 202 correspond only to the audio source 140a, i.e., the left channel of the audio source 140. Similarly, H 2,1 and H 2,2 filter the audio source 140b to produce V3 and V4 played by one or more speakers 160 such that when passing through C 1,2 and C 2,2 are affected by the environment, the resulting signals Z2 and Z4 reaching the right ear of the listener 202 correspond only to the audio source 140b, i.e., the right channel.

[0035] As described above, the crosstalk cancellation application 120 selects a transfer function 132 that is used to configure the filter set H 1,1 、H 1,2 、H 2,1 and H 2,2， The filter set filters the audio source 140a and the audio source 140b based on the position and orientation of the listener 202. The position and orientation of the listener 202 are determined based on sensor data from one or more sensors 150. When the position and / or orientation of the listener 202 changes, the crosstalk cancellation application 120 updates the transfer function 132 used to configure the filters H 1,1 、H 1,2 、H 2,1 and H 2,2 by determining whether the updated position or orientation to which the listener 202 has moved corresponds to a different set of transfer functions 132 defined by the dimensional map 134. In this way, the crosstalk cancellation application 120 performs crosstalk cancellation based on the current position and orientation of the listener 202 and when the listener 202 adjusts its position and / or orientation within a given three-dimensional space characterized by the dimensional map 134.

[0036] Figure 4 illustrates a flowchart of method steps for selecting a transfer function for configuring a filter for performing crosstalk cancellation according to one or more embodiments. Although the method steps are described with reference to Figures 1 to 3 the embodiments, those skilled in the art will understand that any system configured to implement the method steps in any order falls within the scope of the present disclosure.

[0037] Method 400 begins at step 402, where crosstalk cancellation application 120 determines the position and orientation of listener 202 within the environment. The environment includes a space in which one or more speakers 160 play audio, such as a vehicle interior or any other interior or exterior environment. Crosstalk cancellation application 120 determines the position and orientation of listener 202 based on sensor data obtained from sensors 150 associated with audio processing system 100. As described above, sensors 150 include optical sensors, pressure sensors, proximity sensors, and other sensors that obtain information about the environment and the position and orientation of listener 202 within the environment. Based on the sensor data from sensors 150, the position of listener 202 relative to a reference position within the environment is determined. The orientation of listener 202 is also determined relative to a reference orientation within the environment. In some embodiments, crosstalk cancellation application 120 determines the position and orientation of the head and / or ears of listener 202 based on the sensor data.

[0038] At step 404, crosstalk cancellation application 120 identifies a point within dimensional map 134 based on the position and orientation of listener 202 within the environment. In one example, a given position and orientation of the user are characterized by coordinates in a six-dimensional space. Then the point closest to the coordinates is identified within dimensional map 134.

[0039] In some embodiments, the crosstalk cancellation application 120 selects the transfer function 132 associated with the closest point in the dimensionality map 134 to configure the filter 138 that filters the played audio signal 140. In other embodiments, a simplified method for identifying a point based on the position and orientation of the listener 202 includes reducing the number of dimensions of the position and orientation of the user considered when identifying the point associated with the listener 202 in the dimensionality map 134. To reduce mathematical complexity, a simplified set of parameters representing the user's position and orientation can be considered. For example, one or more parameters representing orientation can be removed, and the closest set of points is determined based on the mathematical distance from the coordinates of the position and orientation of the listener's head in the table to one or more points in the set of points in the dimensionality map 134. Examples of coordinates that can be removed include yaw angle, pitch angle, and / or roll angle. As another example, an alternative simplified method for identifying the transfer function 132 includes reducing the dimensionality of the dimensionality map 134. As described above, the dimensionality map 134 includes a set of points in a six-dimensional space to account for three parameters representing position and three parameters representing orientation. To reduce mathematical complexity, a dimensionality map 134 can be generated and utilized that includes a set of points mapped in a three-dimensional, four-dimensional, or five-dimensional space. For example, the dimensionality map 134 can map only the position of the user's head in three-dimensional space and the yaw angle representing orientation, resulting in a four-dimensional map. As another example, the dimensionality map 134 maps only the position of the user's head and two parameters characterizing orientation, reducing the complexity of the dimensionality map 134 to five dimensions. In any of the above scenarios, the crosstalk cancellation application 120 identifies the point within the dimensionality map 134 that is closest to the point representing at least some of the parameters corresponding to the position and orientation of the listener 202.

[0040] At step 406, the crosstalk cancellation application 120 identifies the transfer function 132 specified by the point in the dimensionality map 134 based on the position and orientation of the listener 202. The transfer function 132 is used to configure one or more filters 138 that reduce or eliminate crosstalk in the audio played by one or more speakers 160. In other words, given a particular audio signal provided as input to the filter 138, the transfer function 132 is used to model the output of the filter 138.

[0041] At step 408, the crosstalk cancellation application 120 uses the transfer function 132 identified at step 406 to configure one or more filters 138. The crosstalk cancellation application 120 applies the transfer function 132 to the filter 138 that is used to filter the audio signal that is then provided to one or more speakers 160 for playback within the environment.

[0042] At step 410, the crosstalk cancellation application 120 generates an audio signal for playback based on a filter 138 configured with the identified transfer function 132. The audio signal is generated based on an audio source 140 being played by the audio processing system 100 within the environment, such as a song or other audio input provided to the audio processing system 100. The audio source 140 includes a left channel and a right channel. The crosstalk cancellation application 120 filters the audio source 140 using the filter 138, which is configured with a transfer function 132 selected based on the position and orientation of the listener 202. When played in the environment, the filtered audio signal reaches the left and right ears of the listener 202 respectively, and crosstalk is reduced or eliminated.

[0043] At step 412, the crosstalk cancellation application 120 outputs the filtered audio signal to one or more speakers 160 associated with the audio processing system 100. The one or more speakers 160 play the filtered audio signal in the environment based on the filtered audio signal. The one or more speakers 160 include one or more speakers corresponding to the left channel of the audio processing system 100 and one or more speakers corresponding to the right channel of the audio processing system 100.

[0044] At step 414, the crosstalk cancellation application 120 determines whether the position or orientation of the listener 202 has changed. If the position or orientation of the listener 202 has changed, the method 400 returns to step 402, where the crosstalk cancellation application 120 determines the updated position and orientation of the listener 202 and identifies a new transfer function 132 for updating the filter 138. If the position and orientation of the listener 202 remain unchanged, the method 400 returns to step 412, where the crosstalk cancellation application 120 continues to output the audio signal based on the transfer function 132 identified at step 406.

[0045] Figure 5 A flowchart showing method steps for selecting a transfer function for configuring a filter that performs crosstalk cancellation according to one or more embodiments is shown. Although the method steps are described with reference to Figures 1 to 3 embodiments, those skilled in the art will understand that any system configured to implement the method steps in any order falls within the scope of the present disclosure.

[0046] Method 500 begins at step 402, where crosstalk cancellation application 120 determines the position and orientation of listener 202 within the environment. The environment includes a space in which one or more speakers 160 play audio, such as the interior of a vehicle, a room within a building, or an external environment. Crosstalk cancellation application 120 determines the position and orientation of listener 202 based on sensor data obtained from sensors 150 associated with audio processing system 100. As described above, sensors 150 include optical sensors, pressure sensors, proximity sensors, and other sensors that obtain information about the environment and the position and orientation of listener 202 within the environment. Based on the sensor data from sensors 150, the position of listener 202 relative to a reference position within the environment is determined. The orientation of listener 202 is also determined relative to a reference orientation within the environment. In some embodiments, crosstalk cancellation application 120 determines the position and orientation of the head and / or ears of listener 202 based on the sensor data.

[0047] At step 504, crosstalk cancellation application 120 selects a dimensional map 134 from a plurality of dimensional maps 134. As described above, crosstalk cancellation application 120 may utilize a plurality of dimensional maps 134, which include three dimensions representing positions in a three-dimensional space. Each of the three-dimensional maps is associated with a specific orientation parameter or range of orientation parameters. For example, each of the three-dimensional maps is associated with a yaw angle or a range of yaw angles. Thus, crosstalk cancellation application 120 selects the dimensional map 134 that corresponds to the yaw angle of listener 202 detected based on the sensor data from sensors 150 or based on another orientation parameter for the plurality of dimensional maps 134. As an additional example, the plurality of dimensional maps 134 may include four-dimensional maps or five-dimensional maps representing three position parameters and one or two orientation parameters, respectively.

[0048] At step 506, crosstalk cancellation application 120 identifies a point within the selected dimensional map 134 that corresponds to the position and orientation of listener 202 within the environment and, in some implementations, to some orientation parameters that correspond to the orientation of the listener within the environment. For example, assuming that a dimensional map 134 based on the yaw angle is selected, crosstalk cancellation application 120 identifies a point that characterizes the position and the remaining orientation parameters (such as the roll angle and the pitch angle). Crosstalk cancellation application 120 then identifies the point within dimensional map 134 that is closest to the coordinates representing the position and the remaining orientation parameters representing the position and orientation of listener 202.

[0049] At step 508, the crosstalk cancellation application 120 identifies the transfer function 132 specified by the points in the dimensional map 134 based on the position and orientation of the listener 202. The transfer function 132 is used to configure one or more filters 138 that reduce or eliminate crosstalk in the audio played by one or more speakers 160. In other words, given a particular audio signal provided as the input to the filter 138, the transfer function 132 is used to model the output of the filter 138.

[0050] At step 510, the crosstalk cancellation application 120 configures one or more filters 138 using the transfer function 132 identified at step 406. The crosstalk cancellation application 120 applies the transfer function 132 to the filter 138, which is used to filter an audio signal that is then provided to one or more speakers 160 for playback within the environment.

[0051] At step 512, the crosstalk cancellation application 120 generates an audio signal for playback based on the filter 138 configured with the identified transfer function 132. The audio signal is generated based on an audio source 140 that the audio processing system 100 within the environment is playing, such as a song or other audio input provided to the audio processing system 100. The audio source 140 includes a left channel and a right channel. The crosstalk cancellation application 120 filters the audio source 140 using the filter 138 configured with the transfer function 132 selected based on the position and orientation of the listener 202. When played in the environment, the filtered audio signal will reach the left ear and the right ear of the listener 202 respectively, and crosstalk is reduced or eliminated.

[0052] At step 514, the crosstalk cancellation application 120 outputs the filtered audio signal to one or more speakers 160 associated with the audio processing system 100. One or more speakers 160 play the filtered audio signal in the environment based on the filtered audio signal. One or more speakers 160 include one or more speakers corresponding to the left channel of the audio processing system 100 and one or more speakers corresponding to the right channel of the audio processing system 100.

[0053] At step 516, the crosstalk cancellation application 120 determines whether the position or orientation of the listener 202 has changed. If the position or orientation of the listener 202 has changed, the method 500 returns to step 502, where the crosstalk cancellation application 120 determines the updated position and orientation of the listener 202 and identifies a new transfer function 132 for updating the filter 138. If the position and orientation of the listener 202 remain unchanged, the method 500 returns to step 514, where the crosstalk cancellation application 120 continues to output an audio signal based on the transfer function 132 identified at step 508.

[0054] In summary, the crosstalk cancellation application configures a set of filters for performing crosstalk cancellation between the left and right channels of an audio source played by one or more speakers. The crosstalk cancellation application configures the set of filters by selecting a transfer function for each of the filters in the set of filters. The transfer function is selected by using sensor data from one or more sensors to identify the position and orientation of the user's head in three-dimensional space. A dimensional map specifies a set of points respectively associated with the transfer functions used to configure the filters. Identify the point in the dimensional map that is closest to the position and orientation of the user's head. Identify the transfer function associated with the identified point for each of the filters. The filters filter one or more signals corresponding to the audio source by using the identified transfer functions, and the one or more signals are used to drive one or more speakers to create a sound field. One or more speakers play the corresponding filtered signals. When affected by the environment, once the filtered signals reach the listener's ears, crosstalk is reduced or eliminated.

[0055] At least one technical advantage of the disclosed technology over the prior art is that, using the disclosed technology, an audio processing system can select a transfer function to be applied to each audio channel, and the transfer function modifies the audio output of the one or more speakers to improve the performance of crosstalk cancellation. The transfer function modifies the audio input, which is then played by the one or more speakers of the playback system. By improving the performance of crosstalk cancellation, spectral distortion caused by user movement is reduced. Additionally, the audio intended to be received by the user's left and right ears respectively more accurately represents the audio input output by the audio processing and playback system. These technical advantages provide one or more technological advancements over prior art methods.

[0056] 1. In some embodiments, a computer-implemented method includes: determining a first position and a first orientation of a user in an environment; identifying a first point based on the first position and the first orientation of the user in a dimensional map, the dimensional map associating a plurality of transfer functions with a corresponding plurality of points corresponding to positions and orientations in a multi-dimensional space; determining at least one crosstalk cancellation filter based on the plurality of transfer functions; generating a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; and transmitting the plurality of audio signals to the plurality of speakers for output.

[0057] 2. The computer-implemented method of clause 1, wherein identifying the first point includes selecting the closest point from the plurality of points in the dimensional map based on a mathematical distance from the first point to the closest point.

[0058] 3. The computer-implemented method as described in clause 1 or 2 further includes determining a second position and a second orientation of the user, identifying a second point in the dimensional map based on the second position and the second orientation, and replacing the at least one crosstalk cancellation filter based on the second point in the dimensional map.

[0059] 4. The computer-implemented method as described in any one of clauses 1 to 3, wherein determining the first position and the first orientation of the user in the environment includes receiving sensor data from a plurality of sensors.

[0060] 5. The computer-implemented method as described in any one of clauses 1 to 4, wherein determining the first position and the first orientation of the user in the environment includes calculating three coordinates corresponding to a position relative to a reference position and three coordinates corresponding to an orientation relative to a reference orientation.

[0061] 6. The computer-implemented method as described in any one of clauses 1 to 5, wherein the three coordinates corresponding to the orientation relative to the reference orientation correspond to a roll angle, a pitch angle, and a yaw angle.

[0062] 7. The computer-implemented method as described in any one of clauses 1 to 6, wherein identifying the first point corresponding to the first position and the first orientation is based on three parameters corresponding to the first position and a reduced number of parameters corresponding to the first orientation.

[0063] 8. The computer-implemented method as described in any one of clauses 1 to 7, wherein determining the first position and the first orientation of the user in the environment includes calculating three coordinates corresponding to a position relative to a reference position and at least one of a yaw angle or a pitch angle relative to a reference orientation.

[0064] 9. The computer-implemented method as described in any one of clauses 1 to 8, wherein the dimensional map is selected from a plurality of dimensional maps, and the dimensional map is selected based on a yaw angle relative to a reference orientation corresponding to the first orientation.

[0065] 10. The computer-implemented method as described in any one of clauses 1 to 9, wherein each of the plurality of dimensional maps is associated with a yaw angle range relative to the reference orientation.

[0066] 11. In some embodiments, one or more non-transitory computer-readable media store instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: determine a first position and a first orientation of a user in an environment; identify a first point based on the first position and the first orientation of the user in a dimensionality map that associates a plurality of transfer functions with corresponding pluralities of points corresponding to positions and orientations in a multi-dimensional space; determine at least one crosstalk cancellation filter based on the plurality of transfer functions; generate a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; and transmit the plurality of audio signals to the plurality of speakers for output.

[0067] 12. The one or more non-transitory computer-readable media of clause 11, wherein the plurality of audio signals include a left channel signal and a right channel signal.

[0068] 13. The one or more non-transitory computer-readable media of clause 11 or 12, wherein the at least one crosstalk cancellation filter cancels crosstalk between a left channel signal and a right channel signal at the left ear and the right ear of the user when the user is in the first position and the first orientation.

[0069] 14. The one or more non-transitory computer-readable media of any one of clauses 11 to 13, wherein identifying the first point includes selecting the nearest point from among the plurality of points in the dimensionality map based on a mathematical distance from the first point to the nearest point.

[0070] 15. The one or more non-transitory computer-readable media of any one of clauses 11 to 14, wherein the environment includes the interior of a vehicle compartment.

[0071] 16. The one or more non-transitory computer-readable media of any one of clauses 11 to 15, wherein the steps further include determining a second position and a second orientation of the user, identifying a second point in the dimensionality map corresponding to the second position and the second orientation, and replacing the at least one crosstalk cancellation filter based on the second point in the dimensionality map.

[0072] 17. The one or more non-transitory computer-readable media of any one of clauses 11 to 16, wherein determining the first position and the first orientation of the user in the environment includes calculating three coordinates corresponding to a position relative to a reference position and three coordinates corresponding to an orientation relative to a reference orientation.

[0073] 18. One or more non-transitory computer-readable media as described in any one of clauses 11 to 17, wherein the dimensional map is selected from a plurality of dimensional maps, and wherein the dimensional map is selected based on a yaw angle relative to a reference orientation corresponding to the first orientation.

[0074] 19. One or more non-transitory computer-readable media as described in any one of clauses 11 to 18, wherein each of the plurality of dimensional maps is associated with a range of yaw angles relative to the reference orientation.

[0075] 20. In some embodiments, a system includes: at least one sensor configured to obtain information about a user in an environment; at least one speaker configured to play audio in the environment; a memory storing a crosstalk cancellation application; and a processor coupled to the memory, the processor executing the crosstalk cancellation application by performing the following steps: determining a first position and a first orientation of the user in the environment; identifying a first point corresponding to the first position and the first orientation of the user in the dimensional map, the dimensional map associating a plurality of transfer functions with a corresponding plurality of points corresponding to positions and orientations in a multi-dimensional space; determining at least one crosstalk cancellation filter based on the plurality of transfer functions; generating a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; and transmitting the plurality of audio signals to the plurality of speakers for output.

[0076] Any and all combinations, in any form, of any of the claim elements described in any of the claims and / or any of the elements described in this application fall within the intended scope of the invention and protection.

[0077] The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0078] Aspects of the present implementation can be embodied as a system, method, or computer program product. Thus, aspects of the present disclosure may take the form of an entirely hardware implementation, an entirely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software aspects with hardware aspects, which may generally be referred to herein as a "module", "system", or "computer". Additionally, any hardware and / or software technologies, processes, functions, components, engines, modules, or systems described in the present disclosure can be implemented as a circuit or a collection of circuits. Moreover, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0079] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following media: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing media. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0080] Aspects of the present disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine. When executed by the processor of the computer or other programmable data processing device, these instructions enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor may be, but is not limited to, a general-purpose processor, a special-purpose processor, an application-specific processor, or a field-programmable gate array.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of the possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the accompanying drawings. For example, two blocks shown in succession may, in fact, be executed substantially concurrently depending on the functionality involved, or the blocks may sometimes be executed in the reverse order. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or by combinations of special-purpose hardware and computer instructions.

[0082] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be devised without departing from the basic scope thereof, which is determined by the following claims.

Claims

1. A computer-implemented method, comprising: Determining a first position and a first orientation of a user in an environment; Identifying a first point based on the first position and the first orientation of the user in a dimensionality map that associates a plurality of transfer functions with corresponding pluralities of points corresponding to positions and orientations in a multi-dimensional space; Determining at least one crosstalk cancellation filter based on the plurality of transfer functions; Generating a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; And Transmitting the plurality of audio signals to the plurality of speakers for output.

2. The computer-implemented method according to claim 1, wherein identifying the first point comprises selecting the nearest point from among a plurality of points in the dimensionality map based on a mathematical distance from the first point to the nearest point.

3. The computer-implemented method according to claim 1, further comprising: Determining a second position and a second orientation of the user; Identifying a second point in the dimensionality map based on the second position and the second orientation; And Replacing the at least one crosstalk cancellation filter based on the second point in the dimensionality map.

4. The computer-implemented method according to claim 1, wherein determining the first position and the first orientation of the user in the environment comprises receiving sensor data from a plurality of sensors.

5. The computer-implemented method according to claim 1, wherein determining the first position and the first orientation of the user in the environment comprises calculating three coordinates corresponding to a position relative to a reference position and three coordinates corresponding to an orientation relative to a reference orientation.

6. The computer-implemented method according to claim 5, wherein the three coordinates corresponding to the orientation relative to the reference orientation correspond to a roll angle, a pitch angle, and a yaw angle.

7. The computer-implemented method according to claim 1, wherein identifying the first point corresponding to the first position and the first orientation is based on three parameters corresponding to the first position and a reduced number of parameters corresponding to the first orientation.

8. The computer-implemented method according to claim 1, wherein determining the first position and the first orientation of the user in the environment comprises calculating three coordinates corresponding to a position relative to a reference position and at least one of a yaw angle or a pitch angle relative to a reference orientation.

9. The computer-implemented method according to claim 1, wherein the dimensionality map is selected from a plurality of dimensionality maps, and wherein the dimensionality map is selected based on a yaw angle relative to a reference orientation corresponding to the first orientation.

10. The computer-implemented method according to claim 9, wherein each of the plurality of dimensionality maps is associated with a range of yaw angles relative to the reference orientation.

11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: Determining a first position and a first orientation of a user in an environment; Identify a first point based on the first position and the first orientation of the user in the dimensional map, the dimensional map associating a plurality of transfer functions with corresponding plurality of points corresponding to positions and orientations in a multi-dimensional space; Determine at least one crosstalk cancellation filter based on the plurality of transfer functions; Generate a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; And Transmit the plurality of audio signals to the plurality of speakers for output.

12. One or more non-transitory computer-readable media as recited in claim 11, wherein the plurality of audio signals include a left channel signal and a right channel signal.

13. One or more non-transitory computer-readable media as recited in claim 12, wherein the at least one crosstalk cancellation filter cancels crosstalk between the left channel signal and the right channel signal at the left ear and the right ear of the user when the user is in the first position and the first orientation.

14. One or more non-transitory computer-readable media as recited in claim 11, wherein identifying the first point includes selecting the nearest point from among the plurality of points in the dimensional map based on the mathematical distance from the first point to the nearest point.

15. One or more non-transitory computer-readable media as recited in claim 11, wherein the environment includes the interior of a vehicle cabin.

16. One or more non-transitory computer-readable media as recited in claim 11, wherein the steps further include: Determine a second position and a second orientation of the user; Identify a second point in the dimensional map corresponding to the second position and the second orientation; And Replace the at least one crosstalk cancellation filter based on the second point in the dimensional map.

17. One or more non-transitory computer-readable media as recited in claim 11, wherein determining the first position and the first orientation of the user in the environment includes calculating three coordinates corresponding to a position relative to a reference position and three coordinates corresponding to an orientation relative to a reference orientation.

18. One or more non-transitory computer-readable media as recited in claim 11, wherein the dimensional map is selected from a plurality of dimensional maps, wherein the dimensional map is selected based on a yaw angle relative to a reference orientation corresponding to the first orientation.

19. One or more non-transitory computer-readable media as recited in claim 18, wherein each of the plurality of dimensional maps is associated with a range of yaw angles relative to the reference orientation.

20. A system, comprising: At least one sensor configured to obtain information about a user in an environment; At least one speaker configured to play audio in the environment; A memory storing a crosstalk cancellation application; And A processor coupled to the memory, the processor executing the crosstalk cancellation application by performing the following steps: Determine a first position and a first orientation of the user in the environment; Identify a first point corresponding to the first position and the first orientation of the user in the dimensionality map, the dimensionality map associating a plurality of transfer functions with corresponding plurality of points corresponding to positions and orientations in a multi-dimensional space; Determine at least one crosstalk cancellation filter based on the plurality of transfer functions; Generate a plurality of audio signals for a plurality of speakers based on the at least one crosstalk cancellation filter; And Transmit the plurality of audio signals to the plurality of speakers for output.

Images