Open active noise cancellation system

Abstract

Embodiments of the present disclosure set forth a method of reducing noise in an audio signal. The method includes determining a first location of a user in an environment based on sensor data acquired from a first set of sensors. The method also includes acquiring, by the first set of sensors, one or more audio signals associated with a sound in the environment and identifying one or more noise elements in the one or more audio signals. The method further includes generating a first directional audio signal based on the one or more noise elements. When the first directional audio signal is output by a first speaker, the first speaker produces a first sound field that attenuates the one or more noise elements at the first location.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate generally to audio systems, and more particularly to an open active noise cancellation system. BACKGROUND

[0002] Many corporate offices employ open office environments in which multiple employees work in a common space rather than being separated by physical barriers, such as full walls that provide separate rooms or partition walls that provide separate areas within a common room. Because employees share a common space, open office environments encourage face-to-face communication and collaboration between employees.

[0003] However, one of the drawbacks of open office environments is that the common space forces employees to work in a noisy environment with little privacy. For example, when talking with another person, an employee is forced to speak and listen within a noisy open office environment in which noise from the environment impedes the user's ability to hear the speaker. The noisy environment also impedes the user's ability to speak clearly over other noise sources. Instead, the employee is forced to move to a quieter environment without noise elements. However, such spaces can be limited.

[0004] As described above, an improved system for voice communication within an open office environment would be useful. SUMMARY

[0005] Embodiments of the present disclosure set forth a method of reducing noise in an audio signal. The method includes determining a first location of a user in an environment based on sensor data acquired from a first set of sensors. The method also includes acquiring, by the first set of sensors, one or more audio signals associated with sound in the environment and identifying one or more noise elements in the one or more audio signals. The method further includes generating a first directional audio signal based on the one or more noise elements. When the first directional audio signal is output by a first speaker, the first speaker produces a first sound field that attenuates the one or more noise elements at the first location.

[0006] Among other things, other embodiments provide a system and computer readable storage medium for implementing aspects of the methods set forth above.

[0007] At least one technical advantage of the disclosed technology is that audio signals can be transmitted to a user while also canceling certain noise within an open environment. The open active noise cancellation system identifies and then attenuates or cancels certain noise elements, which enables the user to speak and / or listen to speech within the open environment without the need for additional equipment, such as barriers or headphones, to suppress noise when communicating. BRIEF DESCRIPTION OF DRAWINGS

[0008] So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, can be had by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this inventive concept and are therefore not to be considered limiting of its scope, for the inventive concept can admit to other equally effective embodiments.

[0009] Figure 1 A block diagram illustrating a computer network including an open active noise cancellation system configured to implement one or more aspects of the present disclosure is shown.

[0010] Figure 2 A block diagram of an open active noise cancellation system configured to process speech signals and noise signals in accordance with various embodiments of the present disclosure is shown. Figure 1

[0011] Figure 3 A block diagram of an open active noise cancellation system configured to process audio signals to attenuate noise elements associated with a captured speech signal in accordance with various embodiments of the present disclosure is shown. Figure 1

[0012] Figure 4 A block diagram of an open active noise cancellation system configured to process audio signals to attenuate noise elements in order to emit a directional audio output signal in accordance with various embodiments of the present disclosure is shown. Figure 1

[0013] A flow diagram of method steps for generating a processed audio signal by an open active noise cancellation system in accordance with various embodiments of the present disclosure is shown. Figure 5 Figure 1 A flow diagram of method steps for generating a directional audio output signal by an open active noise cancellation system in accordance with various embodiments of the present disclosure is shown.

[0014] Figure 6 Figure 1 DETAILED DESCRIPTION

[0015] In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one skilled in the art that the inventive concepts can be practiced without one or more of these specific details.

[0016] Figure 1 ​​​​​A block diagram of a computer network 100 including an open active noise cancellation system 110 configured to implement one or more aspects of the present disclosure is shown. As shown, the computer network 100 includes, but is not limited to, the open active noise cancellation system 110, a network 120, a user device 132, a communication server 134, and / or an open active noise cancellation system 136. In alternative embodiments, the computer network 100 can include any number of user devices 132, open active noise cancellation systems 110, 136, and / or communication servers 134.

[0017] The open active noise cancellation system 110 includes one or more sensors 112, an audio input device 114, an audio output device 116, and / or a speech processor 118. In various embodiments, the open active noise cancellation system 110 can include a desktop computer, a laptop computer, a mobile computer, or any other type of computing system suitable for practicing one or more embodiments of the present disclosure and configured to receive data as input, process the data, and emit sound. In various embodiments, the open active noise cancellation system 136 can include one or more components included in the open active noise cancellation system 110. As will be discussed in greater detail below, the open active noise cancellation system 110 is configured to enable a user to communicate through speech with one or more devices through the network 120. In various embodiments, the open active noise cancellation system 110 can execute one or more applications to capture speech of a user and transmit the speech to other devices through the network 120. Additionally or alternatively, the open active noise cancellation system 110 can execute one or more applications to process audio signals received through the network 120 and emit audio signals through one or more audio output devices.

[0018] In various embodiments, in operation, the open active noise cancellation system 110 captures audio signals through the audio input device 114 and / or the sensors 112. The captured audio signals can include speech of a user and one or more noise elements. The speech processor 118 included in the open active noise cancellation system 110 filters the captured audio to attenuate and / or suppress the noise elements in the captured audio signals to produce a processed audio signal. The open active noise cancellation system 110 transmits the processed audio signal to one or more recipients through the network 120. In various embodiments, the one or more recipients include one or more of the user device 132, the communication server 134, and / or a device having the same or similar functionality as the open active noise cancellation system 136.

[0019] In various implementations, the open active noise cancellation system 110 can receive an audio input signal over the network 120. In such cases, the speech processor 118 included in the open active noise cancellation system 110 can process the audio input signal. The one or more sensors 112 can generate location data associated with a location of a user within an environment. The one or more sensors 112 and / or the audio input device 114 can also capture noise signals from one or more noise sources within the environment. The speech processor 118 can receive the location data and / or the noise signals and can produce a corresponding processed directional audio signal. In various implementations, the speech processor 118 can transmit the processed directional audio signal to the audio output device 116. The audio output device 116 can generate a sound field that includes the location of the user within the environment. The audio output device 116 reproduces the processed audio signal within the generated sound field, which enables the user to hear the audio signal while various noise elements within the environment are attenuated within the sound field.

[0020] The network 120 includes a plurality of network communication systems, such as routers and switches, configured to facilitate data communication between the open active noise cancellation systems 110, 136, the user device 132, and / or the communication server 134. Those skilled in the art will recognize that there are many technically feasible techniques for constructing the network 120, including techniques practiced in the deployment of internet communication networks. For example, the network 120 can include a wide area network (WAN), a local area network (LAN), and / or a wireless (Wi-Fi) network, among others.

[0021] The user device 132 can be a desktop computer, a laptop computer, a mobile computer, or any other type of computing system configured to receive input, process data, emit sound, and suitable for practicing one or more embodiments of the present disclosure. The user device 132 is configured to enable a user to communicate by speech with one or more devices over the network 120. In various implementations, the user device 132 can execute one or more applications to capture speech of the user and transmit the speech to other devices over the network 120. Additionally or alternatively, the user device 132 can execute one or more applications to process audio signals received over the network 120 and emit the audio signals through one or more audio output devices.

[0022] The communication server 134 includes a computer system configured to receive data and / or audio signals from one or more user devices 132 and / or open active noise cancellation systems 110, 136. In various implementations, the communication server 134 executes an application to synchronize and / or coordinate data transmission between devices participating in real-time communication.

[0023] Figure 2 FIG. 1 illustrates an open active noise cancellation system 110 in accordance with various embodiments of the present disclosure. Figure 1a block diagram of an open active noise cancellation system 110 configured to process voice signals and noise signals. The open active noise cancellation system 200 includes one or more sensors 112, an audio input device 114, an audio output device 116, and a computing device 210. The computing device 210 includes a processing unit 212 and a memory 214. The memory 214 stores a database 216 and a speech processing application 218.

[0024] In operation, the processing unit 212 receives data from the one or more sensors 112, the audio input device 114, and / or the network 120. In various embodiments, the received data includes audio signals (e.g., speech signals, noise signals, etc.) and / or sensor data. The processing unit 212 executes the speech processing application 218 to analyze the sensor data and the audio signals. In analyzing the audio signals and the sensor data, the speech processing application 218 generates processed audio signals. The processed audio signals attenuate and / or suppress noise elements associated with the audio signals. In various embodiments, the speech processing application 218 can cause the audio output device 116 to emit a sound field.

[0025] In various embodiments, the speech processing application 218 can use various speech recognition and / or noise recognition techniques to identify portions of captured audio. The speech processing application 218 identifies one or more noise elements included in portions of captured audio and filters the captured audio to attenuate and / or remove the identified noise elements. In some embodiments, the speech processing application 218 can attenuate noise elements when processing speech provided by a user prior to generating processed audio signals to be sent to a recipient over the network 120. Additionally or alternatively, the speech processing application 218 can identify noise elements in an environment and generate processed directional audio signals that suppress noise when generating a sound field for a user.

[0026] The one or more sensors 112 include one or more devices that collect data associated with objects in an environment. In various embodiments, the one or more sensors 112 can include a group of sensors that acquire different sensor data. For example, the one or more sensors 112 can include reference sensors, such as microphones and / or accelerometers, that can acquire sound data and / or motion data (e.g., acceleration, velocity, etc.). For example In another example, the one or more sensors 112 can include one or more position trackers, such as one or more cameras, thermal imagers, linear position sensors, etc., that can acquire data corresponding to a position of a user.

[0027] In various embodiments, sensor data is generated by performing measurements and / or collecting other data. For example, one or more sensors 112 can generate sensor data associated with a location of a user within an environment. One or more sensors 112 can perform measurements, such as distance measurements, and generate sensor data (e.g., location data) reflecting the distance measurements. Computing device 210 can analyze sensor data received from one or more sensors 112 in order to track a location of a user. In various embodiments, speech processing application 218 can then determine a target location within the environment at which a sound field is to be generated by audio output device 116.

[0028] In various embodiments, one or more sensors 112 can include a location sensor, such as an accelerometer or an inertial measurement unit (IMU). An IMU can be a device similar to a three-axis accelerometer, a gyroscope sensor, and / or a magnetometer. In some embodiments, one or more sensors 112 can include an optical sensor, such as an RGB camera, a time-of-flight sensor, an infrared (IR) camera, a depth camera, and / or a quick response (QR) code tracking system. Further, in some embodiments, one or more sensors 112 can include wireless sensors, including radio frequency (RF) sensors (e.g., sonar and radar), ultrasonic-based sensors, capacitive sensors, laser-based sensors, and / or wireless communication protocols, including Bluetooth, Bluetooth Low Energy (BLE), wireless local area network (WiFi) cellular protocols, and / or near field communication (NFC).

[0029] As noted above, computing device 210 can include processing unit 212 and memory 214. Computing device 210 can be a device including one or more processing units 212, such as a system on a chip (SoC), or a mobile computing device, such as a tablet computer, a mobile phone, a media player, etc. Generally, computing device 210 can be configured to coordinate the overall operation of open active noise cancellation system 200. In some embodiments, computing device 210 can be coupled to one or more sensors 112, audio input device 114, and / or audio output device 116, but can be separate from them. In such cases, computing device 210 can be included in a separate device. The embodiments disclosed herein contemplate any technically feasible system configured to implement the functionality of open active noise cancellation system 200 through computing device 210.

[0030] The processing unit 212 can include a central processing unit (CPU), a digital signal processing unit (DSP), a microprocessor, an application-specific integrated circuit (ASIC), a neural processing unit (NPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), or the like. In some embodiments, the processing unit 212 can be configured to execute the speech processing application 218 in order to analyze captured audio signals, received audio signals, and / or sensor data and identify noise elements included in the environment. In some embodiments, the processing unit 212 can be configured to execute the speech processing application 218 to identify one or more noise elements and generate a processed audio signal in which the noise elements are attenuated and / or removed.

[0031] The memory 214 can include a memory module or a collection of memory modules. The speech processing application 218 within the memory 214 can be executed by the processing unit 212 to implement the overall functionality of the computing device 210 and thus overall coordinate the operation of the open active noise cancellation system 200.

[0032] The database 216 can store values and other data retrieved by the processing unit 212 to coordinate the operation of the open active noise cancellation system 200. In various embodiments, in operation, the processing unit 212 can be configured to store values in the database 216 and / or retrieve values stored in the database 216. For example, the database 216 can store sensor data, audio content and reference audio (e.g., one or more reference noise signals), digital signal processing algorithms, transducer parameter data, and the like.

[0033] The audio input device 114 can be a device capable of receiving one or more audio inputs. The audio input device 114 can function as a microphone. The audio output device 116 can be a device capable of providing one or more audio outputs. The audio output device 116 can be a speaker system (e.g., one or more sound boxes, loudspeakers, or the like) or other device that generates a sound field. For example, the audio output device 116 can be a speaker array including a plurality of parametric speakers that generate a sound field around a designated location. In various embodiments, one or more of the audio input device 114 and / or the audio output device 116 can be incorporated into the computing device 210 or can be located external to the computing device 210.

[0034] Figure 3 FIG. 1 shows a system 100 for using an open active noise cancellation system 200 in accordance with various embodiments of the present disclosure. Figure 1This describes an open active noise cancellation system that processes audio signals to attenuate noise elements associated with captured speech signals. As shown, the open active noise cancellation system 300 includes an input stack 330 and a processor 118. The input stack 330 includes one or more sensors 112 and audio input devices 114. The processor 118 includes a speech processing application 218, which includes a speech recognition application 344, a noise recognition application 346, a neural network 342, and a filter 348. In various embodiments, the speech processing application 218—including one or more of the speech recognition application 344, the noise recognition application 346, the neural network 342, and the filter 348—may be stored in memory 214 and executed by the processor 118.

[0035] In operation, one or more components included in input stack 330 acquire signals from sources in the surrounding environment. For example, input stack 330 may acquire speech emitted by user 320 and noise emitted by one or more noise sources 310. Processor 118 receives the signals acquired from input stack 330 as captured audio signal 332. Processor 118 executes speech processing application 218 to analyze the captured audio signal 332 and generate a processed audio signal 352 based on the analysis. The processed audio signal 352 is used by one or more devices ( For example The audio output device 116) outputs an electronic or digital signal that presents the audio. The processor 118 can then transmit the processed audio signal 352 to one or more receivers that reproduce the processed audio signal.

[0036] In various embodiments, one or more sensors 112 and / or audio input devices 114 may include microphones that capture one or more physical audio signals. Input stack 330 generates electronic or digital signals as the captured audio signal 332. For example, input stack 330 may acquire one or more noise signals 312 from one or more noise sources 310 in the surrounding environment. Alternatively, input stack 330 may acquire one or more speech signals 322 from one or more users 320 in the surrounding environment. In some embodiments, input stack 330 may receive noise signals 312 and speech signals 322 simultaneously. In such cases, a portion of the captured audio signal 332 includes both noise signals 312 and speech signals 332.

[0037] The processor 118 analyzes the captured audio signal 332 received from the input stack 330 and generates a processed audio signal 352. In various embodiments, the processor 118 executes the speech processing application 218 to analyze the captured audio signal 332. In some embodiments, a neural network 342 included in the speech processing application 218 analyzes the captured audio signal 332 using one or more application programs to identify certain elements included in the captured audio signal 332. For example, the neural network 342 can use a speech recognition application 344 to identify speech elements and / or individual speakers from one or more portions of the captured audio signal 332. Additionally or alternatively, the neural network 342 can also use a noise recognition application 346 to analyze the captured audio signal 332 to identify noise elements included in one or more portions of the captured audio signal 332.

[0038] In analyzing the captured audio signal 332, the speech processing application 218 applies one or more filters 348 to generate a signal based on the captured audio signal 332, where the generated signal emphasizes or attenuates certain portions. In various embodiments, the processor 118 generates the processed audio signal 352 by applying the one or more filters 348 to the captured audio signal 332. In various embodiments, the speech processing application 218 can modify the one or more filters 348 based on identifying noise elements and / or speech elements included in the captured audio signal 332. The speech processing application 218 can then apply the modified filters 348 to the captured audio signal 332 in order to generate the processed audio signal. In such cases, portions of the captured audio 332 can be attenuated in corresponding portions of the processed audio signal 352. In some embodiments, in generating the processed audio signal 352, the processor 118 can transmit the processed audio signal 352 to one or more recipients over the network 120.

[0039] The neural network 342 is an artificial intelligence (AI) computing system that employs one or more machine learning (ML) techniques to analyze input signals. For example, the neural network 342 can employ a speech recognition application 344 that uses one or more ML techniques to learn speech elements and / or characteristics of individual speakers. As the neural network 342 stores the learned speech elements and speaker characteristics, the neural network 342 can identify speech elements in subsequently received captured audio signals 332 based on these stored elements and characteristics. For example, using prior knowledge, the neural network 342 can employ the speech recognition application 344 to analyze the captured audio signals 332. In such cases, the neural network can identify the speech signals 322, individual speakers, speaker characteristics, and / or specific speech elements included in portions of the captured audio signals 332. In various embodiments, the neural network 342 can identify specific speech characteristics and speech elements by retrieving data from the database 216 (e.g., referencing speech elements and / or referencing speech signals) and comparing the retrieved data to portions of the captured speech signals 332. Suitable ML techniques or computing systems employed by the neural network 342 when employing the speech recognition application 344 can include, for example, nearest neighbor classifier processes, Markov chains, deep learning methods, and / or any other technically feasible method. For example

[0040] Additionally or alternatively, the neural network 342 can employ a noise recognition application 346 that uses one or more ML techniques to learn individual noise sources and / or known noise characteristics (e.g., patterns, specific noise sources, etc.) within the surrounding environment. The neural network 342 can similarly employ the noise recognition application 346 to learn noise characteristics and subsequently identify specific noise elements and / or individual speech signals 312 by comparing portions of the captured audio signals 332 to reference data stored in the database 216. For example

[0041] ​​Filter 348 may include one or more filters that modify an audio signal before playback by an audio output device. In various embodiments, filter 348 may include a filter bank of two or more filters that individually adjust each of a plurality of frequency components (e.g., frequency ranges) of the received audio signal. For example, processor 118 may adjust filter 348 to attenuate noise elements and / or some speech elements identified by neural network 342. In such cases, filter 348 may receive captured audio signal 332 and may modify different frequency ranges of the captured audio signal 332 to generate processed audio signal 352. In some embodiments, filter 348 may decompose the captured audio signal 332 into a set of filtered signals, where each filtered signal corresponds to a frequency sub-band of the captured audio signal 332. In such cases, filter 348 may attenuate one or more of the frequency sub-bands to attenuate identified noise elements and / or speech elements of the captured audio signal 352.

[0042] Figure 4 Various embodiments of the present disclosure are shown for use Figure 1 This describes an open active noise cancellation system that processes audio signals to attenuate noise elements in order to transmit directional audio output signals. As shown, the open noise cancellation system 400 includes a processor 118, one or more sensors 112, an audio output device 116, a noise source 410, a user 420, and / or a noise database (DB) 430. The processor 118 includes a speech processing application 218, which includes a neural network 342, a noise recognition application 346, and a filter 348. In various embodiments, the speech processing application 218—including one or more of the neural network 342, the noise recognition application 346, and the filter 348—may be stored in memory 214 and executed by the processor 118.

[0043] In operation, processor 118 (via network 120) receives data from various sources, including one or more sensors 112 and one or more transmitters. The received data includes audio data ( For exampleand position data 424 corresponding to the location of the user 420 within the ambient environment. The processor 118 executes the speech processing application 218 to analyze the received data and generate a processed directional audio signal 432 based on the analysis. The processed directional audio signal 432 has a component corresponding to the input audio signal 402, a component that attenuates the noise signal 422, and a directional component that corresponds to emitting sound waves toward the location of the user 420. The processor 118 then transmits the processed directional audio signal 432 to the audio output device 116. The audio output device 116 outputs the processed directional audio signal 432 by emitting sound waves that produce a sound field 442. The characteristics of the sound field 442 enable the user 420, who is located at a determined location within the ambient environment, to hear the portion of the processed directional audio signal 432 that corresponds to the input audio signal 402 while attenuating the noise signal 422 within the ambient environment.

[0044] The input audio signal 402 is an analog or digital signal for output by the audio output device 116. In various embodiments, the input audio signal 402 can correspond to the processed audio signal 352 provided by another device over the network 120. The noise signal 422 is an analog or digital signal generated by the one or more sensors 112 in response to receiving sound waves from one or more noise sources 410 by the one or more sensors 112. In various embodiments, the processor 118 can receive the noise signal 422 separately from the input audio signal 402.

[0045] The speech processing application 218 analyzes the noise signal 422 in order to identify one or more noise elements. In various embodiments, the neural network 342 included in the speech processing application 218 can employ the noise recognition application 346 in order to identify one or more noise elements included in the noise signal 422. In some embodiments, the neural network 342 can employ the noise recognition application to retrieve one or more reference signals corresponding to a specific noise element (e.g., a cough, one or more speakers, one or more individuals speaking, an HVAC system, a computer interaction, etc.) from the noise database 430. For example, the noise recognition application can compare a portion of the noise signal 422 to reference signals stored in the noise database 430 in order to identify the noise source 410. In such cases, the speech processing application 218 can modify the filter 348 to generate the processed directional audio 432 such that the sound field 442 attenuates the identified noise element within the sound field. For example

[0046] ​In various implementations, the speech processing application 218 provides active noise control (ANC) by generating a noise cancellation signal based on the identified noise elements and / or the noise signal 422. In such cases, the speech processing application 218 generates the noise cancellation signal by applying one or more filters 348 to the noise signal 422. Additionally or alternatively, the speech processing application 218 can incorporate the noise cancellation signal into the characteristics of the processed directional audio signal 432. In such cases, the audio output device 116 can emit sound waves, where the sound waves include an anti-noise portion that provides a destructive interference to the identified noise elements. For example, the speech processing application 218 can receive the noise signal 422 from the one or more sensors 112. The speech processing application 218 can then generate a noise cancellation signal that causes the audio output device 116 to emit sound waves that include an anti-noise component that has the same magnitude as the noise signal 422 and is inverted. In some implementations, the speech processing application 218 can associate the generated anti-noise signal with the corresponding identified noise element, and can store the anti-noise signal in the database 216.

[0047] In various implementations, to generate the processed directional audio signal 432, the speech processing application 218 determines the relative position of the user to the audio output device 116 and includes one or more directional parameters that cause the audio output device 116 to produce a sound field 442 that encompasses the user 420 at the corresponding position. The processor transmits the processed directional audio signal 432 to the audio output device 116, which emits sound waves that correspond to the sound field 442.

[0048] The processor 118 receives position data 424 generated by the one or more sensors 112. In various embodiments, the position data 424 is sensor data related to one or more positions and / or one or more orientations of the one or more users 420 within the surrounding environment. In some embodiments, the position data 424 also includes one or more positions and / or one or more orientations of one or more speakers included in the audio output device 116. In such cases, the processor 118 can execute the speech processing application 218 to generate position parameters, such as direction and distance, based on the relative position of the user 420 to the audio output device 116. In various embodiments, the position data 424 can include data related to the position and / or orientation of the user 420 within the surrounding environment during a specified time period. For example, during a first specified time period of t0-t1, the user 420 has an initial position. In this example, the one or more sensors 112 can acquire position data 424 corresponding to the first position for the first specified time period. When the user 420 moves to a second position during a second specified time period of t1-t2, the one or more sensors 112 can acquire position data corresponding to the second position for the second specified time period.

[0049] In various embodiments, the speech processing application 218 generates the processed directional audio signal 432 to include one or more parameters associated with the audio output device 116 that emits sound waves to produce the sound field 442. In such cases, the parameters specify how the audio output device 116 emits sound waves such that the corresponding sound field 442 encompasses the position of the user 420. The speech processing application 218 produces the one or more parameters based on the position data 424 received from the one or more sensors 112 and includes the parameters in the processed directional audio signal 432. In various embodiments, the processed directional audio signal 432 can include, but is not limited to, a direction in which a target is positioned relative to the audio output device 116 (relative to a center axis of a sound box included in the audio output device 116), For example a sound level to be output by the audio output device 116 in order to generate a desired sound level at the target position (relative to a target position off-axis from the sound box), For example a distance between the audio output device 116 and the target position, a distance and / or an angle between the audio output device 116 and the target position, etc.

[0050] The audio output device 116 receives the processed directional audio signal 432 provided by the speech processing application 218. In various embodiments, the audio output device 116 outputs the processed directional audio signal 432 by emitting sound waves to generate a sound field 442. The sound field 442 is associated with the data included in the processed directional audio signal 432. The sound waves emitted by the audio output device 116 reproduce the input audio signal 402. The sound waves of the sound field 442 have the characteristics of the other noise signals 422 also included in the environment that are attenuated (e.g., canceled by destructive interference). Thus, when the user 420 is within the sound field 442, the user can hear the input audio signal 402 without interference from the one or more noise signals 422.

[0051] Figure 5 is a method step flowchart of a method for generating a processed audio signal by an open active noise cancellation system in accordance with various embodiments of the present disclosure. Figure 1 The method steps are described with respect to the system of Figures 1 to 4 but one of skill in the art will understand that any system configured to perform the method steps in any order is within the scope of various embodiments. In some embodiments, the open active noise cancellation system 200 can continuously perform the method 500 on captured audio in real-time.

[0052] As shown, the method 500 begins at step 501 in which the open active noise cancellation system 110 captures audio including speech and noise signals. In various embodiments, one or more components included in the input stack 330 (e.g., one or more sensors 112, audio input devices 114) acquire signals from sources in the surrounding environment. For example, the input stack 330 can acquire a speech signal 322 generated by a user 320 and noise signals 312 generated by one or more noise sources 310. The processor 118 receives the signals acquired from the input stack 330 as a captured audio signal 332.

[0053] At step 503, the open active noise cancellation system 110 identifies one or more noise elements included in the captured audio signal. Upon receiving the captured audio signal 332, the processor 118 executes the speech processing application 218 to identify one or more noise elements that can be included in the captured audio signal 332. In various embodiments, the neural network 342 can employ various applications (e.g., a speech recognition application 344, a noise recognition application 346, or other ML techniques) to identify noise elements and / or extraneous speech elements included in portions of the captured audio signal 332.

[0054] At step 505, the open active noise cancellation system 110 filters the captured audio to remove the identified noise elements from the captured audio signal. The speech processing application 218 generates a processed audio signal 352 by applying a filter 348 to attenuate and / or remove the noise elements identified by the neural network 342 from the captured audio signal 332. In some embodiments, the filter 348 can decompose the captured audio signal 332 into a set of filtered signals, where each filtered signal corresponds to one or more frequency subbands of the captured audio signal 332. In such cases, the filter 348 can attenuate one or more of the frequency subbands in order to attenuate the identified noise elements and / or speech elements of the captured audio signal 352.

[0055] At step 507, the open active noise cancellation system 110 provides the processed audio signal. Upon generating the processed audio signal 352, the processor 118 transmits the processed audio signal 352 to one or more recipients. In some embodiments, the processor 118 transmits the processed audio to one or more user devices 132, a communication server 134, and / or other devices that employ the open active noise cancellation system 136 over the network 120.

[0056] Figure 6 is a method for generating a directional audio output signal by an open active noise cancellation system in accordance with various embodiments of the present disclosure. Figure 1 is a flowchart of method steps for generating a directional audio output signal by an open active noise cancellation system. While the method steps are described with respect to the system of Figures 1 to 4 of the present disclosure. In some embodiments, the open active noise cancellation system 200 can continuously perform the method 600 on captured and received audio input signals in real-time.

[0057] As shown, the method 600 begins at step 601 in which the open active noise cancellation system 110 captures audio in an ambient environment using one or more sensors. For example, the one or more sensors 112 can acquire sensor data corresponding to sound waves received from one or more noise sources 410. The one or more sensors 112 can then generate a noise signal 322 corresponding to the received sound waves. In various embodiments, the one or more sensors 112 send the noise signal 422 to the processor 118.

[0058] At step 603, the open active noise cancellation system 110 identifies one or more noise elements. In some embodiments, the neural network 342 included in the speech processing application 218 can employ the noise identification application 346 in order to identify one or more noise elements included in the noise signal 422. For example, the neural network 342 can employ the noise identification application to retrieve one or more reference signals from the noise database 430 that correspond to a particular noise element (e.g., a cough, one or more sound boxes, one or more speaking individuals, an HVAC system, a computer keyboard / mouse interaction, etc.). In retrieving the reference signals, the neural network 342 can compare portions of the noise signal 422 to the reference signals and identify portions of the noise signal 422 that match at least one of the reference signals.

[0059] At step 605, the open active noise cancellation system 110 receives an input audio signal. The speech processing application 218 receives the input audio signal 402 from the sender over the network 120. The input audio signal 402 includes a speech signal from the sender device. In some embodiments, the speech processing application 218 can separately acquire and / or analyze the input audio signal 402 and the noise signal 422.

[0060] At step 607, the open active noise cancellation system 110 applies a filter to the noise signal to attenuate the one or more identified noise elements. In various embodiments, the speech processing application 218 can employ the filter 348 to attenuate one or more portions of the noise signal 422. In some embodiments, the speech processing application 218 can employ the filter 348 to generate a new noise cancellation signal that is incorporated into the processed directional signal 432. In such cases, when the audio output device 116 emits sound waves, the sound waves include an anti-noise portion that provides destructive interference to the noise signal 422. Additionally or alternatively, the speech processing application 218 can employ the filter 348 to compensate for only the portions of the noise signal 422 that are identified by the neural network 342. In such cases, the speech processing application only compensates for the portions of the noise signal 422 that are identified as known noise elements. In such cases, the user 420 is able to hear the portions of the noise signal 422 that are not identified as noise elements.

[0061] At step 609, the open active noise cancellation system 110 acquires position data corresponding to the listener. The one or more sensors 112 acquire sensor data related to one or more positions and / or one or more orientations of the one or more users 420 within the surrounding environment. The one or more sensors 112 generate the position data 424 based on the acquired sensor data and transmit the position data 424 to the speech processing application 218.

[0062] At step 611, the open active noise cancellation system 110 generates a processed directional audio signal based on the attenuated noise elements and the acquired position data. The speech processing application 218 analyzes the position data 424 specifying the position of the user 420 and generates position parameters based on the position data 424. In various implementations, the position parameters specify characteristics incorporated into the processed directional audio signal 432, including direction and distance. In such cases, the processed directional audio signal 432 has characteristics corresponding to the input audio signal 402, characteristics compensating for the noise signal 422, and / or characteristics specifying the direction and magnitude of sound waves to emit.

[0063] In generating the processed directional audio signal 432, the speech processing application 218 transmits the processed directional audio signal 432 to the audio output device 116, which outputs the processed directional audio signal 432 by emitting sound waves that produce a sound field 442. The characteristics of the sound field 442 enable the user 420 to hear the portion of the processed directional audio signal 432 corresponding to the input audio signal 402 while attenuating the noise signal 422 within the surrounding environment (e.g., by canceling the noise signal via destructive interference).

[0064] In summary, the open active noise cancellation system includes a speech processor, sensors, and I / O devices. When a user is speaking, an input stack including at least one sensor and one I / O device captures audio that includes the user’s speech signal and one or more noise signals from noise sources in the environment. The speech processor includes a neural network that processes the captured audio and implements a speech recognition and / or noise recognition module to identify portions of the captured audio. The neural network identifies the one or more noise signals included in the portions of the captured audio and causes a filter to remove and / or attenuate the identified noise signals. The speech processor then provides the processed audio signal to one or more devices that reproduce the processed audio signal.

[0065] When a user is listening to an input audio signal, sensors included in an open active noise cancellation system generate location data related to the user's location and one or more noise signals captured from noise sources in the environment. A speech processor receives the input audio signal, the noise signals, and the location data and processes the signals. A neural network identifies the one or more noise signals using a noise recognition module by comparing the received noise signals to one or more stored reference noise signals. The speech processor then generates a processed directional audio signal. The processed directional audio signal causes an output device to emit a sound field that encompasses the user. The processed directional audio signal also attenuates noise signals within the environment, such as by destructively interfering with the noise signals. The processed directional audio signal is transmitted to the output device, which generates the sound field. The user hears the processed directional audio signal within the sound field while noise signals that include the environment are attenuated and / or suppressed within the sound field.

[0066] At least one advantage of the disclosed technology is that audio signals can be transmitted to a user while also canceling certain noise within an open environment. The open active noise cancellation system identifies and then attenuates or cancels certain noise elements in the environment, which enables the user to speak and / or listen to speech in an open environment without the need for additional mechanical equipment, such as barriers, to attenuate the noise elements.

[0067] 1. In one or more embodiments, a method for reducing noise in an audio signal includes: determining a first location of a user in an environment based on sensor data acquired from a first set of sensors; acquiring one or more audio signals associated with sounds in the environment by the first set of sensors; identifying one or more noise elements in the one or more audio signals; and generating a first directional audio signal based on the one or more noise elements, wherein, when the first directional audio signal is output by a first speaker, the first speaker produces a first sound field that attenuates the one or more noise elements at the first location.

[0068] 2. The method of clause 1, wherein identifying the one or more noise elements includes: comparing the one or more audio signals to at least one reference signal, and when the one or more audio signals match the at least one reference signal, classifying the one or more audio signals based on the at least one reference signal.

[0069] 3. The method of clause 1 or 2, wherein identifying the one or more noise elements includes: comparing, by a neural network, a first audio signal included in the one or more audio signals to a first reference signal associated with a first noise element, and based on determining that the first audio signal matches the first reference signal, categorizing the first audio signal as including the first noise element.

[0070] 4. The method of any of clauses 1-3, further comprising comparing a first audio signal included in the one or more audio signals to a first set of reference signals and determining that the first audio signal does not match at least one reference signal included in the first set of reference signals, and storing data associated with the first audio signal as an additional reference signal included in the first set of reference signals.

[0071] 5. The method of any of clauses 1-4, wherein identifying the one or more noise elements comprises comparing the one or more audio signals to each reference signal included in a first set of reference signals and when the one or more audio signals match at least one reference signal included in the first set of reference signals, classifying the one or more audio signals as the one or more noise elements, and when the one or more audio signals do not match at least one reference signal included in the first set of reference signals, determining that the one or more audio signals will not be classified as the one or more noise elements.

[0072] 6. The method of any of clauses 1-5, further comprising determining a second location of a user in an environment based on sensor data acquired from the first set of sensors and generating a second directional audio signal based on the one or more noise elements, wherein when the second directional audio signal is output by the first speaker, the first speaker produces a second sound field that attenuates the one or more noise elements at the second location.

[0073] 7. The method of any of clauses 1-6, further comprising determining a second location of the first speaker, wherein the first directional audio signal is based on the first location and the second location.

[0074] 8. The method of any of clauses 1-7, further comprising receiving an input audio signal from a second device over a first network, wherein the first directional audio signal includes at least a portion of the input audio signal.

[0075] 9. The method of any of clauses 1-8, further comprising generating a first set of directional audio signals based on the one or more noise elements, wherein when the first set of directional audio signals are output by a first plurality of speakers, the first plurality of speakers produce the first sound field.

[0076] 10. In one or more embodiments, an audio system comprises: a first set of sensors that generate sensor data associated with a first location of a user in an environment and generate one or more audio signals associated with sound acquired from the environment; a first speaker; and a processor coupled to the first set of sensors and the first speaker, the processor determines the first location of the user based on the sensor data, receives the one or more audio signals from the first set of sensors, identifies one or more noise elements in the one or more audio signals, and generates a first directional audio signal based on the one or more noise elements, wherein the first speaker outputs the first directional audio signal to produce a first sound field that attenuates the one or more noise elements at the first location.

[0077] 11. The audio system of clause 10, further comprising a first database that stores a first set of reference signals associated with the one or more noise elements.

[0078] 12. The audio system of clause 10 or 11, wherein the processor further compares the one or more audio signals to a first set of reference signals, classifies the one or more audio signals as the one or more noise elements when the one or more audio signals match at least one reference signal included in the first set of reference signals, and determines that the one or more audio signals will not be classified as the one or more noise elements when the one or more audio signals do not match at least one reference signal included in the first set of reference signals.

[0079] 13. The audio system of any one of clauses 10-12, wherein the first set of sensors comprises at least one camera that acquires location data associated with the first location and at least one microphone that acquires the one or more audio signals.

[0080] 14. The audio system of any one of clauses 10-12, wherein the first speaker comprises a parametric speaker.

[0081] 15. The audio system of any one of clauses 10-14, wherein the first speaker is included in a plurality of parametric speakers of the audio system, the processor further generates a first set of directional audio signals based on the one or more noise elements, and each parametric speaker included in the plurality of parametric speakers outputs at least one directional audio signal of the first set of directional audio signals to produce the first sound field.

[0082] 16. The audio system of any of clauses 10-15, wherein the first set of sensors further produces sensor data associated with a second location of the user, the processor further determines the second location of the user based on the sensor data, and generates a second directional audio signal based on the one or more noise elements, and the first speaker outputs the second directional audio signal to produce a second sound field that attenuates the one or more noise elements at the second location.

[0083] 17. In one or more embodiments, one or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: determine a first location of a user in an environment; acquire, by a first set of sensors, one or more audio signals associated with sound in the environment; identify one or more noise elements in the one or more audio signals by comparing the one or more audio signals to each reference signal included in a first set of reference signals, and categorize the one or more audio signals as the one or more noise elements when the one or more audio signals match at least one reference signal included in the first set of reference signals, and generate a first directional audio signal based on the one or more noise elements, wherein, when the first directional audio signal is output by a first speaker, the first speaker produces a first sound field that attenuates the one or more noise elements at the first location.

[0084] 18. The one or more non-transitory computer-readable media of clause 17, wherein generating a first directional audio signal comprises: receiving an input audio signal; generating an anti-noise signal that matches a magnitude of the at least one reference signal and is inversely phased from the at least one reference signal; and combining the input audio signal with the anti-noise signal to generate the first directional audio signal.

[0085] 19. The one or more non-transitory computer-readable media of clause 17 or 18, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following steps: store the anti-noise signal; and associate the anti-noise signal with the at least one reference signal.

[0086] 20. The one or more non-transitory computer-readable media of any of clauses 17-19, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: upon determining that the one or more audio signals are not to be categorized as the one or more noise elements, store data associated with the one or more audio signals as a further reference signal included in the first set of reference signals.

[0087] Any and all combinations of any of the claimed elements and / or any of the elements described in this application in any way possible are within the intended scope of the disclosure and protection.

[0088] The description of various embodiments has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0089] Aspects of the present embodiments can be embodied as a system, a method, or a computer program product. Accordingly, aspects of the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "module," or "system." Furthermore, any of the hardware and / or software technology, processes, functions, components, engines, modules, or systems described in the present disclosure can be implemented as a circuit or a set of circuits. Additionally, aspects of the present disclosure can take the form of a computer program product on one or more computer readable media having computer readable program code embodied in the medium.

[0090] Any combination of one or more computer readable medium can be utilized. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0091] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. Alternatively, computer program implemented processes can be conveyed, implemented, or performed by a general purpose computer, a special purpose computer, an ASIC, a programmable logic device, a signal processor, or other programmable data processing devices.

[0092] The flow and block diagrams in the drawings show the architectural, functional, and operational aspects of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block and combination of blocks in the block and / or flow diagrams can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or a combination of dedicated hardware and computer instructions.

[0093] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure can be devised without departing from the basic scope thereof, and the scope thereof is determined by the appended claims.

Claims

1. A method for reducing noise in an audio signal, the method comprising: determining a first location of a user in an environment based on sensor data acquired from a first set of sensors; acquiring, by the first set of sensors, one or more audio signals associated with sounds in the environment; identifying one or more noise elements in the one or more audio signals; identifying one or more individual loudspeakers in the one or more audio signals by comparing the one or more audio signals to learned speech elements and loudspeaker characteristics via a neural network; decomposing the one or more audio signals into one or more filtered signals, wherein each filtered signal corresponds to one or more frequency subbands of the one or more audio signals; and generating a first directional audio signal based on the one or more filtered signals, wherein, when the first directional audio signal is output by a first loudspeaker, the first loudspeaker produces a first sound field that attenuates the one or more frequency subbands associated with the one or more noise elements at the first location based on the first directional audio signal.

2. The method of claim 1, wherein identifying the one or more noise elements comprises: comparing the one or more audio signals to at least one reference signal; and when the one or more audio signals match the at least one reference signal, classifying the one or more audio signals based on the at least one reference signal.

3. The method of claim 1, wherein identifying the one or more noise elements comprises: comparing, by a neural network, a first audio signal included in the one or more audio signals to a first reference signal associated with a first noise element; and based on determining that the first audio signal matches the first reference signal, categorizing the first audio signal as including the first noise element.

4. The method of claim 1, further comprising: comparing a first audio signal included in the one or more audio signals to a first set of reference signals; and determining that the first audio signal does not match at least one reference signal included in the first set of reference signals; and storing data associated with the first audio signal as an additional reference signal included in the first set of reference signals.

5. The method of claim 1, wherein identifying the one or more noise elements comprises: comparing the one or more audio signals to each reference signal included in a first set of reference signals; and when the one or more audio signals match at least one reference signal included in the first set of reference signals, classifying the one or more audio signals as the one or more noise elements; and when the one or more audio signals do not match at least one reference signal included in the first set of reference signals, determining that the one or more audio signals will not be classified as the one or more noise elements.

6. The method of claim 1, further comprising: ​ determine a second position of a user in an environment based on sensor data acquired from the first set of sensors; and generate a second directional audio signal based on the one or more noise elements, wherein, when the second directional audio signal is output by a first speaker, the first speaker produces a second sound field that attenuates the one or more noise elements at the second position.

7. The method of claim 1, further comprising: determine a second position of the first speaker, wherein the first directional audio signal is based on the first position and the second position.

8. The method of claim 1, further comprising: receive an input audio signal from a second device over a first network, wherein the first directional audio signal includes at least a portion of the input audio signal.

9. The method of claim 1, further comprising: generate a first set of directional audio signals based on the one or more noise elements, wherein, when the first set of directional audio signals is output by a first plurality of speakers, the first plurality of speakers produces the first sound field.

10. An audio system, the audio system comprising: a first set of sensors that: produces sensor data associated with a first position of a user in an environment, and produces one or more audio signals associated with sound acquired from the environment; a first speaker; and a processor coupled to the first set of sensors and the first speaker, the processor that: determines the first position of the user based on the sensor data, receives the one or more audio signals from the first set of sensors, identifies one or more noise elements in the one or more audio signals, identifies one or more individual speakers in the one or more audio signals by comparing the one or more audio signals to learned speech elements and speaker characteristics via a neural network; decomposes one or more audio signals into one or more filtered signals, wherein each filtered signal corresponds to one or more frequency subbands of the one or more audio signals, and generates a first directional audio signal based on the one or more filtered signals, wherein the first speaker outputs the first directional audio signal based on the first directional audio signal to produce a first sound field that attenuates the one or more frequency subbands associated with the one or more noise elements at the first position.

11. The audio system of claim 10, further comprising a first database that stores a first set of reference signals associated with the one or more noise elements.

12. The audio system of claim 11, wherein the processor further: compares the one or more audio signals to the first set of reference signals; when the one or more audio signals match at least one reference signal included in the first set of reference signals, classifies the one or more audio signals as the one or more noise elements; and when the one or more audio signals do not match at least one reference signal included in the first set of reference signals, determines that the one or more audio signals will not be classified as the one or more noise elements.

13. The audio system of claim 10, wherein the first set of sensors comprises: at least one camera that acquires location data associated with the first location; and at least one microphone that acquires the one or more audio signals.

14. The audio system of claim 10, wherein the first speaker comprises a parametric speaker.

15. The audio system of claim 10, wherein: the first speaker is included in a plurality of parametric speakers of the audio system; the processor further generates a first set of directional audio signals based on the one or more noise elements; and each parametric speaker included in the plurality of parametric speakers outputs at least one directional audio signal of the first set of directional audio signals to produce the first sound field.

16. The audio system of claim 10, wherein: the first set of sensors further produces sensor data associated with a second location of the user; the processor further: determines the second location of the user based on the sensor data, and generates a second directional audio signal based on the one or more noise elements; and the first speaker outputs the second directional audio signal to produce a second sound field that attenuates the one or more noise elements at the second location.

17. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: determining a first location of a user in an environment; acquiring, by a first set of sensors, one or more audio signals associated with sounds in the environment; identifying one or more noise elements in the one or more audio signals by: comparing the one or more audio signals to each reference signal included in a first set of reference signals, and classifying the one or more audio signals as the one or more noise elements when the one or more audio signals match at least one reference signal included in the first set of reference signals; identifying one or more individual speakers in the one or more audio signals by comparing the one or more audio signals to learned speech elements and speaker characteristics via a neural network; decomposing one or more audio signals into one or more filtered signals, wherein each filtered signal corresponds to one or more frequency subbands of the one or more audio signals; and generating a first directional audio signal based on the one or more filtered signals, wherein, when the first directional audio signal is output by a first speaker, the first speaker produces a first sound field that attenuates the one or more frequency subbands associated with the one or more noise elements at the first location based on the first directional audio signal.

18. The one or more non-transitory computer-readable media of claim 17, wherein generating a first directional audio signal comprises: receiving an input audio signal; generating an anti-noise signal that matches a magnitude of the at least one reference signal and is inversely phased from the at least one reference signal; and combining the input audio signal with the anti-noise signal to generate the first directional audio signal.

19. The one or more non-transitory computer-readable media of claim 18, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following steps: storing the anti-noise signal; and associating the anti-noise signal with the at least one reference signal.

20. The one or more non-transitory computer-readable media of claim 17, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: upon determining that the one or more audio signals are not to be classified as the one or more noise elements, storing data associated with the one or more audio signals as a further reference signal included in the first set of reference signals.

Images