Audio device with sidetone processing

Abstract

Disclosed is an audio device with sidetone processing, the audio device comprising: a plurality of microphones; an output transducer configured to output a proximal audio output signal; and a processor circuit comprising a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal. The signal processor module is configured to process a first audio input signal and a second audio input signal to provide a plurality of filter parameters, the sidetone module is configured to obtain first data indicative of a plurality of filter parameters and to process the first audio input signal and the second audio input signal using one or more filters based on the first data to provide a sidetone audio output signal, and wherein the sidetone audio output signal comprises a sidetone signal, the near-end audio output signal is based on the sidetone audio output signal and the far-end audio input signal.

Description

[0001] This disclosure relates to the field of audio devices and methods performed by audio devices, and more particularly to audio devices and related methods having sidetone processing. Background Technology

[0002] Sidetone refers to the auditory feedback of one's own voice during a voice call or communication session. In traditional telecommunications systems, sidetone is naturally provided through the handset or headphones, allowing users to hear their own voice as they speak, which helps maintain a natural speaking volume and improves speech quality.

[0003] However, with the advent of modern audio devices such as headphones, in-ear headphones, and headsets, sidetone processing has become more complex. Due to the lack of natural sidetone feedback, users of such audio devices often encounter difficulties in adjusting their speaking volume and maintaining call quality.

[0004] Existing sidetone processing solutions in audio devices typically involve hardware components such as microphones and signal processing circuitry. These solutions aim to capture the user's voice through the microphone, process the audio signal, and feed that audio signal back to the user's in-ear or over-ear headphones in real time, thereby mimicking the natural sidetone experience found in traditional telecommunications systems.

[0005] However, traditional sidetone processing techniques may have drawbacks such as latency, insufficient signal processing accuracy, and limited adaptability to different user environments and preferences. Summary of the Invention

[0006] Therefore, there is a need for improved audio devices with efficient sidetone processing and methods performed by the audio devices, which can mitigate, alleviate or resolve existing shortcomings and provide improved sidetone processing efficiency, thereby achieving reduced latency, lower computational costs and battery savings.

[0007] In other words, there is a need for an improved audio device with sidetone processing capabilities that overcomes these limitations and provides a seamless and natural user experience during voice calls and communication sessions.

[0008] An audio device is disclosed. The audio device can be configured to act as a receiver device and / or a transmitter device. The audio device may include memory, an interface, and one or more processors. Optionally, the audio device includes one or more output transducers (such as one or more speakers) and one or more input transducers (such as one or more microphones). In one or more examples or embodiments, one or more processors are configured to acquire audio data, such as audio input signals. In other words, the audio device can be configured to acquire audio data, such as audio input signals, using one or more processors and / or via an interface.

[0009] The audio device includes a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal.

[0010] The audio device includes an output transducer configured to output a near-end audio output signal.

[0011] The audio device includes a processor circuit, which includes a signal processor module configured to provide a far-end audio output signal and a side-tone module configured to provide a side-tone audio output signal.

[0012] The signal processor module is configured to process a first audio input signal and a second audio input signal to provide multiple filter parameters. The sidetone module is configured to obtain first data indicating the multiple filter parameters and use one or more filters based on the first data to process the first audio input signal and the second audio input signal to provide a sidetone audio output signal. The near-end audio output signal may be based on the sidetone audio output signal and / or the far-end audio input signal.

[0013] Furthermore, a method for sidetone processing is disclosed. This method is performed by an audio device (such as the audio device disclosed herein). The audio device includes a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal. The audio device includes an output transducer configured to output a near-end audio output signal. The audio device includes processor circuitry including a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal.

[0014] The method includes obtaining a first audio input signal and a second audio input signal. The method includes processing the first audio input signal and the second audio input signal using a signal processor module to provide multiple filter parameters. The method includes obtaining first data indicating the multiple filter parameters using a sidetone module. The method includes processing the first audio input signal and the second audio input signal using one or more filters based on the first data using the sidetone module to provide a sidetone audio output signal. The method includes, for example, outputting a near-end audio output signal based on the sidetone audio output signal and a far-end audio input signal.

[0015] The audio devices and methods disclosed herein provide improved sidetone processing efficiency, and consequently achieve reduced latency, lower computational costs, and battery savings.

[0016] Furthermore, the audio device and method of this disclosure have the advantage that the sidetone module has the same or similar noise reduction capabilities as the signal processor module (such as the same or similar noise reduction capabilities as the transmission algorithm operating on the signal processor module). This is achieved by having the signal processor module process the first and second audio input signals to provide multiple filter parameters, which can then be used by the sidetone module for sidetone processing, for example, instead of having the sidetone module process the first and second audio input signals to provide filter parameters. For example, this can reduce the latency of sidetone processing because the filter parameters are provided from the signal processor module. For example, the noise reduction capability can remove background noise from the sidetone signal (such as the sidetone audio output signal), leaving only the user's voice. This provides a clearer and more comfortable user experience, especially in noisy environments. Furthermore, the audio device and method of this disclosure allows the sidetone module to have similar performance to the signal processor module, which helps the user gain a true impression of the call quality and the quality of the user's voice transmitted to the remote end during the call. The audio device and method of this disclosure also allows the signal processor module (such as the transmission algorithm) to run only once, which reduces power consumption. This is achieved by having the signal processor module process the first and second audio input signals to provide multiple filter parameters and simultaneously provide a far-end audio output signal, for example, instead of having both the signal processor module and the sidetone module process the same or similar complex algorithms. It is understood that the audio devices and methods of this disclosure improve sidetone processing, for example, by applying filtering (such as finite impulse response FIR filtering) to the microphone signal. Specifically, the filter or filter parameters (such as FIR filters) may not be computed by the sidetone module (e.g., cannot be computed by a sidetone algorithm operating on the sidetone module), but rather by an algorithm responsible for creating a far-end audio output signal (such as a Tx signal), which is the signal sent to the far end during a call. The filter parameters (such as FIR coefficients) can then be passed to the sidetone path, allowing for a more efficient process. It is understood that this eliminates the need for the signal processor module algorithm to run twice and reduces latency, which is extremely important for one's natural speech experience. Therefore, the techniques of this disclosure can use different algorithms and entities of the audio device to compute filter parameters (such as FIR coefficients) and then apply the filter parameters to the sidetone path. Attached Figure Description

[0017] The above and other features and advantages of the present invention will become apparent to those skilled in the art from the following detailed description of exemplary embodiments thereof with reference to the accompanying drawings, in which:

[0018] Figure 1An exemplary audio device according to this disclosure is schematically illustrated, and

[0019] Figures 2A to 2B A flowchart of an exemplary method according to this disclosure is shown. Detailed Implementation

[0020] Various exemplary embodiments and details are described below with reference to the accompanying drawings (where applicable). It should be noted that the drawings may be drawn to scale or not, and throughout the drawings, elements with similar structures or functions are indicated by similar reference numerals. It should also be noted that the drawings are intended only to facilitate the description of embodiments. The drawings are not intended to be an exhaustive description of the invention or a limitation on the scope of the invention. Furthermore, the illustrated embodiments do not necessarily possess all the aspects or advantages shown. Aspects or advantages described in connection with the specific embodiments are not necessarily limited to that embodiment and may be practiced in any other embodiment even if not so shown or so explicitly described.

[0021] The accompanying drawings are schematic and simplified for clarity, and they show only details that aid in understanding this disclosure, while other details have been omitted. Throughout the specification, the same reference numerals are used for the same or corresponding parts.

[0022] An audio device is disclosed. The audio device can be configured to act as a receiver device and / or a transmitter device. In other words, the audio device is configured to receive input signals (such as audio data) from an audio device configured to act as a transmitter device, or vice versa. The audio device disclosed herein may include one or more interfaces, one or more audio receivers, one or more microphones (e.g., including a first microphone), one or more processors, and one or more memories. The one or more interfaces may include one or more of a wireless interface, a wireless transceiver, an antenna, an antenna interface, a microphone interface, and a receiver interface.

[0023] In addition, the audio device may include one or more microphones, such as a first microphone, optionally a second microphone, optionally a third microphone, and optionally a fourth microphone. The audio device may also include one or more audio receivers, such as an audio receiver (e.g., a speaker).

[0024] An audio device can be considered as an audio device configured to acquire audio data (such as input signals, e.g., audio input signals), output audio signals, and process input signals (such as audio input signals). An audio device can be considered as, or may include, a headset, telephone receiver, hearing aid, and / or video strip. An audio device can be considered, for example, as a conferencing audio device configured for use by one party (such as one or more users at a near end) to communicate with one or more other parties (such as one or more users at a far end). An audio device configured to act as a receiver device can also be configured to act as a transmitter device when sending an output signal back to the far end. Therefore, receiver audio devices and transmitter audio devices can switch between acting as receiver audio devices and acting as transmitter audio devices. An audio device can be considered as a smart audio device. An audio device can be used for large conferences and / or meetings between two or more parties located far apart from each other. An audio device can be used by one or more users in the vicinity of where the audio device is located (also referred to as the near end). An audio device can be configured, for example, to use an audio receiver and output audio device output at the receiver end based on the input signal. The output of an audio device can be considered as an audio output signal, which is the output of the audio device and the audio receiver located near the user of the audio device.

[0025] An audio device can be a single audio device. An audio device can also be considered as multiple interconnected audio devices, such as a system (e.g., an audio device system). This system may include one or more users.

[0026] In one or more exemplary audio devices, the interface includes a wireless transceiver (also referred to as a radio transceiver) and an antenna for wirelessly transmitting and receiving input signals (such as audio signals) (e.g., wireless transmission for output signals and / or wireless reception of wireless input signals). The audio device can be configured to wirelessly communicate with one or more electronic devices (such as another audio device, a smartphone, tablet, computer, and / or smartwatch). The audio device may optionally include an antenna for converting one or more wireless input audio signals into antenna output signals. The audio device system and / or the audio device can be configured to wirelessly communicate via a wireless communication system (such as a short-range wireless communication system, such as Wi-Fi, Bluetooth, Zigbee, IEEE 802.11, IEEE 802.15, infrared, etc.).

[0027] Audio equipment systems and / or audio devices can be configured to conduct wireless communication via wireless communication systems such as 3GPP systems, such as 3GPP systems supporting one or more of the following: New Radio (NR), Narrowband IoT, NB-IoT and LTE-M (Long Term Evolution-Enhanced Machine Type Communication), millimeter wave communication, such as millimeter wave communication in licensed bands, such as device-to-device millimeter wave communication in licensed bands.

[0028] In one or more exemplary audio device systems and / or audio devices, the interface of the audio device includes one or more of the following: a Bluetooth interface, a Bluetooth Low Energy interface, and a magnetic induction interface. For example, the interface of the audio device may include a Bluetooth antenna and / or a magnetic interference antenna.

[0029] In one or more exemplary audio devices, the interface may include a connector for wired communication via a connector, such as by using a cable. The connector can connect one or more microphones to the audio device. The connector can connect the audio device to electronic equipment, such as for wired connections. The connector can be considered an electrical connector, such as a physical connector used to connect an audio device to another device via a wire.

[0030] One or more interfaces may be or include wireless interfaces (such as transmitters and / or receivers) and / or wired interfaces (such as connectors for physical coupling). For example, an audio device may have an input interface configured to receive data (such as microphone input signals). In one or more exemplary audio devices, the audio device can be used in all types of environments and for all forms of use, such as for headsets and / or video conferencing equipment. For example, the audio device may not have specific microphone placement requirements. In one or more exemplary audio devices, the audio device may include an external microphone.

[0031] The audio device includes a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal. The first and second audio input signals may be based on input signals (such as speech and / or sound) from the near end when obtained from the plurality of microphones (such as the first and / or second microphones of the audio device).

[0032] An audio device includes an output transducer configured to output a near-end audio output signal. In other words, an audio device may include one or more output transducers (such as speakers) configured to output a near-end audio output signal (such as outputting a near-end audio output signal to a user of the audio device) at the near end. In one or more exemplary audio devices, processor circuitry is configured to output a near-end audio output signal (such as a near-end audio output) via an interface (such as via an output transducer). In other words, the audio device may be configured to output a near-end audio output signal via a wired and / or wireless interface, via one or more receivers (such as output transducers) at the near end on the audio device itself. In one or more exemplary audio devices, processor circuitry is configured to output a far-end audio output signal (such as a far-end audio output) via an interface. In other words, the audio device may be configured to output a far-end audio output signal via a wired and / or wireless interface to a far end.

[0033] The audio device includes a processor circuit, which includes a signal processor module configured to provide a far-end audio output signal and a side-tone module configured to provide a side-tone audio output signal.

[0034] A signal processor module can be considered as a processor module configured to process one or more audio input signals, such as audio input signals received via a transceiver (such as a wireless transceiver) from one or more microphones at a near end and / or audio input signals from a far end. In one or more exemplary audio devices, the signal processor module can be considered as or includes a digital signal processor (DSP). In one or more examples or embodiments, the signal processor module disclosed herein forms part of a digital signal processor (DSP). In one or more exemplary audio devices, the audio device includes one or more processors having a DSP. The signal processor module can operate on one or more processors of the audio device.

[0035] Audio devices (such as signal processor modules) can be configured to use one or more processors to process audio input signals (such as the first audio input signal, second audio input signal, third audio input signal, fourth audio input signal, and / or far-end audio input signal disclosed herein) to provide audio output signals, such as near-end audio output signals and / or far-end audio output signals. Audio devices (such as signal processor modules) can be configured to operate according to one or more signal processing algorithms. For example, an audio device (such as a signal processor module) can be configured to operate according to a transmitter algorithm (such as a Tx algorithm). In other words, a signal processor module can be configured to operate according to a DSP algorithm. A transmitter algorithm can be considered or represented as a signal processor module algorithm.

[0036] Processing an audio input signal to provide an audio output signal may include performing one or more audio processing steps on the audio input signal. For example, processing an audio input signal to provide an audio output signal may include performing noise reduction (such as background noise reduction) on the audio input signal, such as using a signal processor module, to provide a denoised audio output signal. Other examples may include processing an audio input signal to provide an audio output signal that includes performing filtering on the audio input signal, such as using a signal processor module and / or a Tx algorithm, to provide a filtered audio output signal and / or a speech enhancement task on the audio input signal. Furthermore, processing an audio input signal to provide an audio output signal may include performing compression on the audio input signal. The signal processor module may perform processing according to a Tx algorithm to provide a far-end audio output signal and / or a near-end audio output signal, such as echo control, dereverberation, denoising, and / or beamforming.

[0037] A signal processor module is configured to process a first audio input signal and a second audio input signal to provide multiple filter parameters, such as filtering parameters. For example, the signal processor module may be configured to process the first audio input signal and the second audio input signal to provide a set of filter parameters used by a sidetone module. Filter coefficients can be viewed as numerical values ​​used in a digital filter to adjust the characteristics of the filter's response to an input signal. Filters can be used to modify the frequency content of a signal (such as audio) by selectively amplifying or attenuating certain frequency components. In one or more examples or embodiments, the multiple filter parameters include multiple finite impulse response (FIR) filter coefficients. In one or more exemplary embodiments, the multiple filter parameters include, for example, multiple filter gains in the frequency domain. Filter parameters may include filter coefficients and / or filter gains. In other words, multiple filter parameters can be used in an FIR filter configured to filter audio input signals (such as the first audio input signal and / or the second audio input signal).

[0038] A sidetone module can be viewed as a sidetone processor module configured to process one or more audio input signals, such as audio input signals from one or more microphones at the near end. In one or more exemplary audio devices, the sidetone module can be viewed as or include a signal processor responsible for managing the processing, generation, and control of sidetone feedback during voice communication activities on the audio device. Sidetone can be viewed as the user of the audio device hearing their own voice substantially in real time through the audio device during voice communication. Sidetone can be used to provide a natural and familiar auditory feedback loop to the speaker or user. The sidetone module can be configured to adjust one or more of the amplitude, frequency response, and delay of the audio input signal, such as the user's own voice, to provide a natural sidetone effect that simulates the experience of speaking without an audio device, such as without headphones. Compared to a signal processor module, the sidetone module can be viewed as operating within a low latency audio framework. In other words, the sidetone module can be configured to operate within a lower latency audio framework than a signal processor module.

[0039] In one or more examples or implementations, the sidetone module disclosed herein forms part of one or more processors, such as a digital signal processor (DSP). The sidetone module may operate on one or more processors of an audio device.

[0040] Audio devices (such as sidetone modules) can be configured to process audio input signals (such as the first, second, third, and / or fourth audio input signals disclosed herein) using one or more processors to provide sidetone audio output signals. Audio devices (such as sidetone modules) can be configured to operate according to one or more signal processing algorithms. For example, an audio device (such as a sidetone module) can be configured to operate according to a sidetone algorithm, such as being responsible for managing the processing, generation, and control of sidetone feedback during voice communication activities on the audio device. The sidetone module can process the audio input signals in a sidetone processing path, while the signal processor module can process the audio input signals in a transmitter processing path, such as a digital signal processor processing path.

[0041] The sidetone module is configured to acquire indications and / or first data based on multiple filter parameters. In other words, the sidetone module can be configured to receive, retrieve, and / or determine indications or first data based on multiple filter parameters. In one or more examples or implementations, the first data includes filter parameters, and / or the first data is derived based on the filter parameters. For example, the first data may include or be based on multiple FIR coefficients from a signal processor module.

[0042] A sidetone module is configured to process a first audio input signal and a second audio input signal using one or more filters based on first data to provide a sidetone audio output signal. In one or more examples or embodiments, the one or more filters include multiple filters from a signal processor module. For example, the one or more filters may include multiple FIR filters from a signal processor module. Processing the first audio input signal and the second audio input signal may include applying one or more filters, such as filter parameters, during a filtering process at the sidetone module. For example, the sidetone module may include a filtering module, such as an FIR filter, for filtering the audio input signal. Processing the first audio input signal and the second audio input signal may include applying FIR filter coefficients to the first audio input signal and the second audio input signal during a filtering process at the sidetone module. The sidetone module (such as a filtering module) may be configured to process the audio input signal according to one or more filters to provide a filtered audio output signal.

[0043] In one or more exemplary audio devices, a sidetone module is configured to determine one or more filters based on first data. In other words, the sidetone module may be configured to determine one or more filters based on a plurality of filter parameters. In one or more examples or implementations, the one or more filters include filter parameters, and / or the one or more filters are derived based on filter parameters. For example, the one or more filters may include or be based on a plurality of filter coefficients and / or filter gains. For example, the one or more filters may include or be based on a plurality of FIR coefficients from a signal processor module.

[0044] By obtaining first data indicating multiple filter parameters, the sidetone module has the same or similar noise reduction capabilities as the signal processor module (such as the same or similar noise reduction capabilities as the transmission algorithm operating on the signal processor module). This can be achieved by having the signal processor module process the first and second audio input signals to provide multiple filter parameters, which can then be used by the sidetone module for sidetone processing, for example, instead of having the sidetone module process the first and second audio input signals to provide filter parameters. For example, this can reduce the latency of sidetone processing because the filter parameters are provided from the signal processor module. For example, the noise reduction capability can remove background noise from the sidetone signal (such as the sidetone audio output signal), leaving only the user's voice. This provides a clearer and more comfortable user experience, especially in noisy environments. Furthermore, the audio device and method of this disclosure allow the sidetone module to have similar performance to the signal processor module, which helps the user gain a true impression of the call quality and the quality of the user's voice transmitted to the remote end during the call. The audio device of this disclosure allows the signal processor module (such as the transmission algorithm) to run only once, which reduces power consumption. This is achieved by having the signal processor module process the first and second audio input signals to provide multiple filter parameters and simultaneously provide a far-end audio output signal, for example, instead of having both the signal processor module and the sidetone module process the same or similar complex algorithms. It is understood that the audio devices and methods of this disclosure improve sidetone processing, for example, by applying filtering (such as finite impulse response FIR filtering) to the microphone signal. Specifically, the filter or filter parameters (such as FIR filters) may not be computed by the sidetone module (e.g., cannot be computed by a sidetone algorithm operating on the sidetone module), but rather by an algorithm responsible for creating a far-end audio output signal (such as a Tx signal), which is the signal sent to the far end during a call. The filter parameters (such as FIR coefficients) can then be passed to the sidetone path, allowing for a more efficient process. It is understood that this eliminates the need for the signal processor module algorithm to run twice and reduces latency, which is extremely important for one's natural speech experience. Therefore, the techniques of this disclosure can use different algorithms and entities of the audio device to compute filter parameters (such as FIR coefficients) and then apply the filter parameters to the sidetone path.

[0045] The near-end audio output signal is based on the side-tone audio output signal and the far-end input signal. In other words, the audio device can be configured to determine the near-end audio output signal based on the side-tone audio output signal and / or the far-end audio input signal received from the far end. Therefore, the audio device can provide auditory feedback of the user's own voice during a voice call or communication session, which in turn allows the user of the audio device to hear their own voice as they speak. In one or more examples or embodiments, the audio device includes a mixer configured to mix the side-tone audio output signal and the far-end audio input signal to provide the near-end audio output signal. In one or more audio devices, the audio device (such as a mixer) is configured to combine the side-tone audio output signal and the far-end audio output signal to provide a mixer output signal. The mixer can, for example, be configured to add the side-tone audio output signal to the far-end audio input signal to provide the mixer audio output signal. The mixer audio output signal can be considered as the near-end audio output signal. For example, a mixer can be configured to generate a mixer output signal that includes a portion of the side-tone audio output signal and a portion of the far-end audio input signal.

[0046] In one or more exemplary audio devices, obtaining first data includes performing windowing on the first data to reduce the sample size of the first data. The audio device (such as a sidetone module) can be configured to perform windowing on multiple filter parameters obtained from a signal processor module. The audio device (such as a sidetone module) may include a windowing module configured to perform windowing on the first data. The output of the windowing module can be considered a windowed output. Windowing can be performed before processing a first audio input signal and a second audio input signal using one or more filters. In other words, the sidetone module can be configured to multiply the first data (such as multiple filter parameters) by a window function before applying one or more filters. It is understood that for an FIR filter with finite-length filter coefficients, a window function can be applied to the FIR filter coefficients to modify the characteristics of the FIR filter coefficients. For example, FIR filter coefficients can be derived from an ideal frequency response. Therefore, directly using FIR filter coefficients can lead to problems such as spectral leakage and ripple effects in the frequency domain. Using windowing allows the filter coefficients to gradually decrease towards zero at the edges, for example, reducing the abruptness of the transition from the passband to the stopband. Windowing the first data can include, for example, applying an attenuation window or a Hanning window to the first data, such as to multiple filter coefficients.

[0047] In one or more exemplary audio devices, a sidetone module is configured to perform smoothing of first data. The audio device (such as the sidetone module) may be configured to perform smoothing of multiple filter parameters obtained from a signal processor module. The audio device (such as the sidetone module) may include a smoothing module configured to perform smoothing of the first data. The output of the smoothing module can be considered a smoothed output. Smoothing may be performed before processing the first audio input signal and the second audio input signal using one or more filters, but after windowing the first data. The sidetone module may perform smoothing on the windowed output. In other words, the sidetone module may be configured to reduce and / or smooth abrupt transitions and / or irregularities in multiple filter parameters (such as filter frequency response). It is understood that for an FIR filter with finite-length filter coefficients, smoothing may be applied to the FIR filter coefficients to reduce or smooth abrupt transitions, ripples, or irregularities in the actual frequency response of the filter. For example, smoothing techniques may include modifying filter parameters to achieve a smoother frequency response curve with reduced ripples or abrupt transitions. Smoothing the initial data can include performing interpolation in the time domain, for example, by using new filter parameters that are a smaller percentage than the previous filter parameters (such as those from previous iterations) to avoid abrupt changes. For example, a sidetone module can be configured to use 10% of the new filter parameters (such as those from a new iteration where the filter parameters were determined) and 90% of the previous filter parameters.

[0048] In one or more exemplary audio devices, the input buffer size of the sidetone module is smaller than or equal to the input buffer size of the signal processor module. In other words, the audio device (such as the sidetone module) can be configured with an input buffer size smaller than or equal to the input buffer size of the signal processor module. Reducing the input buffer size of the sidetone module allows for a reduction in latency in the sidetone path, which is advantageous because the sidetone path may be more sensitive to latency than the signal processor module path. In one or more examples or embodiments, the input buffer sizes of the first and second audio input signals of the sidetone module are smaller than the input buffer sizes of the first and second audio input signals of the signal processor module. For example, the input buffer size of the sidetone module can be in the range of 16 to 64 times smaller than the input buffer size of the signal processor module. In one or more exemplary embodiments, the maximum size of the input buffer of the sidetone module can be the size when the input buffer size of the sidetone module is equal to the input buffer size of the signal processor module.

[0049] In one or more exemplary audio devices, the output buffer size of the sidetone module is smaller than or equal to the output buffer size of the signal processor module. In other words, the audio device (such as the sidetone module) can be configured with an output buffer size smaller than or equal to the output buffer size of the signal processor module. Reducing the output buffer size of the sidetone module allows for a reduction in latency in the sidetone path, which is advantageous because the sidetone path may be more sensitive to latency than the signal processor module path. In one or more examples or embodiments, the output buffer size of the first and second audio input signals of the sidetone module is smaller than the output buffer size of the first and second audio input signals of the signal processor module. For example, the output buffer size of the sidetone module can be in the range of 16 to 64 times smaller than the output buffer size of the signal processor module. In one or more exemplary embodiments, the maximum size of the output buffer of the sidetone module can be the size when the output buffer size of the sidetone module is equal to the output buffer size of the signal processor module.

[0050] For example, reducing the input buffer size and / or output buffer size of the sidetone module (such as compared to the signal processor module) can help reduce the impact of algorithmic latency at the audio device (such as the deterministic algorithmic latency attributable to multiple filter parameters).

[0051] In one or more exemplary audio devices, the sidetone module includes a downsampler and / or an upsampler configured to reduce the amount of computation at the sidetone module. For example, the downsampler and / or upsampler may be configured to reduce the amount of computation at the sidetone module (e.g., reducing millions of operations per second) to make the sidetone module computationally efficient. As discussed earlier, it is advantageous to have the smallest possible input buffer size. However, there may be a trade-off between buffer size and the number of operations. The smaller the buffer size, the higher the number of operations. However, for the sidetone module, lower latency and therefore lower buffer size may take precedence over computational efficiency.

[0052] In one or more exemplary audio devices, the audio device includes an active noise cancellation (ANC) module configured to acquire and process a sidetone audio output signal based on a first audio input signal and / or a second audio input signal to provide an ANC audio output signal. The ANC module can be considered as a module configured to process audio input signals to perform ANC on them. For example, the ANC module may perform ANC on the sidetone audio output signal and / or the far-end audio input signal before they are output at the audio device (such as at an output transceiver). The sidetone audio output signal can be considered as the audio input signal of the ANC module. In one or more examples or embodiments, the ANC module may be configured to acquire and process the sidetone audio output signal and the far-end audio input signal to provide an ANC audio output signal. In one or more examples or embodiments, the ANC module may acquire a mixed signal of the sidetone audio output signal and the far-end audio input signal from a mixer.

[0053] In one or more exemplary audio devices, the audio device includes a third microphone configured to provide a third audio input signal and a fourth microphone configured to provide a fourth audio input signal. In one or more exemplary audio devices, the third microphone is a feedforward microphone, and the fourth microphone is a feedback microphone. In one or more exemplary audio devices, an ANC module is configured to process a sidetone audio output signal based on the third audio input signal and the fourth audio input signal to provide an ANC audio output signal. In other words, the ANC module may be able to perform feedforward ANC and / or feedback ANC based on the third audio input signal and / or the fourth audio input signal. For example, the ANC module may be configured to perform mixed ANC based on a first audio input signal, a second audio input signal, a third audio input signal, and / or the fourth audio input signal. In one or more examples or embodiments, the ANC module is configured to process a sidetone audio output signal based on the first audio input signal, the second audio input signal, the third audio input signal, and / or the fourth audio input signal to provide an ANC audio output signal.

[0054] In one or more examples or implementations, an audio device (such as an ANC module) includes an ANC regulator, such as an ANC control mechanism that manages or regulates the operation of the ANC process. For example, the ANC regulator may be configured to determine the gain to be applied to a sidetone audio output signal, such as the gain that the sidetone module would apply when processing an audio input signal before providing the sidetone audio output signal. The ANC regulator may be configured to determine the gain based on an audio input signal (such as based on a fourth audio input signal) (e.g., based on a feedback microphone audio input signal).

[0055] In one or more exemplary audio devices, the audio device is configured to determine a near-end audio output signal based on an ANC audio output signal. For example, the audio device may be configured to apply gain to the ANC audio output signal to provide a near-end audio output signal.

[0056] In one or more exemplary audio devices, a sidetone module is configured to process a third audio input signal and / or a fourth audio input signal to provide a sidetone audio output signal. Processing the third and fourth audio input signals may include applying one or more filters, such as filter parameters, during a filtering process at the sidetone module. For example, the sidetone module may include a filtering module, such as an FIR filter, for filtering the audio input signals. Processing the third and fourth audio input signals may include applying FIR filter coefficients to the third and fourth audio input signals during a filtering process at the sidetone module. The sidetone module (such as a filtering module) may be configured to process the audio input signals (such as a first audio input signal, a second audio input signal, a third audio input signal, and / or a fourth audio input signal) according to one or more filters to provide a filtered audio output signal.

[0057] In one or more exemplary audio devices, a signal processor module is configured to process a third audio input signal and / or a fourth audio input signal to provide multiple filter parameters. For example, the signal processor module may be configured to process the third and fourth audio input signals to provide a set of filter parameters used by a sidetone module. In other words, multiple filter parameters can be used in an FIR filter configured to filter audio input signals such as a first audio input signal, a second audio input signal, a third audio input signal, and / or a fourth audio input signal.

[0058] In one or more exemplary audio devices, a sidetone module is initialized using one or more predetermined filters to process a first audio input signal and a second audio input signal. In other words, a signal processor module can be configured to obtain or determine one or more predetermined filter parameters to initialize the sidetone module for processing the first and second audio input signals. First data may include one or more predetermined filter parameters, such as predetermined filter coefficients and / or filter gains. The sidetone module can obtain or determine one or more filters, such as one or more predetermined filters, based on one or more predetermined filter parameters. The predetermined filter parameters and predetermined filters can be used, for example, to initialize the sidetone module at the start of a voice conversation. It is understood that the predetermined filters can be adaptive or constant.

[0059] In one or more exemplary audio devices, a signal processor module includes a noise reduction module and a first signal processor configured to operate according to a first processing algorithm. The noise reduction module may, for example, include a denoiser and / or an echo controller. The noise reduction module can be considered as a module configured to perform noise reduction (such as background noise reduction) on an audio input signal. The noise reduction module may be configured to perform noise reduction, speech pickup, or a combination of both.

[0060] In one or more exemplary audio devices, a noise reduction module is configured to process a first audio input signal and a second audio input signal to provide a noise-reduced output, such as a denoised audio output signal. The noise reduction module may be configured to process the first audio input signal, the second audio input signal, the third audio input signal, and / or a fourth audio input signal. In one or more exemplary audio devices, a first signal processor is configured to process the noise-reduced output according to a first processing algorithm to provide a far-end audio output signal. The noise-reduced output may be considered as an input to the first signal processor. The noise reduction module may provide a noise-reduced output for each processed audio input signal. For example, the noise reduction module may provide a first noise-reduced output for a processed first audio input signal, a second noise-reduced output for a processed second audio input signal, a third noise-reduced output for a processed third audio input signal, and / or a fourth noise-reduced output for a processed fourth audio input signal. The first processing algorithm may be configured to process the audio input signals (such as the noise-reduced output) to provide a far-end audio output signal and / or one or more filter parameters, such as one or more FIR filter coefficients. The first processing algorithm may be considered as a transmission algorithm as disclosed herein, such as a DSP algorithm. The first processing algorithm can be configured to provide both the remote audio output signal and filter parameters as disclosed herein.

[0061] In one or more exemplary audio devices, a first signal processor is configured to determine filter parameters and send the filter parameters to a sidetone module. In other words, the first signal processor may be configured to use a first processing algorithm to determine the filter parameters. For example, the first signal processor may be configured to determine FIR filter coefficients and send the FIR filter coefficients to the sidetone module. In other words, the first signal processor may be configured to send first data, including or indicating the filter parameters, to the sidetone module.

[0062] In one or more exemplary audio devices, a signal processor module is configured to perform wind noise detection on a first audio input signal and / or a second audio input signal and provide a wind noise detection indication to a sidetone module. In one or more examples or embodiments, a first signal processor is configured to perform wind noise detection on the first audio input signal, the second audio input signal, the third audio input signal, and / or the fourth audio input signal, for example, using a first processing algorithm. The signal processor module may indicate a wind noise detection flag to the sidetone module. It is understood that the wind noise detection flag may be triggered based on the satisfaction of one or more criteria, such as wind speed, direction, or other environmental factors. The sidetone module may be configured to perform a hysteresis on the wind noise detection indication. For example, performing a hysteresis on the wind noise detection indication (such as a wind noise detection flag) may involve introducing a delayed and historically relevant response to wind noise conditions, such as improving system stability, reliability, and the smoothness of transitions between wind noise detection states.

[0063] In one or more exemplary audio devices, the audio device is configured to perform one or more of in-ear detection, sidetone user gain, and mute button detection. A sidetone module may be configured to obtain information about one or more of in-ear detection, sidetone user gain, and mute button. The sidetone module may be configured to process a first audio input signal, a second audio input signal, a third audio input signal, and / or a fourth audio input signal based on the in-ear detection information, the sidetone user gain information, and / or the mute button information, to, for example, control the gain of the sidetone audio output signal.

[0064] For example, an audio device can be configured to detect whether a user is wearing the audio device, such as whether the audio device is inserted into and / or on the user's ear. For example, an audio device can be configured to detect changes or adjustments in sidetone gain, such as when the user manually changes the sidetone gain. For example, an audio device can be configured to detect when a mute button is present and when the mute function is activated by the user of the audio device.

[0065] A method for sidetone processing is disclosed. This method is performed by an audio device. The audio device includes a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal. The audio device includes an output transducer configured to output a near-end audio output signal. The audio device includes processor circuitry, including a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal.

[0066] The method includes obtaining a first audio input signal and a second audio input signal. The method includes processing the first and second audio input signals using a signal processor module to provide multiple filter parameters. The method includes obtaining first data indicating the multiple filter parameters using a sidetone module. The method includes processing the first and second audio input signals using the sidetone module with one or more filters based on the first data to provide a sidetone audio output signal. The method includes outputting a near-end audio output signal based on the sidetone audio output signal and the far-end input signal.

[0067] In one or more exemplary methods, obtaining the first data includes performing windowing on the first data to reduce the sample size of the first data.

[0068] In one or more exemplary methods, the processing includes performing smoothing of the first data.

[0069] In one or more exemplary methods, the processing includes using downsamplers and / or upsamplers to reduce computation at the sidetone module.

[0070] In one or more exemplary methods, the method includes obtaining and processing a sidetone audio output signal based on a first audio input signal and a second audio input signal to provide an ANC audio output signal.

[0071] In one or more exemplary methods, the audio device includes a third microphone configured to provide a third audio input signal and a fourth microphone configured to provide a fourth audio input signal. In one or more exemplary methods, the third microphone is a feedforward microphone, and the fourth microphone is a feedback microphone. In one or more exemplary methods, the processing includes processing a sidetone audio output signal based on the third audio input signal and the fourth audio input signal to provide an ANC audio output signal.

[0072] In one or more exemplary methods, outputting a near-end audio output signal includes determining the near-end audio output signal based on an ANC audio output signal.

[0073] In one or more exemplary methods, the processing includes processing a third audio input signal and / or a fourth audio input signal to provide a sidetone audio output signal.

[0074] In one or more exemplary methods, the processing includes processing a third audio input signal and / or a fourth audio input signal to provide multiple filter parameters.

[0075] In one or more exemplary methods, the method includes using a sidetone module to determine one or more filters based on first data.

[0076] In one or more exemplary methods, the method includes initializing a sidetone module to process a first audio input signal and a second audio input signal using one or more predetermined filters.

[0077] In one or more exemplary methods, the signal processor module includes a noise reduction module and a first signal processor configured to operate according to a first processing algorithm. In one or more exemplary methods, the method includes processing a first audio input signal and a second audio input signal using the noise reduction module for S104 to provide a noise-reduced output. In one or more exemplary methods, processing the noise-reduced output using the first signal processor according to the first processing algorithm for S108 to provide a far-end audio output signal.

[0078] In one or more exemplary methods, the process includes determining filter parameters using a first signal processor and sending the filter parameters to a sidetone module using the first signal processor.

[0079] In one or more exemplary methods, the method includes performing wind noise detection on a first audio input signal and / or a second audio input signal and providing a wind noise detection indication to a sidetone module.

[0080] In one or more exemplary methods, the method includes performing one or more of in-ear detection, sidetone user gain, and mute button detection.

[0081] It should be understood that the description of the characteristics of the audio device also applies to the corresponding characteristics in the methods of operating the audio device disclosed herein, and vice versa.

[0082] Figure 1 An exemplary audio device (such as audio device 10) according to this disclosure is schematically illustrated. Audio device 10 can be considered as an audio communication device. Audio device 10 can be considered as a communication device for performing calls (such as audio and / or video calls). Audio device 10 can be considered as an audio device with sidetone processing.

[0083] Audio device 10 can be configured to act as a receiver device and / or a transmitter device. In other words, audio device 10 can be configured to receive input signals from other audio devices configured to act as transmitter devices and / or configured to send output signals to other audio devices. Audio device 10 includes an interface and memory (not shown). Optionally, audio device 10 includes an output transducer 10D and one or more microphones, such as a first microphone 10E1 and a second microphone 10E2. Optionally, audio device 10 includes one or more transceivers, such as a first wireless transceiver 10F and a second wireless transceiver 10G. Audio device 10 can be considered as an audio device configured to acquire audio signals, output audio signals, and process audio signals. Audio device 10 can be considered, for example, a conference audio device or headset audio device configured for one party (such as one or more users at a near end) to communicate with one or more other parties (such as one or more users at a far end). Audio device 10 can be considered as a smart audio device. Audio device 10 can be used for communication between two or more parties at a distance from each other, large conferences, and / or meetings. Audio device 10 can be used by one or more users in the vicinity of where audio device 10 is located (also referred to as the near end). In this example, the receiver end can be considered as the near end, and the transmitter end can be considered as the far end.

[0084] The audio device 10 includes a plurality of microphones, including a first microphone 10E1 configured to provide a first audio input signal 50 and a second microphone 10E2 configured to provide a second audio input signal 52. The first audio input signal 50 and the second audio input signal 52 may be based on input signals (such as speech and / or sound) from the near end when obtained from the plurality of microphones (such as the first microphone 10E1 and / or the second microphone 10E2 of the audio device 10).

[0085] In one or more examples or embodiments, the audio device 10 may include a first calibration module 4 for calibrating the microphone gain of a first audio input signal 50, a second audio input signal 52, a third audio input signal 54, and / or a fourth audio input signal 56 before being input to the signal processor module 12. The output of the first calibration module 4 may be considered as gain-calibrated audio input signals, such as gain-calibrated first audio input signals 50, second audio input signals 52, third audio input signals 54, and / or fourth audio input signals 56.

[0086] In one or more examples or embodiments, the audio device 10 may include a second calibration module 6 for calibrating the microphone gain of the first audio input signal 50, the second audio input signal 52, the third audio input signal 54, and / or the fourth audio input signal 56 before being input to the sidetone module 14. The output of the second calibration module 6 may be considered as gain-calibrated audio input signals, such as gain-calibrated first audio input signal 50, second audio input signal 52, third audio input signal 54, and / or fourth audio input signal 56 (not shown).

[0087] Audio device 10 includes an output transducer 10D configured to output a near-end audio output signal 82. In other words, audio device 10 may include one or more output transducers (such as speakers) configured to output a near-end audio output signal 82 at the near end (such as outputting a near-end audio output signal 82 to a user of audio device 10). In one or more exemplary audio devices, processor circuitry 10C is configured to output the near-end audio output signal 82 (such as near-end audio output) via an interface (such as via output transducer 10D). In other words, audio device 10 may be configured to output the near-end audio output signal 82 via a wired and / or wireless interface, via one or more receivers (such as output transducers) at the near end on audio device 10 itself. In one or more exemplary audio devices, processor circuitry 10C is configured to output a far-end audio output signal 80 (such as far-end audio output) via an interface (such as a second wireless transceiver 10G). In other words, the audio device 10 can be configured to output the remote audio output signal 80 to a remote location via a wired and / or wireless interface.

[0088] The audio device 10 includes a processor circuit 10C, which includes a signal processor module 12 configured to provide a far-end audio output signal 80 and a side-tone module 14 configured to provide a side-tone audio output signal 60.

[0089] Audio device 10 (such as signal processor module 12) can be configured to process audio input signals (such as the first audio input signal 50, the second audio input signal 52, the third audio input signal 54, the fourth audio input signal 56 and / or the far-end audio input signal 76 disclosed herein) to provide audio output signals, such as the near-end audio output signal 82 and / or the far-end audio output signal 80. Audio device 10 (such as signal processor module 12) can be configured to operate according to one or more signal processing algorithms. For example, audio device 10 (such as signal processor module 12) can be configured to operate according to a transmitter algorithm (such as the Tx algorithm). In other words, signal processor module 12 can be configured to operate according to a DSP algorithm. The transmitter algorithm can be considered or represented as a signal processor module algorithm.

[0090] Signal processor module 12 is configured to process the first audio input signal 50 and the second audio input signal 52 to provide a plurality of filter parameters, such as filtering parameters. For example, signal processor module 12 may be configured to process the first audio input signal 50 and the second audio input signal 52 to provide a set of filter parameters used by sidetone module 14. Filter coefficients can be thought of as numerical values ​​used in digital filters to adjust the characteristics of the filter's response to an input signal. Filters can be used to modify the frequency content of a signal (such as audio) by selectively amplifying or attenuating certain frequency components. In one or more examples or embodiments, the plurality of filter parameters includes a plurality of finite impulse response (FIR) filter coefficients. In other words, the plurality of filter parameters can be used in an FIR filter (such as filter 14D) configured to filter audio input signals (such as the first audio input signal 50 and / or the second audio input signal 52).

[0091] Sidetone module 14 can be viewed as a sidetone processor module configured to process one or more audio input signals, such as audio input signals from one or more microphones at the near end. In one or more exemplary audio devices, sidetone module 14 can be viewed as or includes a signal processor responsible for managing the processing, generation, and control of sidetone feedback during voice communication activities on the audio device. Sidetone can be viewed as the user of the audio device hearing their own voice substantially in real time through audio device 10 during voice communication. Sidetone can be used to provide a natural and familiar auditory feedback loop to the speaker or user. Sidetone module 14 can be configured to adjust one or more of the amplitude, frequency response, and delay of the audio input signal, such as the user's own voice, to provide a natural sidetone effect that simulates the experience of speaking without an audio device, such as without headphones. Compared to signal processor module 12, sidetone module 14 can be viewed as operating in a low latency audio frame. In other words, sidetone module 14 can be configured to operate in a lower latency audio frame than signal processor module 12.

[0092] In one or more examples or implementations, the sidetone module 14 disclosed herein forms part of one or more processors, such as a digital signal processor (DSP). The sidetone module 14 may operate on one or more processors of the audio device 10.

[0093] Audio device 10 (such as sidetone module 14) can be configured to process audio input signals (such as the first audio input signal 50, the second audio input signal 52, the third audio input signal 54, and / or the fourth audio input signal 56 disclosed herein) using one or more processors to provide a sidetone audio output signal 60. Audio device 10 (such as sidetone module 14) can be configured to operate according to one or more signal processing algorithms. For example, audio device 10 (such as sidetone module 14) can be configured to operate according to a sidetone algorithm, such as being responsible for managing the processing, generation, and control of sidetone feedback during voice communication activities on audio device 10. Sidetone module 14 can process audio input signals in a sidetone processing path, while signal processor module 12 can process audio input signals in a transmitter processing path, such as a digital signal processor processing path.

[0094] The sidetone module 14 is configured to acquire indications and / or first data 70 based on a plurality of filter parameters. In other words, the sidetone module 14 may be configured to receive, retrieve, and / or determine the first data 70 indicating or based on a plurality of filter parameters. In one or more examples or embodiments, the first data 70 includes filter parameters, and / or the first data 70 is derived based on filter parameters. For example, the first data 70 may include or be based on a plurality of FIR coefficients from the signal processor module 12.

[0095] Sidetone module 14 is configured to process a first audio input signal 50 and a second audio input signal 52 using one or more filters based on first data 70 to provide a sidetone audio output signal 60. In one or more examples or embodiments, the one or more filters include multiple filters from signal processor module 12. For example, the one or more filters may include multiple FIR filters from signal processor module 12. Processing the first audio input signal 50 and the second audio input signal 52 may include applying one or more filters, such as filter parameters, during a filtering process at sidetone module 14 (e.g., at filter module 14D). For example, sidetone module 14 may include a filter module 14D, such as an FIR filter, for filtering the audio input signals. Processing the first audio input signal 50 and the second audio input signal 52 may include applying FIR filter coefficients to the first audio input signal 50 and the second audio input signal 52 during a filtering process at sidetone module 14. Sidetone module 14 (such as filter module 14D) may be configured to process the audio input signals according to one or more filters to provide a filtered audio output signal 66.

[0096] In one or more exemplary audio devices, the sidetone module 14 is configured to determine one or more filters based on first data 70. In other words, the sidetone module 14 may be configured to determine one or more filters based on a plurality of filter parameters. In one or more examples or embodiments, the one or more filters include filter parameters, and / or the one or more filters are derived based on filter parameters. For example, the one or more filters may include or be based on a plurality of FIR coefficients from the signal processor module 12.

[0097] By obtaining first data 70 indicating multiple filter parameters, the sidetone module 14 has the same or similar noise reduction capabilities as the signal processor module 12 (such as the same or similar noise reduction capabilities as the transmission algorithm operating on the signal processor module 12). This can be achieved by having the signal processor module 12 process the first audio input signal 50 and the second audio input signal 52 to provide multiple filter parameters, which can then be used by the sidetone module 14 for sidetone processing, for example, instead of having the sidetone module 14 process the first audio input signal 50 and the second audio input signal 52 to provide filter parameters. For example, this can reduce the latency of sidetone processing because the filter parameters are provided from the signal processor module 12. For example, the noise reduction capability can remove background noise from the sidetone signal (such as the sidetone audio output signal), leaving only the user's voice. This provides a clearer and more comfortable user experience, especially in noisy environments. Furthermore, the audio devices and methods of this disclosure allow the sidetone module 14 to have similar performance to the signal processor module 12, which helps the user gain a true impression of the call quality and the quality of the user's voice transmitted to the remote end during a call. The audio device of this disclosure allows the signal processor module 12 (such as a transmission algorithm) to run only once, which reduces power consumption. This is achieved by having the signal processor module 12 process the first audio input signal 50 and the second audio input signal 52 to provide multiple filter parameters and simultaneously provide the far-end audio output signal 80, for example, instead of having both the signal processor module 12 and the sidetone module 14 process the same or similar complex algorithms. It will be understood that the audio device and method of this disclosure improve sidetone processing, for example, by applying filtering (such as finite impulse response FIR filtering) to microphone signals (such as 50, 52, 54, 56). Specifically, the filter or filter parameters (such as an FIR filter) may not be computed by the sidetone module 14 (e.g., cannot be computed by a sidetone algorithm operating on the sidetone module), but rather by an algorithm responsible for creating the far-end audio output signal 80 (such as a Tx signal), which is the signal sent to the far end during a call. The filter parameters (such as FIR coefficients) can then be passed to the sidetone path, allowing for a more efficient process. Understandably, this eliminates the need for the signal processor module 12 algorithm to run twice and reduces waiting time, which is extremely important for one's natural speech experience. Therefore, the techniques disclosed herein can use different algorithms and entities of the audio device 10 to calculate filter parameters (such as FIR coefficients) and then apply the filter parameters to the sidetone path.

[0098] The near-end audio output signal 82 is based on the side-tone audio output signal 60 and the far-end input signal 76. In other words, the audio device 10 can be configured to determine the near-end audio output signal 82 based on the side-tone audio output signal 60 and / or the far-end audio input signal 76 received from the far end. Therefore, the audio device 10 can provide auditory feedback of the user's own voice during a voice call or communication session, which in turn allows the user of the audio device to hear their own voice as they speak. In one or more examples or embodiments, the audio device 10 includes a mixer 90 configured to mix the side-tone audio output signal 60 and the far-end audio input signal 76 to provide the near-end audio output signal 82. In one or more audio devices, the audio device 10 (such as the mixer 90) is configured to combine the side-tone audio output signal 60 and the far-end audio output signal 76 to provide a mixer output signal 92. The mixer 90 can, for example, be configured to add the side-tone audio output signal 60 to the far-end audio input signal 76 to provide the mixer audio output signal 92. In one or more examples or implementations, the mixer audio output signal 92 can be considered as the near-end audio output signal 82. For example, the mixer 90 can be configured to generate a mixer output signal 92 that includes a portion of the side-tone audio output signal 60 and a portion of the far-end audio input signal 76. The mixer 90 can be configured to control the gain of the side-tone audio output signal 60 and the far-end audio input signal 76.

[0099] In one or more exemplary audio devices, obtaining first data 70 includes performing windowing on the first data 70 to reduce the sample size of the first data 70. Audio device 10 (such as sidetone module 14) may be configured to perform windowing on a plurality of filter parameters obtained from signal processor module 12. Audio device 10 (such as sidetone module 14) may include a windowing module 14B configured to perform windowing on the first data 70. The output of windowing module 14B can be considered as windowed output 61. Windowing may be performed before processing the first audio input signal 50 and the second audio input signal 52 using one or more filters. In other words, sidetone module 14 may be configured to multiply the first data 70 (such as a plurality of filter parameters) by a window function before applying one or more filters. It is understood that for an FIR filter with finite-length filter coefficients, a window function can be applied to the FIR filter coefficients to modify the characteristics of the FIR filter coefficients. For example, FIR filter coefficients can be derived from an ideal frequency response. Therefore, directly using FIR filter coefficients may lead to problems such as spectral leakage and ripple effects in the frequency domain. Using windowing allows filter coefficients to gradually decrease towards zero at the edges, for example, reducing the abruptness of the transition from the passband to the stopband. Windowing the first data 70 can, for example, involve applying an attenuation window or a Hanning window to the first data 70, such as to multiple filter parameters.

[0100] In one or more exemplary audio devices, a sidetone module 14 is configured to perform smoothing of first data 70. An audio device 10 (such as sidetone module 14) may be configured to perform smoothing of multiple filter parameters obtained from a signal processor module 12. An audio device 10 (such as sidetone module 14) may include a smoothing module 14C configured to perform smoothing of the first data 70. The output of the smoothing module 14C may be considered a smoothed output 62. Smoothing may be performed before processing the first audio input signal 50 and the second audio input signal 52 using one or more filters, but after windowing the first data 70. The smoothing module 14C may be configured to perform smoothing of the windowed output 61 to provide a smoothed output 62. The sidetone module 14 may perform smoothing on the windowed output 61. In other words, the sidetone module 14 may be configured to reduce abrupt transitions and / or irregularities and / or smooth abrupt transitions and / or irregularities in multiple filter parameters (such as filter frequency responses). Understandably, for an FIR filter with finite-length filter coefficients, smoothing can be applied to the FIR filter coefficients to reduce or smooth abrupt transitions, ripples, or irregularities in the filter's actual frequency response. For example, smoothing techniques may include modifying the filter coefficients to achieve a smoother frequency response curve with reduced ripples or abrupt transitions. Smoothing of the first data may include performing interpolation in the time domain, for example, by using new filter parameters that are a smaller percentage than previous filter parameters (such as those from previous iterations) to avoid sudden changes. For example, the sidetone module 14 may be configured to use 10% new filter parameters (such as those from a new iteration where the filter parameters were determined) and 90% of the previous filter parameters.

[0101] In one or more exemplary audio devices, the input buffer size of the sidetone module 14 is less than or equal to the input buffer size of the signal processor module 12. In other words, the audio device 10 (such as the sidetone module 14) may be configured with an input buffer size less than or equal to the input buffer size of the signal processor module 12. Reducing the input buffer size of the sidetone module 14 allows for a reduction in latency in the sidetone path, which is advantageous because the sidetone path may be more sensitive to latency than the signal processor module path. In one or more examples or embodiments, the input buffer sizes of the first audio input signal 50 and the second audio input signal 52 of the sidetone module 14 are less than the input buffer sizes of the first audio input signal 50 and the second audio input signal 52 of the signal processor module 12.

[0102] In one or more exemplary audio devices, the output buffer size of the sidetone module 14 is smaller than or equal to the output buffer size of the signal processor module 12. In other words, the audio device 10 (such as the sidetone module 14) may be configured with an output buffer size smaller than or equal to the output buffer size of the signal processor module 12. Reducing the output buffer size of the sidetone module 14 allows for a reduction in latency in the sidetone path, which is advantageous because the sidetone path may be more sensitive to latency than the path of the signal processor module 12. In one or more examples or embodiments, the output buffer sizes of the first audio input signal 50 and the second audio input signal 52 of the sidetone module 14 are smaller than the output buffer sizes of the first audio input signal 50 and the second audio input signal 52 of the signal processor module 12.

[0103] For example, reducing the input buffer size and / or output buffer size of the sidetone module 14 (such as compared to the signal processor module 12) can help reduce the impact of algorithmic latency (such as determined algorithmic latency attributable to multiple filter parameters) at the audio device 10.

[0104] In one or more exemplary audio devices, the sidetrack module 14 includes a downsampler 14A and / or an upsampler 14E configured to reduce the amount of computation at the sidetrack module 14. For example, the downsampler 14A and / or the upsampler 14E may be configured to reduce the amount of computation at the sidetrack module 14 (such as reducing millions of operations per second) to make the computation of the sidetrack module 14 efficient. As discussed above, it is advantageous to have the smallest possible input buffer size. However, there may be a trade-off between buffer size and the number of operations. The smaller the buffer size, the higher the number of operations. However, for the sidetrack module 14, lower latency and therefore lower buffer size may take precedence over computational efficiency. The output of the downsampler 14A can be considered as a downsampled audio input signal 64. The output of the upsampler 14E can be considered as an upsampled audio output signal 68.

[0105] In one or more exemplary audio devices, audio device 10 includes an active noise cancellation (ANC) module 16 configured to acquire and process a sidetone audio output signal 60 based on a first audio input signal 50 and / or a second audio input signal 52 to provide an ANC audio output signal 84. ANC module 16 can be considered as a module configured to process audio input signals to perform ANC on them. For example, ANC module 16 may perform ANC on the sidetone audio output signal and / or the far-end audio input signal 76 before they are output at audio device 10 (such as at output transceiver 10D). Sidetone audio output signal 60 and / or mixer output signal 92 can be considered as audio input signals to the ANC module. In one or more examples or embodiments, ANC module 16 may be configured to acquire and process the sidetone audio output signal 60 and the far-end audio input signal 76 to provide an ANC audio output signal 84. In one or more examples or embodiments, ANC audio output signal 84 can be considered as near-end audio output signal 82. In one or more examples or implementations, the ANC module 16 can obtain a mixed signal 92 of the side tone audio output signal 60 and the far-end audio input signal 76 from the mixer 90.

[0106] In one or more exemplary audio devices, audio device 10 includes a third microphone 10E3 configured to provide a third audio input signal 54 and a fourth microphone 10E4 configured to provide a fourth audio input signal 56. In one or more exemplary audio devices, the third microphone 10E3 is a feedforward microphone, and the fourth microphone 10E4 is a feedback microphone. In one or more exemplary audio devices, ANC module 16 is configured to process the sidetone audio output signal 60 based on the third audio input signal 54 and the fourth audio input signal 56 to provide an ANC audio output signal 84. In other words, ANC module 16 may be able to perform feedforward ANC and / or feedback ANC based on the third audio input signal 54 and / or the fourth audio input signal 56. For example, ANC module 16 may be configured to perform hybrid ANC based on a first audio input signal 50, a second audio input signal 52, a third audio input signal 54, and / or the fourth audio input signal 56. In one or more examples or embodiments, ANC module 16 includes a hybrid ANC module 16C for performing hybrid ANC. In one or more examples or embodiments, the ANC module 16 is configured to process the sidetone audio output signal 60 based on a first audio input signal 50, a second audio input signal 52, a third audio input signal 54, and / or a fourth audio input signal 56 to provide an ANC audio output signal 84. In one or more exemplary embodiments, the ANC module 16 includes a filter module 16B. It is understood that the ANC module 16 (such as a hybrid ANC module 16C) may cancel the sidetone audio output signal 60. To avoid this, the filter module 16B may apply a filter to the sidetone audio output signal 60 to provide a filter module output 83. The filter module 16B may be configured to provide a filter tuned to match the receiver's response to the in-ear microphone. Thus, the filter may simulate a sidetone signal (such as the sidetone audio output signal) at the in-ear microphone and subtract that sidetone signal from the feedback loop to avoid canceling the sidetone audio output signal.

[0107] In one or more examples or implementations, audio device 10 (such as ANC module 16) includes an ANC regulator 16A, such as an ANC control mechanism that manages or regulates the operation of the ANC process. For example, ANC regulator 16A may be configured to determine the gain to be applied to the sidetone audio output signal 60, such as the gain that will be applied by the sidetone module 14 when processing the audio input signal before providing the sidetone audio output signal 60. ANC regulator 16A may be configured to determine the gain based on the audio input signal (such as based on a fourth audio input signal 56) (e.g., based on a feedback microphone audio input signal).

[0108] In one or more exemplary audio devices, audio device 10 is configured to determine a near-end audio output signal 82 based on an ANC audio output signal 84. For example, the audio device may be configured to apply gain to the ANC audio output signal 84 to provide a near-end audio output signal. In one or more exemplary embodiments, filter module 16B may control the gain 85 that will be applied to the ANC audio output signal 84.

[0109] In one or more exemplary audio devices, a sidetone module 14 is configured to process a third audio input signal 54 and / or a fourth audio input signal 56 to provide a sidetone audio output signal 60. Processing the third audio input signal 54 and the fourth audio input signal 56 may include applying one or more filters, such as filter parameters, during a filtering process at the sidetone module 14. For example, the sidetone module 14 may include a filtering module 14D, such as an FIR filter, for filtering the audio input signals. Processing the third audio input signal 54 and the fourth audio input signal 56 may include applying FIR filter coefficients to the third audio input signal 54 and the fourth audio input signal 56 during a filtering process at the sidetone module 14. The sidetone module 14 (such as the filtering module 14D) may be configured to process audio input signals (such as a first audio input signal 50, a second audio input signal 52, a third audio input signal 54, and / or a fourth audio input signal 56) according to one or more filters to provide a filtered audio output signal 66. In one or more examples or embodiments, the filtered audio output signal 66 may be considered as an input to an upsampler 14E.

[0110] In one or more examples or implementations, audio device 10 (such as sidetone module 14) includes an equalizer 14F configured to equalize the upsampler audio output signal 68 to provide an equalized audio output signal 69.

[0111] In one or more examples or implementations, the audio device 10 (such as the sidetone module 14) includes a gain controller 14H configured to control the gain of the sidetone audio output signal 60.

[0112] In one or more exemplary audio devices, signal processor module 12 is configured to process a third audio input signal 54 and / or a fourth audio input signal 56 to provide a plurality of filter parameters. For example, signal processor module 12 may be configured to process the third audio input signal 54 and the fourth audio input signal 56 to provide a set of filter parameters used by sidetone module 14. In other words, the plurality of filter parameters may be used in an FIR filter configured to filter audio input signals such as a first audio input signal 50, a second audio input signal 52, a third audio input signal 54, and / or a fourth audio input signal 56.

[0113] In one or more exemplary audio devices, the sidetone module 14 is initialized using one or more predetermined filters to process a first audio input signal 50 and a second audio input signal 52. In other words, the signal processor module 12 can be configured to obtain or determine one or more predetermined filter parameters to initialize the sidetone module 14 to process the first audio input signal 50 and the second audio input signal 52. First data 70 may include one or more predetermined filter parameters. The sidetone module 14 can obtain or determine one or more filters, such as one or more predetermined filters, based on one or more predetermined filter parameters. The predetermined filter parameters and predetermined filters can be used, for example, to initialize the sidetone module 14 at the start of a voice conversation. It is understood that the predetermined filters can be adaptive or constant.

[0114] In one or more exemplary audio devices, signal processor module 12 includes a noise reduction module 12A and a first signal processor 12B configured to operate according to a first processing algorithm. Noise reduction module 12A may, for example, include a denoiser and / or an echo controller. Noise reduction module 12A can be considered as a module configured to perform noise reduction (such as background noise reduction) on an audio input signal. Noise reduction module 12A may be configured to perform noise reduction, speech pickup, or a combination of both.

[0115] In one or more exemplary audio devices, a noise reduction module 12A is configured to process a first audio input signal 50 and a second audio input signal 52 to provide a noise-reduced output, such as a denoised audio output signal 72. The noise reduction module 12A may be configured to process the first audio input signal 50, the second audio input signal 52, the third audio input signal 54, and / or the fourth audio input signal 56. In one or more exemplary audio devices, a first signal processor 12B is configured to process the noise-reduced output (such as the noise-reduced output signal 72) according to a first processing algorithm to provide a remote audio output signal 80. The noise-reduced output (such as the noise-reduced output signal 72) may be considered as an input to the first signal processor. The noise reduction module 12A may provide a noise-reduced output for each processed audio input signal. For example, the noise reduction module 12A may provide a first noise-reduced output for a processed first audio input signal, a second noise-reduced output for a processed second audio input signal, a third noise-reduced output for a processed third audio input signal, and / or a fourth noise-reduced output for a processed fourth audio input signal. The first processing algorithm can be configured to process an audio input signal (such as a noise-reduced output) to provide a remote audio output signal 80 and / or one or more filter parameters, such as one or more FIR filter coefficients included in the first data 70. The first processing algorithm can be considered as a transmission algorithm as disclosed herein, such as a DSP algorithm. The first processing algorithm can be configured to provide both the remote audio output signal 80 as disclosed herein and the filter parameters (such as the first data 70).

[0116] In one or more exemplary audio devices, a first signal processor 12B is configured to determine filter parameters and send the filter parameters to a sidetone module 14. In other words, the first signal processor 12B may be configured to determine the filter parameters using a first processing algorithm. For example, the first signal processor may be configured to determine FIR filter coefficients and send the FIR filter coefficients to the sidetone module 14. In other words, the first signal processor 12B may be configured to send first data 70, including or indicating the filter parameters, to the sidetone module 14.

[0117] In one or more exemplary audio devices, signal processor module 12 is configured to perform wind noise detection on a first audio input signal 50 and / or a second audio input signal 52 and provide a wind noise detection indication 74 to sidetone module 14. In one or more examples or embodiments, first signal processor 12B is configured to perform wind noise detection on the first audio input signal 50, the second audio input signal 52, the third audio input signal 54, and / or the fourth audio input signal 56, for example, using a first processing algorithm. Signal processor module 12 may indicate a wind noise detection flag to sidetone module 14. It is understood that the wind noise detection flag may be triggered based on meeting one or more criteria, such as wind speed, direction, or other environmental factors. Sidetone module 14 may be configured to perform a hysteresis on wind noise detection indication 74. Sidetone module 14 may include a hysteresis module 14G configured to perform a hysteresis on wind noise detection indication 74. For example, implementing a hysteresis on wind noise detection indications (such as wind noise detection flags) can involve introducing a delayed and historically relevant response to wind noise conditions, such as improving system stability, reliability, and the smoothness of transitions between wind noise detection states.

[0118] In one or more exemplary audio devices, audio device 10 is configured to perform one or more of in-ear detection, sidetone user gain, and mute button detection. In other words, audio device 10 may include an in-ear detection module 18 for performing in-ear detection, a sidetone user gain module 20 for performing sidetone user gain detection, and / or a mute button detection module 22 for performing mute button detection. Sidetone module 14 may be configured to obtain information about one or more of in-ear detection, sidetone user gain detection, and mute button detection. Sidetone module 14 may be configured to process a first audio input signal 50, a second audio input signal 52, a third audio input signal 54, and / or a fourth audio input signal 56 based on in-ear detection information, sidetone user gain information, and / or mute button information, to control the gain of the sidetone audio output signal 60, for example, using gain controller 14H.

[0119] For example, audio device 10 can be configured to detect whether a user is wearing the audio device, such as whether the audio device 10 is inserted into and / or on the user's ear. For example, audio device 10 can be configured to detect changes or adjustments in sidetone gain, such as when the user of audio device 10 manually changes the sidetone gain. For example, audio device can be configured to detect when a mute button is present and when the mute function is activated by the user of the audio device.

[0120] Audio device 10 can be configured to perform Figures 2A to 2B Any method disclosed herein.

[0121] The operation of the audio device 10 can be embodied in the form of an executable logic routine (e.g., a line of code, a software program, etc.), which is stored on a non-transitory computer-readable medium (e.g., memory) and executed by the processor circuit 10C.

[0122] Furthermore, the operation of the audio device 10 can be considered as a method configured to be performed by the audio device 10. Moreover, although the described functions and operations can be implemented in software, such functions can also be implemented via dedicated hardware or firmware, or some combination of hardware, firmware and / or software.

[0123] The memory of an audio device can be one or more of a buffer, flash memory, hard disk drive, removable media, volatile memory, non-volatile memory, random access memory (RAM), or other suitable devices. In a typical arrangement, the memory may include non-volatile memory for long-term data storage and volatile memory used as system memory for the processor circuit 10C. The memory can exchange data with the processor circuit 10C on a data bus. Control lines and an address bus may also exist between the memory and the processor circuit 10C. Figure 1 (Not shown in the image). Memory is considered a non-transitory computer-readable medium.

[0124] The memory can be configured to store information, such as filter parameters, filter coefficients, filter gains, processing filters, and processing algorithms, in a portion of the memory.

[0125] Figures 2A to 2B A flowchart of an exemplary method (such as method 100) is shown.

[0126] A method for sidetone processing is disclosed. This method is performed by an audio device. The audio device includes a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal. The audio device includes an output transducer configured to output a near-end audio output signal. The audio device includes processor circuitry, including a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal.

[0127] The method includes obtaining a first audio input signal and a second audio input signal in step S102. The method includes processing the first audio input signal and the second audio input signal using a signal processor module in step S106 to provide multiple filter parameters. The method includes obtaining first data indicating the multiple filter parameters using a sidetone module in step S110. The method includes processing the first audio input signal and the second audio input signal using one or more filters based on the first data in step S112 to provide a sidetone audio output signal. The method includes outputting a near-end audio output signal based on the sidetone audio output signal and the far-end input signal in step S122.

[0128] In one or more exemplary methods, obtaining the first data in S110 includes performing windowing of the first data in S110A to reduce the sample size of the first data.

[0129] In one or more exemplary methods, processing S112 includes performing S112A smoothing of the first data.

[0130] In one or more exemplary methods, processing S112 includes using a downsampler and / or an upsampler to reduce the computational load at the sidetone module of S112B.

[0131] In one or more exemplary methods, the method includes obtaining and processing an S120 side tone audio output signal based on a first audio input signal and a second audio input signal to provide an ANC audio output signal.

[0132] In one or more exemplary methods, the audio device includes a third microphone configured to provide a third audio input signal and a fourth microphone configured to provide a fourth audio input signal. In one or more exemplary methods, the third microphone is a feedforward microphone, and the fourth microphone is a feedback microphone. In one or more exemplary methods, processing S120 includes processing a side-tone audio output signal based on the third audio input signal and the fourth audio input signal (S120A) to provide an ANC audio output signal.

[0133] In one or more exemplary methods, outputting the S122 near-end audio output signal includes determining the S122A near-end audio output signal based on the ANC audio output signal.

[0134] In one or more exemplary methods, processing S112 includes processing S112C on a third audio input signal and / or a fourth audio input signal to provide a sidetone audio output signal.

[0135] In one or more exemplary methods, processing S106 includes processing the third audio input signal and / or the fourth audio input signal S106A to provide multiple filter parameters.

[0136] In one or more exemplary methods, the method includes using a sidetone module to determine one or more filters S111 based on first data.

[0137] In one or more exemplary methods, the method includes initializing the sidetone module with one or more predetermined filters in step S107 and processing the first audio input signal and the second audio input signal in pairs.

[0138] In one or more exemplary methods, the signal processor module includes a noise reduction module and a first signal processor configured to operate according to a first processing algorithm. In one or more exemplary methods, the method includes processing a first audio input signal and a second audio input signal using the noise reduction module for S104 to provide a noise-reduced output. In one or more exemplary methods, processing the noise-reduced output using the first signal processor according to the first processing algorithm for S108 to provide a far-end audio output signal.

[0139] In one or more exemplary methods, processing S108 includes using a first signal processor to determine filter parameters for S108A and using the first signal processor to send the filter parameters to the sidetone module.

[0140] In one or more exemplary methods, the method includes performing S114 wind noise detection on a first audio input signal and / or a second audio input signal and providing S116 wind noise detection indication to a sidetone module.

[0141] In one or more exemplary methods, the method includes performing one or more of S118 in-ear detection, sidetone user gain, and mute button detection.

[0142] Examples of audio devices, systems, and methods based on this disclosure are illustrated in the following items:

[0143] Project 1. An audio device, comprising:

[0144] - Multiple microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal;

[0145] - Output transducer, configured to output a near-end audio output signal; and

[0146] - Processor circuitry, including a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal.

[0147] The signal processor module is configured to process the first audio input signal and the second audio input signal to provide a plurality of filter parameters. The sidetone module is configured to obtain first data indicating the plurality of filter parameters and use one or more filters based on the first data to process the first audio input signal and the second audio input signal to provide a sidetone audio output signal. The near-end audio output signal is based on the sidetone audio output signal and the far-end audio input signal.

[0148] Project 2. The audio device according to Project 1, wherein obtaining the first data includes performing windowing of the first data to reduce the sample size of the first data.

[0149] Item 3. The audio device according to any one of the preceding items, wherein the sidetone module is configured to perform smoothing of the first data.

[0150] Item 4. The audio device according to any one of the preceding items, wherein the input buffer size of the sidetone module is less than or equal to the input buffer size of the signal processor module.

[0151] Item 5. The audio device according to any one of the preceding items, wherein the output buffer size of the sidetone module is less than or equal to the output buffer size of the signal processor module.

[0152] Item 6. The audio device according to any one of the preceding items, wherein the sidetone module includes a downsampler and / or an upsampler configured to reduce the computational load at the sidetone module.

[0153] Item 7. The audio device according to any one of the preceding items, wherein the audio device includes an active noise cancellation (ANC) module configured to obtain and process a sidetone audio output signal based on a first audio input signal and / or a second audio input signal to provide an ANC audio output signal.

[0154] Item 8. The audio device according to Item 7, wherein the audio device includes a third microphone configured to provide a third audio input signal and a fourth microphone configured to provide a fourth audio input signal, wherein the third microphone is a feedforward microphone and the fourth microphone is a feedback microphone, wherein the ANC module is configured to process a side-tone audio output signal based on the third audio input signal and the fourth audio input signal to provide an ANC audio output signal.

[0155] Item 9. The audio device according to any one of Items 7 to 8, wherein the audio device is configured to determine a near-end audio output signal based on an ANC audio output signal.

[0156] Item 10. The audio device according to any one of Items 7 to 9, wherein the sidetone module is configured to process a third audio input signal and / or a fourth audio input signal to provide a sidetone audio output signal.

[0157] Item 11. The audio device according to any one of Items 7 to 9, wherein the signal processor module is configured to process a third audio input signal and / or a fourth audio input signal to provide a plurality of filter parameters.

[0158] Item 12. The audio device according to any one of the preceding items, wherein the sidetone module is configured to determine one or more filters based on first data.

[0159] Item 13. The audio device according to any one of the preceding items, wherein the sidetone module is initialized using one or more predetermined filters to process the first audio input signal and the second audio input signal.

[0160] Item 14. The audio device according to any one of the preceding items, wherein the signal processor module includes a noise reduction module and a first signal processor configured to operate according to a first processing algorithm, wherein the noise reduction module is configured to process a first audio input signal and a second audio input signal to provide a noise-reduced output, and wherein the first signal processor is configured to process the noise-reduced output according to the first processing algorithm to provide a far-end audio output signal.

[0161] Item 15. The audio device according to Item 14, wherein a first signal processor is configured to determine filter parameters and send the filter parameters to a sidetone module.

[0162] Item 16. The audio device according to any one of the preceding items, wherein the signal processor module is configured to perform wind noise detection on a first audio input signal and / or a second audio input signal and provide a wind noise detection indication to a sidetone module.

[0163] Item 17. The audio device according to any one of the preceding items, wherein the audio device is configured to perform one or more of in-ear detection, sidetone user gain, and mute button detection.

[0164] Item 18. A method for sidetone processing performed by an audio device, wherein the audio device includes: a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal; an output transducer configured to output a near-end audio output signal; and processor circuitry including a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal, wherein the method includes:

[0165] - Obtain (S102) the first audio input signal and the second audio input signal;

[0166] - The first audio input signal and the second audio input signal are processed using a signal processor module (S106) to provide multiple filter parameters;

[0167] - Use the sidetone module to obtain (S110) the first data indicating multiple filter parameters;

[0168] - The sidetone module uses one or more filters based on the first data to process the first audio input signal and the second audio input signal (S112) to provide a sidetone audio output signal; and

[0169] - Output a near-end audio output signal based on the side tone audio output signal and the far-end input signal (S122).

[0170] Item 19. The method according to Item 18, wherein obtaining (S110) the first data includes performing (S110A) windowing of the first data to reduce the sample size of the first data.

[0171] Item 20. The method according to any one of items 18 to 19, wherein the processing (S112) includes performing (S112A) smoothing of the first data.

[0172] Item 21. The method according to any one of Items 18 to 20, wherein the processing (S112) includes using a downsampler and / or an upsampler to reduce (S112B) the computational load at the sidetone module.

[0173] Item 22. The method according to any one of items 18 to 21, wherein the method comprises:

[0174] - Obtain and process (S120) the side tone audio output signal based on the first audio input signal and the second audio input signal to provide the ANC audio output signal.

[0175] Item 23. The method according to Item 22, wherein the audio device includes a third microphone configured to provide a third audio input signal and a fourth microphone configured to provide a fourth audio input signal, wherein the third microphone is a feedforward microphone and the fourth microphone is a feedback microphone, wherein the processing (S120) includes processing the side tone audio output signal based on the third audio input signal and the fourth audio input signal (S120A) to provide an ANC audio output signal.

[0176] Item 24. The method according to any one of Items 22 to 23, wherein the output (S122) near-end audio output signal includes determining (S122A) the near-end audio output signal based on the ANC audio output signal.

[0177] Item 25. The method according to any one of Items 23 to 24, wherein the processing (S112) includes processing (S112C) the third audio input signal and / or the fourth audio input signal to provide a sidetone audio output signal.

[0178] Item 26. The method according to any one of items 23 to 25, wherein the processing (S106) includes processing the third audio input signal and / or the fourth audio input signal (S106A) to provide a plurality of filter parameters.

[0179] Item 27. The method according to any one of items 18 to 26, wherein the method comprises:

[0180] - Use the sidetone module to determine (S111) one or more filters based on the first data.

[0181] Item 28. The method according to any one of items 18 to 27, wherein the method comprises:

[0182] - Initialize the sidetone module using one or more predetermined filters (S107) and process the first audio input signal and the second audio input signal in pairs.

[0183] Item 29. The method according to any one of items 18 to 28, wherein the signal processor module includes a noise reduction module and a first signal processor configured to operate according to a first processing algorithm, wherein the method includes:

[0184] - The first and second audio input signals are processed using a noise reduction module (S104) to provide a noise-reduced output; and

[0185] - The noise reduction output is processed by the first signal processor according to the first processing algorithm (S108) to provide a remote audio output signal.

[0186] Item 30. The method according to Item 29, wherein the processing (S108) includes determining (S108A) filter parameters using a first signal processor and sending the filter parameters to the sidetone module using the first signal processor.

[0187] Item 31. The method according to any one of items 18 to 30, wherein the method comprises:

[0188] - Perform (S114) wind noise detection on the first audio input signal and / or the second audio input signal and provide (S116) wind noise detection indication to the side sound module.

[0189] Item 32. The method according to any one of items 18 to 31, wherein the method comprises:

[0190] - Perform (S118) one or more of in-ear detection, side tone user gain and mute button detection.

[0191] The use of terms such as "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary" does not imply any specific order, but is included to identify individual elements. Furthermore, the use of terms such as "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary" does not indicate any order or importance, but is used to distinguish one element from another. Note that the terms "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary" are used here and elsewhere solely for labelling purposes and are not intended to indicate any specific spatial or temporal order. Moreover, the labeling of a first element does not imply the existence of a second element, and vice versa.

[0192] It is understood that the accompanying drawings include some circuits or operations shown in solid lines and some circuits, components, features, or operations shown in dashed lines. The circuits or operations included in solid lines are those included in the most broad examples. The circuits, components, features, or operations included in dashed lines are examples that can be included in the circuits, components, features, or operations of the solid-line examples, or are part of the circuits, components, features, or operations of the solid-line examples, or are additional circuits, components, features, or operations that may be employed besides those of the solid-line examples. It should be understood that these operations do not need to be performed in the order presented. Furthermore, it should be understood that not all operations need to be performed. The example operations can be performed in any order and in any combination. It should be understood that these operations do not need to be performed in the order presented. The circuits, components, features, or operations included in dashed lines can be considered optional.

[0193] Other operations not described herein may be incorporated into the exemplary operations. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the described operations.

[0194] Some features discussed above as individual implementations can also be implemented in combination as a single implementation. Conversely, features described as a single implementation can also be implemented individually or in any suitable sub-combination in multiple implementations. Furthermore, although features may be described above as functioning in certain combinations, in some cases, one or more features from the claimed combination can be removed from that combination, and that combination can be claimed as any sub-combination or a variation of any sub-combination.

[0195] It should be noted that the word "including" does not necessarily exclude the presence of other elements or steps besides those listed.

[0196] It should be noted that the words "a" or "an" preceding an element do not preclude the existence of multiple such elements.

[0197] It should be noted that the term "indicator" can be considered as "associated with," "related to," "describe," "represent," and / or "define." The terms "indicator," "associated with," "related to," "describe," "represent," and "define" are used interchangeably. The term "indicator" can be considered as indicating a relation. For example, weight data indicating weights may include one or more weight parameters.

[0198] It should be noted that the word "based on" can be considered as "as a function of" and / or "derived from". The terms "based on" and "as a function of" are used interchangeably. For example, parameters determined "based on" a dataset can be considered as parameters determined "as a function of the dataset". In other words, parameters can be the output of one or more functions, with the dataset as input.

[0199] Functions can represent the relationship between inputs and outputs, such as mathematical relationships, database relationships, hardware relationships, logical relationships, and / or other suitable relationships.

[0200] It should also be noted that no reference numerals in the drawings limit the scope of the claims, examples can be implemented at least in part by means of both hardware and software, and several “tools,” “units,” or “devices” can be represented by the same piece of hardware.

[0201] The various exemplary methods, devices, nodes, and systems described herein are described in the general context of method steps or processes. In one aspect, these method steps or processes can be implemented by a computer program product embodied in a computer-readable medium, including computer-executable instructions, such as program code, that are executed by a computer in a networked environment. Computer-readable media can include removable and non-removable storage devices, including but not limited to read-only memory (ROM), random access memory (RAM), optical disc (CD), digital versatile disc (DVD), etc. Generally, program circuitry can include routines, programs, objects, components, data structures, etc., that perform a specified task or implement a particular abstract data type. Computer-executable instructions, associated data structures, and program circuitry represent examples of program code for performing steps of the methods disclosed herein. A particular sequence of such executable instructions or associated data structures represents examples of corresponding actions for implementing the functionality described in such steps or processes.

[0202] Although features have been shown and described, it should be understood that they are not intended to limit the claimed disclosure, and it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the scope of the claimed invention. Therefore, this specification and drawings should be considered illustrative rather than restrictive. The claimed disclosure is intended to cover all alternatives, modifications, and equivalents.

Claims

1. An audio device, comprising: - Multiple microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal; - Output transducer, configured to output near-end audio output signal; as well as - Processor circuitry, including a signal processor module configured to provide a far-end audio output signal and a sidetone module configured to provide a sidetone audio output signal. The signal processor module is configured to process the first audio input signal and the second audio input signal to provide a plurality of filter parameters. The sidetone module is configured to obtain first data indicating the plurality of filter parameters and to process the first audio input signal and the second audio input signal using one or more filters based on the first data to provide the sidetone audio output signal. The near-end audio output signal is based on the sidetone audio output signal and the far-end audio input signal.

2. The audio device according to claim 1, wherein, Obtaining the first data includes: performing windowing on the first data to reduce the sample size of the first data.

3. The audio device according to any one of the preceding claims, wherein, The sidetone module is configured to perform smoothing of the first data.

4. The audio device according to any one of the preceding claims, wherein, The input buffer size of the sidetone module is less than or equal to the input buffer size of the signal processor module.

5. The audio device according to any one of the preceding claims, wherein, The output buffer size of the sidetone module is less than or equal to the output buffer size of the signal processor module.

6. The audio device according to any one of the preceding claims, wherein, The sidetone module includes a downsampler and / or an upsampler configured to reduce the computational load at the sidetone module.

7. The audio device according to any one of the preceding claims, wherein, The audio device includes an active noise cancellation (ANC) module, which is configured to obtain and process the sidetone audio output signal based on the first audio input signal and / or the second audio input signal to provide an ANC audio output signal.

8. The audio device according to claim 7, wherein, The audio device includes a third microphone configured to provide a third audio input signal and a fourth microphone configured to provide a fourth audio input signal, wherein the third microphone is a feedforward microphone and the fourth microphone is a feedback microphone, wherein the ANC module is configured to process the side-tone audio output signal based on the third audio input signal and the fourth audio input signal to provide the ANC audio output signal.

9. The audio device according to any one of claims 7 to 8, wherein, The sidetone module is configured to process a third audio input signal and / or a fourth audio input signal to provide the sidetone audio output signal.

10. The audio device according to any one of claims 7 to 8, wherein, The signal processor module is configured to process the third audio input signal and / or the fourth audio input signal to provide the plurality of filter parameters.

11. The audio device according to any one of the preceding claims, wherein, The sidetone module is configured to determine the one or more filters based on the first data.

12. The audio device according to any one of the preceding claims, wherein, The sidetone module is initialized using one or more predetermined filters to process the first audio input signal and the second audio input signal.

13. The audio device according to any one of the preceding claims, wherein, The signal processor module includes a noise reduction module and a first signal processor configured to operate according to a first processing algorithm, wherein the noise reduction module is configured to process the first audio input signal and the second audio input signal to provide a noise-reduced output, and wherein the first signal processor is configured to process the noise-reduced output according to the first processing algorithm to provide the far-end audio output signal.

14. The audio device according to claim 13, wherein, The first signal processor is configured to determine the filter parameters and send the filter parameters to the sidetone module.

15. A method for sidetone processing performed by an audio device, wherein, The audio device includes: a plurality of microphones, including a first microphone configured to provide a first audio input signal and a second microphone configured to provide a second audio input signal; an output transducer configured to output a near-end audio output signal; and a processor circuit, including a signal processor module configured to provide a far-end audio output signal and a side-tone module configured to provide a side-tone audio output signal, wherein the method includes: - Obtain (S102) the first audio input signal and the second audio input signal; - The signal processor module is used to process the first audio input signal and the second audio input signal (S106) to provide multiple filter parameters; - Use the sidetone module to obtain (S110) first data indicating the plurality of filter parameters; - The sidetone module uses one or more filters based on the first data to process the first audio input signal and the second audio input signal (S112) to provide the sidetone audio output signal; and - Output the near-end audio output signal based on the side-tone audio output signal and the far-end input signal (S122).

Images