Adaptive filter banks using scale-dependent nonlinearities for psychoacoustic frequency range extension

Abstract

The system provides psychoacoustic frequency range extension. The system generates rotated spectral orthogonal components by generating orthogonal components from an audio channel and applying a forward transform that rotates the spectrum of the orthogonal components from a standard basis to a rotated basis. In the rotated basis, the system generates weighted phase-coherent harmonic spectral orthogonal components by isolating components of the rotated spectral orthogonal components at a target frequency and applying a nonlinearity to the isolated components having a scale dependence according to a constraint. The circuit generates harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase-coherent harmonic spectral orthogonal components from the rotated basis to a standard basis. The circuit combines frequencies and harmonic spectral components of the audio channel outside the range of the target frequency to generate an output channel and provides the output channel to a speaker.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates generally to audio processing, and more specifically to creating frequency effects beyond the bandwidth of a physical driver. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 222,370, filed July 15, 2021, and U.S. Application No. 17 / 471,012, filed September 9, 2021, which applications are incorporated by reference in their entireties.

[0003] The bandwidth of loudspeakers, headphones, and other acoustic actuators is often limited to a subdomain of the bandwidth of the human auditory system. This is most often a problem in the low frequency region of the audible spectrum, roughly between 18 Hz and 250 Hz. It is desirable to modify the audio signal to produce effects at frequencies beyond the bandwidth of the physical driver.

[0004] Some embodiments include a system including a circuit (e.g., one or more processors) that provides psychoacoustic frequency range extension for a speaker. The circuit generates orthogonal components from the audio channels that define an orthogonal representation of the audio channels, and generates rotated spectral orthogonal components by applying a forward transform that rotates the spectrum of the orthogonal components from a standard basis to a rotated basis. In the rotated basis, the circuit generates weighted phase-coherent harmonic spectral orthogonal components by isolating components of the rotated spectral orthogonal components at a target frequency and applying a nonlinearity to the isolated components having a scale dependence according to a constraint. The circuit generates harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase-coherent harmonic spectral orthogonal components from the rotated basis to a standard basis. The circuit combines frequencies and harmonic spectral components of the audio channels outside the range of the target frequency to generate an output channel and provides the output channel to the speaker.

[0005] In some embodiments, the nonlinearity comprises a weighted mixture of constituent nonlinearities, and each of the constraints comprises a constraint on a gain modification applied to an input of each constituent nonlinearity.

[0006] In some embodiments, the nonlinearity includes a weighted sum of Chebyshev polynomials of the first type with amplitudes selectively factored according to constraints.

[0007] In some embodiments, the circuitry is further configured to generate a plurality of harmonic spectral components, each of the harmonic spectral components being generated using a different frequency band of the audio channel, and the circuitry is configured to generate an output channel by combining the plurality of harmonic spectral components.

[0008] In some embodiments, the circuitry is configured to generate a plurality of harmonic spectral components in series with each downstream harmonic spectral component generated using the residual of the upstream harmonic spectral component as an input.

[0009] In some embodiments, the circuitry is configured to generate multiple harmonic spectral components in parallel.

[0010] In some embodiments, the circuitry is configured to filter odd-order harmonic spectral components. Non It is further configured to apply linearity.

[0011] In some embodiments, the harmonic spectral components include frequencies that differ from the target frequency of the audio channel and produce the psychoacoustic impact of the target frequency when rendered by the loudspeaker.

[0012] In some embodiments, the forward transform rotates the spectrum of the orthogonal components such that the target frequency is mapped to 0 Hertz, and the inverse transform rotates the spectrum of the weighted phase coherent harmonic spectrum orthogonal components such that 0 Hertz is mapped to the target frequency.

[0013] In some embodiments, the target frequencies include frequencies between 18 Hertz and 250 Hertz.

[0014] In some embodiments, the circuitry determines the target frequency based on the reproducible range of the speaker, reduced power consumption of the speaker, or increased life span of the speaker.

[0015] In some embodiments, the speaker is a component of the mobile device.

[0016] In some embodiments, the circuitry is further configured to separate the components at the target amplitude using a gating function, hi some embodiments, the circuitry is further configured to apply a smoothing function to the separated components.

[0017] Some embodiments include a method that includes generating, by a circuit, orthogonal components from an audio channel that defines an orthogonal representation of the audio channel, generating rotated spectral orthogonal components by applying a forward transform that rotates the spectrum of the orthogonal components from a standard basis to a rotated basis, isolating components of the rotated spectral orthogonal components at a target frequency in the rotated basis, generating weighted phase-coherent harmonic spectral orthogonal components by applying a nonlinearity to the separated components having a scale dependence according to a constraint, generating harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase-coherent harmonic spectral orthogonal components from the rotated basis to a standard basis, combining frequencies and harmonic spectral components of the audio channel outside the range of the target frequency to generate an output channel, and providing the output channel to a speaker.

[0018] Some embodiments include a non-transitory computer-readable medium including stored instructions that, when executed by at least one processor, configure the at least one processor to: generate orthogonal components from the audio channels defining an orthogonal representation of the audio channels; generate rotated spectral orthogonal components by applying a forward transform that rotates a spectrum of the orthogonal components from a standard basis to a rotated basis; generate weighted phase-coherent harmonic spectral orthogonal components in the rotated basis by separating components of the rotated spectral orthogonal components at a target frequency and applying a nonlinearity to the separated components having a scale dependence according to a constraint; generate harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase-coherent harmonic spectral orthogonal components from the rotated basis to a standard basis; combine frequencies of the audio channels outside the range of the target frequency and the harmonic spectral components to generate an output channel; and provide the output channel to a speaker. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a block diagram of an audio system according to some embodiments. [Diagram 2] FIG. 2 is a block diagram of a harmonic processing module according to some embodiments. [Diagram 3] FIG. 2 is a block diagram of a forward transformation module, according to some embodiments. [Figure 4] FIG. 2 is a block diagram of a coefficient operator module according to some embodiments. [Diagram 5] FIG. 2 is a block diagram of an inverse transform module according to some embodiments. [Figure 6] FIG. 2 is a block diagram of a combiner module according to some embodiments. [Figure 7] FIG. 2 is a block diagram of a filter bank module according to some embodiments. [Figure 8]1 is a flowchart of a process for psychoacoustic frequency range extension, according to some embodiments. [Figure 9] FIG. 1 is a block diagram of a computer according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The drawings depict various embodiments for purposes of illustration only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be used without departing from the principles described herein.

[0021] The drawings and the following description relate to preferred embodiments for purposes of illustration only. It should be noted that from the following description, alternative embodiments of the structures and methods disclosed herein will be readily recognized as feasible alternatives that may be used without departing from the principles claimed.

[0022] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying drawings. It is noted that similar or similar reference numerals may be used anywhere in the drawings, and may indicate similar or similar functionality wherever used. The drawings represent embodiments of the disclosed system (or method) for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be used without departing from the principles described herein.

[0023] An embodiment relates to providing psychoacoustic frequency range extension. Psychoacoustic phenomena can be used to create virtual stimuli where real stimuli are not feasible because the human auditory system responds to cues in a nonlinear manner. The audio system may include a circuit providing an adaptive nonlinear filter bank using highly tunable nonlinearities with scale-dependence according to constraints. The nonlinearities are used to generate weighted phase-coherent harmonic spectra from one or more subbands of the audio channel. The nonlinearities may include weighted mixtures of constituent nonlinearities. The constraints may each include constraints on gain modifications applied to the input of each constituent nonlinearity. Independent constraints may be applied to each of the constituent nonlinearities in the sum defining nonlinearity, which allows selective spectral animation among a selected subset of the generated harmonics. This allows much more non-artifactual effects to be realized, thereby succeeding in generalizing across content. Furthermore, it reduces the perceptual salience of intermodulation artifacts, potentially allowing fewer filters with wider bandwidths to be used. In some embodiments, the nonlinearity includes a weighted sum of first type Chebyshev polynomials with amplitudes selectively factored according to constraints. The phase-coherent harmonic spectrum of one or more subbands produces subband effects when the frequencies of the subbands exceed the bandwidth of the physical driver.

[0024] In some embodiments, the adaptive nonlinear filter bank may include multiple harmonic processors. Each of the harmonic processors includes a nonlinear filter that analyzes a target subband in the audio signal and resynthesizes the data of the subband with a configurable spectral transformation. Each of the harmonic processors uses a different frequency band of the audio channel to generate harmonic spectral components, which are combined to generate an output channel. The harmonic spectral components may be generated in parallel or in series. In the serial case, the downstream harmonic spectral component uses the residual of the upstream harmonic spectral component as an input. The parallel case is conceptually simple, but sometimes results in a difficult tuning process, for example, when the parallel design did not constrain the power spectrum of the analyzed content. By utilizing a serial architecture where subsequent filters operate only on the residual of the input signal, the total spectral power is preserved at the input to the filter bank. The result is a filter bank architecture where the constituent filters are not subject to constructive interference.

[0025] Advantages of frequency range extension include allowing speakers that cannot render certain frequencies (e.g., low quality) to provide the psychoacoustic effects of those frequencies. Thus, low-cost speakers, such as those often found in mobile devices, can provide a high-quality listening experience. Psychoacoustic frequency range extension is achieved by processing the audio signal, such as by processing circuitry found in mobile devices, without requiring hardware modifications to the speaker. Frequency range extension and frequency response improvement can also be useful for improving the power consumption characteristics and lifetime of speaker drivers, when achieved without resorting to increasing the amount of physical energy in suboptimal subbands.

[0026] Audio Processing System 1 is a block diagram of an audio system 100 according to some embodiments. The audio system 100 uses a nonlinear filter bank module 120 to provide frequency range extension to a speaker 110. The system 100 includes the filter bank module 120, which includes harmonic processing modules 104(1), 104(2), 104(3), and 104(4), an all-pass filter network module 122, and a combiner module 106. Some embodiments of the audio system 100 may include different components than those described herein.

[0027] The filter bank module 120 generates a phase-coherent harmonic spectrum from the audio channel a(t) using a highly tunable nonlinearity with scale-dependence according to constraints. In some embodiments, the harmonic processing modules 104 may be connected in parallel as shown. Some embodiments may include a series implementation of the filter bank module, where the residual of each of the upstream harmonic processing modules is passed to the downstream harmonic processing module. The series implementation is described in more detail in connection with FIG. 7. The system 100 generates an output channel o(t) that is provided to the speaker 110 for rendering. The harmonic processing modules 104(1)-104(4) of the filter bank module 120 provide a psychoacoustic frequency range extension for the audio channel a(t) beyond the physical bandwidth of the speaker 110.

[0028] The filter bank module 120 includes multiple harmonic processing modules 104(n) that generate harmonic spectral components h(t)(n). In some embodiments, each of the harmonic processing modules 104(1)-104(4) analyzes the entire audio channel a(t) and synthesizes a respective harmonic spectral component h(t)(1)-h(t)(4). In some embodiments, each of the harmonic processing modules may analyze a different target subband of the audio channel. Each of the harmonic spectral components h(t)(n) is a phase-coherent spectral transform of the data in a(t). Each of the harmonic spectral components h(t)(n) has a weighted phase-coherent harmonic spectrum that includes frequencies that are different from the frequencies of the data in the respective target subband of a(t), and when output by the speaker 110, produces the psychoacoustic effect of the frequencies of the respective target subband. One or more of the harmonic processing modules 104(n) may be selected to generate harmonic spectral components h(t)(n) to provide psychoacoustic frequency range extension for the speaker 110. In some embodiments, the selection of the target subbands may be based on the capabilities of the speaker 110, such as the frequency response of the speaker 110. For example, if the speaker 110 is not capable of efficiently rendering low frequencies of the sound, then the harmonic processing module 104 may be configured to target frequency subband components corresponding to low frequencies, which may be converted to harmonic spectral components h(t)(n). The audio system 100 may include one or more harmonic processing modules 104. Additional details regarding the harmonic processing modules 104 are discussed in connection with FIGS. 2-5.

[0029] The all-pass filter network module 122 generates a filtered audio channel a(t) to ensure that the audio channel a(t) remains coherent with the output of the filter bank module 120. Module122 compensates for the phase change resulting from the application of the harmonic processing module 104(n) by applying a matching phase change to the input signal a(t), allowing for coherent addition between the phase-manipulated signal, but perceptually indistinguishable from a(t), and the harmonic spectral components h(t)(n) generated by the filter bank module 120.

[0030] The combiner module 106 generates an output channel o(t) by combining the filtered audio channel a(t) from the all-pass filter network module 122 and one or more harmonic spectral components h(t)(n) from the filter bank module 120. The combiner module 106 provides the output channel o(t) to the speaker 110. In some embodiments, the combiner module 106 performs additional processing on the summed harmonic spectral components h(t)(n), as described in more detail in connection with FIG. 6.

[0031] FIG. 2 is a block diagram of the harmonic processing module 104 according to some embodiments. The harmonic processing module 104 provides a nonlinear filter that analyzes the audio channel and resynthesizes the data of the target subband with a configurable spectral transformation. The harmonic processing module 104 includes an all-pass network module 202, a forward transformer module 204, a coefficient operator module 206, and an inverse transformer module 208. The all-pass network module 202 applies a pair of transformations in phase to the audio channel x(t) to generate quadrature components. The forward transformer module 204 applies a forward transformation to the quadrature components that rotates the entire spectrum, so that the selected frequency is mapped to 0 Hz to generate rotated spectral quadrature components. The shift of the selected frequency to 0 Hz is referred to as a change from a standard basis to a rotated basis. The selected frequency may be the center frequency of the target subband or another frequency. The coefficient operator module 206 performs operations in the rotation basis, including selectively filtering the data based on frequency, magnitude, or phase, and applying a nonlinearity to the separated components with scale dependence according to the constraints to generate weighted phase coherent harmonic spectral orthogonal components. The inverse transformer module 208 applies an inverse transform to rotate the spectrum of the weighted phase coherent rotated spectral orthogonal components, so that 0 Hertz is mapped to a selected frequency to generate the harmonic spectral components.

[0032]

number

[0033] The shift of 0 Hz to the selected frequency is called a change from the rotational basis to the standard basis.

[0034]

number

[0035] may contain frequencies different from the target subbands of audio channel x(t), but when rendered by a loudspeaker, will have the psychoacoustic impact of the frequencies of the target subbands of audio channel x(t).

[0036] In some embodiments, the audio components x(t) input to the harmonic processing module 104 may be subband components a(t)(n). In this example, the selective filtering by the coefficient operator module 206 to select the target frequency may be skipped.

[0037] Allpass Network Module 202 converts the audio channel x(t) into a vector y(t) containing orthogonal components y1(t) and y2(t). The orthogonal components y1(t) and y2(t) have a 90 degree phase relationship. The orthogonal components y1(t) and y2(t) and the input signal x(t) have a unified amplitude relationship for all frequencies. The real-valued input signal x(t) is converted into orthogonal values ​​by a matched pair of all-pass filters H1 and H2. This operation may be defined via a continuous-time prototype as shown in equation (1).

[0038]

number

[0039] Some embodiments do not necessarily guarantee a phase relationship between the input (mono) signal and either of the two (stereo) quadrature components y1(t) and y2(t), but result in quadrature components y1(t) and y2(t) that include a 90 degree phase relationship, and quadrature components y1(t), y2(t), and input signal x(t) that include a unity amplitude relationship for all frequencies.

[0040] 3 is a block diagram of the forward transformer module 204 according to some embodiments. The forward transformer module 204 includes a rotation matrix module 302 and a matrix multiplier 304. The forward transformer module 204 receives the quadrature components y1(t) and y2(t) and applies a forward transform to generate a vector u(t) containing rotated spectral quadrature components u1(t) and u2(t). This transform is applied by generating a time-varying rotation matrix via the rotation matrix module 302 and applying it to the quadrature components via the matrix multiplier 304, resulting in the rotated spectral quadrature component u(t). The vector u(t) defines a coefficient space that is a frequency-shifted form of the spectrum of the audio signal x(t) and each of u at different times t is defined as a rotated spectral quadrature component. The coefficients defined by the vector u(t) are the result of rotating the spectrum of x(t) so that the desired center frequency θc is at 0 Hertz.

[0041] The forward transform may be applied as a time-varying two-dimensional rotation on the orthogonal signals, as defined by equation (2).

[0042]

number

[0043] H1 is an all-pass filter, rotated

[0044]

number

[0045] teeth, angular circumference is the rotation of the wave number θc and is defined by equation (3).

[0046]

number

[0047] Equations (2) and (3) contain iterative calls to trigonometry functions. The forward transform may be computed by a recursive 2D rotation rather than iterative calls to trigonometry functions through the interval where θc is constant. When this optimization strategy is used, calls to sin and cos are only made when θc is initialized or changed. The optimization is performed as successive powers of an infinitesimal rotation matrix, i.e.,

[0048]

number

[0049] As the respective matrices

[0050]

number

[0051] Since multiplying two 2×2 matrices together is a highly optimized computation for most architectures, this definition may provide a performance advantage over repeated calls to the trigonometric functions shown in equation (3), which are nevertheless equivalent.

[0052] 4 is a block diagram of the coefficient operator module 206 according to some embodiments. The coefficient operator module 206 includes a filter module 402, an amplitude module 404, a gate module 406, division operators 408 and 410, a harmonic generator module 412, multiplication operators 414 and 416, and a maximum module 420. The coefficient operator module 206 uses a vector u(t) including rotated spectral quadratures u1(t) and u2(t) to generate weighted phase-coherent rotated spectral quadratures u1(t) and u2(t).

[0053]

number

[0054] and

[0055]

number

[0056] Rotational spectrum, including

[0057]

number

[0058] Generate.

[0059] In some embodiments, the filter module 402 is a two channel low pass filter. In this case, the harmonic processing module 104 is configured to perform a spectral transformation on a target subband centered at θc, with a bandwidth that doubles the cutoff frequency of the filter module 402. The filter module 402 may apply a low pass filter F(x) that results in a tunable band pass filter after the inverse transformation. In this case, the cutoff frequency of F(x) corresponds to half the bandwidth of the analysis domain of the nonlinear filter.

[0060] The amplitude module 404 determines the length of the 2D vector, which is used as a measure of instantaneous amplitude that may be selectively factored out from the filtered signal vector using division operators 408 and 410. For example, division operator 408 may perform division on the u(t) component of u(t), and division operator 410 may perform division on the u(t) component of u(t). A constraint on scale independence, as defined by the max() function in equation (9), is applied by maximum value module 420, which effectively constrains the action of division operators 408 and 410. In some embodiments, the amplitude may be factored without regard to scale to enable harmonic generator module 412 to provide harmonics based on signals whose relationship is scale independent.

[0061] The harmonic generator module 412 generates a nonlinearity that includes a sum of weighted constituent nonlinearities based on the target subband of the rotated spectrum orthogonal components to provide a harmonic spectrum. For example, the harmonic generator module 412 generates constituent nonlinearities for different harmonics and assigns weights a to the constituent nonlinearities. n to generate the nonlinearity as a sum of weighted component nonlinearities.

[0062] The amplitude provided by the amplitude module 404 is then used again as it passes through the gating module 406, which generates an envelope whose instantaneous slope is limited by a slew limiter 418. The resulting slew-limited envelope is then applied to the output of the harmonic generator module 412 via multiplication operators 414 and 416. For example, the multiplication operator 416 may perform a multiplication on the u1(t) component of u(t), and the multiplication operator 414 may perform a multiplication on the u2(t) component of u(t). The nonlinearity defined by the sum of weighted harmonics results in a rotating spectrum

[0063] [Number]

[0064] is multiplied by a time-varying envelope so as to generate

[0065] The coefficients of u(t) may be expressed in polar coordinates using Equation (4).

[0066] [Number]

[0067] Here, the term ||u(t)|| is the instantaneous amplitude of the coefficient signal, and ∠u(t) is the instantaneous phase. Here, these terms can be manipulated before the inverse transformation stage.

[0068] The coefficients defined by u(t) are selectively filtered based on their instantaneous amplitudes. The filtering may include a gate function applied by the gate module 406 and a slew limiting filter applied by the slew limiter 418. The gate function based on the threshold n may be defined by Equation (5).

[0069] [Number]

[0070] Here, the case x >= n results in maintaining the coefficient, and the case x < n results in removing the coefficient. In some embodiments, the case x < n Alternatively may result in attenuation rather than complete removal of the coefficient. Since the gate function operates on the estimation of the instantaneous amplitude, it is generally more responsive and has fewer artifacts than a gate based on real-valued amplitude.

[0071] To further tailor the envelope characteristics of the nonlinear filter's response, time-domain smoothing can be achieved via a slew limiting filter. A slew limiting filter is a nonlinear filter that saturates the maximum (positive) and minimum (negative) slopes of a function. Various types of slew limiting filters or elements may be used, such as a nonlinear filter with independent control for the positive and negative saturation points, symbolized below as S(x). Applying slew limiting to the output of the gating function results in a time-varying envelope: S(G(||u[t]||)). This may be used to sculpt the envelope of the coefficients.

[0072]

number

[0073] To generate the phase-coherent harmonic spectrum of x, the harmonic generator module 412 may use a first type Chebyshev polynomial as defined by equation (6).

[0074]

number

[0075] These polynomials allow the controlled generation of harmonics by summing their outputs, as defined by equation (7) or equation (8) for scale independent nonlinearities.

[0076]

number

[0077] Or, equivalently,

[0078]

number

[0079] Here, a n = [a0, a1, a2 ... aN] are harmonic weights applied to each of the harmonics n in the phase-coherent harmonic spectrum, where N is the highest harmonic generated. In both expressions of equations (7) and (8), the nonlinearity (e.g., defined by the summation product) is independent of the input scale. This prevents the output spectrum from varying with the input volume, instead allowing only a variation determined by the spectral weights a. The weights are generally arranged as a descending series, emulating the harmonic series of naturally occurring sounds to which the human auditory system is adapted. The weight series is independent of the scale of the incoming audio channel.

[0080] While equivalent, equation (7) has the advantage of allowing direct manipulation of the output stage, while equation (8) only operates on amplitudes, omitting potentially costly trigonometric functions.

[0081] In equations (7) and (8), the output spectrum of the nonlinearity does not vary as a function of the amplitude ||u(t)|| of the input coefficients. This results in a tightly controlled and predictable nonlinearity, but its uniformity can produce textures that sound unnatural in some cases. This puzzling phenomenon is particularly noticeable with certain input content, such as spoken vocals, and the phenomenon is exacerbated when low-frequency content is also present.

[0082] For example, movie content may often use low frequency effects (LFE) content in conjunction with dialogue, which is exactly the type of content one would want to reproduce using this technology, but the resulting intermodulation distortion can affect voice intelligibility and realism.

[0083] To address this, various degrees of control may be applied to each of the constituent nonlinearities, allowing the resulting harmonic mixing to be (e.g., somewhat) active in response to the input content. The degree to which the incoming amplitude is reduced to unity determines the degree of spectral stability. At amplitudes below unity, the harmonic contributions of the constituent nonlinearities include a mixture of lower integer harmonics. Even polynomials produce a mixture of even integer harmonics, while odd polynomials produce a mixture of odd integer harmonics.

[0084] Since the instantaneous amplitude calculation is directly applied in equation (8), the algorithm can be trivially modified to apply constraints on its application, as defined in equation (9).

[0085]

number

[0086] where b n = "b0,b1,b2...bN" is the function max(||u(t)||,b n ), where N is the highest harmonic generated. For each harmonic n, the amplitude-correction factor max(||u(t)||,bn) defines a constraint on the gain modification applied to the input u(t) of the component nonlinearity, as defined by equation (10).

[0087]

number

[0088] Therefore, the nonlinearity defined by equation (11) is

[0089]

number

[0090] Weighted nonlinearity components for different harmonics (n=0 to N) (e.g., a n (by

[0091] b n At lower amplitudes of u(t), the signal amplitude used for compensation is allowed to vary. n For amplitudes of u(t) above B, the harmonic content is defined as the sum of harmonics corresponding to the polynomial order, as for all possible amplitudes in equation (8). For amplitudes of u(t) between B and 0, the upper harmonic content roughly decreases with decreasing amplitude, but for mixtures of higher order polynomials the relationship can be more complicated than simply monotonic.

[0092] For example, a transfer function containing the third Chebyshev polynomial as defined by equation (12) is

[0093]

number

[0094] If x is a unit amplitude cosine wave, as defined in equation (13), this results in a pure third harmonic (and -∞dB of the first) of:

[0095]

number

[0096] However, when x is instead a cosine wave of amplitude -6 dB, as defined by equation (14), this results in harmonic mixing.

[0097]

number

[0098] Or in spoken terms, this results in a third harmonic at -18 dB and a first (fundamental) harmonic at +1 dB. This mixing also demonstrates the oddness of all the components that result in harmonics. Furthermore, the first harmonic amplifies relative to the input, resulting in a positive dB value.

[0099] The same transfer function when applied to a -12 dB cosine wave produces a result as defined by equation (15).

[0100]

number

[0101] This includes a decreasing third harmonic and non-monotonic behavior of the first harmonic.

[0102] By constraining the degree of spectral clipping, the algorithm can generalize better across content, and potentially fewer frequency bands need to be calculated since any intermodulation effects will be perceptually less present.

[0103] Intermodulation effects are a typical by-product of applying a nonlinear transfer function to a signal with multiple frequencies. Typically, these intermodulation effects include frequencies that are sums and differences of the input signal frequencies. In the unconstrained case, these intermodulation effects have a given additional weight and persistence. By constraining the spectral clipping function, the resulting spectrum is less stable and emphasizes more the dominant frequencies due to intermodulation effects.

[0104] As a result, extending the frequency range via constrained spectral clipping may achieve a similar effect using fewer individual nonlinear filters than using unconstrained methods. This may result in increased computational efficiency. Furthermore, parameter reduction may also result in algorithms that are easier to tune, since managing the interactions between many filters can sometimes be difficult.

[0105] As shown in equation (14), the processing of the third Chebyshev polynomial applied to the cosine of amplitude -6 dB may result in amplification rather than attenuation. This fact, combined with the relatively unintuitive behavior of harmonic mixing, may cause clipping if care is not taken to avoid clipping. In some embodiments, odd nonlinearity may be applied to the harmonic spectral components generated by the filter bank module 120 to manage this resulting dynamic, as described in more detail in connection with FIG. 6.

[0106] 5 is a block diagram of the inverse transformer module 208 according to some embodiments. The inverse transformer module 208 includes a rotation matrix module 502, a matrix multiplier 504, a projection operator 506, and a matrix transpose operator 508. The inverse transformer module 208 generates phase-coherent rotated spectral quadrature components.

[0107]

number

[0108] and

[0109]

number

[0110] Rotational spectrum including

[0111]

number

[0112] to harmonic spectral components

[0113]

number

[0114] The rotation matrix module 502 generates rotate A rotation matrix identical to the rotation matrix generated by the matrix module 302 is transposed by a matrix transpose operator 508 and multiplied by a matrix multiplier 504 to generate phase-coherent rotated spectral quadrature components.

[0115]

number

[0116] and

[0117]

number

[0118] The resulting 2D vector is projected to a single dimension by the projection operator 506.

[0119] To perform the inverse transformation from the rotated basis back to the standard basis, the output spectrum is shifted so that 0 Hz is returned to its original position θ c as defined by equation (16).

[0120]

number

[0121] where P is the projection from the two-dimensional real coefficient space into a single dimension, as defined by equation (17).

[0122]

number

[0123] Forward transformation

[0124]

number

[0125] Since involves an orthonormal rotation, the inverse transform is a transpose. This algebraic structure allows for caching of the forward transform matrix and for it to be easily inverted by changing the order by which the coefficients are multiplied. In this sense, the Rotation Matrix module 302 in FIG. 3 and the Rotation Matrix module 502 in FIG. 5 are said to be identical. Harmonic Spectral Content

[0126]

number

[0127] is an example of harmonic spectral content h(t)(n) and thus may be the response of a nonlinear filter in a larger filter bank.

[0128] 6 is a block diagram of the combiner module 106 according to some embodiments. The combiner module 106 performs further processing on the harmonic spectral components h(t)(n) from the filter bank module 120, combines the harmonic spectral components h(t)(n) to generate a combined component z(t), performs further processing on the combined component z(t), and combines the combined component z(t) with the filtered audio channel a(t) from the all-pass filter network module 122 to generate an output channel o(t).

[0129] The combiner module 106 includes component processors 602(1)-602(4) (individually referred to as component processors 602 or 602(n)), a harmonic spectral component combiner 604, a combined component processor 605, and a Module The combiner module 106 may include a component processor 602 for each of the harmonic processing modules 104 of the filter bank module 120. As described above, the filter bank module 120 may selectively generate one or more harmonic spectral components h(t)(n), with each of the harmonic spectral components h(t)(n) being generated using a different frequency band n of the audio channel a(t).

[0130] With a constrained nonlinearity as defined by equation (10), the larger fluctuations in the output level that can result suggest that more can be done to limit the instantaneous peak levels.

[0131]

number

[0132] Following the generation of the signal, component processor 602(n) applies a nonlinearity to the signal that constrains it to the range (-1,1). This nonlinearity may be an odd-order function, such as a sigmoid function. Non The nonlinearity may be linear. The nonlinearity may generally be sign-preserving and may slope gently towards either extreme of the range. The scaling factor

[0133]

number

[0134] The hyperbolic tangent, with

[0135]

number

[0136] When used to reduce peaks, this nonlinearity may also add odd harmonics to the harmonic spectral content h(t)(n). These odd harmonics are in phase with the harmonics of the harmonic spectral content h(t)(n). The odd harmonics at this stage shift the overall amplitude change into a change in timbre in a manner that respects common human hearing effects on loudness.

[0137] When combined with a peak limiter, the peak limiting threshold can be set slightly below that of equation (18) so that the harmonic character of the limiting function is influenced by a more perceptually meaningful hyperbolic tangent rather than the sharp curve of the peak limiter.

[0138] In some embodiments, one or more of the component processors 602(n) attenuate (e.g., by independent tuning) each harmonic spectral component h(t)(n) to produce a combined component Achieve the desired nonlinear characteristics for z(t).

[0139] The harmonic spectral component combiner 604 combines the harmonic spectral components h(t)(n), such as harmonic spectral components h(t)(1) through h(t)(n), to generate a combined component z(t).

[0140] connected component Processor Module 606 processes the combined component z(t). Processor Module 606 may also apply various types of processing, such as high-pass filtering, dynamic range processing (eg, limiting or compression), etc.

[0141] The output combiner 608 combines the combined components z(t) with the filtered audio channels a(t) from the all-pass filter network module 122 to generate output channels o(t). In some embodiments, the output combiner 608 also combines the filtered audio channels a(t) with the filtered audio channels a(t) from the all-pass filter network module 122 before combining. channel Either a(t) or the combined component z(t) may be attenuated.

[0142] FIG. 7 is a block diagram of a filter bank module 700 according to some embodiments. The filter bank module 700 is an embodiment of the filter bank module 120. The filter bank module 700 uses a serial implementation in which each of the downstream harmonic spectral components is generated using the residual of the upstream harmonic spectral components as input. Although the construction of a filter bank module with independent filters applied in parallel is relatively intuitive, tuning such a filter bank module can be a complex task. This difficulty is a result of losing power spectrum preservation. In practice, tuning a filter bank with problematic power spectrum preservation often results in short-term delay or comb filter effects at low frequencies, hindering the listener's ability to determine timing. This occurs because the envelope of percussive low-frequency content often drops simultaneously in both amplitude and fundamental frequency. Thus, the power spectrum discontinuity results in the perception of multiple transient events where only one was present earlier.

[0143] In the sequential paradigm, each of the filters of the filter bank module 700 splits the signal between the frequency band to be analyzed and the residual of the incoming content. This is done by replacing the low-pass filter F(x) with a two-band crossover network. Note that in some cases, this can be achieved simply by subtracting the low-pass signal from the broadband signal just before the low-pass operation. Subsequent filters then operate only on the remaining high-pass signal, excluding the spectral data previously acted on by the upstream filter. As a result, the total spectral energy analyzed by the filter bank module 700 is identical to the total spectral energy at the input.

[0144] As in the parallel case, each of the serial filters uses independent forward and inverse transforms. This can be achieved in several ways. In a first example, the forward and inverse transforms of each of the filters are applied before moving on to the forward and inverse transforms of the downstream filters, and so on. In a second example, a pyramid algorithm is used in which the coordinates of the forward transform of the subsequent filter are transformed, and the pyramid algorithm involves calculating a transformation matrix using the difference between the frequency shift θcn-1 of the upstream filter and the frequency shift of the next θcn upstream filter. After all the forward transforms have been applied, the inverse transforms can be applied in reverse order, starting with the most downstream filter and moving up the series. This allows caching of frequency deltas between the forward and reverse steps.

[0145] The filter bank module 700 uses a pyramid algorithm of forward and inverse transform. In this example, there are N subbands of the audio channel a(t) that are processed serially, from subband 1 to subband N. Blocks op1 718, op2 734, and opM 752 perform coefficient operations on the first subband, the second subband, and the Nth subband, respectively. Each of op1 718, op2 734, and opM 752 may perform coefficient operations as described herein for the coefficient operator module 206.

[0146] Blocks R 704, R 720, and R 736 each multiply the right-hand two-dimensional signal with a time-varying rotation matrix R2, as described herein for the rotation matrix module 302. Block H 702 indicates the orthogonal filter operation described in equation (1), while blocks H and R jointly perform the operation defined by equation (2).

[0147] As described herein for filter module 402, blocks F706, F708, F722, F724, F740, and F742 each perform a low-pass filter operation F(x).

[0148] block * (-1) 710, * (-1)712, * (-1)726, * (-1)728, * (-1)744, and * (-1) 746 inverts the received input. Blocks +714, +716, +730, +732, +748, +750, +774, and +776 combine the received inputs to produce outputs.

[0149] Block R -1 754, R -1 756, R -1 762, R -1 766, R -1 764, and R -1772 performs the inverse transform of the R block. For example, blocks R 704, R -1 7 72, and R -1 Blocks R 720, R 766 use a rotation of -(θc1t). -1 7 64, and R -1 Blocks R736, R762 use a rotation of -(θc2-θc1)t. -1 7 54, and R -1 756 uses a rotation of -(θcN-θc(N-1))t.

[0150] Block P778 performs the one-dimensional projection operation described in equation (17).

[0151] Note that we use the difference between adjacent values ​​of θcn rather than the angular frequency θc. For a particular choice of θcn, the pyramid algorithm

[0152]

number

[0153] Limiting the number of times that is calculated can provide a computationally more efficient implementation. A particularly computationally efficient choice for the θcn distribution is linear (the difference between θc for adjacent filters is held constant) and therefore, since the matrices are identical to each other,

[0154]

number

[0155] Minimize recalculation of completely.

[0156] The final residual contains the data that is not affected by the entire filter bank, eliminating the possibility of constructive or destructive interference between affected and unaffected signals. The transfer function of this residual signal fits perfectly into the filter bank analysis domain. This does not necessarily imply a perfect reconstruction of the power spectrum of the output signal, since the coefficient operations are likely to result in modifications of dynamic behavior or the synthesis of entirely new content. In many cases, this final residual can be discarded entirely, and the output of the H702 can be used to mix in the unaffected content back into the final sum.

[0157] The filter bank module 700 uses the residuals of the upstream harmonic spectral components as input to generate each of the downstream harmonic spectral components. The filter bank topology including M all nonlinear filters can be described in this case as a serial architecture. The nonlinear filters can therefore be defined by an index m having a value from 1 to M. For example, blocks +714 and +716 output the residuals of the first harmonic spectral components (e.g., m=1) that are used to generate the second harmonic spectral components (e.g., m=2). Here, the residuals of the first harmonic spectral components represent the parts of the audio channel that were removed by blocks F706 and F708 and thus not processed by block OP1 718. These residual parts are then filtered out by blocks +714 and +716. * (-1)710 and * The residual is generated by inverting the portion filtered by (-1) 712 and adding the inverted filtered portion to the portion filtered by blocks + 714 and + 716. Further downstream processing operates in a similar manner. For example, blocks + 730 and + 732 output a residual of the second harmonic spectral component, which is then used to generate the third harmonic spectral component (e.g., m=3), and so on.

[0158] Processing of the Example Figure 8 is a flow chart of a process 800 for psychoacoustic frequency range extension according to some embodiments. The process shown in Figure 8 may be performed by components of an audio system (e.g., audio system 100). Other entities may perform some or all of the steps in Figure 8 in other embodiments. Implementations may include different and / or additional steps or may perform steps in a different order.

[0159] The audio system generates (805) quadrature components that define an orthogonal representation of the audio channels. The audio channels may be channels of a multi-channel audio signal, such as the left or right channel of a stereo audio signal. The quadrature components include a 90 degree phase relationship. The quadrature components and the audio channels include a unity amplitude relationship for all frequencies. In some embodiments, the real-valued input signal is converted to quadrature values ​​by a matched pair of all-pass filters.

[0160] The audio system generates (810) rotated spectral orthogonal components by applying a forward transform that rotates the spectrum of the orthogonal components (e.g., the entire spectrum) from a standard basis to a rotated basis. The standard basis represents the frequencies of the input audio channels before the rotation. The rotation may result in a target frequency being mapped to 0 Hertz. This target frequency may be the center of the analysis region of a harmonic processing module, such as the center frequency of a target subband for psychoacoustic range extension. The forward transform may be calculated using iterative calls to a trigonometric function as defined by equation (3) or using an equivalent recursive 2D rotation.

[0161] The audio system separates the components of the rotated spectral quadrature components at the target frequency and target amplitude (815). Separating the components may be performed in a rotated basis. For example, the target frequency may be separated using a filter F(x), where x includes the components defined by u(t). In some embodiments, the filter removes frequencies above a threshold, which has the effect of separating the target subband, spanning twice the threshold symmetrically around the center frequency θc to which the forward transform was tuned. In some embodiments, the audio system determines the target frequency based on factors such as the reproducible range of the speaker, reduced power consumption of the speaker, or increased life of the speaker.

[0162] The audio system may also separate components at the target amplitude from the rotated spectral orthogonal components, such as by using a gating function. The gating function can be configured to either discard undesired information in the subband or to preserve the amplitude envelope. The gating function may further include a slew limiting filter or similar smoothing function.

[0163] The audio system generates weighted phase-coherent harmonic spectral orthogonal components by applying a nonlinearity to the separated components with a scale dependency according to the constraint (820). The weighted phase-coherent rotated spectral orthogonal components may be generated in a rotation basis. This rotation basis is well suited to the designer's generation of the spectrum because it represents the standard basis signal as a two-dimensional vector and because it is centered around the target frequency of zero. The vector can then be further decomposed into polar coordinates as seen in equation (4), which is similar to calculating the amplitude and argument of a single bin in a short-time Fourier transform (STFT), which is a natural descriptor of information about a particular frequency. This implementation has several distinct advantages over the STFT representation. The first advantage is that the bin information is calculated only as needed, rather than for the entire spectrum. Another advantage is that the results are calculated at the time resolution required for an appropriate representation of the temporal data. Furthermore, the filter, which operates similarly to the window function in the STFT technique, is easily adjusted for the purpose of separating the target spectral content from its residual, and in the case of multiple harmonic processing modules, may have a non-uniform adjustment.

[0164] The nonlinearity, whose function is to generate a phase coherent spectrum to first order given the phase information in the rotated spectrum orthogonal components, may have a dependence on scale according to a constraint as defined by equation (11). The nonlinearity comprises a weighted mixture of constituent nonlinearities, each of which corresponds to a different harmonic n, defined by equation (10). The application of the nonlinearity to separate components is defined by equation (9). For each harmonic n, an amplitude correction factor max(||u(t)||,bn) defines a constraint on the gain modification applied to the input u(t) of the constituent nonlinearity. The scale indicates the amplitude of the input component u(t) as defined by ||u(t)||, which represents the energy present in the signal at time t. Different harmonics n correspond to different minimum constraints b nFor example, the lower harmonics (e.g., fundamental n=1) may be unconstrained (e.g., b n =0), whereas higher harmonics have higher b n can be further constrained by the value of

[0165] The nonlinearity itself may include a weighted sum of Chebyshev polynomials of the first kind with amplitudes selectively factored according to the constraints. Each of the component nonlinearities of the nonlinearity has a predetermined harmonic weight a, as defined by equation (9). n It can be weighted by:

[0166] The audio system generates harmonic spectral components by applying an inverse transform, which rotates the spectrum of the weighted phase-coherent rotated spectral orthogonal components from a rotated basis to a standard basis ( 8 25). The inverse transform may rotate the spectrum so that 0 Hertz is mapped to the target frequency. Harmonic spectral components include frequencies that are different from the target frequency, but produce the psychoacoustic effects of the target frequency when rendered by a speaker. The frequencies of the harmonic spectral components may be within the bandwidth of the speaker, while the subband frequencies may be outside the bandwidth of the speaker. In some embodiments, the subband frequencies are lower than the frequencies of the harmonic spectral components. In some embodiments, the subband frequencies include frequencies between 18 and 250 Hertz. In some embodiments, the target subbands or frequencies may be within the reproducible range of the speaker, but may have been selected for application-specific reasons, such as to reduce power consumption of the audio system or improve speaker life.

[0167] The audio system combines (830) the frequencies and harmonic spectral components of the audio channel outside the range of the target frequency to generate an output channel and provides (835) the output channel to the speaker. In some embodiments, the audio system generates the output channel by combining the original audio channel with the harmonic spectral components and provides the output channel to the speaker. In some embodiments, the audio system filters the audio channel or other subband components of the audio channel to ensure that the audio channel or other subband components remain coherent with the harmonic spectral components (e.g., eliminates subband component(s) used for frequency range extension), and combines the harmonic spectral components with the filtered audio channel or other subband components to generate an output channel for the speaker. In some embodiments, the combination of the filtered channel or original audio channel and the harmonic spectral components may be further processed, e.g., equalized, compressed, etc., to generate an output channel for the speaker.

[0168] In steps 805-825, harmonic spectral components are generated for a frequency band of an audio channel. In some embodiments, multiple harmonic spectral components are generated and combined (830), where each of the harmonic spectral components is generated using a different frequency band of the audio channel. An output channel may be generated by combining frequencies of the audio channel that are outside the range of target frequencies of the harmonic spectral components. The harmonic spectral components may be generated in parallel or in series. In the serial case, each of the downstream harmonic spectral components may be generated using the residual of the upstream harmonic spectral component as an input. In some embodiments, different speakers may have different available bandwidths or frequency responses. For example, a mobile device (e.g., a mobile phone) may include unbalanced speakers. Different sub-band components may be used for frequency range extension for different speakers.

[0169] Computer of the embodiment 9 is a block diagram of a computer 900 according to some embodiments. The computer 900 is an example of a circuit for implementing the audio system 100 or an audio system and its components, such as the filter bank module 120 or the filter bank module 700. Illustrated is at least one processor 902 coupled to a chipset 904. The chipset 904 includes a memory controller hub 920 and an input / output (I / O) controller hub 922. The memory 906 and the graphics adapter 912 are coupled to the memory controller hub 920, and the display device 918 is coupled to the graphics adapter 912. The storage device 908, the keyboard 910, the pointing device 914, and the network adapter 916 are coupled to the I / O controller hub 922. The computer 900 may include various types of input or output devices. Other embodiments of the computer 900 have different architectures. For example, the memory 906 is directly coupled to the processor 902 in some embodiments.

[0170] The storage device 908 includes one or more non-transitory computer-readable storage media, such as a hard drive, a compact disc read-only memory (CD-ROM), a DVD, or a solid-state memory device. The memory 906 holds program code (comprised of one or more instructions) and data used by the processor 902. The program code may correspond to the processing aspects described with reference to FIGS. 1-8.

[0171] 9. A pointing device 914 is used in combination with the keyboard 910 to input data into the computer system 900. The graphics adapter 912 displays images and other information on the display device 918. In some embodiments, the display device 918 includes touch screen capabilities for receiving user inputs and selections. The network adapter 916 couples the computer system 900 to a network. Some embodiments of the computer 900 have different and / or other components than those shown in FIG.

[0172] The circuitry may include one or more processors executing program code stored on a non-transitory computer readable medium, which, when executed by the one or more processors, configures the one or more processors to implement an audio processing system or module of an audio processing system. Other examples of circuitry implementing an audio processing system or module of an audio processing system may include integrated circuits, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other types of computer circuitry.

[0173] Additional Considerations Exemplary benefits and advantages of the disclosed configurations include allowing a speaker to effectively render (e.g., lower) frequencies beyond the speaker's physical capabilities. By processing the audio signal as discussed herein, the rendered sound produces frequency effects that exceed the bandwidth of the physical driver.

[0174] Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Individual operations of one or more methods are illustrated and described as separate operations, and one or more of the individual operations may be performed simultaneously and do not require the operations to be performed in the order illustrated. Structures and functionality presented as separate components in an example configuration may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter herein.

[0175] Certain embodiments are described herein as including logic or some components, modules, blocks, or mechanisms. A module may constitute either a software module (e.g., code embodied in a machine-readable medium or transmission signal) or a hardware module. A hardware module is a tangible unit capable of performing a particular operation and may be configured or arranged in a particular manner. In example embodiments, one or more computer systems (e.g., standalone, client, or server computer systems) or one or more hardware modules (e.g., processors or groups of processors) of a computer system may be configured by software (e.g., applications or application portions) as hardware modules that operate to perform particular operations as described herein.

[0176] Various operations of example methods described herein may be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the associated operations. Such processors, whether temporarily or permanently configured, may constitute processor-implemented modules that operate to perform one or more operations or functions. Modules referred to herein may, in some example embodiments, include processor-implemented modules.

[0177] Similarly, the methods described herein may be at least partially processor-implemented. For example, at least some of the operations of the methods may be performed by one or more processors or processor-implemented hardware modules. The execution of certain operations may be distributed among one or more processors and may be spread across several machines as well as within a single machine. In some example embodiments, the processor or processor(s) may be located at a single location (e.g., in a home environment, an office environment, or as a server farm), while in other embodiments, the processor may be distributed across several locations.

[0178] Unless otherwise specifically stated, discussions herein using terms such as "processing," "computing," "calculating," "determining," "presenting," "displaying," and the like may refer to machine (e.g., computer) actions or operations that manipulate or transform data represented as physical (e.g., electronic, magnetic, or optical) quantities in one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0179] As used herein, any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0180] Some embodiments may be described using the phrases "coupled" and "connected," along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. For example, some embodiments may be described using the term "connected" to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other even further. The embodiments are not limited in this context.

[0181] As used herein, the terms "comprises," "comprises," "includes," "including," "has," "having," or any other variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, article, or apparatus that includes a list of elements is not necessarily limited to only those elements and may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless expressly stated to the contrary, "or" indicates an inclusive or, not an exclusive or. For example, a condition A or B is satisfied by any one of A being true (or present) and B being false (or not present), A being false (or not present) and B being true (or present), and both A and B being true (or present).

[0182] In addition, the use of "a" or "an" is used to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the invention. The description should be read to include one or at least one, and the singular also includes the plural, unless it is clear that something else is meant.

[0183] Some portions of this description will describe embodiments in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to effectively convey the substance of their work to others skilled in the art. While these operations have been described functionally, computationally, or logically, it will be understood that they may be implemented by computer programs, or equivalent electronic circuits, microcode, or the like. Further, it has also proven convenient at times to depict these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combination thereof.

[0184] Any step, operation, or process described herein may be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module may be implemented using a computer program product that includes a computer-readable medium that includes computer program code that can be executed by a computer processor to perform any or all of the described steps, operations, or processes.

[0185] The embodiments may also refer to an apparatus for performing the operations herein. The apparatus may be specially constructed for the required purposes and / or the apparatus may include a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer-readable storage medium, or any type of medium suitable for storing electronic instructions that may be coupled to a computer system bus. Furthermore, any computing system referred to herein may include a single processor or may be an architecture using a multiple processor design to increase computing power.

[0186] Embodiments may also relate to products produced by the computational processes described herein. Such products may include information resulting from the computational processes, where the information is stored in a non-transitory, tangible computer-readable storage medium, and may include any embodiment of the computer program product or other data combination described herein.

[0187] Upon reading this disclosure, those skilled in the art will still appreciate additional alternative structural and functional designs for systems and processes through the principles disclosed herein. Thus, while specific embodiments and applications have been shown and described, it will be understood that the disclosed embodiments are not limited to the precise structures and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the spirit and scope as defined in the appended claims.

[0188] Finally, the language used herein has been chosen primarily for readability and educational purposes, and may not be chosen to elaborate or limit the patent rights. Accordingly, it is intended that the scope of the patent rights be limited not by this detailed description, but by any claims published in an application based on this specification. Accordingly, the disclosure of the embodiments is intended to be illustrative, and not limiting, of the scope of the patent rights, as set forth in the following claims.

Claims

1. generating orthogonal components from the audio channels defining the orthogonal representation of the audio channels; generating rotated spectral orthogonals by applying a forward transform that rotates the spectrum of the orthogonals from a standard basis to a rotated basis; In the rotation basis, Separating components of the rotated spectrum orthogonal components at a target frequency; and generating weighted phase-coherent harmonic spectral orthogonal components by applying the nonlinearity to the separated components in a scale-dependent manner subject to a constraint on a gain modification applied to an input of the nonlinearity; generating harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase-coherent harmonic spectral orthogonal components from the rotated basis to the standard basis; combining said harmonic spectral components with frequencies of said audio channel outside said target frequency range to generate an output channel; providing said output channels to a speaker; The system includes a circuit configured as follows.

2. 2. The system of claim 1, wherein the nonlinearity comprises a weighted mixture of constituent nonlinearities, and each of the constraints comprises a constraint on a gain modification applied to an input of each constituent nonlinearity.

3. The system of claim 2 , wherein the nonlinearity comprises a weighted sum of Chebyshev polynomials of a first type with amplitudes selectively factored according to the constraints.

4. 2. The system of claim 1, wherein the circuitry is further configured to generate a plurality of harmonic spectral components, each of the harmonic spectral components being generated using a different frequency band of the audio channel, and the circuitry is configured to generate the output channel by combining the plurality of harmonic spectral components.

5. The system of claim 4 , wherein the circuitry is configured to use a residual of an upstream harmonic spectral component as an input to generate the plurality of harmonic spectral components in series with each of the downstream harmonic spectral components.

6. The system of claim 4 , wherein the circuitry is configured to generate the multiple harmonic spectral components in parallel.

7. The system of claim 1 , wherein the circuitry is further configured to apply an odd-order nonlinearity to the harmonic spectral components.

8. The system of claim 1 , wherein the harmonic spectral components include frequencies different from the target frequency of the audio channel and produce a psychoacoustic effect of the target frequency when rendered by the speaker.

9. the forward transform rotates the spectrum of the quadrature components such that a target frequency is mapped to 0 Hertz; and the inverse transform rotates the spectrum of the weighted phase coherent harmonic spectral orthogonal components such that 0 Hertz is mapped to the target frequency; The system of claim 1 .

10. The system of claim 1 , wherein the target frequency comprises a frequency between 18 Hertz and 250 Hertz.

11. The circuit comprises: The reproducible range of the speaker; reducing the power consumption of the speaker; or increased life span of the speaker; The system of claim 1 , further configured to determine the target frequency based on at least one of:

12. The system of claim 1 , wherein the speaker is a component of a mobile device.

13. The system of claim 1 , wherein the circuitry is further configured to isolate the components at a target amplitude using a gating function.

14. The system of claim 1 , wherein the circuitry is further configured to apply a smoothing function to the separated components.

15. A non-transitory computer-readable medium containing instructions stored thereon, the instructions, when executed by at least one processor, generating orthogonal components from the audio channels defining orthogonal representations of the audio channels; generating rotated spectral orthogonals by applying a forward transform that rotates the spectrum of the orthogonals from a standard basis to a rotated basis; In the rotation basis, Separating components of the rotated spectrum orthogonal components at a target frequency; and generating weighted phase-coherent harmonic spectral orthogonal components by applying the nonlinearity to the separated components in a scale-dependent manner subject to a constraint on a gain modification applied to an input of the nonlinearity; generating harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase-coherent harmonic spectral orthogonal components from the rotated basis to the standard basis; combining said harmonic spectral components with frequencies of said audio channel outside said target frequency range to generate an output channel; providing said output channels to a speaker; and a non-transitory computer-readable medium for configuring the at least one processor to:

16. 16. The non-transitory computer-readable medium of claim 15, wherein the non-linearities include a weighted mixture of constituent non-linearities, and each of the constraints includes a constraint on a gain modification applied to an input of each constituent non-linearity.

17. 17. The non-transitory computer-readable medium of claim 16, wherein the non-linearity comprises a weighted sum of Chebyshev polynomials of a first type with amplitudes selectively factored according to the constraints.

18. The instructions further configure the at least one processor to generate a plurality of harmonic spectral components, each of the harmonic spectral components being generated using a different frequency band of the audio channel; the output channel is generated by combining the plurality of harmonic spectral components; and 16. The non-transitory computer-readable medium of claim 15, wherein the multiple harmonic spectral components are generated serially for each downstream harmonic spectral component using a residual of an upstream harmonic spectral component as an input.

19. 20. The non-transitory computer-readable medium of claim 15, wherein the instructions further configure the at least one processor to apply an odd order non-linearity to the harmonic spectral components.

20. By circuit, generating orthogonal components from the audio channels defining orthogonal representations of the audio channels; generating rotated spectral orthogonals by applying a forward transform that rotates the spectrum of the orthogonals from a standard basis to a rotated basis; In the rotation basis, separating components of the rotated spectral orthogonals at a target frequency; generating weighted phase-coherent harmonic spectral orthogonal components by applying a nonlinearity to the separated components that is scale-dependent subject to a constraint on a gain modification applied to an input of the nonlinearity; generating harmonic spectral components by applying an inverse transform that rotates the spectrum of the weighted phase coherent harmonic spectral orthogonal components from the rotated basis to the standard basis; combining said harmonic spectral content with frequencies of said audio channel outside said target frequency range to generate an output channel; providing said output channel to a speaker; The method comprises:

21. 21. The method of claim 20, wherein the nonlinearity comprises a weighted mixture of constituent nonlinearities, and each of the constraints comprises a constraint on a gain modification applied to an input of each constituent nonlinearity.

22. 22. The method of claim 21, wherein the nonlinearity comprises a weighted sum of Chebyshev polynomials of a first type with amplitudes selectively factored according to the constraints.

23. generating, by said circuitry, a plurality of harmonic spectral components, each of said harmonic spectral components being generated using a different frequency band of said audio channel; the output channel is generated by combining the plurality of harmonic spectral components; and 21. The method of claim 20, further comprising: generating the plurality of harmonic spectral components in series for each downstream harmonic spectral component using a residual of an upstream harmonic spectral component as an input.

24. 21. The method of claim 20, further comprising applying, by said circuitry, an odd-order nonlinearity to said harmonic spectral components.

Images