Dynamic processing effect architecture

Abstract

A method includes providing, for each respective audio channel of a plurality of audio channels provided by an operating system of a computing device, a set of successive audio processing stages for application to the respective audio channel. The method also includes providing, by the operating system, an application programming interface (API) configured to set a plurality of parameters for adjusting a set of successive audio processing stages for each respective audio channel. The method also includes receiving, via the API and from an application running on the computing device, one or more values for one or more of the plurality of parameters. The method also includes adjusting, by the operating system, the plurality of audio channels based on the received one or more values for the one or more parameters.

Description

[0001] This application is a divisional application of the PCT invention patent application with application date of May 2, 2019, application number 201980030467.1, and invention name “Dynamic Processing Effects System Architecture”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 668,142, filed May 7, 2018, which is incorporated herein by reference in its entirety. Background Art

[0004] The present disclosure generally relates to processing audio signals. Summary of the invention

[0005] Aspects of the subject technology relate to dynamic audio processing effects and software applications that can utilize such effects. Each audio channel provided by the operating system of a computing device is provided with a set of successive audio levels (which can be referred to as "audio processing levels") to apply to or process the audio channel (e.g., Pre-EQ, multi-band compression, Post-EQ, and limiter). The operating system provides an application programming interface (i.e., an application programming interface, or simply referred to as an API) to set parameters to adjust the set of successive audio levels for each audio channel. A set of parameters for adjusting the set of successive audio levels can be received from an application running on the device (e.g., developed and provided by a third-party developer) via the application programming interface. The operating system can then adjust the audio channel based on the received parameter set. In other words, the operating system can process the audio channel using a set of successive audio levels adjusted according to the received parameter set. In this way, multiple different applications can each utilize these configurable audio channels to achieve various desired audio processing effects. Therefore, these applications do not have to implement any audio processing algorithm separately to achieve the desired effect. In some examples, this can result in a reduction in latency in the audio output. The subject technology described herein can be utilized in a variety of different ways, some of which are described below, one such use being for enhancing or compensating for a user's hearing.

[0006] Thus, in a first example embodiment, a computer-implemented method is provided, the method comprising providing a set of sequential audio levels for each respective audio channel of a plurality of audio channels provided by an operating system of a computing device to be applied to the respective audio channel. The computer-implemented method further comprises providing, by the operating system, an API configured to set a plurality of parameters for adjusting the set of sequential audio levels for each respective audio channel. The computer-implemented method further comprises receiving, via the API, one or more values ​​for one or more parameters of the plurality of parameters from an application running on the computing device. The computer-implemented method further comprises adjusting, by the operating system, the plurality of audio channels based on the received one or more values ​​for the one or more parameters.

[0007] In a second example embodiment, a computing system is provided, the computing system comprising an application configured to be executed by the computing system and an operating system configured to provide a plurality of audio channels. The computing system also comprises a set of sequential audio levels for each respective audio channel in the plurality of audio channels to be applied to the respective audio channels. The computing system also comprises an API configured to (i) set a plurality of parameters for adjusting the set of sequential audio levels for each respective audio channel, and (ii) receive one or more values ​​for one or more parameters in the plurality of parameters from the application. The operating system is configured to adjust the plurality of audio channels based on the received one or more values ​​for the one or more parameters.

[0008] In a third example embodiment, a non-transitory computer-readable storage medium having instructions stored thereon is provided, the instructions causing the computing device to perform operations when executed by a computing device. The operations include providing a set of sequential audio levels for each of a plurality of audio channels provided by an operating system of the computing device to be applied to the respective audio channel. The operations also include providing, by the operating system, an API configured to set a plurality of parameters for adjusting the set of sequential audio levels for each respective audio channel. The operations also include receiving, via the API, one or more values ​​for one or more of the plurality of parameters from an application executing on the computing device. The operations also include adjusting, by the operating system, the plurality of audio channels based on the received one or more values ​​for the one or more parameters.

[0009] In a fourth example embodiment, a system is provided that includes means for providing a set of sequential audio levels for each of a plurality of audio channels provided by an operating system of a computing device to be applied to the respective audio channel. The system also includes means for providing, by the operating system, an application programming interface (API) configured to set a plurality of parameters for adjusting the set of sequential audio levels for each respective audio channel. The system also includes means for receiving, via the API, one or more values ​​for one or more of the plurality of parameters from an application executing on the computing device. The system also includes means for adjusting, by the operating system, the plurality of audio channels based on the received one or more values ​​for the one or more parameters.

[0010] It should be understood that other configurations of the present technology will be readily apparent to those skilled in the art from the following detailed description, wherein various configurations of the present technology are shown and described by way of illustration. As will be appreciated, the subject technology is capable of other and different configurations, and several of its details are capable of being modified in various other respects, all without departing from the scope of the subject technology. Thus, the drawings and detailed description should be considered illustrative rather than restrictive in nature. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are included to provide a further understanding and are incorporated in and constitute a part of this specification, illustrate aspects of the disclosure and together with the description serve to explain the principles of the aspects of the disclosure.

[0012] Figure 1 A sample flexible audio processing architecture according to example aspects of the present disclosure is illustrated.

[0013] Figure 2 A sample flexible audio processing architecture according to example aspects of the present disclosure is illustrated.

[0014] Figure 3 A sample nested parameter structure according to example aspects of the present disclosure is illustrated.

[0015] Figure 4 An instantiation example according to example aspects of the present disclosure is illustrated.

[0016] Figure 5 A sample process of a multi-band compressor according to example aspects of the present disclosure is illustrated.

[0017] Figure 6 A sample listing of parameter getters and setters in the MBC level according to example aspects of the present disclosure is illustrated.

[0018] Figure 7A sample list of parameter getters and setters in a slicer and a sample process of a slicer stage according to example aspects of the present disclosure are illustrated.

[0019] Figure 8 Illustrated are exemplary multi-channel audio processing effects according to example aspects of the present disclosure.

[0020] Fig. 9 Illustrated is a list of control parameters for a smart hearing feature according to example aspects of the present disclosure.

[0021] Fig.10 Illustrated are sample accessibility services user interface screens according to example aspects of the present disclosure.

[0022] Fig.11 A sample smart hearing user interface according to example aspects of the present disclosure is illustrated.

[0023] Fig.12 A sample multi-dimensional map according to example aspects of the present disclosure is illustrated.

[0024] Fig.13A and Fig. 13B A sample multi-dimensional map according to example aspects of the present disclosure is illustrated.

[0025] Fig.14 An example network environment for dynamic audio processing effects in accordance with the subject technology is illustrated.

[0026] Fig.15 An example electronic system is conceptually illustrated with which some embodiments of the subject technology may be implemented.

[0027] Fig.16 A flow chart according to example aspects of the present disclosure is illustrated. DETAILED DESCRIPTION

[0028] The detailed description set forth below is intended to be a description of various configurations of the subject technology, and is not intended to represent the only configuration in which the subject technology can be practiced. The accompanying drawings are incorporated herein and constitute a part of the detailed description. In order to provide a thorough understanding of the subject technology, the detailed description includes specific details. However, the subject technology is not limited to the specific details set forth herein, and can be practiced without these specific details. In some cases, structures and components are shown in block diagram form to avoid blurring the concept of the subject technology.

[0029] The operating system of a computing device (e.g., an operating system for a mobile device) has simple built-in audio processing effects and provides limited options to application developers to expand the capabilities of the built-in audio processing effects. Specifically, the operating system may be provided with an audio processing architecture that is limited to a fixed number of bands (e.g., frequency bands) for equalization (EQ), a multi-band compressor (MBC), and a fixed size and frequency of the band. However, utilizing the built-in audio processing effects available in the operating system platform may allow the developer of the application to access certain audio data paths (e.g., telephone audio signal paths, etc.) of the computing device that the developer cannot directly access for security reasons. There may be audio processing solutions that can be implemented using common built-in audio processing effects that are not flexible enough for developers to implement similar but still sufficiently different architectures / modules. Therefore, in order to achieve the desired audio processing effects, developers who do not have the privilege of using built-in audio processing effects are required to build their own solutions, which increases the complexity of application design.

[0030] According to one or more embodiments, a method and system for dynamically processing the effects of audio signals are disclosed herein. According to various aspects of the subject technology, a flexible audio processing architecture is built into the operating system of a computing device, and an interface (e.g., an application program interface (API)) is provided for applications to configure the built-in flexible audio processing architecture and achieve the desired audio signal processing effects. That is, applications are allowed to access and configure the audio processing architecture to generate a desired audio processing chain and use this chain to process audio waveforms. Thus, when developing these applications, these applications do not have to implement independent and / or separate audio processing tools, algorithms and / or processes. By assigning different values ​​to parameters exposed by the API, applications can alternatively rely on the operating system and API to achieve a wide range of desired audio processing effects.

[0031] Figure 1 A sample flexible audio processing architecture 130 is illustrated according to example aspects of the present disclosure. The flexible audio processing architecture 130 may be built into an operating system of a computing device. The flexible audio processing architecture 130 and an API associated with the flexible audio processing architecture 130 may describe an audio algorithm, process, and / or audio effects chain for utilizing a solution (e.g., a software application) for multi-channel equalization and / or multi-band compression. The flexible audio processing architecture 130 may include a plurality of audio channels (e.g., K audio channels). Each audio channel may include successive audio stages. Successive audio stages may include an input gain (InputGain) stage ( Figure 1 ), a pre-equalization (Pre-EQ) stage 120, a multi-band compressor (MBC) stage 122, a post-equalization (Post-EQ) stage 124 and a limiter stage 126.

[0032] The Pre-EQ stage 120 can allow the developer of the application to programmatically adjust the audio frequency, for example, from 20Hz to 20kHz. The Pre-EQ stage 120 can also allow the developer to adjust the balance between different frequency components present in the audio signal. The MBC stage 122 can reduce loud sounds while boosting quiet sounds without distorting the characteristics of the original audio input. The Post-EQ stage 124 can fine-tune the sound before the limiter stage, allowing further adjustment of the balance between different frequency components present in the audio signal generated by the MBC stage 122. The limiter stage 126 can prevent additional gain above a certain threshold to protect the output audio from being loud and disruptive. That is, the limiter 126 can help avoid saturation of the audio signal, thereby avoiding distortion.

[0033] In some embodiments, the input gain stage and the Pre-EQ stage 120 can be combined into one stage. Audio data can be successively input to the input gain stage, the Pre-EQ stage 120, the MBC stage 122, the Post-EQ stage 124, and the limiter stage 126. The flexible audio processing architecture 130 provides EQ stages (e.g., Pre-EQ 120 and Post-EQ 124) before and after the MBC stage 122 to allow a wide range of audio processing effects for applications. Each stage in the successive audio stages can be enabled or disabled (e.g., turned on / off). For example, an enabled stage modifies the audio signal according to parameters set by the application, while a disabled stage passes the audio signal. Providing the flexibility of turning on / off each stage (which will be discussed in detail later) can allow the flexible audio processing architecture 130 to be utilized by various applications.

[0034] Figure 2 A sample flexible audio processing architecture is illustrated according to example aspects of the present disclosure. Of note, Figure 2 The audio processing architecture includes Figure 1 , thereby forming multiple audio channels. The flexible audio processing architecture can be constructed or implemented in an operating system, or as part of an operating system, and can include multiple audio channels (e.g., channel 0 and channel 1 to channel K-1) that receive audio inputs (e.g., input 0 and input 1 to input K-1) from, for example, an audio source (e.g., a multi-channel audio source).

[0035] Each audio channel can include Figure 1That is, channel 0 includes Pre-EQ 200, MBC 202, Post-EQ 204, and limiter 206, channel 1 includes Pre-EQ 210, MBC 212, Post-EQ 214, and limiter 216, and channel K-1 includes Pre-EQ 220, MBC 222, Post-EQ 224, and limiter 226. After being processed by the successive audio stages, audio data is output from the audio channels (e.g., output 0 and output 1 to output K-1).

[0036] Each of the Pre-EQ, MBC, and Post-EQ stages may support multiple bands (e.g., band 0, band 1, band M-2, band M-1). Each channel and each level in the channel may be configured based on a parameter set, for example, by an application via an API. In one or more embodiments, all parameters may be changed at runtime, or only some parameters (or no parameters) may be changed at runtime. In addition, each channel and / or each level thereof may be modifiable independently of other channels and levels. In one example, for each of channels 0 and 1 to K-1, the number of bands N in the Pre-EQ stage may be independently modifiable. That is, different N values ​​may be assigned to each of Pre-EQ 200, 210, and 220 using parameters. Similarly, the number of bands M of the MBC stage and the number of bands O of the Post-EQ stage may be independently modifiable for each channel by parameters. Alternatively, in another example, the number of bands N in the Pre-EQ stage may be the same for each of channels 0 and 1 to K-1. That is, the same value can be assigned to each of the Pre-EQ 200, 210, and 220 using the parameter. Similarly, for each channel, the number of bands M of the MBC stage and the number of bands O of the Post-EQ stage can be the same. It is worth noting that the number of bands N, M, and O can each be assigned different values ​​independently of each other.

[0037] Figure 3 A sample nested parameter structure according to an example aspect of the present disclosure is illustrated. Audio processing effects can be instantiated and controlled through a channel-based nested structure with parameters. Figure 4 An instantiation example according to an example aspect of the present disclosure is illustrated in FIG. An application programming interface (API) may be provided to application developers, through which a built-in flexible audio processing architecture may be configured to achieve a desired audio signal processing effect. That is, Figure 4 The program code 400 is shown, which calls the functions provided by the API to set Figure 3In some embodiments, the details of the implementation of the API and / or API functions can be hidden (e.g., from application developers). Thus, the same API can be shared between different machines, computer processing unit (CPU) architectures, and / or vendors for different implementations. That is, applications can use a unified interface that allows developers to ignore the implementation details of the API or its functions.

[0038] In one example, this method can allow frequency domain and / or time domain implementation of the effect. In some cases, some additional parameters (e.g., additional parameters whose values ​​determine whether to use frequency domain or time domain implementation) can be used to select and fine-tune the specific implementation utilized by the application. The setPreferredFrameDuration() method called in program code 400 can be used to set the expected frame duration (e.g., in milliseconds) and set the effect variable to time domain or frequency domain. The time domain implementation can have less algorithmic delay, but may be less flexible for specific frequency band cutoffs. The frequency domain implementation may have more delays, but can be more flexible for working with different numbers of frequency bands, unexpected changes in frequency bands, specific frequency band cutoffs and parameters for these frequency bands. The frequency domain implementation using the short-time Fourier transform algorithm and the overlap-add method can help select any number of frequency bands with any crossover to find suitable control parameters. Additional implementations can be included as needed or desired.

[0039] The API allows the application to set parameters for each channel and each level in the channel. The API is provided for the application to select the number of channels for processing the audio signal (e.g., setting the value of K) according to the number of channels required to achieve the desired audio signal processing effect. Each mono channel can be regarded as an independent channel. For example, a stereo signal may include two channels: 0 and 1, while a 5.1 surround sound signal may have six independent channels.

[0040] Back to Figure 3 , each stage that supports bands (e.g., Pre-EQ 200, MBC 202, and Post-EQ 204) may include a different number of bands. Therefore, EQ parameters 330 include a variable BAND_COUNT 336 for specifying the number of bands. The API allows the application to select the number of bands in the Pre-EQ, MBC, and Post-EQ stages by setting parameter values ​​for "BAND_COUNT" (i.e., BAND_COUNT 336 and BAND_COUNT 346) in each stage. In some embodiments, the same number of bands may be provided for the same type of stage on each of the K channels.

[0041] Each band may have a different cutoff frequency specified by the application (e.g., FREQUENCY_CUT-OFF 376 indicated by EQ band parameter 370 and FREQUENCY_CUT-OFF 383 specified by MBC band parameter 380). The API allows the application to select the cutoff frequency for each band in the stage. Each band may be turned on / off as indicated by variables ENABLED 372 and 381.

[0042] Similarly, each stage can be turned on / off. The API allows the application to enable or disable any stage by setting a parameter in each stage to "ENABLED", as indicated by the variables ENABLED 334, 344, and 354. For example, the application can turn each stage on / off by specifying specific values ​​for these parameters. An enabled stage will modify the audio signal according to the parameters specified by the application, while a disabled stage will pass the audio signal through without modification.

[0043] If the sound exceeds a certain saturation threshold, then the limiter at the end of each channel chain can help limit the output of the sound, thereby avoiding oversaturation in the channel. For example, the API allows the application to set the saturation threshold in the limiter by setting a parameter for "THRESHOLD". Limiters from different channels are linked via the LINK_GROUP parameter (represented by the variable LINK_GROUP 356) to allow the channels to react in a similar manner when any of the linked limiters are engaged. That is, if any one of the linked limiters is engaged, then all other linked channels are affected in the same manner. Therefore, for example, when a stereo signal saturates one of the channels, the stereo image does not suddenly move to one channel, but rather remains stable by controlling all channels in a similar manner based on the corresponding limiters. For example, when the output of the left stereo channel is attenuated to 90% of its original value by its corresponding limiter, the output of the right stereo channel can also be attenuated to 90% by its limiter, thereby preserving the relative amplitude between the channels.

[0044] EQ parameters 330, MBC parameters 340, and limiter parameters 350 may additionally include variables IN_USE 332, IN_USE 342, and IN_USE 352, respectively, which may indicate whether the corresponding level is being actively used or configured to process audio. In some embodiments, when variables IN_USE 332, IN_USE 342, and / or IN_USE 352 indicate (e.g., when an operating system, application, and / or audio processing effect is activated) that one or more of the corresponding audio levels are deactivated, the operating system may de-allocate computing resources that should have been dedicated to one or more corresponding audio levels. Limiter parameters 350 may also specify ATTACK_TIME 358, RELEASE_TIME 360, RATIO 362, THRESHOLD 364, and POST_GAIN 366, each of which may further define, for example, how the limiter 206 processes the audio signal. Similarly, EQ band parameters 370 for band 338 may include BAND_COUNT 374 (e.g., band 5 of N) and gain (GAIN) 378. MBC band parameters 380 for band 348 may include BAND_COUNT 382, ​​ATTACK_TIME 384, RELEASE_TIME 385, ratio 386, threshold 387, KNEEW_WIDTH 388, NOISE_GATE_THRESHOLD 389, EXPANDER_RATIO 390, PRE-GAIN 391, and POST-GAIN 392. Additionally, each of channel 0, channel 1, and channel K-1 may be associated with a corresponding input gain 300, 310, and 320. Notably, other embodiments may include ratios 386, 387, and 388. Figure 3 More parameters or fewer parameters as shown in .

[0045] Figure 5 A sample process of a multi-band compressor according to an example aspect of the present disclosure is illustrated. Specifically, Figure 5 The transfer function for one of the frequency bands of the MBC is graphically illustrated, showing the relationship between the input to the MBC (i.e., MBC input 514 shown on the horizontal axis) and the output of the MBC (i.e., MBC output 516 shown on the vertical axis). It is noteworthy that the smaller the number along the horizontal or vertical axis, the quieter the physical sound represented thereby (e.g., -10 represents a louder sound than -90). The API allows parameters to be set for the MBC stage to process the audio signal input from the Pre-EQ stage and output to the Post-EQ stage. For example, if Figure 5As shown in , the noise gate threshold 512, the knee width 504, the threshold 508, the expansion ratio 510, and the compression ratio 506 may be specified by parameters from the application.

[0046] Figure 5 Additionally shown are an input waveform 500 and an output waveform 502 of an audio signal to illustrate the difference in the audio signal before and after the signal processing of MBC is applied, respectively. Although the input waveform 500 includes a portion with a relatively high amplitude (e.g., a left portion thereof) and a portion with a relatively low amplitude (e.g., a right portion thereof), the output waveform 502 compresses and expands these portions of the input waveform 500, respectively. Therefore, the output waveform 502 represents a smaller amplitude range. It is noteworthy that Figure 5 The graph of may correspond to a single frequency band supported by MBC. Thus, additional similar graphs may be used to illustrate the input-output transfer functions of other frequency bands of MBC.

[0047] Figure 6 600 (e.g., an API function for obtaining the current value of a parameter) and a setter 602 (e.g., an API function for modifying the value of a parameter) in the MBC stage according to an example aspect of the present disclosure. It is noteworthy that the parameter getter 600 and the parameter setter 602 are similar to Figure 3 corresponds to a subset of the MBC band parameters 380 shown in . Figure 7 1 illustrates a sample listing of a parameter getter 720 and a setter 722 in a slicer stage according to an example aspect of the present disclosure. The parameter getter 720 and the parameter setter 722 are Figure 3 corresponds to a subset of the limiter parameters 350 shown in . Figure 7 Also illustrated is a graph 724 of an example input-output transfer function of the limiter stage. That is, the graph 724 illustrates a limiter input 726 along the horizontal axis, a limiter output 728 along the vertical axis, a threshold 730 of the limiter stage, and a limiter ratio 732 of the limiter stage.

[0048] The flexible audio processing architecture allows application developers to easily configure audio processing effects by enabling / disabling stages and configuring band and frequency limits along with other available parameters to suit the actual signal flow required by the application.

[0049] In some implementations, audio for different media may be post-processed in a studio to have enhanced dynamic range and equalization, depending on the distribution medium. However, with a flexible audio processing architecture that can be applied to high-resolution audio files, the dynamic range and equalization can be adapted to different outputs on a computing device to play back the audio file, or can be saved for further distribution of the audio file.

[0050] In some embodiments, algorithms that help enhance hearing can be adapted to a flexible audio processing architecture. Using configurable band and multi-band compressors and EQ for compensation, various types of hearing enhancement algorithms can be implemented. A similar configuration can be used in a flexible audio processing architecture for a personal sound amplifier. That is, various audio processing algorithms / processes similar to those provided by a hearing aid or other sound amplifier can be generated through an API.

[0051] In some implementations, the dynamic audio processing effects of the subject technology can be utilized to implement an algorithm that adapts the output level to a target level to compensate for loudness.

[0052] Figure 8 An exemplary multi-channel audio processing effect according to an exemplary aspect of the present disclosure is illustrated. Specifically, Figure 8 The diagram illustrates a microphone 820 configured to capture and / or generate an audio signal, left parameters 800 specifying the operation of Pre-EQ and gain 804, MBC 806, Post-EQ and gain 808, and limiter 810, and right parameters 802 specifying the operation of Pre-EQ and gain 812, MBC 814, Post-EQ and gain 816, and limiter 818, and a speaker 822 configured to generate an audible representation of the processed audio signal.

[0053] In one or more implementations, a computing device may include a smart hearing feature. The smart hearing feature may utilize post-processing audio effects (e.g., dynamic audio processing effects) that use a flexible audio processing architecture to implement audio processing to enhance sound. The smart hearing feature may also utilize an accessibility service that provides an accessibility service user interface (UI) screen to initiate the smart hearing feature and control parameters and presets of the sound.

[0054] In some embodiments, the smart hearing feature may be invoked when a user enables the smart hearing feature from an accessibility menu provided by an accessibility service, when a headset or earphone (e.g., wired, wireless) is connected to the computing device, when a particular application is launched on the computing device, and / or when a button or combination of buttons is pressed. The sound may be captured by a microphone of the computing device or a microphone of a headset connected to the computing device (e.g., either may be represented by microphone 820). The accessibility service UI screen may allow the user to control the level and overall effect and process of the sound as well as the source of the microphone.

[0055] The API for a flexible audio processing architecture allows an application (e.g., an accessibility service) to set parameters (e.g., parameters 800 and 802) for each channel (e.g., an audio channel for the left ear and an audio channel for the right ear) separately to enhance or compensate for the user's hearing. For example, in compensated hearing, each ear can be treated independently with its own set of parameters to process the audio signal. For the left and right ears, the source signal (e.g., the signal from a single microphone 820) can be the same (i.e., the left and right outputs of microphone 820 can be the same), but the parameters are set independently for each ear (i.e., left ear parameters 800 and right parameters 802) to provide personalization and fine-tuning of the audio signal to enhance or compensate for the user's hearing.

[0056] like Figure 8 As shown in , the source audio signal (e.g., generated by a microphone 820 connected to a computing device) can be the same for the left and right channels and can be input through a flexible audio processing architecture. The left channel processes the audio signal through successive audio stages 804, 806, and 808 in the left channel based on parameters set by the application (i.e., left parameters 800), and the audio signal reaches a limiter 810 in the left channel. The right channel processes the audio signal through successive audio stages 812, 814, and 816 in the right channel based on parameters set by the application (i.e., right parameters 802), and the audio signal reaches a limiter 818 in the right channel.

[0057] The limiters 810 and 818 of the left and right channels can be connected, respectively (e.g., via LinkGroup parameters) to synchronize and preserve the stereo image when one of the channels is engaged. For example, when a saturation threshold is set for the limiter of the left channel and the limiter 810 of the left channel detects that the audio signal exceeds the set threshold, the limiter 818 of the right channel can be configured to limit the output of the audio signal in a manner similar to the limiter 810 of the left channel. For example, when the left limiter 810 attenuates the amplitude of the signal provided thereto by 15%, the right limiter 818 can similarly attenuate the amplitude of the signal provided thereto by 15%. Therefore, in the event that one of the two limiters is saturated, the limiters of the left and right channels can be connected to synchronize and preserve the stereo image.

[0058] Fig. 9 900 for a list of control parameters for a smart hearing feature according to an example aspect of the present disclosure. Notably, Fig. 9 Pictured Figure 3 A subset of the parameters shown in are selected to be modified to achieve the desired audio processing characteristics for the smart hearing feature. Fig. 9 The parameters selected for each of the pre-EQ stage, MBC stage, Post-EQ stage, and limiter stage are illustrated. Fig. 9 Parameters not shown in the drawings may remain set to their respective default or otherwise predetermined values.

[0059] Fig.10 Sample accessibility service UI screens 1000, 1002, and 1004 are illustrated according to example aspects of the present disclosure. Accessibility services provided by a computing device or its operating system allow a computing device to improve sound. The accessibility service may allow real-time microphone input to pick up and modify sounds from the immediate environment of the computing device. Specifically, the sound may be improved using, for example, an application called "Sound Amplifier." As shown in UI 1000, this application may be enabled through an "Accessibility" menu provided by the operating system. That is, when UI icon 1006 is selected, the operating system may provide UI 1002, allowing a user to enable the "Sound Amplifier" application / service through UI icon 1008 (e.g., by selecting a check box therein).

[0060] Enabling the "Sound Amplifier" accessibility service through UI icon 1008 may cause the operating system to provide a UI 1004 through which parameters of the service may be adjusted. That is, the accessibility service UI screen 1004 provided by the accessibility service may be provided with sliders (e.g., two sliders per channel, including a loudness slider and a tuning slider) that represent the parameters in a simplified form and provide intuitive manipulation of the parameters to achieve the purpose of satisfying the audio signal processing effect for the computing device user.

[0061] It is worth noting that application developers may be able to directly use the API for the flexible audio processing architecture. That is, developers may be able to directly modify, for example, Figure 3 However, directly controlling the parameters of the flexible audio processing architecture of the computing device may require technical knowledge of audio signal processing. Thus, in some embodiments, the user may be presented with Fig.10 1004) to simplify the selection of parameters. For example, the values ​​selected by the "left loudness" slider, the "left tuning" slider, the "right loudness" slider, and the "right tuning" slider in UI 1004 can be mapped to multiple different parameters that can be used for adjustment (e.g., Figure 3 ).

[0062] Fig.11Another sample smart hearing UI 1100 and 1102 according to example aspects of the present disclosure are illustrated. The accessibility service allows a user of a computing device to access a smart hearing feature, for example, by selecting icon 1112 (“Personal Sound Amplification”) through UI 1100. Alternatively, in some embodiments, the icon may be presented on a display of the computing device (e.g., on its home screen) as a shortcut to access the smart hearing feature. Thus, while the accessibility service (e.g., UI 1100) may be the primary access point for smart hearing functionality, a shortcut to accessing the accessibility service may be available to the user outside of the accessibility menu. Selection of the shortcut and / or icon 1112 may cause the operating system to provide UI 1102, through which the parameters of the smart hearing feature may be modified. Notably, UI 1102 may be an alternative implementation of UI 1004.

[0063] like Fig.11 As shown in FIG. 1 , smart hearing UI 1102 can provide a user of a computing device with the ability to control audio channels for both the left and right ears simultaneously. That is, by unchecking the checkbox in UI element 1110, the user can indicate that the left and right channels are to be adjusted simultaneously, rather than independently as in UI 1004. In this case, the number of sliders can be increased from two sliders per ear (e.g., Fig.10 1004) to a total of two sliders 1104 and 1106 (as shown in the UI 1004 of FIG. Fig.11 1102). However, Fig.10 As shown in FIG. 1 , the smart hearing UI 1004 may also allow independent control of parameters of the audio channels for the left and right ears (e.g., when the checkbox in UI element 1110 is selected). The user interface 1102 may also present an option (e.g., via UI element 1108) for selecting whether the application uses a built-in microphone of the computing device or a head-mounted microphone connected to the computing device.

[0064] The accessibility service of the operating system (e.g., represented by UI 1000, 1002, 1004, 1100, and 1102) can utilize the API to dynamically adjust the flexible audio processing architecture. The accessibility service can detect and verify that a headphone device is connected to the computing device. When a headphone device is detected, the accessibility service can determine whether to use a microphone built into the computing device or to use the detected headphone device. The accessibility function service can control the start and stop of recording ambient sound through the selected microphone. The accessibility service can (e.g., based on user selection) adjust multiple audio channels (e.g., right ear and left ear) at the same time, or can adjust multiple audio channels independently of each other. The accessibility service can store preferred parameters used or applied in certain situations (e.g., at a specific location, at a specific time, etc.) in the memory of the computing device.

[0065] Fig.12 A sample multidimensional map is illustrated according to an example aspect of the present disclosure. Fig.10 UI 1004, Fig.11 1102 and / or Fig.12 1202, the accessibility service can utilize the API to map the simplified parameters provided by these UIs (e.g., 2 to 4 parameters provided with slider settings) to all parameters of the flexible audio processing architecture (e.g., Figure 3 ). For example, UI 1202 may provide a user with the ability to control two audio channels (e.g., left and right) via two sliders 1204 and 1206 in UI 1202. However, these two sliders 1204 and 1206 have been reduced from (i.e., represent) the more than 100 parameters available in the flexible audio processing architecture. Thus, an operating system and / or software application may utilize mapping 1200 to determine the values ​​of multiple parameters exposed through the API based on the values ​​set for one or more UI sliders (e.g., 1204 and 1206), and vice versa.

[0066] Thus, mapping 1200 may represent a mathematical transformation or embedding that associates the value of an API parameter with the value of 2, 4, or other number of sliders (or other UI elements, through which values ​​may be provided) provided by UI 1004, 1102, and / or 1202. It is noteworthy that mapping 1200 may be configurable to accommodate various possible numbers of sliders or UI elements. That is, mapping 1200 may be an R:P mapping, where R represents the number of values ​​obtained by the UI, R represents the number of API parameters, and the values ​​of P and R may vary between implementations. For example, UI 1202 illustrates an additional noise reduction slider 1208 that allows the degree of noise reduction applied to an audio signal to be adjusted by a flexible audio processing architecture. The noise reduction slider 1208 may, for example, set a noise amplitude threshold, below which the signal is considered noise. Depending on the values ​​of loudness 1204 and tuning 1206, the value set by the noise reduction slider 1208 may affect different frequencies to different degrees as indicated by mapping 1200. Thus, the example of user UI 1202 may indicate that R is equal to three. In other examples, additional similar sliders or UI elements may be provided to adjust other aspects of the audio signal, thereby allowing other R values ​​to be used.

[0067] Fig.13A A sample multidimensional map 1300 according to an example aspect of the present disclosure is illustrated. Specifically, the map 1300 represents the projection of various candidate combinations of values ​​of multiple parameters exposed through the API on a two-dimensional space defined by boost 1302 and pitch 1304. It is worth noting that in some embodiments, boost 1302 can correspond to loudness set through UI 1004, 1102 and / or 1202, and pitch 1304 can correspond to pitch set by these UIs. Therefore, each point in the map 1300 corresponds to a specific combination of values ​​of audio parameters exposed by the API. By collecting a large number of such projections for various different combinations of parameter values, data points can be densely populated in the boost-pitch space. Thus, when target values ​​are specified for boost 1302 and pitch 1304, these values ​​can define the target point within the map 1300. By finding the point closest to the target point in the map 1300 and determining the parameter value corresponding to this closest point, the parameter to be set through the API can be determined. In another example, a plurality of points surrounding a target point may be determined (eg, within a threshold distance of the target point), and parameter values ​​associated with each of the points may be interpolated to determine parameters to be set for the target point via the API.

[0068] In some embodiments, dimensionality reduction and principal component analysis can be performed to reduce the total number of parameters exposed by the API to the total number of parameters provided by the user interface (e.g., to determine mapping 1300), thereby allowing a user to easily navigate the parameter space of audio parameters via the UI. Notably, mapping 1300 can be an example of mapping 1200.

[0069] Fig. 13B Results of analyses of thresholds for hearing sounds emitted by various sources under various conditions are illustrated. For example, a large amount of data (e.g., collected under a number of different ambient noise conditions) related to hearing thresholds for hearing-impaired users and normal / normal hearing users is analyzed. Threshold shifts (i.e., changes in the minimum audible amplitude of sounds having a particular frequency) that occur based on ambient noise (e.g., restaurants, cafeterias, theaters, etc.) and audio content (e.g., dialogue, movie audio, music, performances, etc.) are analyzed.

[0070] Specifically, Fig. 13B A graph of audible sound (i.e., sound pressure level in decibels (dB SPL) 1308, represented along the vertical axis) is illustrated as a function of the frequency of the sound (i.e., Hertz (HZ) 1306, represented along the horizontal axis). Curve 1310 illustrates a baseline threshold for hearing in a quiet environment. That is, curve 1310 indicates how loud a sound needs to be before a human ear hears a given frequency. Similarly, curve 1312 illustrates a threshold shift for hearing in an environment containing audible noise (e.g., relative to curve 1310). Additionally, curve 1314 illustrates a threshold shift for hearing due to listening conditions (e.g., relative to curve 1310).

[0071] Based on the analysis of the collected data, a formula for navigating the parameter space can be developed and maximized. This formula allows Fig.10 and Fig.11 The parameters exposed by the UI in are related to the multiple parameters exposed by the API. Fig. 13B The analysis shown in , can relate parameters in a frequency-dependent and environmental condition-dependent manner, so that certain frequencies can be boosted to different degrees depending on the listening conditions and / or noise conditions existing at a given time.

[0072] Fig.14 An example network environment 100 for dynamic audio processing effects according to the subject technology is illustrated. The network environment 100 includes computing devices 102, 104, and 106, a server 110, and a storage device 112. In some aspects, the network environment 100 may have more than Fig.14 More or fewer computing devices (e.g., 102-106) and / or servers (e.g., 110) as shown.

[0073] Each of computing devices 102, 104, and 106 may represent various forms of processing devices having a processor, memory, and communication capabilities. Computing devices 102, 104, and 106 may communicate with each other, with server 110, and / or with Fig.14 As non-limiting examples, the processing device may include a desktop computer, a laptop computer, a handheld computer, a personal digital assistant (PDA), a cellular phone, a network device, a camera, a smart phone, an Enhanced General Packet Radio Service (EGPRS) mobile phone, a media player, a navigation device, an email device, a game console, a wired / wireless headset / headset, a wearable device, or a combination of any of these or other processing devices.

[0074] Each of the computing devices 102, 104, and 106 may be provided with a flexible audio processing architecture and an API for an application to configure the built-in flexible audio processing architecture to achieve a desired audio signal processing effect. The application may be installed as a client application on the computing devices 102, 104, and 106. The computing devices 102, 104, and 106 may be associated with a single user. Preferred API parameters set for the flexible audio processing architecture may be transmitted to and received from the server 110 via the network 108.

[0075] The network 108 can be a computer network, such as a local area network (LAN), a wide area network (WAN), the Internet, a cellular network, or a combination thereof, connecting any number of mobile clients, fixed clients, and servers. In addition, the network 108 can include, but is not limited to, any one or more of the following network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, tree or hierarchical networks, etc. In some aspects, the communication between each client (e.g., computing devices 102, 104, and 106) and a server (e.g., server 110) can occur via a virtual private network (VPN), a secure shell (SSH) tunnel, a secure socket layer (SSL) communication, or other secure network connection. In some aspects, the network 108 can also include a corporate network (e.g., an intranet) and one or more wireless access points.

[0076] Server 110 may represent a single computing device, such as a computer server including a processor and a memory. The processor may execute computer instructions stored in the memory. Server 110 is configured to communicate with client applications (e.g., applications) on client devices (e.g., computing devices 102, 104, and 106) via network 108. For example, when a user switches a device from computing device 102 to computing device 106, server 110 may transmit preferred API parameters received from computing device 102 to computing device 106. In one or more embodiments, computing device 102, computing device 104, computing device 106, or server 110 may be the following about Fig.15 The electronic system components discussed may include all or part of the electronic system components.

[0077] For example, preferred parameters may be associated with a user profile (e.g., a user of computing device 102, 104, or 106) or a device profile (e.g., computing device 102, 104, or 106). Preferred parameters associated with a user profile may be shared between various devices (e.g., computing devices 102, 104, or 106) used by a user. For example, when a user switches from a first device (e.g., computing device 102) to a second device (e.g., computing device 106), if the second device has a music playback application installed, the preferred parameters of the music playback application on the first device (e.g., computing device 102) may be shared from the second device (e.g., computing device 106). In some embodiments, preferred parameters may be shared via server 110. Preferred parameters associated with a device (e.g., computing device 104) profile may be globally applied to applications installed on computing devices (e.g., computing device 104).

[0078] To the extent that the systems discussed herein collect personal information about users or can utilize personal information, users may be provided with the opportunity to control whether a program or feature collects user information (e.g., information about a user's contacts, a user's preferences, or a user's current location). Users may also be provided with the option of turning on or off certain features or functions provided by the system. In addition, before certain data is stored or used, it may be processed in one or more ways to remove personally identifiable information. For example, the user's identity may be processed so that no personally identifiable information can be determined for the user, or the user's geographic location may be generalized to a place where location information is obtained (such as generalized to a city, zip code, or state) so that the user's specific location cannot be determined. Thus, users may control how information about the user is collected and how the system uses it.

[0079] In some embodiments, when the speakers on the computing device may not accommodate low frequencies, a speaker tuning application provided by the application can utilize the flexible audio processing architecture and API to set parameters to utilize the EQ features of the flexible audio processing architecture to compensate for the lack of low frequencies of the speakers.

[0080] In some embodiments, applications can take advantage of the flexible audio processing architecture and API to accommodate ambient noise by setting parameters for MBC to reduce loud noises and tune out quiet sounds.

[0081] In some embodiments, the API of the flexible audio processing architecture can access a database including information about the type of speakers or microphones. Based on the type of speakers or microphones used in the application, the API can include parameters to compensate for the sound output by the speakers or picked up by the microphone. For example, when the speakers on the computing device do not accommodate low frequencies, the flexible audio processing architecture can be set to compensate for the lack of low frequencies of the speakers.

[0082] In some embodiments, a microphone connected to the computing device can pick up sound output from the computing device. Based on the sound picked up from the microphone, the flexible audio processing architecture can be adjusted to dynamically tune the sound to improve the quality of the sound.

[0083] In some embodiments, a user can set a mode so that the flexible audio processing architecture processes the audio signal so that when sound (e.g., a movie) is output through a speaker of a computing device (e.g., a mobile device), certain sounds (e.g., explosions in a movie) are reduced while certain other sounds (e.g., dialogue in a movie) are maintained or emitted.

[0084] In some embodiments, the operating system of the computing device prohibits applications from directly accessing the phone audio data path.Having a flexible audio processing architecture and API built into the operating system of the computing device allows applications to set parameters to modify the phone audio signal without directly accessing the phone audio data path.

[0085] In some embodiments, the subject technology of the flexible audio processing architecture built into the operating system provides for reducing the delay of, for example, playing a movie on a computing device. Because the flexible audio processing architecture is provided within the operating system, the audio of the movie does not need to be enhanced in the application and transmitted to the operating system. That is, the application can transmit the audio and necessary parameters to the flexible audio processing architecture via an API, thereby reducing the delay caused by the audio being processed in the application. Therefore, the image and audio of the movie are synchronized.

[0086] Fig.15An example electronic system 700 is conceptually illustrated, with which some embodiments of the subject technology can be implemented. Electronic system 700 can be a computer, a phone, a personal digital assistant (PDA), or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. Electronic system 700 includes bus 708, (one or more) processors 712, system memory 704, read-only memory (ROM) 710, permanent storage device 702, input device interface 714, output device interface 706, and network interface 716.

[0087] Bus 708 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of electronic system 700. For example, bus 708 communicatively connects processor(s) 712 with ROM 710, system memory 704, and permanent storage device 702.

[0088] Processor(s) 712 retrieve instructions to execute and data to process from these various memory units in order to perform the processes of the subject disclosure. In various implementations, the processing unit(s) may be a single processor or a multi-core processor.

[0089] ROM 710 stores static data and instructions needed by processor(s) 712 and other modules of the electronic system. On the other hand, permanent storage device 702 is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 700 is turned off. Some embodiments of the subject disclosure use a mass storage device (e.g., a magnetic or optical disk or flash memory) as permanent storage device 702.

[0090] Other embodiments use a removable storage device (e.g., a floppy disk, a flash drive) as the permanent storage device 702. Like the permanent storage device 702, the system memory 704 is a read-write memory device. However, unlike the storage device 702, the system memory 704 is a volatile read-write memory, such as a random access memory. The system memory 704 stores some instructions and data required by the processor at run time. In some embodiments, the processes disclosed in the subject matter are stored in the system memory 704, the permanent storage device 702, or the ROM 710. For example, the various memory units include instructions for displaying graphical elements and identifiers associated with the corresponding application, receiving predetermined user input to display a visual representation of a shortcut associated with the corresponding application, and displaying a visual representation of the shortcut. The processor (one or more) 712 retrieves instructions to be executed and data to be processed from these various memory units in order to perform the processes of some embodiments.

[0091] The bus 708 is also connected to input and output device interfaces 714 and 706. The input device interface 714 enables a user to transmit information to the electronic system and select commands. Input devices used with the input device interface 714 include, for example, an alphanumeric keyboard, a pointing device (also referred to as a "cursor control device"), and a sound capture device (e.g., a microphone). The output device interface 706 enables, for example, the display of images generated by the electronic system 700. The output devices used with the output device interface 706 include, for example, a printer, a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)), and a sound playback and / or transmission device (e.g., a speaker). Some embodiments include devices, for example, a touch screen that acts as both an input device and an output device.

[0092] Finally, if Fig.15 As shown in , bus 708 also couples electronic system 700 to a network (not shown) via network interface 716. In this manner, the computer can be part of a computer network, such as a LAN, WAN, or intranet or a network of networks (e.g., the Internet). Any or all components of electronic system 700 may be used in conjunction with the subject disclosure.

[0093] Fig.16 Illustrated is a flow chart 1600 of operations associated with using an audio processing stage provided by an operating system. The operations may be performed by and / or used with computing devices 102-106, electronic system 700, or any other device in the aforementioned example embodiments.

[0094] Block 1602 involves providing, for each respective audio channel of a plurality of audio channels provided by an operating system of a computing device, a set of sequential audio levels to apply to the respective audio channel.

[0095] Block 1604 involves providing, by the operating system, an application programming interface (API) configured to set a plurality of parameters for adjusting a set of successive audio levels for each respective audio channel.

[0096] Block 1606 involves receiving, via the API and from an application running on the computing device, one or more values ​​for one or more parameters of the plurality of parameters.

[0097] Block 1608 involves adjusting, by the operating system, the plurality of audio channels based on the received one or more values ​​for the one or more parameters.

[0098] In some embodiments, the application may be provided by a third party other than the provider of the operating system of the computing device.

[0099] In some embodiments, the set of successive audio stages for each respective audio channel may include one or more of: (i) an input gain stage, (ii) a pre-equalization stage, (iii) a multi-band compression stage, (iv) a post-equalization stage, or (v) a limiter stage.

[0100] In some embodiments, the plurality of parameters for adjusting the set of successive audio levels for each respective audio channel may include, for each respective one of the successive audio levels, a first parameter for enabling or disabling the respective audio level.

[0101] In some embodiments, when the corresponding audio stage is enabled, audio data received by the corresponding audio channel may be modified according to a subset of the plurality of parameters for the effect audio stage. When the corresponding audio stage is disabled, audio data may pass through the corresponding audio stage without modification.

[0102] In some embodiments, a first audio level in a set of consecutive audio levels may be adjustable to support a first number of frequency bands. A second audio level in a set of consecutive audio levels may be adjustable to support a second number of frequency bands. One or more values ​​for one or more parameters may indicate (i) a first subset of the first number of frequency bands to which the first audio level is adjusted, and (ii) a second subset of the second number of frequency bands to which the second audio level is adjusted.

[0103] In some embodiments, the first number of frequency bands may be different from the second number of frequency bands.

[0104] In some embodiments, one or more values ​​for one or more parameters may indicate (i) a first cutoff frequency for a first frequency band among a plurality of frequency bands supported by a first audio stage in a set of consecutive audio stages, and (ii) a second cutoff frequency for a second frequency band among the plurality of frequency bands.

[0105] In some embodiments, the first cutoff frequency may be different from the second cutoff frequency.

[0106] In some embodiments, the set of sequential audio stages for each respective audio channel may include a limiter stage. A first limiter stage for a first audio channel may be linked to a second limiter stage for a second audio channel such that, when the first limiter stage attenuates the output of the first audio channel by a first degree, the second limiter stage attenuates the output of the second audio channel by a second degree proportional to the first degree.

[0107] In some embodiments, a stereo audio signal may be defined by a first audio channel and a second audio channel.

[0108] In some embodiments, the API may provide a plurality of API functions through which a plurality of parameters may be adjusted. Receiving one or more values ​​for the one or more parameters may involve receiving (i) a request to execute one or more API functions of the plurality of API functions corresponding to the one or more parameters, and (ii) one or more values ​​as respective inputs to the one or more API functions. Adjusting the plurality of audio channels based on the received one or more values ​​may involve executing the one or more API functions with the one or more values ​​as respective inputs.

[0109] In some embodiments, the API may be configured to adjust one or more additional values ​​of one or more additional parameters of the plurality of parameters based on at least one of: (i) a type of microphone used by the application or (ii) a type of speaker used by the application.

[0110] In some embodiments, the plurality of audio channels may include a left audio channel and a right audio channel. The application may be configured to enhance the user's sense of hearing by receiving an audio signal from a microphone connected to the computing device. The application may provide the audio signal to (i) the left audio channel and (ii) the right audio channel. The audio signal may be modified by the left audio channel according to a first parameter set for the left audio channel. The audio signal may be modified by the right audio channel according to a second parameter set for the right audio channel. The application may receive a first output of (i) the left audio channel and a second output of (ii) the right audio channel. The application may also generate a left output sound based on the first output by a first speaker connected to the computing device, and generate a right output sound based on the second output by a second speaker connected to the computing device.

[0111] In some embodiments, the plurality of parameters may be a plurality of API parameters. The one or more values ​​of the one or more parameters may be a first one or more values ​​of the one or more API parameters. The application may be configured to provide a user interface through which a plurality of application parameters for adjusting a set of successive audio levels of each corresponding audio channel may be adjusted. The plurality of application parameters may include fewer parameters than the plurality of API parameters. The application may receive a second one or more values ​​of one or more application parameters in the plurality of application parameters through the user interface. The application may also determine a mapping between (i) one or more application parameters and (ii) one or more API parameters. The application may also determine a first one or more values ​​based on the mapping and the second one or more values.

[0112] In some embodiments, the second one or more values ​​may be received via one or more sliders provided via a user interface.

[0113] In some embodiments, the application can be configured to determine one or more values ​​for one or more parameters based on one or more of: (i) the geographic location of the computing device, (ii) noise conditions in the environment of the computing device, (iii) the time of day, (iv) user preferences associated with the computing device, or (v) the output of an artificial intelligence algorithm.

[0114] Many of the above features and applications are implemented as software processes specified as sets of instructions recorded on computer-readable storage media (also referred to as computer-readable media). When these instructions are executed by one or more processing units (e.g., one or more processors, cores of processors, or other processing units), they cause the (one or more) processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, magnetic media, optical media, electronic media, etc. Computer-readable media do not include carrier waves and electronic signals passed wirelessly or through wired connections.

[0115] In this specification, the term "software" is intended to include, for example, firmware residing in a read-only memory or other form of electronic storage, or applications that can be stored in a magnetic storage device, optical, solid-state, etc., which can be read into a memory for processing by a processor. Moreover, in some embodiments, multiple software aspects disclosed in this subject matter can be implemented as sub-parts of a larger program while retaining different software aspects disclosed in this subject matter. In some embodiments, multiple software aspects can also be implemented as separate programs. Finally, any combination of separate programs that implement the software aspects described herein together are within the scope of this subject matter disclosure. In some embodiments, the software program, when installed to operate on one or more electronic systems, defines a specific machine implementation of one or more executions and operations of the software program.

[0116] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages, declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store portions of one or more modules, subroutines, or code). A computer program may be deployed to execute on one computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0117] These functions can be implemented in digital electronic circuits, computer software, firmware or hardware. One or more computer program products can be used to implement these techniques. Programmable processors and computers can be included in mobile devices or packaged as mobile devices. Processes and logic flows can be performed by one or more programmable processors and by one or more programmable logic circuits. General and special computing devices and storage devices can be interconnected through communication networks.

[0118] Some embodiments include electronic components such as microprocessors, storage devices, and memories that store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), read-only digital versatile disks (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro-SD card, etc.), magnetic or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer codes include machine code, such as produced by a compiler, and files including high-level code that is executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0119] Although the above discussion refers primarily to microprocessors or multi-core processors that execute software, some implementations are performed by one or more integrated circuits (e.g., application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs)). In some implementations, such integrated circuits execute instructions stored on the circuits themselves.

[0120] As used in this specification and any claims of this application, the terms "computer", "server", "processor" and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms "display" or "displaying" mean displaying on an electronic device. As used in the specification and any claims of this application, the terms "computer-readable medium" and "computer-readable medium (media)" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other temporary signals.

[0121] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT or LCD monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, a computer may interact with a user by sending documents to or receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0122] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server) or includes a middleware component (e.g., an application server) or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), interconnections (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0123] A computing system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., an HTML page) to a client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input from it). Data generated at the client device (e.g., a result of a user interaction) may be received from the client device at the server.

[0124] It should be understood that any specific order or hierarchy of steps in the disclosed process is an illustration of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged, or all the steps shown can be performed. Some of the steps can be performed simultaneously. For example, in some cases, multitasking and parallel processing can be advantageous. Moreover, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0125] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should conform to the full scope consistent with the language claims, wherein elements mentioned in the singular are not intended to mean "one and only one", unless explicitly stated, but rather "one or more". Unless otherwise specifically stated, the term "some" refers to one or more. Male pronouns (e.g., his) include female and neutral genders (e.g., her and its), and vice versa. Titles and subtitles (if any) are used merely for convenience and do not limit the subject disclosure.

[0126] As used herein, the phrase "at least one of" preceding a series of items, the use of the terms "and" or "or" to separate any items, modifies the list as a whole, rather than each member (e.g., each item) of the list. The phrase "at least one of" does not require selection of at least one of each listed item; rather, the phrase permits a meaning that includes at least one of any one item and / or at least one of any combination of items and / or at least one of each item. For example, the phrase "at least one of A, B, and C" or "at least one of A, B, or C" each refers only to A, B, or C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0127] Phrases (such as aspect, this aspect, another aspect, some aspects, one or more aspects, implementation, this implementation, another implementation, some implementations, one or more implementations, example, this example, another example, some examples, one or more examples, configuration, this configuration, another configuration, some configurations, one or more configurations, subject technology, the disclosure, the present disclosure, other variations thereof, etc.) are for convenience and do not mean that the disclosure associated with such (one or more) phrases is essential to the subject technology or that such disclosure applies to all configurations of the subject technology. The disclosure associated with such (one or more) phrases may apply to all configurations, or one or more configurations. The disclosure associated with such (one or more) phrases may provide one or more examples. Phrases (such as aspect or some aspects) may refer to one or more aspects, and vice versa, and this applies similarly to the other aforementioned phrases.

[0128] All structural and functional equivalents of the elements throughout the various aspects of the present disclosure description that are known or will be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the subject technology. Moreover, regardless of whether such disclosure is explicitly described in the above description, anything disclosed herein is not intended to be dedicated to the public. It is not intended to interpret any claimed element content according to the provisions of the sixth paragraph of 35 U.S.C. § 112, unless the phrase "component for..." or the phrase "step for..." is used in the case of a method claim to explicitly describe the element. In addition, to the extent that the terms "including", "having", etc. are used in the description or claims, such terms are intended to be inclusive in a manner similar to the term "including", as explained when "including" is used as a transitional word in the claims.

Claims

1. A computer-implemented method comprising: providing, for each respective audio channel of a plurality of audio channels provided by an operating system of the computing device, a set of sequential audio levels to be applied to the respective audio channel; providing, by the operating system, an application programming interface (API) configured to set a plurality of parameters for adjusting a set of successive audio levels for each respective audio channel, wherein the plurality of parameters for adjusting the set of successive audio levels for each respective audio channel comprises, for each respective audio level of the successive audio levels, a first parameter for enabling or disabling the respective audio level, wherein when a respective audio level is enabled, audio data received by the respective audio channel is modified according to a subset of the plurality of parameters for the respective audio level, and wherein when a respective audio level is disabled, audio data passes through the respective audio level without modification; receiving, via the API and from an application executing on a computing device, one or more values ​​for one or more of the plurality of parameters; and The plurality of audio channels are adjusted, by the operating system, based on the received one or more values ​​for the one or more parameters.

2. The computer-implemented method of claim 1, wherein: The application is provided by a third party other than the provider of the operating system of the computing device.

3. The computer-implemented method of claim 1, wherein: The set of sequential audio stages for each respective audio channel includes one or more of: (i) an input gain stage, (ii) a pre-equalization stage, (iii) a multi-band compression stage, (iv) a post-equalization stage, or (v) a limiter stage.

4. The computer-implemented method of claim 1, wherein: A first audio level in a set of consecutive audio levels is adjustable to support a first number of frequency bands, wherein a second audio level in the set of consecutive audio levels is adjustable to support a second number of frequency bands, and wherein one or more values ​​for the one or more parameters indicate (i) a first subset of the first number of frequency bands to which the first audio level is adjusted, and (ii) a second subset of the second number of frequency bands to which the second audio level is adjusted.

5. The computer-implemented method of claim 4, wherein: The first number of frequency bands is different from the second number of frequency bands.

6. The computer-implemented method of claim 1, wherein: One or more values ​​for the one or more parameters indicate (i) a first cutoff frequency for a first frequency band of a plurality of frequency bands supported by a first audio stage in a set of consecutive audio stages, and (ii) a second cutoff frequency for a second frequency band of the plurality of frequency bands.

7. The computer-implemented method of claim 6, wherein: The first cutoff frequency is different from the second cutoff frequency.

8. The computer-implemented method of claim 1, wherein: The set of successive audio stages for each respective audio channel includes a limiter stage, wherein a first limiter stage for a first audio channel is linked to a second limiter stage for a second audio channel such that when the first limiter stage attenuates an output of the first audio channel by a first degree, the second limiter stage attenuates an output of the second audio channel by a second degree that is proportional to the first degree.

9. The computer-implemented method of claim 8, wherein: The stereo audio signal is defined by a first audio channel and a second audio channel.

10. The computer-implemented method of claim 1, wherein: The API provides a plurality of API functions through which the plurality of parameters are adjustable, wherein receiving one or more values ​​for the one or more parameters comprises receiving (i) a request for executing one or more API functions of the plurality of API functions corresponding to the one or more parameters, and (ii) the one or more values ​​as respective inputs of the one or more API functions, and wherein adjusting the plurality of audio channels based on the received one or more values ​​comprises executing the one or more API functions with the one or more values ​​as respective inputs.

Images